Lots of Logs

Linux/Debian:

This entry is one of those notes I talked about in my intro.

I was trying to forward my logs from a firewall that I setup using IPTables to a splunk server, and I ended up learning a good bit about the logging system. I'm sure I have not done things in the most efficient manner but for my purposes it worked well.

First I started logging some things using IPTables. I logged at the info level. As soon as I restarted the firewall the log messages started flooding my console. I logged in with putty and continued from there.

First to forward the logs I edited /etc/syslog.conf
I added:
kern.* @192.168.1.5 (Replace IPAddress with that of the server you are forwarding to)

This will forward all alerts coming from the kernal. You could be much more specific or do a *.* to forward EVERYTHING. See man syslog.conf for more information.

Next I restarted sysklogd:
/etc/init.d/sysklogd restart

That did it. The logs were forwarding just fine to the splunk server.

At this point I was getting all of my logs from the firewall showing up on my console, which made the console basically useless. First I checked my syslog.conf to see if there was an entry that was sending those logs to the console, but there weren't any. So I did some digging and found a few sites that talked about printk. It seems that printk controls what gets sent to your console along with any entries in syslog.conf that may point to your console.

So to fix it I typed:
echo "4 1 1 7" > /proc/sys/kernel/printk

That fixed the info level logs showing up on my console and to fix it for future reboots I edited /etc/sysctl.conf and added:
kernel.printk = 4 4 1 7

I then restarted sysklogd again:
/etc/init.d/sysklogd restart

A strange thing happened though. The alerts weren't showing up on the console or in the log files. I messed around for quite a while and finally fixed the problem by restarting klogd:
/etc/init.d/klogd restart

Update: It looks like depending on the version of klogd, and some other things I'm not yet aware of, when you restart klogd it might reset printk = 7 1 1 7 (Happened on one server but not the other). To fix this run "/etc/init.d/klogd stop" then run "/sbin/klogd -c 4" then run "/etc/init.d/klogd start"

That fixed things. Everything seems to work well now. Check out these web sites for further information and reference.

Information about changing printk
http://old.nabble.com/Suppressing-kernel-%27printk%27s.-td15280888.html

More information about printk
http://www.de-brauwer.be/wiki/wikka.php?wakka=printk

Great information about IPTables
http://www.faqs.org/docs/iptables

My First Blog (Yeah, thats the title I thought up all by myself!)

If you have randomly stumbled on this blog, I first apologize... This is a blog that I am writing not for the masses but just to keep some record of the things I have learned in Linux. I often spend long periods of time looking for stupid little bits of how-to on Linux just to find out that all I needed was a quick command... The problem comes later when I what to do the same thing and of course I have completely forgotten everything that I learned two days ago... So... This will be random comments on things that I find out, and a notebook of sorts for commands and other things I learn as I use Linux...

Edit: I guess this thing has turned out to be more then Linux... So it's a blog on Random things in IT (Information Technology)... We'll see if it morphs again later...