log rolling the easy way

By Paul Murphy, author of The Unix Guide to Defenestration

As a unix sysadmin a big part of your responsibility involves checking and, if necessary, responding to, logs.

There's the log for the Sybase backup server, for Informix, for mail, for the system, and so on. Lots of the things. In real life you probably do something like "% tail -50 proc_log" to look at the less critical ones and have some kind of on-line or web monitor running on critical processes.

For non critical logs you can save yourself a lot of typing by automating the review process.

The first step is grab the log entries that apply to the period you want to review; typically 24 hours. You can do that by rotating the logs but I like leaving logs intact for ninty days or so at a time because that makes it easy to review things if, or when, they go wrong.

So I select rows from the log's time stamp entries. Since no two seem to do it the same way, you need a different script for each log. In general you use awk or perl to process the output from "% date" to print a selector expression like
"sed -n '/Apr 17/p' /var/log/syslog >>~/logs.rv"
and call the thing in a script like:

#!/bin/csh
cd /usr/murph
touch logs.rv; rm logs.rv
##yeah, I know, it's because I used to work with HP-UX 10 :-)
echo "system log" > ~/logs.rv
date | syslg.pl | csh
echo "Sybase log" > > logs.rv
date | syblg.pl | csh


and run this as a 7:20 AM cron job.

You can review the combined list manually or build up an error vocabulary file and use it to auto-search the log extracts. For example if:
% cat errors.wrds
/Error/p
/Group Writeable/p
/Device not responding/p


then
% sed -n -f errors.wrds logs.rv | sort | uniq
shows show whether to look for these problems and, if so, where.