httpd logs
Posted by: grymwulf
Posted on: 2006-12-09 23:53:00
Couldn't find a thread through extensive searching that answered these questions, and unfortunately I'm a little new to the concept I want to pursue. What I'd like to do is take the server generated log files for all of my sub-domains, combine them as one larger log, then load those into a DB, or even a server side MySQL db. I'm trying to experiment a bit with data mining using Transact SQL (don't ask, but it's what the place I work at uses), and I've got a local, test instance of SQL Server I can load the info into. Since what I would be doing for the job I do would be quering and analysis of time-indexed data, I figure learning off of time-indexed log files of some sort would be ideal.
Now I know where the files are located, $HOME/logs/domain.com, $HOME/logs/sub1.domain.com, etc... what I need to do is append them together. Unfortunately I don't know too much concerning shell scripting.
Basic concept is a shell script that goes something like this:
(please note this is not written in an actual language, but just summarizes my idea. I'd have to match it of course to a usable language)
dim $sites as array
$sites() = ("domain.com", "sub1.domain.com", "sub2.domain.com", "sub3.domain.com");
log.copy("/home/grymwulf/dailylog.blank", "/home/grymwulf/dailylog.log");
dim log as FILEHANDLER();
foreach $sites() as $site do {
log.append("/home/grymwulf/logs/" & $site & "/httpd/logfile." & $TEXT(($DATE() - $DAYS(1)), "yyyymmdd"), "/home/grymwulf/dailylog.log"); }
And then just build another script that emails the daily log to an email address, and deletes the daily log.
. It's a shame, as it would make what you are trying to do a *lot* easier.