Splunk for Log Searching...

by swjohnson 7/6/2009 4:39:00 AM

It's been awhile but wanted to post something that I found very useful.  

I encountered a situation where I need to parse through tens of thousands of log files for SQL Server and IIS and needed to do an analysis of what was contained within for a possible security incident and to create a standard auditing process for some compliance requirements for a new contract.  I thought about pulling the logs to a secure location, creating a SQL Server database and importing the files and using full text search but then I came across Splunk.   They have a nice little free version that allows you to index up to 500MB per day.  

With Splunk, I was able to pull down the logs to a directory, point Splunk at it and within minutes, it was all indexed and it automatically did the rest as new logs were added.   From there I was able to do research for XSSSQL Injection, Login/Logout and failed/succesful information.  

Very nice and fast!  Check Splunk out as it has tons of possiblities besides log management! 


Feeds