I recently consolidated my Python code bits for dealing with Snort and Suricata unified2 log files into a project called idstools. While I’ll be adding more than just unified2 reading support, that is about it for now.
While it can be installed with pip (pip install idstools), if you just want to play around with it I suggest cloning the repo (git clone https://github.com/jasonish/idstools.py). You can then use the REPL or write test scripts from within the idstools.py directory without having to install the library (yeah, basic stuff for Python developers).
idstools does come with a few example programs that demonstrate unified2 file reading, namely, u2fast.py, u2tail.py and u2spewfoo.py (a simple clone of the Snort provided u2spewfoo).
Basic Unified2 File Reading
from idstools import unified2 reader = unified2.FileEventReader("tests/merged.log") for event in reader: print("Event:n%s" % str(event))
This few lines of code will iterate through each record in the specified unified2 log files, aggregate the records into events and return each event as a dict.
If straight up record is reading is more what you are after then check out unified2.FileRecordReader, or the lower level unified2.read_record function.
Each event is represented as a dict containing the fields of a unified2 event record, with the associated packets represented as a list in event[“packets”] and extra data records represented as a list in event[“extra-data”].
Resolving Event Message and Classification Names
from idstools import maps # Create and populate the signature message map. sigmap = maps.MsgMap() sigmap.load_genmsg_file("gen-msg.map") sigmap.load_sidmsg_file("sid-msg.map") # Get the description for 1:498. print("Message for 1:498: %s" % (sigmap.get(1, 498).msg)) # Create and populate the classification map. classmap = maps.ClassificationMap() classmap.load_classification_file("classification.config") print("The description for classification id 9 is %s, with priority %d." % ( classmap.get(9).description, classmap.get(9).priority))
Spool Reading
from idstools import spool def my_open_hook(reader, filename): print("File %s has been opened." % (filename)) def my_close_hook(reader, filename): print("File %s has been closed." % (filename)) reader = spool.Unified2EventSpoolReader( "/var/log/snort", "merged.log", delete_on_close=False, bookmark=True, open_hook=my_open_hook, close_hook=my_close_hook) for event in reader: print("Read event with generator-id %d, signature-id %d." % ( event["signature-id"], event["generator-id"]))