EveBox Oneshot Demo

Following up on EveBox support for SQLite, I’ve created a screencast demo of oneshot mode. Oneshot is an interactive run of EveBox to process a single eve.json file and view it in the EveBox UI.

Advertisements

EveBox with SQLite

The latest builds of EveBox support an embedded SQLite database that allow it to be used without Elastic Search for lighter loads. The SQLite support was added to support two use cases that may be of interest to some.

One Shot Mode

One shot mode is the loading of a single eve.json into a temporary database and allowing the user to work with it, then cleaning up on exit. Probably most useful for loading up the Suricata log file after running over a PCAP, or just trying out EveBox for the first time. Example usage:

./evebox oneshot /path/to/eve.json

If all goes well your browser should eventually open up and display the EveBox Inbox.

Self Contained Mode

For lack of a better name, self contained mode is the usage of EveBox without any external dependencies. This is suitable for lighter loads when running EveBox on the same machine that is running Suricata. Example usage:

./evebox server --datastore sqlite --input /var/log/suricata/eve.log

The idea here is just a simple way to get a GUI for your Suricata events without messing around with any configuration or databases. However, you may want to create a configuration file and setup a retention period to keep your SQLite database size in check (more documentation coming soon).

If you have multiple Suricata instances, and believe the load to be light, you can configure an EveBox agent to send events to the SQLite enabled server, but your mileage will vary as you add more load.

Using Elastic Search?

If using Elastic Search the agent and/or the –input option may still be interesting as alternatives for shipping eve logs to Elastic Search, and open up future options for dealing with the real time event feeds from your Suricata instances.

Download here.

inbox

EveBox – Stable Repositories

I’ve been asked a few times now for “stable” APT and Yum repositories as the current ones are marked “development”, in fact they contain the packages created on Travis-CI runs of the master branch.

So I’ve added stable repos for Yum and Apt. For the short term they still contain builds out of the master branch, but uploaded by me instead of the output CI, and they will transition to only tagged releases after the next release, 0.6.0 which I will probably tag soon.

The builds in the repos above should work with any modern x86_64 Fedora, CentOS, EL, Debian or Ubuntu distribution.

And if you’d rather just get at the files, I’ve made it a little easier than the Bintray URLs make it — https://evebox.org/files/development/.

eve2pcap – Eve Packet and Payload Conversion to pcap

I’ve added a new tool to my idstools package to convert the packets (or the payloads) found in Suricata eve logs to a pcap file.

To just grab the script, download eve2pcap.py and make it executable, or to install the complete idstools package (will install as idstools-eve2pcap):

pip install –upgrade idstools

Usage is pretty simple:

./eve2pcap.py -o output.pcap /path/to/eve.json

Or to use the payload field instead of the packet field:

./eve2pcap.py -o output.pcap –payload /path/to/eve.json

For straight packet conversion no dependencies are required other than Python and libpcap. Scapy is used for conversion of the payload field, so make sure to install it before trying to convert the payload.

It is also important to note that eve logs do not contain all the information to recreate the packet headers, so when converting payloads to pcap the headers are “manufactured” and may not always produce the best packet for the payload, so YMMV.

Suricata Quick Start for Fedora 21 and 22

Fedora Linux 21 and 22 have Suricata 2.0.8 in their default repositories making it very easy to get started.  The following was done on Fedora 22, but should be applicable to Fedora 21 as well.

Install Suricata

yum install suricata

Configure Interface

By default, Suricata will be configured to run on eth0, if you need to change this, edit /etc/sysconfig/suricata and change eth0 to your desired interface.

Start Suricata

To start Suricata one time:

systemctl start suricata

To have it restarted on each boot:

systemctl enable suricata

But you’ll probably want to have it started after your network is ready. This can be done by including network-online.target in the unit file /usr/lib/systemd/system/suricata.service

[Unit]
Description=Suricata Intrusion Detection Service
After=syslog.target network-online.target

[Service]
ExecStart=/sbin/suricata -c /etc/suricata/suricata.yaml $OPTIONS
EnvironmentFile=-/etc/sysconfig/suricata

[Install]
WantedBy=multi-user.target

Verify That Suricata is Running

Even though we have not downloaded any rules yet, Suricata will still log HTTP requests, DNS requests, TLS certificates and SSH connection by default. These can be observed by monitoring /var/log/suricata/eve.log.

tail -f /var/log/suricata/eve.log

Download Some Rules

To get the most out of Suricata you will want to download some rules. The Emerging Threats Open rules are freely available and can be installed with the following commands:

cd /etc/suricata
curl http://rules.emergingthreats.net/open/suricata-2.0/emerging.rules.tar.gz | tar zxvf -
systemctl restart suricata

Another IDS rule downloader – rulecat

I didn’t mean to do it, but I created yet another tool to download IDS rules, with a specific focus on Suricata.  Like some other people, I’ve always used my own scripts for downloading rules instead of using existing tools, and I finally pulled some of that together and dropped it into my idstools project.

Installation

With pip (as its written in Python):

pip install idstools

From source, https://github.com/jasonish/py-idstools/releases/download/0.5.0/idstools-0.5.0.tar.gz, where idstools-rulecat can be run from the exploded tarball without installing (also works with a git clone).

Usage

Probably the simplest usage, assuming you have Suricata already installed is something like:

idstools-rulecat --rules-dir /etc/suricata/rules

This will download the ET Open rules for your version of Suricata and drop them in /etc/suricata/rules.

Other useful output options include:

  • –merged to merge all the rules into a single file making it easier to include into your suricata.yaml.
  • –yaml-fragment to dump a fragment of YAML for inclusion in your suricata.yaml that lists each rule file downloaded.

Of course you can also disable, enable and modify rules.  To get sample configuration files for doing so, run:

idstools-rulecat --dump-sample-configs

which will drop the following files into your current directory:

  • disable.conf – allows disabling of rules by ID or regular expression.
  • enable.conf – allows enabling of rules by ID or regular expression.
  • modify.conf – rule modifications
  • threshold.in – an input threshold.conf that idstools-rulecat will expand (experimental)

As with many Python programs using the argparse module, a configuration file can be created by dropping command line arguments in a file and calling idstools-rulecat like:

idstools-rulecat @/path/to/rulecat.conf

For example, I use a rulecat.conf like:

--suricata=/usr/bin/suricata
--merged=rules/merged.rules
--disable=disable.conf
--enable=enable.conf
--modify=modify.conf
--threshold-in=threshold.in
--threshold-out=etc/threshold.config
--post-hook=sudo kill -USR2 $(cat /var/run/suricata.pid)
--etpro=ETPRO_CODE

Note the –etpro which will ET Pro rules instead of the ET Open rules.

For full usage, head over to http://idstools.readthedocs.org/en/latest/tools/rulecat.html.

This is a work in progress, and is primarily being developed to satisfy my needs, perhaps it will be useful to others as well.