Threat Level: green Handler on Duty: Guy Bruneau

SANS ISC: SANS Internet Storm Center SANS Internet Storm Center


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!

Latest Diaries

Building an IDS Sensor with Suricata & Zeek with Logs to ELK

Published: 2021-04-10
Last Updated: 2021-04-10 17:11:01 UTC
by Guy Bruneau (Version: 1)
0 comment(s)


Over the past several years I have used multiple pre-built sensors using readily available ISO images (rockNSM, SO, OPNSense, etc) but what I was really looking for was just a sensor to parse traffic (i.e Zeek) and IDS alerts (Suricata) to ELK. To speed up the deployment of each sensors, I created a basic CentOS7 server VM where I copied all the scripts and files, I need to get Suricata & Zeek up and running. Since my ELK cluster is the recipient of these logs, it includes Elastic filebeat. I saved all the important scripts and changes into two tarballs (installation and sensor). The sensor tarball has a copy of the softflowd (netflow binary) that can be use to capture netflow data.

Using this document as a template, to build the sensor, it is time to download and extract the installation tarball on the sensor to install Suricata & Zeek as well as the Elasticsearch applications filebeate, metricbeat and packetbeat if using ELK to analyze the traffic. Refer to the document to configure each of the ELK applications.

There are two tarball, the first installation.tgz is to setup all the scripts listed below to install the software and the second tarball is to preconfigure some of the sensor configuration files (Suricata, Zeek, softflowd, Filebeat, Metricbeat & Packetbeat).

  • $ wget https://handlers.sans.edu/gbruneau/scripts/installation.tgz
  • $ wget https://handlers.sans.edu/gbruneau/scripts/sensor.tgz
  • Extract the tarball with the scripts as follow: $ sudo tar zxvf installation.tgz -C /
  • Install Suricata: $ sudo yum -y install suricata
  • Install Zeek: $ sudo yum -y install zeek

After Suricata & Zeek have been installed, if you plan to send the logs to Elasticsearch, install filebeat (metricbeat & packetbeat are optional).

  • Install Filebeat: $ sudo yum -y install filebeat (metricbeat and packetbeat)

The sensor.tgz tarball has Zeek configured to save the logs in JSON format which has support by most commercial products like ELK, RSA NetWitness, Splunk, etc.

  • Extract this tarball after installing all the packages: $ sudo tar zxvf sensor.tgz -C /

If the packet capture interface is other than ens160 (ifconfig), update the following files:

  • /opt/zeek/etc/node.cfg
  • /etc/suricata/suricata.yaml

If using packetbeat:

  • /etc/packetbeat/packetbeat.yml

If using softflowd (make script executable: chmod 755 /etc/rc.local):

  • /etc/rc.local

Enable Suricata & Zeek to start on reboot:

  • $ sudo systemctl enable suricata
  • $ sudo systemctl enable zeek

Update Suricata's rules:

  • $ sudo /usr/bin/suricata-update update --reload-command "/usr/bin/systemctl kill -s USR2 suricata"

Lets start some services:

  • $ sudo systemctl start suricata
  • $ sudo systemctl status suricata
  • $ sudo systemctl start zeek
  • $ sudo systemctl status zeek

Last, configure filebeat (metricbeat & packetbeat are optional) Elasticsearch server section to send the logs to the server. To make sure nothing is missed to configure Elasticsearch applications, review this document Logging Data to Elasticsearch which contains all the steps to configure these Elastic Beats.

  • /etc/filebeat/filebeat.yml
  • /etc/metricbeat/metricbeat.yml
  • /etc/packetbeat/packetbeat.yml

If using any of the Beats, enable them to start on reboot:

  • $ sudo systemctl enable filebeat
  • $ sudo systemctl enable metricbeat
  • $ sudo systemctl enable packetbeat

Let's start Filebeat:

  • $ sudo systemctl start filebeat
  • $ sudo systemctl status filebeat

Note: Because Suricata logs are sent to ELK with filebeat, there is an hourly cronjob that delete the previous hour logs from the /nsm/suricata directory to keep it clean and in the end requires a minimal /nsm/suricata partition documented in [4].

Since I use VMs as sensors, I exported this sensor template as an OVA, which requires minimum configuration changes for the next deployment.

[1] https://github.com/irino/softflowd
[2] https://handlers.sans.edu/gbruneau/elastic.htm
[3] https://handlers.sans.edu/gbruneau/elk/TLS_elasticsearch_configuration.pdf
[4] https://handlers.sans.edu/gbruneau/elk/Building_Custom_IDS_Sensor.pdf
[5] https://handlers.sans.edu/gbruneau/scripts/installation.tgz
[6] https://handlers.sans.edu/gbruneau/scripts/sensor.tgz

-----------
Guy Bruneau IPSS Inc.
My Handler Page
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu

0 comment(s)

If you have more information or corrections regarding our diary, please share.

Recent Diaries

No Python Interpreter? This Simple RAT Installs Its Own Copy
Apr 9th 2021
1 day ago by Xme (0 comments)

Simple Powershell Ransomware Creating a 7Z Archive of your Files
Apr 8th 2021
2 days ago by Xme (0 comments)

WiFi IDS and Private MAC Addresses
Apr 7th 2021
3 days ago by Johannes (0 comments)

Malspam with Lokibot vs. Outlook and RFCs
Apr 6th 2021
4 days ago by Jan (0 comments)

YARA and CyberChef: ZIP
Apr 4th 2021
6 days ago by DidierStevens (0 comments)

Video: YARA and CyberChef
Apr 4th 2021
6 days ago by DidierStevens (0 comments)

View All Diaries →

Latest Discussions

Handler's Diary (Full text) RSS Feeds stopt working due to a typo
created Mar 5th 2021
1 month ago by bas.auer@auerplace.nl (0 replies)

port_scan issue in Snort3
created Feb 23rd 2021
1 month ago by astraea (0 replies)

PFSense
created Dec 23rd 2020
3 months ago by bas.auer@auerplace.nl (6 replies)

Port 23 & 2323 107.173.58.179
created Nov 15th 2020
4 months ago by Anonymous (0 replies)

Gmail hacked vis MS Outlook / request.zip virus/malware
created Oct 13th 2020
5 months ago by Anonymous (3 replies)

View All Forums →

Latest News

Top Diaries

An infection from Rig exploit kit
Jun 17th 2019
1 year ago by Brad (0 comments)

Qakbot infection with Cobalt Strike
Mar 3rd 2021
1 month ago by Brad (0 comments)

Fun with DNS over TLS (DoT)
Mar 1st 2021
1 month ago by Rob VandenBrink (0 comments)

Adversary Simulation with Sim
Mar 2nd 2021
1 month ago by Russ McRee (0 comments)

Maldocs: Protection Passwords
Feb 28th 2021
1 month ago by DidierStevens (0 comments)