Auditd is your friend
Recently I’ve been working on several incidents that included attackers getting shell access to the compromised host and somehow elevating their privileges to root. Of course, once they have access to the box, one of the first things they want to do is to be able to establish that same level of access again.
While there are many, many ways for an attacker to plant a backdoor that will allow him access to the server later, the easiest way is, of course, to create a new, privileged account that the attacker can use to access the server.
Now, when analyzing what happened during the attack we figured that this was exactly what the attacker did, however there were no logs on the system and subsequently the central logging system and SIEM implemented by the victim were of no use.
As this was a Linux server, you can probably already guess what the attacker did: they simply opened the /etc/passwd and /etc/shadow files and added their backdoor accounts (with an UID of 0). Of course, since they did this directly by modifying the system files there were absolutely no logs.
Very simple, yet also very effective!
So, what can we do against this? One obvious answer is to monitor any changes to these two and some other important files (for example, /etc/hosts, /etc/sudoers .. there are actually many). On Linux servers it is actually really easy to do this thanks to auditd – the Linux Audit daemon.
Auditd is the userspace part of the Linux Auditing system, which integrates deeply with the kernel. Being integrated with the kernel allows it to inspect every little detail of what’s happening on the system. In fact, many administrators turn auditd off due to huge amounts of logs it can create, and potential performance impact on the system. However, it is a true gem in auditing, if used correctly.
In our example on monitoring /etc/passwd we just need to add one rule to the /etc/audit/audit.rules file:
-w /etc/passwd -p wa
This tells auditd to monitor and log any changes (trigger on write and attribute change of the file). So when our attacker tries to modify this file directly, we will get log similar to the following:
type=SYSCALL msg=audit(1360781410.961:24122): arch=40000003 syscall=15 success=yes exit=0 a0=8357590 a1=81a4 a2=1 a3=1 items=1 ppid=17480 pid=8437 auid=510 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=210 comm="vim" exe="/usr/bin/vim" key=(null)
type=CWD msg=audit(1360781410.961:24122): cwd="/etc/audit"
type=PATH msg=audit(1360781410.961:24122): item=0 name="/etc/passwd" inode=4786344 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00
Pretty cool! So we had the UID and GID of the user who modified the /etc/passwd file, as well as the full path to the process (command) that was used to edit it.
Since auditd creates logs in /var/log/audit/audit.log, we can now send the same file to our SIEM and create a rule to trigger an alert on such actions, so we can catch any modifications to this (and other) system critical files.
For the bonus part, we can even setup a simple cron job (it will suffice in most cases) that will calculate an SHA1 sum of the /etc/passwd file, trigger on any changes and do a diff on the old file (saved before) and the new one.
What other things do you do to monitor your critical files? Let us know!
--
Bojan
INFIGO IS
Web App Penetration Testing and Ethical Hacking | Munich | Oct 14th - Oct 19th 2024 |
Comments
nk
Feb 14th 2013
1 decade ago
Actually, at the moment I'm using two: Samhain and Tripwire. Both have their individual strengths & weaknesses, and I keep meaning to explore another competitor - AIDE - but never seem to get the time.
At this first cursory glance, I'd say that the main difference between these tools and auditd is that auditd is better at telling you what happened and who did it.
But a system like Samhain/Tripwire/AIDE is better at the actual forensics of what happened - their DB is like a snapshot of the system state, so is much more useful after the fact when doing forensics.
It's a subtle distinction, and it doesn't mean that either one is better than the other.
In fact, there is no reason not to run both...
Which is probably what I'll be doing shortly. :-)
Philip Storry
Feb 14th 2013
1 decade ago
samhain can be coupled with auditd to check for files modification (via inotify) and report changes immediately. distributed installation of samhain/yule is nice because central server can keep both config and database
tk
Feb 14th 2013
1 decade ago
Steven Chamberlain
Feb 14th 2013
1 decade ago
Steven Chamberlain
Feb 14th 2013
1 decade ago
Too bad more SIEM vendors don't support auditd logs as of yet.
Peter
Feb 14th 2013
1 decade ago
Snoopy generates a lot of information and can be bypassed. The forwarding of history in real time can also be easily bypassed using another shell (sh, ksh, etc.) or for example launching a connection from metasploit.
Does anyone know a better alternative?
Thanks!
Javi
Feb 14th 2013
1 decade ago
A simple cron script can be written to monitor the addition of UID0 accounts to /etc/passwd and send an alert via email or logger to /var/log/messages, but it has the same drawback as auditd. It's a passive reporting tool.
http://forkbomb.org/ninja/
jbmoore
Feb 15th 2013
1 decade ago
https://docs.google.com/file/d/0B6CZu7FowekfNlkyS3g5UXV1SUU/edit?usp=sharing
ddjohnson
Feb 17th 2013
1 decade ago