SANS ISC: The special case of TCP RST

• SANS Site Network
  • Current Site
  • SANS Internet Storm Center
  • Other SANS Sites Help
  • Graduate Degree Programs
  • Security Training
  • Security Certification
  • Security Awareness Training
  • Penetration Testing
  • Industrial Control Systems
  • Cyber Defense Foundations
  • DFIR
  • Software Security
  • Government OnSite Training

SANS ISC InfoSec Forums

In TCP, packets with the "Reset" (RST or R) flag are sent to abort a connection. Probably the most common reason you are seeing this is that an SYN packet is sent to a closed port.

But RST packets may be sent in other cases to indicate that a connection should be closed. 

Lets first look at the reset in response to an SYN to a closed port:

IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto TCP (6), length 64)
    192.0.2.1.65007 > 192.0.2.2.81: Flags [S], seq 3972116176, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 447631150 ecr 0,sackOK,eol], length 0
16:24:59.928936 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    192.0.2.2.81 > 192.0.2.1.65007: Flags [R.], seq 0, ack 3972116177, win 0, length 0

Interesting here: The sequence number of the RST packet is 0. If you are looking at "unusually frequent" sequence numbers, "0" may show up at the top if you had a lot of failed connections that resulted in resets.

For an RST in response to an SYN, the sequence number is not really used. This is the first packet arriving from that source, and no further packets will be sent. Also, there is nothing to acknowledge. So the sequence number, if there is one, would be ignored and as a result, the few operating systems I tested (BSD, macOS, Linux, Windows 10) all use a sequence number of 0.

Could this be used to help with spoofing TCP Resets? Not really. As there is no initial sequence number yet, the RST sequence number wouldn't add anything.

A second issue that sometimes causes confusion: RST packets with payload

IP (tos 0x0, ttl 64, id 49111, offset 0, flags [DF], proto TCP (6), length 51)
    192.0.2.43.37444 > 10.0.28.20.25: Flags [R.], seq 1:12, ack 1, win 57352, length 11 [RST 220 mailgat]

The RST packet above has a payload length of 11 bytes. tcpdump is nice enough to display the payload with a "RST" prefix. The actual payload here was "220 mailgat". This behavior is typical for security devices (the trace above is a bit old, but I believe the RST was created by a Cisco IPS at the time). The idea is that the payload will provide more information about why the IPS (and in some cases firewall) sent the RST. Someone once told me that there is an RFC describing this behaviour, but I haven't found it yet (if you know: please comment ;-) ).

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|