I was asked if I could provide mote details on the following sentence from my diary entry "nslookup's Debug Options":

     (notice that in my nslookup query, I terminated the FQDN with a dot: "example.com.", I do that to prevent Windows from adding suffixes)

A DNS suffix is a configuration of the Windows DNS client (locally, via DHCP, ...) to have it append suffixes when doing domain lookups.

For example, if a DNS suffix local is configured, then Windows' DNS client will not only do a DNS lookup for example.com, but also for example.com.local.

As an example, let me configure mylocalnetwork as a suffix on a Windows machine:

With DNS suffix mylocalnetwork configured, nslookup will use this suffix. For example, when I perform a lookup for "example.com", nslookup will also do a lookup for "example.com.mylocalnetwork".

I can show this with nslookup's debug option d2:

You can see in these screenshots DNS type A and AAAA resolutions for example.com.mylocalnetwork and example.com.

One of the ideas behind DNS suffixes, is to reduce typing. If you have a NAS, for example, named mynas, you can just access it with https://mynas/login. No need to type the fully qualified domain name (FQDN) https://mynas.mylocalnetwork/login.

Notice that the suffix also applies for AAAA queries, while in the screenshots above I only configured it for IPv4. That's because the DNS suffix setting applies both to IPv4 and IPv6:

Before I show the results with "example.com." (notice the dot character at the end), let me show how I can summarize the lookups by grepping for "example" (findstr):

If I terminate my DNS query with a dot character (.), suffixes will not be appended:

Notice that there are no resolutions for mylocalnetwork in this last example. That's because the trailing dot instructs Windows' DNS client to start resolving from the DNS root zone.

A domain name consists of domain labels separated by dots:

If you are adding a trailing dot, you are actually adding an empty domain label:

The empty label represents the DNS root zone, and no suffixes are appended to the DNS root zone, as it is the top-level (root) DNS zone.

A small tip if you want to restrict nslookup's resolutions to A records, for example. There is an option for that.

If you use nslookup's help option /?, you will see that you can provide options, but the actual options are not listed:

To see the available options, start nslookup, and then type "?" at its prompt, like this:

Now you can see that option "type" allows you to specify which type of records to query. Here is an example for A records:

Didier Stevens
Senior handler
blog.DidierStevens.com

Occasionaly, Xavier and Jim will ask me specific students' questions about my tools when they teach FOR610: Reverse-Engineering Malware.

Recently, a student wanted to know if my pdf-parser.py tool can extract all the PDF streams with a single command.

Since version 0.7.9, it can.

A stream is (binary) data, part of an object (optional), and can be compressed, or otherwise transformed. To view a single stream with pdf-parser, one selects the object of interest and uses option -f to apply the filters (like zlib decompression) to the stream:

I added a feature that is present in several of my tools, like oledump.py and zipdump.py: extract al of the "stored items" into a single JSON document.

When you use pdf-parser's option -j (--jsonoutput), all objects with a stream, will have the raw data (e.g., unfiltered) extracted and put into a JSON document that is sent to stdout:

To have the filtered (e.g., decompressed data), use option -f together with option -j:

What can you do with this JSON data? It depends on what your goals are. I have several tools that can take this JSON data as input, like file-magic.py and strings.py.

Here I use file-magic.py to identify the type of each raw data stream:

From this we can learn, for example, that object 143's stream contains a JPEG image.

And here I use file-magic.py to identify the type of each filtered data stream:

From this we can learn, for example, that object 881's stream contains a compressed TrueType Font file.

What if you want to write all stream data to disk, in individual files, for further analysis (that's what the student wanted to do, I guess)?

Then you can use my tool myjson-filter.py. It's a tool designed to filter JSON data produced by my tools, but it can also write items to disk.

When you use option -l, this tool will just produce a listing of the items contained in de JSON data:

And you can use option -W to write the streams to disk. -W takes a value that specifies what aming convention must be used to write the file to disk. vir will write items to disk with their sanitized name and extension .vir:

hashvir will write items to disk with their sha256 value as name and extension .vir:

Didier Stevens
Senior handler
Microsoft MVP
blog.DidierStevens.com

Synology NAS solutions are popular devices. They are also used in many organizations. Their product range goes from small boxes with two disks (I’m not sure they still sell a single-disk enclosure today) up to monsters, rackable with plenty of disks. They offer multiple disk management options but rely on many open-source software (like most appliances). For example, there are no expensive hardware RAID controllers in the box. They use the good old “MD” (“multiple devices”) technology, managed with the well-known mdadm tool[1]. Synology NAS run a Linux distribution called DSM. This operating system has plenty of third-party tools but lacks pure forensics tools.

In a recent investigation, I had to investigate a NAS that was involved in a ransomware attack. Many files (backups) were deleted. The attacker just deleted some shared folders. The device had two drives configured in RAID0 (not the best solution I know but they lack storage capacity). The idea was to mount the file system (or at least have the block device) on a Linux host and run forensic tools, for example, photorec.

In such a situation, the biggest challenge will be to connect all the drivers to the analysis host! Here, I had only two drives but imagine that you are facing a bigger model with 5+ disks. In my case, I used two USB-C/SATA adapters to connect the drives. Besides the software RAID, Synology volumes also rely on LVM2 (“Logical Volume Manager”)[2]. In most distributions, the packages mdadm and lvm2 are available (for example on SIFT Workstation). Otherwise, just install them:

# apt install mdadm lvm2

Once you connect the disks (tip: add a label on them to replace them in the right order) to the analysis host, verify if they are properly detected:

# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
sda       8:0    0 465.8G  0 disk
|-sda1    8:1    0 464.8G  0 part  /
|-sda2    8:2    0     1K  0 part
`-sda5    8:5    0   975M  0 part  [SWAP]
sdb       8:16   0   3.6T  0 disk
|-sdb1    8:17   0     8G  0 part
|-sdb2    8:18   0     2G  0 part
`-sdb3    8:19   0   3.6T  0 part
sdc       8:32   0   3.6T  0 disk
|-sdc1    8:33   0   2.4G  0 part
|-sdc2    8:34   0     2G  0 part
`-sdc3    8:35   0   3.6T  0 part
sr0      11:0    1  1024M  0 rom

"sdb3" and "sdc3" are the NAS partitions used to store data (2 x 4TB in RAID0). The good news, the kernel will detect that these disks are part of a software RAID! You just need to rescan them and "re-assemble" the RAID:

# mdadm --assemble --readonly --scan --force --run 

Then, your data should be available via a /dev/md? device:

# cat /proc/mdstat
Personalities : [raid0]
md0 : active (read-only) raid0 sdb3[0] sdc3[1]
      7792588416 blocks super 1.2 64k chunks

unused devices: <none>

The next step is to detect how data are managed by the NAS. Synology provides a technology called SHR[3] that uses LVM:

# lvdisplay
  WARNING: PV /dev/md0 in VG vg1 is using an old PV header, modify the VG to update.
  --- Logical volume ---
  LV Path                /dev/vg1/syno_vg_reserved_area
  LV Name                syno_vg_reserved_area
  VG Name                vg1
  LV UUID                08g9nN-Etde-JFN9-tn3D-JPHS-pyoC-LkVZAI
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              NOT available
  LV Size                12.00 MiB
  Current LE             3
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto

  --- Logical volume ---
  LV Path                /dev/vg1/volume_1
  LV Name                volume_1
  VG Name                vg1
  LV UUID                fgjC0Y-mvx5-J5Qd-Us2k-Ppaz-KG5X-tgLxaX
  LV Write Access        read/write
  LV Creation host, time ,
  LV Status              NOT available
  LV Size                <7.26 TiB
  Current LE             1902336
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto

You can see that the NAS has only one volume created ("volume_1" is the default name in DSM).

From now on, you can use /dev/vg1/volume_1 in your investigations. Mount it, scan it, image it, etc...

[1] https://en.wikipedia.org/wiki/Mdadm
[2] https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)
[3] https://kb.synology.com/en-br/DSM/tutorial/What_is_Synology_Hybrid_RAID_SHR

Xavier Mertens (@xme)
Xameco
Senior ISC Handler - Freelance Cyber Security Consultant
PGP Key

ISPs have a history of intercepting DNS. Often, DNS interception is done as part of a "value add" feature to block access to known malicious websites. Sometimes, users are directed to advertisements if they attempt to access a site that doesn't exist. There are two common techniques how DNS spoofing/interception is done:

1. The ISP provides a recommended DNS server. This DNS server will filter requests to known malicious sites.
2. The ISP intercepts all DNS requests, not just requests directed at the ISPs DNS server.

The first method is what I would consider a "recommended" or "best practice" method. The customer can use the ISP's DNS server, but traffic is left untouched if a customer selects a different recursive resolver. The problem with this approach is that malware sometimes alters the user's DNS settings.

Comcast, as part of its "Business Class" offer, provides a tool called "Security Edge". It is typically included for free as part of the service. Security Edge is supposed to interface with the customer's modem but can only do so for specific configurations. Part of the service is provided by DNS interception. Even if "Security Edge" is disabled in the customer's dashboard, DNS interception may still be active.

One issue with any filtering based on blocklists is false positives. In some cases, what constitutes a "malicious" hostname may not even be well defined. I could not find a definition on Comcast's website. But Bleeping Computer (www.bleepingcomputer.com) recently ended up on Comcast's "naughty list". I know all to well that it is easy for a website that covers security topics to end up on these lists. The Internet Storm Center website has been on lists like this before. Usually, sloppy signature-based checks will flag a site as malicious. An article may discuss a specific attack and quote strings triggering these signatures.

Comcast offers recursive resolvers to it's customers: 75.75.75.75, 75.75.76.76, 2001:558:feed:1 and 2001:558:feed:2. There are advantages to using your ISP's DNS servers. They are often faster as they are physically closer to your network, and you profit from responses cached by other users. My internal resolver is configured as a forwarding resolver, spreading queries among different well performing resolvers like Quad9, Cloudflare and Google.

So what happened to bleepingcomputer.com? When I wasn't able to resolve bleepingcomputer.com, I checked my DNS logs, and this entry stuck out:

broken trust chain resolving 'bleepingcomputer.com/A/IN': 8.8.8.8#53 

My resolver verifies DNSSEC. Suddenly, I could not verify DNSSEC, which is a good indication that either DNSSEC was misconfigured or someone was modifying DNS responses. Note that the response appeared to come from Google's name server (8.8.8.8).

My first step in debugging this problem was dnsviz.net, a website operated by Sandia National Laboratory. The site does a good job of visualizing DNSSEC and identifying configuration issues. Bleepingcomputer.com looked fine. Bleepingcomputer didn't use DNSSEC. So why the error? There was another error in my resolver's logs that shed some light on the issue:

no valid RRSIG resolving 'bleepingcomputer.com/DS/IN': 8.8.8.8#53

DNSSEC has to establish somehow if a particular site supports DNSSEC or not. The parent zone should offer an "NSEC3" record to identify zones that are not signed or not signed. DS records, also offered by the parent zone, verify the keys you may receive for a zone. If DNS is intercepted, the requests for these records may fail, indicating that something odd is happening.

So, someone was "playing" with DNS. And it affected various DNS servers I tried, not just Comcast or Google. Using "dig" to query the name servers directly, and skipping DNSSEC, I received a response:

8.8.8.8.53 > 10.64.10.10.4376: 35148 2/0/1 www.bleepingcomputer.com. A 192.73.243.24, www.bleepingcomputer.com. A 192.73.243.36 (85)

Usually, www.bleepingcomputer.com resolved to:

% dig +short www.bleepingcomputer.com
104.20.185.56
172.67.2.229
104.20.184.56

It took a bit of convincing, but I was able to pull up the web page at the wrong IP address:

The problem with these warning pages is that you usually never see them. Even if you resolve the IP address, TLS will break the connection, and many sites employ strict transport security. As part of my Comcast business account, I can "brand" the page, but by default, it is hard to tell that this page was delivered by Comcast.

But how do we know if someone is interfering with DNS traffic? A simple check I am employing is to look for the DNS timing and compare the TTL values for different name servers.

(1) Check timing

Send the same query to multiple public recursive DNS servers. For example:

% dig www.bleepingcomputer.com @75.75.75.75

; <<>> DiG 9.10.6 <<>> www.bleepingcomputer.com @75.75.75.75
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8432
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.bleepingcomputer.com.    IN    A

;; ANSWER SECTION:
www.bleepingcomputer.com. 89    IN    A    104.20.185.56
www.bleepingcomputer.com. 89    IN    A    104.20.184.56
www.bleepingcomputer.com. 89    IN    A    172.67.2.229

;; Query time: 59 msec
;; SERVER: 75.75.75.75#53(75.75.75.75)
;; WHEN: Tue May 07 20:00:05 EDT 2024
;; MSG SIZE  rcvd: 101

Dig includes the "Query time" in its output. In this case, it was 59 msec. We expect a speedy time like this for Comcast's DNS server while connected to Comcast's network. But let's compare this to other servers:

8.8.8.8: 59 msec
1.1.1.1: 59 msec
9.9.9.9: 64 msec
11.11.11.11: 68 msec
113.113.113.113: 69 msec

The results are very consistent. In particular, the last one is interesting. This server is located in China. 

(2) check TTLs

A recursive resolver will add a response it receives from an authoritative DNS server to its cache. The TTL for records bulled from the cache will decrease with the time the response sits in the resolver's cache. If all responses come from the same resolver, the TTL should decrement consistently. This test is a bit less telling. Often, several servers are used, and with anycast, it is not always easy to tell which server the response comes from. These servers do not always have a consistent cache.

Final Words

DNS interception, even if well-meaning, does undermine some of the basic "internet trust issues". Even if it is used to block users from malicious sites, it needs to be properly declared to the user, and switches to turn it off will have to function. This could be a particular problem if queries to other DNS filtering services are intercepted. I have yet to test this for Comcast and, for example, OpenDNS.

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|

A friend was having unexpected results with DNS queries on a Windows machine. I told him to use nslookup's debug options.

When you execute a simple DNS query like "nslookup example.com. 8.8.8.8", you get an answer like this (notice that in my nslookup query, I terminated the FQDN with a dot: "example.com.", I do that to prevent Windows from adding suffixes):

You see the result of a reverse DNS lookup (8.8.8.8 is dns.google) and you get 2 IP addresses for example.com in your answer: an IPv6 address and an IPv4 address.

If my friend would have been able to run packet capture on the machine, he would have seen 3 DNS queries and answers:

A PTR query to do a reverse DNS lookup for 8.8.8.8, an A query to lookup IPv4 addresses for example.com, and an AAAA query to lookup IPv6 addresses for example.com.

One can use nslookup's debug options to obtain equivalent information, without doing a packet capture.

Debug option -d displays extra information for each DNS response packet:

Here is nslookup's parsed DNS response packet for the PTR query:

Here is Wireshark's dissection of this packet:

You can see that the debug output contains the same packet information as Wireshark's, but presented in another form.

The same applies for the A query:

And the AAAA query:

If you also want to see the DNS query packets, you can use debug option -d2:

Besides the parsed DNS query, you now also see the length in bytes of each DNS packet (the UDP payload).

Here is the A query:

And here is the AAAA query:

Didier Stevens
Senior handler
blog.DidierStevens.com

Before diving into the vulnerability, a bit about the affected devices. LB-Link, the make of the devices affected by this vulnerability, produces various wireless equipment that is sometimes sold under different brands and labels. This will make it difficult to identify affected devices. These devices are often low-cost "no name" solutions or, in some cases, may even be embedded, which makes it even more difficult to find firmware updates.

Before buying any IoT device, WiFi router, or similar piece of equipment, please make sure the vendor does:

1. Offer firmware updates for download from an easy-to-find location.
2. Provide an "end of life" policy stating how long a particular device will receive updates.

Alternatively, you may want to verify if the device can be "re-flashed" using an open source firmware.

But let us go back to this vulnerability. There are two URLs affected, one of which showed up in our "First Seen URLs":

/goform/sysTools
/goform/set_LimitClient_cfg

The second one has been used more in the past, the first is relatively new in our logs. The graph below shows how "set_LimitClient.cfg" is much more popular. We only saw a significant number of scans for "sysTools" on May 1st.

The full requests we are seeing:

POST /goform/set_LimitClient_cfg HTTP/1.1
Cookie: user=admin

And yes, the vulnerability evolves around the "user=admin" cookie and a command injection in the password parameter. This is too stupid to waste any more time on, but it is common enough to just give up and call it a day. The NVD entry for the vulnerability was updated last week, adding an older PoC exploit to it. Maybe that got some kids interested in this vulnerability again.

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|