Check Link Count - chklinkcnt

Chklinkcnt is a small tool I wrote to find evidence that a kernel based rootkit tried to hide something in the local filesystem. The name is meant to be spoken "check link count". Many unix and linux rootkits allow directories to be hidden. But often the link count of the father directory is not corrected. The idea of chklinkcnt is to traverse all directories of a linux or unix filesystem and in each directory count the directory entries and compare the number with the link count of the current directory. If these don't match something might be wrong. Sounds simple? It really is!

Important: we use the assumption that the link count of a directory is the number of subdirectories plus 2 (for '.' and '..'). This is dependent on the filesystem and might not hold. It currently works for ext2/3, though. This also means that special file systems like /proc will give many false positives.

Example:

  bibo:~ # chklinkcnt
  dir '/proc/bus' has 5 hard links but 6 dir entries!
  25022 dirs checked
  bibo:~ #
Here /proc/bus is not an ordinary directory and checking it results in a false positive.

Another example:

linux:~ # chklinkcnt
dir '/proc' has 48 hard links but 46 dir entries!
dir '/proc/bus' has 4 hard links but 5 dir entries!
dir '/proc/bus/usb' has 1 hard links but 2 dir entries!
dir '/usr/share/locale/sk' has 4 hard links but 3 dir entries!
2692 dirs checked
linux:~ # ls -al /usr/share/locale/sk
total 3
drwxr-xr-x    4 root     root         1024 Sep  2  2006 .
drwxr-xr-x   42 root     root         1024 Aug 31  2006 ..
drwxr-xr-x    2 root     root         1024 Aug 31  2006 LC_MESSAGES
linux:~ #
Again, the findings in /proc are better ignored. But the directory /usr/share/locale/sk should not produce a false positive. Looking closer reveals a subdirectory, a reference to the father directory and a reference to itself: a sum of 3. The link count of '.' is 4, though. There might be a problem.

What to do if something seems to be wrong?

You better investigate further. For example with debugfs: 
linux:~ # debugfs /dev/sda1
 debugfs 1.37 (21-Mar-2005)
 debugfs: ls -l /usr/share/locale/sk
  808001       41777    (2)        0        0       4096   2-Sep-2006    00:52   .
          2    40755    (2)        0        0       4096  31-Aug-2006    02:11   ..
  938006       40755    (2)    21037    27421       4096  17-Okt-2006    15:30   owned
  340099       40700    (2)        0        0       4096  31-Aug-2006    15:35   LC_MESSAGES
There really was something wrong! The directory 'owned' was hidden - probably by changing the user and group-id to some "magic" values which are debugfs conviniently provides. You can now dump the directory with debugfs or just try to enter it by using the full path.

Does it work with my filesystem?

I tested the program using linux and ext2 / ext3 filesystems. The implementation of readdir() of Reiserfs does not give you the type of the entry like the aforementioned fs do. This is not against the rules but makes life a lot more difficult for us. You can call chklinkcnt with the option '-s' which makes it call stat on every entry. This is a lot slower and destroys evidence :-/

Destroying evidence

The potential problem is taht we might destroy the access times of files and directories which might be usefull for forensics later on. But since we only call readdir on every directory we will not change the access times of the normal files but just of the directory entries. I think the benefit is higher than the loss of directory access times but you have to judge for yourself.

The source

Direct download links via the DFN mirror of source forge for the source chklinkcnt.c and the detached signature chklinkcnt.c.sig. You might want to check the pgp signature: 
gpg --verify chklinkcnt.c.sig chklinkcnt.c
If you don't know my key (0x7C06FD25) you have to import it and you might want to check if there is a path in the web of trust between your key and mine (key stats at uu.nl): 
gpg --keyserver pgpkeys.pca.dfn.de --recv-key 0x7C06FD25
I recommend to compile a static binary: 
gcc -Wall -static chklinkcnt.c -o chklinkcnt

Referenzes