Parsing the hiberfil.sys, searching for slack space

Nytro · December 10, 2014

Parsing the hiberfil.sys, searching for slack space

Implementing functionality that is already available in an available tool is something that has always taught me a lot, thus I keep on doing it when I encounter something I want to fully understand. In this case it concerns the ‘hiberfil.sys’ file on Windows. As usual I first stumbled upon the issue and started writing scripts to later find out someone had written a nice article about it, which you can read here (1). For the sake of completeness I’m going to repeat some of the information in that article and hopefully expand upon it, I mean it’d be nice if I could use this entry as a reference page in the future for when I stumble again upon hibernation files. Our goal for today is going to be to answer the following question:

What’s a hiberfil.sys file, does it have slack space and if so how do we find and analyze it?

To answer that question will hopefully be answered in the following paragraphs; we are going to look at the hibernation process, hibernation file, it’s file format structure, how to interpret it and finally analyze the found slack space. As usual you can skip the post and go directly to the code.

Hibernation process

When you put your computer to ‘sleep’ there are actually several ways in which it can be performed by the operating system one of those being the hibernation one. The hibernation process puts the contents of your memory into the hiberfil.sys file so that the state of all your running applications is preserved. By default when you enable hibernation the hiberfil.sys is created and filled with zeros. To enable hibernation you can run the following command in an elevated command shell:

powercfg.exe -H on If you want to also control the size you can do:

powercfg.exe -H -Size 100 An interesting fact to note is that Windows 7 sets the size of the hibernation file size to 75% of your memory size by default. According to Microsoft documentation (2) this means that hibernation process could fail if it’s not able to compress the memory contents to fit in the hibernation file. This of course is useful information since it indicates that the contest of the hibernation file is compressed which usually will make basic analysis like ‘strings’ pretty useless.

if you use strings always go for ‘strings -a <inputfile>’ read

this

post if you are wondering why.

The hibernation file usually resides in the root directory of the system drive, but it’s not fixed. If an administrators wants to change the location he can do so by editing the following registry key as explained by this (3) msdn article:

Key Name: HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\

Value Name: PagingFiles

Type: REG_MULT_SZ

Data: C:\pagefile.sys 150 500

In the Data field, change the path and file name of the pagefile, along with the minimum and maximum file size values (in megabytes). So if you are performing an incident response or forensic investigation make sure you check this registry key before you draw any conclusion if the hiberfil.sys file is absent from it’s default location. Same goes for creating memory images using hibernation, make sure you get the 100% and write it to a location which doesn’t destroy evidence or where the evidence has already been collected.

Where does the slack space come from you might ask? That’s an interesting question since you would assume that each time the computer goes into hibernation mode it would create a new hiberfil.sys file, but it doesn’t. Instead it will overwrite the current file with the contents it wants to save. This is what causes slack space, since if the new data is smaller in size than the already available files the data at the end of the file will still be available even if it’s not referenced by the new headers written to the file.

From a forensic standpoint that’s pretty interesting since the unreferenced but available data might contain important information to help the investigation along. If you are working with tools that automatically import / parse or analyse the hiberfil.sys file you should check / ask / test how they handle slack space. In a best case scenario they will inform you about the slack space and try to recover the information, in a less ideal scenario they will inform you that there is slack space but it’s not able to handle the data and in the worst case scenario it will just silently ignore that data and tell you the hibernation file has been processed successfully.

Hibernation File structure

Now that we know that slack space exists how do we find and process it? First of all we should start with identifying the file format to be able to parse it, which unfortunately isn’t available from Microsoft directly. Luckily for us we don’t need to reverse it (yet?) since there are pretty smart people out there who have already done so. You could read this or this or this to get some very good information about the file format. Let’s look at the parts that are relevant for our purpose of retrieving and analyzing the hibernation slack space. The general overview of the file format is as follow:

Hibernation File Format (9)

Like you can see in the image above the file format seems reasonably easy to parse if we have the definition of all the headers. The most important headers for us are the “Hibernation File Header” which starts at page zero, the “Memory Table” header which contains a pointer to the next one and the amount of Xpress blocks and the “Xpress image” header which contains the actual memory data. The last two are the headers actually create the chain we want to follow to be able to distinguish between referenced Xpress blocks and blocks which are just lingering around in slack space. The thing to keep in mind is that even though the “Hibernation File Header” contains a lot of interesting information to make a robust tool, it might not be present when we recover a hibernation file. The reason for this is that when Windows resumes from hibernation the first page is zeroed. Luckily it isn’t really needed if you just assume a few constants like page size being 4096 bytes and finding the first Xpress block with a bit of searching around. Let’s have a look at the headers we have been talking about:

typedef struct
{
ULONG Signature;
ULONG Version;
ULONG CheckSum;
ULONG LengthSelf;
ULONG PageSelf;
UINT32 PageSize;
ULONG64 SystemTime;
ULONG64 InterruptTime;
DWORD FeatureFlags;
DWORD HiberFlags;
ULONG NoHiberPtes;
ULONG HiberVa;
ULONG64 HiberPte;
ULONG NoFreePages;
ULONG FreeMapCheck;
ULONG WakeCheck;
UINT32 TotalPages;
ULONG FirstTablePage;
ULONG LastFilePage;
PO_HIBER_PREF PerfInfo;
ULONG NoBootLoaderLogPages;
ULONG BootLoaderLogPages[8];
ULONG TotalPhysicalMemoryCount;
} PO_MEMORY_IMAGE, *PPO_MEMORY_IMAGE;

The FirstTablePage member is the most important one for us since it contains a pointer to the first memory table. Then again, knowing that the first page might be wiped out, do we really want to parse it when it’s available? Let’s look at the memory table structure:

struct MEMORY_TABLE
{
DWORD PointerSystemTable;
UINT32 NextTablePage;
DWORD CheckSum;
UINT32 EntryCount;
MEMORY_TABLE_ENTRY MemoryTableEntries[EntryCount];
};

That’s neat as already expected it contains the NextTablePage member which points to the next memory table. Directly following the memory table we’ll find the Xpress block which have the following header:

struct IMAGE_XPRESS_HEADER
{
CHAR Signature[8] = 81h, 81h, "xpress";
BYTE UncompressedPages = 15;
UINT32 CompressedSize;
BYTE Reserved[19] = 0;
};

It seems this header contains the missing puzzle pieces, since it tells us how big each Xpress block is. So if you followed along, here is the big picture on how it all fits in to be parsed and discover if there is any slack space and if so how much.

Find the first MemoryTable
Follow pointers until you find the last one
Follow all the xpress blocks until the last one
Calculate distance from the end of the last block until the end of the file
Slack space found

There are a few caveats that we need to be aware of (you know this if you already read the references):

Every OS version might change the structures and thus their size
Everything is page aligned
Every memory table entry should correspond to an Xpress block
The get the actual compressed size calculate as follow:
- CompressedSize / 4 + 1 and round it up to 8

Parsing and interpreting the hibernation file

The theory sounds pretty straight forward right? The practice however actually made me waste some hours. For one I was assuming the “Hibernation File Header” to be the same across all operating systems, stupid I know. Just didn’t realize it until I was brainstorming with Mitchel Sahertian about why the pointers where not pointing at the correct offsets. This however taught me that when you are writing some proof of concept code you should parse the entire structure, not just reference the structure members you are interested in. Since when you parse the entire structure it gives you more context and the ability to quickly spot that a lot of members contain garbage data. When you are just directly referencing a single member like I was doing, you loose the context and you only get to see one pointer which could virtually be anything. So even after learning this lesson, I still decided to implement the script by just parsing the minimum needed data, even though I used full structures while debugging the code. The most important snippets of the script are highlighted hereafter, the script probably contains bugs although in the few tests I’ve performed it seems to work fine:

Finding the first MemoryTable, first we search for an xpress block and then we subtract a full page from it.

[TABLE]

[TR]

[TD=class: gutter]1

2

3[/TD]

[TD=class: code]firstxpressblock = fmm.find(XPRESS_SIG)

print "Found Xpress block @ 0x%x" % firstxpressblock

firstmemtableoffset = firstxpressblock - PAGE_SIZE[/TD]

[/TR]

[/TABLE]

Finding the pointer to the next MemoryTable in a dynamic way, we want to avoid reversing this structure for every new operating system version or service pack. We start at the beginning of the MemoryTable and force-interpret every four bytes as a pointer, then we check the pointer destination. The pointer destination is checked by verifying that an xpress block follows it immediately, if so it’s valid.

[TABLE]

[TR]

[TD=class: gutter]1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23[/TD]

[TD=class: code]def verify_memorytable_offset(offset):

"""

Verify the table pointer to be valid

valid table pointer should have an Xpress block

on the next page

"""

fmm.seek(offset+PAGE_SIZE)

correct = False

if fmm.read(8) == XPRESS_SIG:

correct = True

fmm.seek(offset)

return correct

#could go horribly wrong, seems to work though

def find_memorytable_nexttable_offset(data):

"""

Dynamically find the NextTablePage pointer

Verification based on verify_memorytable_offset function

"""

for i in range(len(data)):

toffset = unpack('<I',data[i:(i+4)])[0]*PAGE_SIZE

if verify_memorytable_offset(toffset):

return i[/TD]

[/TR]

[/TABLE]

After we find the last MemoryTable, we just have to walk all the xpress blocks until the last xpress block and from it’s end till the end of the file we found the slack space:

[TABLE]

[TR]

[TD=class: gutter]1

2

3

4

5

6

7

8

9

10

11[/TD]

[TD=class: code]while True:

xsize = xpressblock_size(fmm,nxpress)

fmm.seek(xsize,1)

xh = fmm.read(8)

if xh != XPRESS_SIG:

break

fmm.seek(-8,1)

if VERBOSE:

print "Xpress block @ 0x%x" % nxpress

nxpress = fmm.tell()

print "Last Xpress block @ 0x%x" % nxpress[/TD]

[/TR]

[/TABLE]

Analyzing the found hibernation slack space

So after all this how do we even know this slack space is really here and can we even extract something useful from it? First of all let’s compare our output to that of volatility, after all it’s the de facto, and in my opinion best, memory analysis tool out there. One of the things you can for example do with volatility is convert the hibernation file into a raw memory file, volatility doesn’t support direct hibernation file analysis. Before converting it however I added two debug lines to the file ‘volatility/plugins/addrspaces/hibernate.py’

Printing the memory table offset:

[TABLE]

[TR]

[TD=class: gutter]1

2

3

4[/TD]

[TD=class: code]NextTable = MemoryArray.MemArrayLink.NextTable.v()

print "memtableoff %x" % NextTable

# This entry count (EntryCount) should probably be calculated[/TD]

[/TR]

[/TABLE]

Printing the xpress block offset:

[TABLE]

[TR]

[TD=class: gutter]1

2

3

4[/TD]

[TD=class: code]XpressHeader = obj.Object("_IMAGE_XPRESS_HEADER", XpressHeaderOffset, self.base)

XpressBlockSize = self.get_xpress_block_size(XpressHeader)

print "xpressoff %x" % XpressHeaderOffset

return XpressHeader, XpressBlockSize[/TD]

[/TR]

[/TABLE]

With this small modification I’m able to compare the output of my script to the output of volatility:

Volatility output:

./vol.py -f ~/p/find_hib_slack/hibfiles/a.sys –profile=Win7SP0x86 imagecopy -O ~/p/find_hib_slack/hibfiles/raw.a.img

[…]

memtableoff 10146

xpressoff 10147000

xpressoff 1014f770

xpressoff 101589f0

xpressoff 10160240

xpressoff 10166a98

xpressoff 1016c7a8

xpressoff 10172448

xpressoff 10179b60

xpressoff 1017ff50

xpressoff 10181af8

xpressoff 10181d58

xpressoff 10183790

xpressoff 101877e0

xpressoff 1018c290

xpressoff 10192270

xpressoff 10195890

xpressoff 1019b680

xpressoff 101a0d50

xpressoff 101a5258

xpressoff 101a8608

xpressoff 101aa3e8

xpressoff 101adc70

xpressoff 101b2de0

That looks pretty clear, the last memory table is at offset 0x10146 and the last xpress block is at offset 0x101b2de0

My script:

./find_hib_slack.py hibfiles/a.sys

Last MemoryTable @ 0x10146000

Xpress block @ 0x10147000

Xpress block @ 0x1014f770

Xpress block @ 0x101589f0

Xpress block @ 0x10160240

Xpress block @ 0x10166a98

Xpress block @ 0x1016c7a8

Xpress block @ 0x10172448

Xpress block @ 0x10179b60

Xpress block @ 0x1017ff50

Xpress block @ 0x10181af8

Xpress block @ 0x10181d58

Xpress block @ 0x10183790

Xpress block @ 0x101877e0

Xpress block @ 0x1018c290

Xpress block @ 0x10192270

Xpress block @ 0x10195890

Xpress block @ 0x1019b680

Xpress block @ 0x101a0d50

Xpress block @ 0x101a5258

Xpress block @ 0x101a8608

Xpress block @ 0x101aa3e8

Xpress block @ 0x101adc70

Last Xpress block @ 0x101b2de0

Start of slack space @ 270223752

Total file size 1073209344

Slackspace size 765 megs

At least we know that the parsing seems to be ok, since our last MemoryTable offset and our last xpress offset match the ones from volatility. We can also see that the end of the last xpress block is way before the end of the file. This indicates that the space in between might contain some interesting data. From a memory forensic perspective it’s logical that volatility doesn’t parse this since the chances of being able to extract any meaningful structured data from it are reduced in comparison with a normal memory image or hibernation file. You can use the output to carve out the slack space with dd if you want to further analyze it, for example like this:

dd if=<inputfile> of=<slack.img> bs=1 skip=<start_of_slackspace>

The thing is, like with all slack space it can contain virtually anything. If you are really lucky you’ll find nice new MemoryTables and xpress blocks. If you are less lucky you’ll only find partial xpress blocks. For now I’ve opted for medium optimism and assumed we’ll at least be able to find full xpress blocks. So I wrote another scripts which you can use to extract and decompress blocks from a blob of data and write it to a file. After this you can try your luck with strings for example or volatility plugins yarascan or psscan. Here is some example output:

./bulk_xpress_decompress.py hibfiles/a.sys 270223752

Advancing to offset 270223752

Xpress block @ 0x101b65c0 size: 20824

Xpress block @ 0x101bb718 size: 17760

Xpress block @ 0x101bfc78 size: 17240

Xpress block @ 0x101c3fd0 size: 20424

Xpress block @ 0x101c8f98 size: 21072

Xpress block @ 0x101ce1e8 size: 16712

The script also writes all the decompressed output to a file called ‘decompressed.slack’. I use the decompression file from volatility, hope I didn’t mess up any license requirements, since I just included it in it’s entirety.

Conclusion

Sometimes you really have to dive into a file format to fully understand it, it won’t always end in glorious victory, but you’ll learn a lot during the development of your own script. Also the slack space is a nice place to store your malicious file. I’m taking a guess here, but I assume that if you enlarge the hibernation file, Windows will be fine with it. As long as the incident response or forensic investigator doesn’t look at it you’ll be fine with it and get away without your stash being discovered due to the fact that most tools ignore the slack space.

References

[*]http://msdn.microsoft.com/en-us/library/windows/desktop/aa373229(v=vs.85).aspx