Reversing Microsoft Visual C++ Part I: Exception Handling

Nytro · August 7, 2013

Author: igorsk

Abstract

Microsoft Visual C++ is the most widely used compiler for Win32 so it is important for the Win32 reverser to be familiar with its inner working. Being able to recognize the compiler-generated glue code helps to quickly concentrate on the actual code written by the programmer. It also helps in recovering the high-level structure of the program.

In part I of this 2-part article (see also: Part II: Classes, Methods and RTTI), I will concentrate on the stack layout, exception handling and related structures in MSVC-compiled programs. Some familiarity with assembler, registers, calling conventions etc. is assumed.

Terms:

Stack frame: A fragment of the stack segment used by a function. Usually contains function arguments, return-to-caller address, saved registers, local variables and other data specific to this function. On x86 (and most other architectures) caller and callee stack frames are contiguous.
Frame pointer: A register or other variable that points to a fixed location inside the stack frame. Usually all data inside the stack frame is addressed relative to the frame pointer. On x86 it's usually ebp and it usually points just below the return address.
Object: An instance of a (C++) class.
Unwindable Object: A local object with auto storage-class specifier that is allocated on the stack and needs to be destructed when it goes out of scope.
Stack UInwinding: Automatic destruction of such objects that happens when the control leaves the scope due to an exception.

There are two types of exceptions that can be used in a C or C++ program.

SEH exceptions (from "Structured Exception Handling"). Also known as Win32 or system exceptions. These are exhaustively covered in the famous Matt Pietrek article[1]. They are the only exceptions available to C programs. The compiler-level support includes keywords __try, __except, __finally and a few others.
C++ exceptions (sometimes referred to as "EH"). Implemented on top of SEH, C++ exceptions allow throwing and catching of arbitrary types. A very important feature of C++ is automatic stack unwinding during exception processing, and MSVC uses a pretty complex underlying framework to ensure that it works properly in all cases.

In the following diagrams memory addresses increase from top to bottom, so the stack grows "up". It's the way the stack is represented in IDA and opposite to the most other publications.

Basic Frame Layout

The most basic stack frame looks like following:

... Local variables Other saved registers Saved ebp Return address Function arguments ...

Note: If frame pointer omission is enabled, saved ebp might be absent.

SEH

In cases where the compiler-level SEH (__try/__except/__finally) is used, the stack layout gets a little more complicated.

SEH3 Stack Layout

When there are no __except blocks in a function (only __finally), Saved ESP is not used. Scopetable is an array of records which describe each __try block and relationships between them:

    struct _SCOPETABLE_ENTRY {

DWORD EnclosingLevel;

void* FilterFunc;

void* HandlerFunc;

}

For more details on SEH implementation see[1]. To recover try blocks watch how the try level variable is updated. It's assigned a unique number per try block, and nesting is described by relationship between scopetable entries. E.g. if scopetable entry i has EnclosingLevel=j, then try block j encloses try block i. The function body is considered to have try level -1. See Appendix 1 for an example.

Buffer Overrun Protection

The Whidbey (MSVC 2005) compiler adds some buffer overrun protection for the SEH frames. The full stack frame layout in it looks like following:

SEH4 Stack Layout

The GS cookie is present only if the function was compiled with /GS switch. The EH cookie is always present. The SEH4 scopetable is basically the same as SEH3 one, only with added header:

    struct _EH4_SCOPETABLE {

DWORD GSCookieOffset;

DWORD GSCookieXOROffset;

DWORD EHCookieOffset;

DWORD EHCookieXOROffset;

_EH4_SCOPETABLE_RECORD ScopeRecord[1];

};

struct _EH4_SCOPETABLE_RECORD {

DWORD EnclosingLevel;

long (*FilterFunc)();

union {

void (*HandlerAddress)();

void (*FinallyFunc)();

};

GSCookieOffset = -2 means that GS cookie is not used. EH cookie is always present. Offsets are ebp relative. Check is done the following way: (ebp+CookieXOROffset) ^ [ebp+CookieOffset] == _security_cookie Pointer to the scopetable in the stack is XORed with the _security_cookie too. Also, in SEH4 the outermost scope level is -2, not -1 as in SEH3.

C++ Exception Model Implementation

When C++ exceptions handling (try/catch) or unwindable objects are present in the function, things get pretty complex.

C++ EH Stack Layout

EH handler is different for each function (unlike the SEH case) and usually looks like this:

     (VC7+)    mov eax, OFFSET __ehfuncinfo    jmp ___CxxFrameHandler

__ehfuncinfo is a structure of type FuncInfo which fully describes all try/catch blocks and unwindable objects in the function.

    struct FuncInfo {

// compiler version.

// 0x19930520: up to VC6, 0x19930521: VC7.x(2002-2003), 0x19930522: VC8 (2005)

DWORD magicNumber;

// number of entries in unwind table

int maxState;

// table of unwind destructors

UnwindMapEntry* pUnwindMap;

// number of try blocks in the function

DWORD nTryBlocks;

// mapping of catch blocks to try blocks

TryBlockMapEntry* pTryBlockMap;

// not used on x86

DWORD nIPMapEntries;

// not used on x86

void* pIPtoStateMap;

// VC7+ only, expected exceptions list (function "throw" specifier)

ESTypeList* pESTypeList;

// VC8+ only, bit 0 set if function was compiled with /EHs

int EHFlags;

};

Unwind map is similar to the SEH scopetable, only without filter functions:

    struct UnwindMapEntry {

int toState; // target state

void (*action)(); // action to perform (unwind funclet address)

};

Try block descriptor. Describes a try{} block with associated catches.

    struct TryBlockMapEntry {

int tryLow;

int tryHigh; // this try {} covers states ranging from tryLow to tryHigh

int catchHigh; // highest state inside catch handlers of this try

int nCatches; // number of catch handlers

HandlerType* pHandlerArray; //catch handlers table

};

Catch block descriptor. Describes a single catch() of a try block.

struct HandlerType {

// 0x01: const, 0x02: volatile, 0x08: reference

DWORD adjectives;

// RTTI descriptor of the exception type. 0=any (ellipsis)

TypeDescriptor* pType;

// ebp-based offset of the exception object in the function stack.

// 0 = no object (catch by type)

int dispCatchObj;

// address of the catch handler code.

// returns address where to continues execution (i.e. code after the try block)

void* addressOfHandler;

};

List of expected exceptions (implemented but not enabled in MSVC by default, use /d1ESrt to enable).

    struct ESTypeList {

// number of entries in the list

int nCount;

// list of exceptions; it seems only pType field in HandlerType is used

HandlerType* pTypeArray;

};

RTTI type descriptor. Describes a single C++ type. Used here to match the thrown exception type with catch type.

struct TypeDescriptor {

// vtable of type_info class

const void * pVFTable;

// used to keep the demangled name returned by type_info::name()

void* spare;

// mangled type name, e.g. ".H" = "int", ".?AUA@@" = "struct A", ".?AVA@@" = "class A"

char name[0];

};

Unlike SEH, each try block doesn't have a single associated state value. The compiler changes the state value not only on entering/leaving a try block, but also for each constructed/destroyed object. That way it's possible to know which objects need unwinding when an exception happens. You can still recover try blocks boundaries by inspecting the associated state range and the addresses returned by catch handlers (see Appendix 2).

Throwing C++ Exceptions

throw statements are converted into calls of _CxxThrowException(), which actually raises a Win32 (SEH) exception with the code 0xE06D7363 ('msc'|0xE0000000). The custom parameters of the Win32 exception include pointers to the exception object and its ThrowInfo structure, using which the exception handler can match the thrown exception type against the types expected by catch handlers.

    struct ThrowInfo {

// 0x01: const, 0x02: volatile

DWORD attributes;

// exception destructor

void (*pmfnUnwind)();

// forward compatibility handler

int (*pForwardCompat)();

// list of types that can catch this exception.

// i.e. the actual type and all its ancestors.

CatchableTypeArray* pCatchableTypeArray;

};

struct CatchableTypeArray {

// number of entries in the following array

int nCatchableTypes;

CatchableType* arrayOfCatchableTypes[0];

};

Describes a type that can catch this exception.

    struct CatchableType {

// 0x01: simple type (can be copied by memmove), 0x02: can be caught by reference only, 0x04: has virtual bases

DWORD properties;

// see above

TypeDescriptor* pType;

// how to cast the thrown object to this type

PMD thisDisplacement;

// object size

int sizeOrOffset;

// copy constructor address

void (*copyFunction)();

};

// Pointer-to-member descriptor.

struct PMD {

// member offset

int mdisp;

// offset of the vbtable (-1 if not a virtual base)

int pdisp;

// offset to the displacement value inside the vbtable

int vdisp;

};

We'll delve more into this in the next article.

Prologs and Epilogs

Instead of emitting the code for setting up the stack frame in the function body, the compiler might choose to call specific prolog and epilog functions instead. There are several variants, each used for specific function type:

[TABLE]

[TR]

[TD=class: table_sub_header]Name[/TD]

[TD=class: table_sub_header]Type[/TD]

[TD=class: table_sub_header, align: right]EH Cookie[/TD]

[TD=class: table_sub_header, align: right]GS Cookie[/TD]

[TD=class: table_sub_header, align: right]Catch Handlers[/TD]

[/TR]

[TR=class: table_row_1]

[TD]_SEH_prolog/_SEH_epilog [/TD]

[TD]SEH3 [/TD]

[TD=align: right]-[/TD]

[TD=align: right]

[/TD]

[/TR]

[TR=class: table_row_2]

[TD]_SEH_prolog4/_SEH_epilog4 S [/TD]

[TD]EH4 [/TD]

[TD=align: right]+[/TD]

[TD=align: right]-[/TD]

[TD=align: right]

[/TD]

[/TR]

[TR=class: table_row_1]

[TD]_SEH_prolog4_GS/_SEH_epilog4_GS [/TD]

[TD]SEH4 [/TD]

[TD=align: right]+[/TD]

[TD=align: right]

[/TD]

[/TR]

[TR=class: table_row_2]

[TD]_EH_prolog [/TD]

[TD]C++ EH[/TD]

[TD=align: right]-[/TD]

[TD=align: right]+/-[/TD]

[/TR]

[TR=class: table_row_1]

[TD]_EH_prolog3/_EH_epilog3 [/TD]

[TD]C++ EH[/TD]

[TD=align: right]+[/TD]

[TD=align: right]-[/TD]

[TD=align: right]- [/TD]

[/TR]

[TR=class: table_row_2]

[TD]_EH_prolog3_catch/_EH_epilog3 [/TD]

[TD]C++ EH[/TD]

[TD=align: right]+[/TD]

[TD=align: right]-[/TD]

[TD=align: right]+ [/TD]

[/TR]

[TR=class: table_row_1]

[TD]_EH_prolog3_GS/_EH_epilog3_GS [/TD]

[TD]C++ EH[/TD]

[TD=align: right]+[/TD]

[TD=align: right]- [/TD]

[/TR]

[TR=class: table_row_2]

[TD]_EH_prolog3_catch_GS/_EH_epilog3_catch_GS[/TD]

[TD]C++ EH[/TD]

[TD=align: right]+[/TD]

[TD=align: right]+ [/TD]

[/TR]

[/TABLE]

SEH2

Apparently was used by MSVC 1.XX (exported by crtdll.dll). Encountered in some old NT programs.

... Saved edi Saved esi Saved ebx Next SEH frame Current SEH handler (__except_handler2) Pointer to the scopetable Try level Saved ebp (of this function) Exception pointers Local variables Saved ESP Local variables Callee EBP Return address Function arguments ...

Appendix I: Sample SEH Program

Let's consider the following sample disassembly.

func1           proc near
_excCode        = dword ptr -28hbuf             = byte ptr -24h_saved_esp      = dword ptr -18h_exception_info = dword ptr -14h_next           = dword ptr -10h_handler        = dword ptr -0Ch_scopetable     = dword ptr -8_trylevel       = dword ptr -4str             = dword ptr  8
  push    ebp  mov     ebp, esp  push    -1  push    offset _func1_scopetable  push    offset _except_handler3  mov     eax, large fs:0  push    eax  mov     large fs:0, esp  add     esp, -18h  push    ebx  push    esi  push    edi
  ; --- end of prolog ---
  mov     [ebp+_trylevel], 0 ;trylevel -1 -> 0: beginning of try block 0  mov     [ebp+_trylevel], 1 ;trylevel 0 -> 1: beginning of try block 1  mov     large dword ptr ds:123, 456  mov     [ebp+_trylevel], 0 ;trylevel 1 -> 0: end of try block 1  jmp     short _endoftry1
_func1_filter1:                         ; __except() filter of try block 1  mov     ecx, [ebp+_exception_info]  mov     edx, [ecx+EXCEPTION_POINTERS.ExceptionRecord]  mov     eax, [edx+EXCEPTION_RECORD.ExceptionCode]  mov     [ebp+_excCode], eax  mov     ecx, [ebp+_excCode]  xor     eax, eax  cmp     ecx, EXCEPTION_ACCESS_VIOLATION  setz    al  retn
_func1_handler1:                        ; beginning of handler for try block 1  mov     esp, [ebp+_saved_esp]  push    offset aAccessViolatio ; "Access violation"  call    _printf  add     esp, 4  mov     [ebp+_trylevel], 0 ;trylevel 1 -> 0: end of try block 1
_endoftry1:  mov     edx, [ebp+str]  push    edx  lea     eax, [ebp+buf]  push    eax  call    _strcpy  add     esp, 8  mov     [ebp+_trylevel], -1 ; trylevel 0 -> -1: end of try block 0  call    _func1_handler0     ; execute __finally of try block 0  jmp     short _endoftry0
_func1_handler0:                        ; __finally handler of try block 0  push    offset aInFinally ; "in finally"  call    _puts  add     esp, 4  retn
_endoftry0:  ; --- epilog ---  mov     ecx, [ebp+_next]  mov     large fs:0, ecx  pop     edi  pop     esi  pop     ebx  mov     esp, ebp  pop     ebp  retnfunc1           endp
_func1_scopetable  ;try block 0  dd -1                      ;EnclosingLevel  dd 0                       ;FilterFunc  dd offset _func1_handler0  ;HandlerFunc
  ;try block 1  dd 0                       ;EnclosingLevel  dd offset _func1_filter1   ;FilterFunc  dd offset _func1_handler1  ;HandlerFunc

The try block 0 has no filter, therefore its handler is a __finally{} block. EnclosingLevel of try block 1 is 0, so it's placed inside try block 0. Considering this, we can try to reconstruct the function structure:

    void func1 (char* str)

{

char buf[12];

__try // try block 0

{

__try // try block 1

{

*(int*)123=456;

}

__except(GetExceptCode() == EXCEPTION_ACCESS_VIOLATION)

{

printf("Access violation");

}

strcpy(buf,str);

}

__finally

{

puts("in finally");

}

Appendix II: Sample Program with C++ Exceptions

func1           proc near
_a1             = dword ptr -24h_exc            = dword ptr -20he               = dword ptr -1Cha2              = dword ptr -18ha1              = dword ptr -14h_saved_esp      = dword ptr -10h_next           = dword ptr -0Ch_handler        = dword ptr -8_state          = dword ptr -4
  push    ebp  mov     ebp, esp  push    0FFFFFFFFh  push    offset func1_ehhandler  mov     eax, large fs:0  push    eax  mov     large fs:0, esp  push    ecx  sub     esp, 14h  push    ebx  push    esi  push    edi  mov     [ebp+_saved_esp], esp
  ; --- end of prolog ---
  lea     ecx, [ebp+a1]  call    A::A(void)  mov     [ebp+_state], 0          ; state -1 -> 0: a1 constructed  mov     [ebp+a1], 1              ; a1.m1 = 1  mov     byte ptr [ebp+_state], 1 ; state 0 -> 1: try {  lea     ecx, [ebp+a2]  call    A::A(void)  mov     [ebp+_a1], eax  mov     byte ptr [ebp+_state], 2 ; state 2: a2 constructed  mov     [ebp+a2], 2              ; a2.m1 = 2  mov     eax, [ebp+a1]  cmp     eax, [ebp+a2]            ; a1.m1 == a2.m1?  jnz     short loc_40109F  mov     [ebp+_exc], offset aAbc  ; _exc = "abc"  push    offset __TI1?PAD         ; char *  lea     ecx, [ebp+_exc]  push    ecx  call    _CxxThrowException       ; throw "abc";
loc_40109F:  mov     byte ptr [ebp+_state], 1 ; state 2 -> 1: destruct a2  lea     ecx, [ebp+a2]  call    A::~A(void)  jmp     short func1_try0end
; catch (char * e)func1_try0handler_pchar:  mov     edx, [ebp+e]  push    edx  push    offset aCaughtS ; "Caught %s\n"  call    ds:printf       ;  add     esp, 8  mov     eax, offset func1_try0end  retn
; catch (...)func1_try0handler_ellipsis:  push    offset aCaught___ ; "Caught ...\n"  call    ds:printf  add     esp, 4  mov     eax, offset func1_try0end  retn
func1_try0end:  mov     [ebp+_state], 0          ; state 1 -> 0: }//try  push    offset aAfterTry ; "after try\n"  call    ds:printf  add     esp, 4  mov     [ebp+_state], -1         ; state 0 -> -1: destruct a1  lea     ecx, [ebp+a1]  call    A::~A(void)  ; --- epilog ---  mov     ecx, [ebp+_next]  mov     large fs:0, ecx  pop     edi  pop     esi  pop     ebx  mov     esp, ebp  pop     ebp  retnfunc1           endp
func1_ehhandler proc near  mov     eax, offset func1_funcinfo  jmp     __CxxFrameHandlerfunc1_ehhandler endp
func1_funcinfo  dd 19930520h            ; magicNumber  dd 4                    ; maxState  dd offset func1_unwindmap ; pUnwindMap  dd 1                    ; nTryBlocks  dd offset func1_trymap  ; pTryBlockMap  dd 0                    ; nIPMapEntries  dd 0                    ; pIPtoStateMap  dd 0                    ; pESTypeList
func1_unwindmap  dd -1  dd offset func1_unwind_1tobase ; action  dd 0                    ; toState  dd 0                    ; action  dd 1                    ; toState  dd offset func1_unwind_2to1 ; action  dd 0                    ; toState  dd 0                    ; action
func1_trymap  dd 1                    ; tryLow  dd 2                    ; tryHigh  dd 3                    ; catchHigh  dd 2                    ; nCatches  dd offset func1_tryhandlers_0 ; pHandlerArray  dd 0
func1_tryhandlers_0dd 0                    ; adjectivesdd offset char * `RTTI Type Descriptor' ; pTypedd -1Ch                 ; dispCatchObjdd offset func1_try0handler_pchar ; addressOfHandlerdd 0                    ; adjectivesdd 0                    ; pTypedd 0                    ; dispCatchObjdd offset func1_try0handler_ellipsis ; addressOfHandler
func1_unwind_1tobase proc neara1 = byte ptr -14h  lea     ecx, [ebp+a1]  call    A::~A(void)  retnfunc1_unwind_1tobase endp
func1_unwind_2to1 proc neara2 = byte ptr -18h  lea     ecx, [ebp+a2]  call    A::~A(void)  retnfunc1_unwind_2to1 endp

Let's see what we can find out here. The maxState field in FuncInfo structure is 4 which means we have four entries in the unwind map, from 0 to 3. Examining the map, we see that the following actions are executed during unwinding:

state 3 -> state 0 (no action)
state 2 -> state 1 (destruct a2)
state 1 -> state 0 (no action)
state 0 -> state -1 (destruct a1)

Checking the try map, we can infer that states 1 and 2 correspond to the try block body and state 3 to the catch blocks bodies. Thus, change from state 0 to state 1 denotes the beginning of try block, and change from 1 to 0 its end. From the function code we can also see that -1 -> 0 is construction of a1, and 1 -> 2 is construction of a2. So the state diagram looks like this:

Where did the arrow 1->3 come from? We cannot see it in the function code or FuncInfo structure since it's done by the exception handler. If an exception happens inside try block, the exception handler first unwinds the stack to the tryLow value (1 in our case) and then sets state value to tryHigh+1 (2+1=3) before calling the catch handler.

The try block has two catch handlers. The first one has a catch type (char*) and gets the exception object on the stack (-1Ch = e). The second one has no type (i.e. ellipsis catch). Both handlers return the address where to resume execution, i.e. the position just after the try block. Now we can recover the function code:

    void func1 ()

{

A a1;

a1.m1 = 1;

try {

A a2;

a2.m1 = 2;

if (a1.m1 == a1.m2) throw "abc";

}

catch(char* e)

{

printf("Caught %s\n",e);

}

catch(...)

{

printf("Caught ...\n");

}

printf("after try\n");

}

Appendix III: IDC Helper Scripts

I wrote an IDC script to help with the reversing of MSVC programs. It scans the whole program for typical SEH/EH code sequences and comments all related structures and fields. Commented are stack variables, exception handlers, exception types and other. It also tries to fix function boundaries that are sometimes incorrectly determined by IDA. You can download it from MS SEH/EH Helper.

Links and References

[1] Matt Pietrek. A Crash Course on the Depths of Win32 Structured Exception Handling.

A Crash Course on theDepths of Win32 Structured Exception Handling, MSJ January 1997

Still THE definitive guide on the implementation of SEH in Win32.

[2] Brandon Bray. Security Improvements to the Whidbey Compiler.

Security Improvements to the Whidbey Compiler - Visual C++ Internals and Practices - Site Home - MSDN Blogs

Short description on changes in the stack layout for cookie checks.

[3] Chris Brumme. The Exception Model.

The Exception Model - cbrumme's WebLog - Site Home - MSDN Blogs

Mostly about .NET exceptions, but still contains a good deal of information about SEH and C++ exceptions.

[4] Vishal Kochhar. How a C++ compiler implements exception handling.

How a C++ compiler implements exception handling - CodeProject