The "Ultimate" Anti-debugging reference

[Nytro]

Contents
1. NtGlobalFlag ...................................................................................................................... 5
2. Heap flags ........................................................................................................................... 8
3. The Heap ............................................................................................................................. 15
4. Thread Local Storage ................................................................................................... 19
5. Anti-Step-Over ................................................................................................................ 25
6. Hardware ............................................................................................................................. 29
A. Hardware breakpoints ............................................................................................... 29
B. Instruction Counting ............................................................................................... 30
C. Interrupt 3 ................................................................................................................... 34
D. Interrupt 0x2d ............................................................................................................ 35
E. Interrupt 0x41 ............................................................................................................ 36
F. MOV SS .............................................................................................................................. 37
7. APIs ...................................................................................................................................... 38
A. Heap functions ............................................................................................................ 38
B. Handles ............................................................................................................................ 41
i. OpenProcess ............................................................................................................... 41
ii. CloseHandle ............................................................................................................. 44
iii. CreateFile ............................................................................................................. 48
iv. LoadLibrary ............................................................................................................. 53
v. ReadFile ...................................................................................................................... 55
C. Execution Timing ........................................................................................................ 57
D. Process-level............................................................................................................... 62
i. CheckRemoteDebuggerPresent .............................................................................. 62
ii. Parent Process ...................................................................................................... 63
iii. CreateToolhelp32Snapshot .............................................................................. 65
iv. DbgBreakPoint......................................................................................................... 79
v. DbgPrint ...................................................................................................................... 80
vi. DbgSetDebugFilterState ..................................................................................... 82
vii. IsDebuggerPresent.............................................................................................. 83
viii. NtQueryInformationProcess .......................................................................... 84
ix. OutputDebugString ................................................................................................ 88
x. RtlQueryProcessHeapInformation ..................................................................... 90
xi. NtQueryVirtualMemory ......................................................................................... 91
xii. RtlQueryProcessDebugInformation ............................................................... 92
xiii. SwitchToThread .................................................................................................. 94
xiv. Toolhelp32ReadProcessMemory........................................................................ 95
xv. UnhandledExceptionFilter ................................................................................ 97
xvi. VirtualProtect .................................................................................................... 98
E. System-level ............................................................................................................... 100
i. FindWindow ............................................................................................................... 100
ii. NtQueryObject....................................................................................................... 102
iii. NtQuerySystemInformation ............................................................................ 105
iv. Selectors ............................................................................................................... 115
F. User-interface .......................................................................................................... 118
i. BlockInput ............................................................................................................... 118
ii. FLD............................................................................................................................. 120
iii. NtSetInformationThread................................................................................. 121
iv. SuspendThread....................................................................................................... 122
v. SwitchDesktop ......................................................................................................... 123
G. Uncontrolled execution ......................................................................................... 124
i. CreateProcess ......................................................................................................... 125
ii. CreateThread ......................................................................................................... 130
iii. DebugActiveProcess ......................................................................................... 131
iv. Enum... .................................................................................................................... 134
v. GenerateConsoleCtrlEvent................................................................................. 134
vi. NtSetInformationProcess................................................................................. 136
vii. NtSetLdtEntries ................................................................................................ 137
viii. QueueUserAPC .................................................................................................... 138
ix. RaiseException .................................................................................................... 139
x. RtlProcessFlsData ................................................................................................ 141
xi. WriteProcessMemory............................................................................................ 142
xii. Intentional exceptions................................................................................. 143
H. Conclusion ................................................................................................................... 146

Download: http://pferrie.host22.com/papers/antidebug.pdf