Introduction to Malware & Malware Analysis

Nytro · September 5, 2011

Introduction to Malware & Malware Analysis

by Rajesh Nikam

Introduction

Reverse engineering is the process of analyzing a subject system to identify the system's components and their relationships, and to create representations of the system in another form or at a higher level of abstraction. The process of reverse engineering, which is part of malware analysis, is accomplished using specific tools that are categorized as hex editors, disassemblers/debuggers, decompiles and monitoring tools.

Disassemblers/debuggers occupy important position in the list of reverse engineering tools. A disassembler converts binary code into assembly code. Disassemblers also extract strings, used libraries, and imported and exported functions. Debuggers expand the functionality of disassemblers by supporting the viewing of the stack, the CPU registers, and the hex dumping of the program as it executes. Debuggers allow breakpoints to be set and the assembly code to be edited at runtime.

One must be familiar with the Portable Executable (PE)[1]file format before diving into reverse engineering for Windows executables. In this article we will get into important aspects of Hiew, OllyDbg and IDA Pro from reverse engineer's perspective.

Hiew

Hiew[2] short for Hacker's view is a great disassembler (not that this is not debugger) designed for hackers, asthe name suggests.

It supports three modes - Text, Hexadecimal and Decode (Dis-assembly) mode. Enter/F4 key is used to switch between these modes. In each mode the Function Line, corresponding to function keys from F1 to F12, which appears at the bottom of the Hiew screen, changes and its functionality with CTRL, SHIFT and ALT combinations.

PE Header

PE Header could be viewed by pressing F8 from Hex or Decode view. In this mode we could see important properties of PE file using following shortcuts:

F6
Sections Table

F7
Import Table

F9
Export Table

F10
Data Directories

F5
Jump to Entry Point

Alt-F2
Jump to end of last section

Search in file

Hiew supports to search in a file for ASCII or HEX sequence of bytes by pressing F7 key. It also supports byte wild character.

Alt-?
Wild character

Shift-F7   
To repeat search

Alt-F7 
To change search direction

Strings

ASCII and Unicode strings are viewed from Text/Hex mode by pressing Alt-F6 key. This helps to search for juicy strings like suspicious urls, FTP, SMTP or IRC commands, files names, registry keys etc in the file. You could jump to selected string from string window by pressing ENTER key. +/- keys are used to change the minimum length of displayed strings, this will help to filter out smaller strings. You could apply filter for displayed strings using F9 key.

Moving around

You could directly jump to specific location by pressing F5 key and providing offset (offset values are hexadecimal?). To specify relative offset + or - sign could be used as prefix to offset. When specified offset is a Virtual Address, it should start with ".". Alt-F1 key is used to toggle between Virtual Address and file offset.

If you want jump to specific function or offset which appears as part of control transfer instruction like call, jmp or conditional jump, you could press the key that appears at the end of instruction. Please see Fig.1 marked for label 4. In this case if you press key “4”, it will take you to offset 0x010073DA.

0 or Backspace key is used to jump back the previous instruction.

Simple Decryption

Hiew supports decryption of block using simple encryptions like xor, add, rol etc. Press F3 from Hex or Decode view to enter in edit mode and then press F7 to add simple decryption routine. You could set operand size as byte, word or dword by pressing F2.

Hiew works great when used in combination with File Manager like FAR[3] by configuring its command line. This is very helpful disassembler to quickly get different aspects of file under analysis like file header, section information, data directories, imported / exported functions and strings.

OllyDbg

OllyDbg[4][5]is an application-level debugger. OllyDbg interface shows the disassembly, hex dump, stack, and CPU registers. Additionally, OllyDbg supports run tracing, conditional breakpoints, PE header viewing, hex editing, and plug-in support.

At first Startup, OllyDbg asks to setup User Data Directory (UDD) and Plugins directory.UDD is used to keep debugged application specific information like breakpoints and other information and obviously you need to save plugins in Plugins directory. It provides wide Debugging Options like break on new module or when thread is created, how to process exceptions etc. OllyDbg supports setting of Hardware Breakpoints, Software Breakpoints, Memory Breakpoints and even Conditional Breakpoints.

OllyDbg supports plugins to enhance its functionality.

Olly Advanced Plugin

There were some bugs reported with Olly v1.10 related to string parsing routine, parsing of faulty executables. This plugin fixes most of these bugs. Some malware samples are loaded with Anti-Debugging techniques [7], Olly Advanced plugin helps to counter most of them.

Olly DumpPlugin

Olly Dump is used to dump debugged process memory. You could trace the packed file till it reaches original entry point and then dump unpacked version of file from process memory. It provides options to rebuild Import Address Table (IAT).

Olly ScriptPlugin

OllyScriptis a plugin to that lets you to automate OllyDbg by writing scripts in an assembly-like language. Many tasks involve a lot of repetitive work just to get to some point in the debugged application. By using this plugin you could write a script once and it could be used with other similar samples. OpenRCE[8]hosts dozens of scripts that helpful to find original entry point (OEP) of many packers.

IDA Pro

IDA Pro is a powerful disassembler that presents the disassembly in well-organized format, shows Graph view of selected function. However, it is less frequently used as a debugger in reverse engineers community where OllyDbg steals the top rank. IDA Pro's features include hex editing, string extraction, and import and export viewing. IDA Pro also features a window for viewing all of the functions called by a program, and provides accurate analyses of the program, summarizing them in a color-coded bar at the top of the screen, which classifies the various sections of the program's code. Below figure shows IDA Pro's interface, including the disassembly and the color-coded analysis bar at the top of the screen. The titles of the other windows are visible on the tabs above the disassembly.

IDA Pro supports wide variety of processors like ARM, DEC, Intel, Motorola etc.

IDA Pro provides selection of debuggers

• Bochs

• Win Debugger

• GDB

• WinDbg

IDA Pro with Boch semulator make an interesting combination that is used to debug Operating system starting from booting process and it is helpful in debugging even ROM BIOS and Master Boot Record code.

Analysis done on particular sample, comments added, functions marked could be saved as an .idb file.

IDA Shortcuts

Below is the list of some important IDA Shortcuts, for complete list please visit reference [9].

Enter
Goto address or variable

Esc
Go back to previous location

;
Add inline comment

INSERT | SHIFT ;
Add comment

N
Rename label, variable, functions etc.

X
Show cross reference

M
Substitute enum

CTRL W
*Dont forget to* Save changes

Extending IDA

IDA supports writing IDC Scripts which is very similar to C like language on top of powerful IDA disassembler. The functionality of disassembler could be utilized even through python scripts and by writing plugins.

FLIRT

Fast Library Identification and Recognition Technology

One of the challenges with disassembly of programs developed with modern high level languages is to identify library functions. One may end up in spending considerable time to go through these functions. On the other hand identification of library functions can considerably ease the analysis of a program. IDA comes with FLIRT to recognize the standard library functions.

One must understand the power of each tool to choose appropriate tool for specific requirement during reverse engineering.

References

1. Portable Executable File Format – A Reverse Engineer View

Tuts 4 You: Downloads / Portable Executable Format (PE) / Portable Executable File Format

2. Hiew

Hiew homepage

3. FAR Manager

Far Manager Official Site : main

4. OllyDbg

OllyDbg v1.10

5. OllyDbg Quick Start Guide

Tuts 4 You: Downloads / OllyDbg Tutorials / OllyDbg Quick Start Guide