Measuring virtual machine detection in malware using DSD tracer

Report
Boris Lau, Vanja Svajcer
Sophoslabs, Journal in Computer Virology, 2008
報告者:張逸文
Outline
 Introduction
 Virtual machine detection methods
 Methodology of our study with DSD-Tracer
 Results
 Conclusion
2
Introduction(#1)
 Virtual machine technology is first implemented by IBM
 More attention from virus writers & computer security
researchers
 If in VM,malware will behave like a normal program
 If the proportion is > 0.1%,developing an environment
to successfully analyze VM-aware malware is important
3
Introduction(#2)
 The most common security use cases with VM
 Software vulnerability research
 Malware analysis
 Honeypots
4
Virtual machine detection methods(#1)
 If VM is detected, the malware will
 stop its execution or
 launch a specially crafted payload
 Zlob Trojans
 IRC bots
 Executable packers
5
Virtual machine detection methods(#2)
 Detection of running under MS virtual PC using VPC
communication channel
 Communication between guest OS & VMM
 Exceptions due to opcode:0x0f, 0x3f / 0x0f, 0xc7, 0xc8
 Call different VMM services: 0x07, 0x0B
6
Invalid instruction VPC communication channel detection
7
Virtual machine detection methods(#3)
 Detection of running under VMware using VMWare
control API
 VMWare backdoor communication
 guest ↔ host communication
 IN instruction
 port 0x5658
 eax:0x564D5868(VMXh)
 ebx :function number
8
9
Anti-VMWare prevention virtual machine
initialization settings
10
Virtual machine detection methods(#4)
 Redpill(using SIDT, SGDT or SLDT)
 SxxT x86 instruction
 Return the contests of the sensitive register
 IDT in VMWare is 0xffXXXXXX
 IDT in Virtual PC is 0xe8XXXXXX
 Compare with 0xd0
 Invalid in multi processor system
11
Redpill
12
Virtual machine detection methods(#5)
 SMSW VMWare detection
 Store Machine Specific Word instruction
 Return 16-bit result
 32 bits register(16-bit undefined + 16-bit result)
 In VMWare, the top 16-bits doesn’t change
13
SMSW VMWare detection code
14
Methodology of our study with
DSD-Tracer(#1)
 DSD-Tracer
 identify obfuscation packers
 dynamic & static analysis
15
Methodology of our study with
DSD-Tracer(#2)
16
Methodology of our study with
DSD-Tracer(#3)
 Dynamic component
 Instructions decoded before its execution
 All CPU registers
 Reads / writes to virtual / physical memory
 Interrupts / exceptions generated
 Instrumented virtual machine
 Low-level information
17
Methodology of our study with
DSD-Tracer(#4)
 Static component
 C++ interface
 Python Script
 Match known techniques for detecting VM
 Automatic replication harness
 Web-based automatic replication harness
18
Methodology of our study with
DSD-Tracer(#5)
 Case study:DSD-Tracer on Themida
 Analyzing Themida by traditional debugger/static technique
is troublesome
 recording memory-io
 “dump” sample in static environment
19
Methodology of our study with
DSD-Tracer(#6)
 Justification for using DSD-Tracer
 Coverage of packed samples
 Low-level accuracy
 Circumventing armour techniques
 Mitigating factors in using DSD-Tracer
 No Bochs detect techniques in any sample
 4 samples/hour, 5 samples from each set of packed file
 85% of Themida samples with VM-aware techniques
20
Methodology of our study with
DSD-Tracer(#7)
 Proof of concept experiment for DSD-Tracer on VMware
 Cross-verified multiple dynamic analysis
 Implemented on VMware Workstation 6
 Invisible breakpoint
 GDB script for printing the assembly execution trace in user
mode
21
Results(#1)
 VM detection in packers
 193 different packers, 400 packed samples
 Overall VM detection rate is 1.15%
 Themida accounting for 1.03%
 ExeCryptor accounting for 0.15%
 EncPk:custom packers
22
Results(#2)
 VM detection in malware families
 Static analysis rules – disassembly
 Dynamic analysis rules – Sophos virus engine emulation
 2 million known malicious files
 A large set of knows clean files
 VM-aware samples < 1%
 Method breakdown(Table 1.)
 Family breakdown (Table 2.)
 Dial/FlashL
23
Results(#3)
24
Results(#4)
 VMWare backdoor detection method  50% VPC illegal
instruction detection method
 VPC illegal instruction detection method  93%
VMWare backdoor detection method
25
Results(#5)
 Fig. 7 VMWare backdoor detection in 2007
26
Results(#6)
 Fig. 8 VPC backdoor detections in 2007
27
Conclusion
 Combination of dynamic and static analysis is better
 2.13% VM-aware samples
28
Q&A
29
Appendix
 VMWare backdoor I/O port
 On the Cutting Edge:Thwarting Virtual MachineDetection
 Trapping worm in a virtual net
 VM、Virtual PC、Bochs比較
 http://hi.baidu.com/%CC%FA%D0%AC%B9%C3%C4%E
F/blog/item/085cc609b215f3226b60fba5.html 大陸版
 http://www.osnews.com/story/1054 國外版
30
Thanks ~
31

similar documents