### SMT Solvers for Software Security

```SMT Solvers for Software Security
George Nosenko,
Security researcher at Digital Security
SMT Solvers for Software Security
#whoami
• Member of DSecRG.
• System Developer
• Reverse Engineer
• Security Researcher
SMT Solvers in very simple terms
What is a SMT Solver?
Just like the first time using a SMT
constraint solver
SMT Solvers in very simple terms
What is a SMT Solver?
 Solver is a program
“What is the meaning of life?”
“42”
SMT Solvers in very simple terms
How can I ask a question?
 Question is a logical formula
b+2 = c, f(read(write(a,b,3), c-2)) ≠ f(c-b+1)
 SMT-LIB: Language for expressing formulas
http://smtlib.org/
 All solvers understand this language
SMT Solvers in very simple terms
What solver should I choose?
There are many SMT-solvers (over 20)
CVC3, CVC4, STP, Alt-Ergo, Yices, Z3, etc
Z3 is my choice
 Efficient SMT solver
 Open Source Project: http://z3.codeplex.com
 Python, C/C++, .NET binding
 Available online
 Support Windows & Linux
SMT Solvers in very simple terms
SAT or not SAT? Ask a question.
 structure of formula
 declaration
 precondition
 postcondition
 sat, unsat, unknown
 satisfiability
 validity
 model
Taint Nobody Got Time for Crash
(declare-const work Int)
(declare-const sleep Int)
(declare-const fun Int)
(assert (>= work 40))
(assert (>= sleep 42))
(assert (>= fun work))
(assert (= (+ work (+ sleep fun)) 168))
(check-sat)
(get-model)
sat (model (sleep: 42, fun: 63, work 63)
http://rise4fun.com/Z3/pLpMc
SMT Solvers in very simple terms
Properties of SMT solvers
 Mathematical precision
 Expressive power
 Data model
 Efficient implementation
Support Bit-vector & Array
SMT Solvers for Software Security
SMT Solvers for Software Security
SMT Solvers for Software Security
Applications
 Bug Hunting
 Fuzzing (whitebox or blackbox)
 Program Verification & Analysis
 Exploit Generation
 PoC, AEG, APEG
 Protection Analysis
 Obfuscation
 Crypto Analysis
 Malware Analysis
SMT Solvers for Software Security
What’s the point?
Idea: convert portions of code into logical formulas,
and use SMT solver to prove properties about them
xor ebx, ebx
sub ecx, 0x123
setz bl
Is this snippet equivalent to “add eax, ebx”?
sub
bl, bl
What value must EAX have at the beginning
movzx ebx, bl
ebx, 0xbbbbbbbb of this snippet in order for EAX to be
0x12345678 after the snippet executes?
eax, ebx
http://recon.cx/2012/schedule/attachments/52_semantics-based-methods.pdf
Taint Nobody Got Time for Crash
SMT Solvers for Software Security
BV Operations in SMT-LIB 2.0
=/bvcomp
distinct
ite
bvand
bvor
bvxor
bvnot
bvnand
bvnor
bvxnor
bvneg
bvmul
bvudiv
bvurem
bvsub
bvsdiv
bvsrem
bvsmod
bvshl
bvlshr
bvashr
bvult
bvule
bvugt
bvuge
bvslt
bvsle
bvsgt
bvsge
concat
extract
bvshl
bvlshr
bvashr
repeat
zero_extend
sign_extend
rotate_left
rotate_right
Slides - SMT Workshop 2013
SMT Solvers for Software Security
Array Operations in SMT-LIB 2.0: select-store axioms
1. Expression (select a i) returns the value stored at position i of the array a;
2. And (store a i v) returns a new array identical to a, but on position i it
contains the value v.
(declare-const x Int)
(declare-const y Int)
(declare-const a1 (Array Int Int))
(assert (= (select a1 x) x))
(assert (= (store a1 x y) a1))
(check-sat)
SMT Solvers for Software Security
Binary Analysis Platform: http://bap.ece.cmu.edu/
BIL code for add %rax, %rbx
label pc_0x0
T_t1:u64 = R_RBX:u64
T_t2:u64 = R_RAX:u64
R_RBX:u64 = R_RBX:u64 + T_t2:u64
R_CF:bool = R_RBX:u64 < T_t1:u64
R_OF:bool = high:bool((T_t1:u64 ^ ~T_t2:u64) & (T_t1:u64 ^ R_RBX:u64))
R_AF:bool = 0x10:u64 == (0x10:u64 & (R_RBX:u64 ^ T_t1:u64^T_t2:u64))
R_PF:bool =
~low:bool(let T_acc:u64 := R_RBX:u64 >> 4:u64 ^ R_RBX:u64 in
let T_acc:u64 := T_acc:u64 >> 2:u64 ^ T_acc:u64 in
T_acc:u64 >> 1:u64 ^ T_acc:u64)
R_SF:bool = high:bool(R_RBX:u64) R_ZF:bool = 0:u64 == R_RBX:u64
SMT Solvers for Software Security
Bug Hunting
Bug Hunting
Vulnerability related with Integer
 CWE-190,191,192,194,196
 May cause:
 Bypass sanity check
 Buffer Overflow
 Dangling Pointer
 Use after free
 Application specific
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
static int fb_mmap(struct file *file, struct vm_area_struct * vma){
if (!info) return -ENODEV;
...
off = vma->vm_pgoff << PAGE_SHIFT;
fb = info->fbops;
if (!fb)
return -ENODEV;
...
/* frame buffer memory */
start = info->fix.smem_start;
len = PAGE_ALIGN((start & ~PAGE_MASK) + info->fix.smem_len);
if (off >= len) {
/* memory mapped io */
off -= len;
...
start = info->fix.mmio_start;
len = PAGE_ALIGN((start & ~PAGE_MASK) + info>fix.mmio_len);
}
mutex_unlock(&info->mm_lock);
if ((vma->vm_end - vma->vm_start + off) > len)
return -EINVAL;
...
fb_pgprotect(file, vma, off);
if (io_remap_pfn_range(vma, vma->vm_start, off >> PAGE_SHIFT,
vma->vm_end - vma->vm_start, vma>vm_page_prot))
return -EAGAIN;
return 0;
}
Bug Hunting
How does Motochopper work?
1728 open("/dev/graphics/fb0", O_RDWR) = 6
...
1728 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = 0x400f2000
...
1728 munmap(0x4015b000, 9433088)
= 0
1728 mmap2(NULL, 9437184, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = 0x4015b000
1728 munmap(0x4015b000, 9437184)
= 0
1728 mmap2(NULL, 9441280, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = -1 EINVAL (Invalid
argument)
1728 mmap2(NULL,
1728 mmap2(NULL,
(Out of memory)
1728 mmap2(NULL,
(Out of memory)
...
(Out of memory)
1728 mmap2(NULL,
0x4015b000
NAME
2415919104, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x70900) = -1 ENOMEM
2231369728, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x7b900) = -1 ENOMEM
2214592512, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x7c900) = -1 ENOMEM
2113929216, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x82900) =
mmap2 - map files or devices into memory
#include <sys/mman.h>
void *mmap2(void *addr, size_t length, int prot,
int flags, int fd, off_t pgoffset);
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
input_userauth_info_response(){
...
u_int nresp;
...
nresp = packet_get_int();
if (nresp > 0) {
response = xmalloc(nresp * sizeof(char*));
for (i = 0; i < nresp; i++)
response[i] = packet_get_string(NULL);
}
packet_check_eom();
}
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
(declare-const
sizeof (_ BitVec 32))
(declare-const
nresp
(_ BitVec 32))
(declare-const
mult
(_ BitVec 32))
(assert ( =
4
sizeof
(assert ( = mult
nresp*sizeof
(_ bv4 32)))
(bvmul nresp sizeof)))
(assert ( bvugt
nresp
(assert ( bvult
nresp
mult nresp))
(get-model)
;
(_ bv0 32) )) ; nresp > 0
(assert ( = mult (_ bv256 32)))
nresp*sizeof = 256
(check-sat)
; sizeof (char*) =
; nresp*sizeof <
;
Bug Hunting
Verification & Static analyze with SMT
 Single collaborative framework
 It’s not heuristic bug-finding
 It allows user to manipulate
 Functional specification
 Prove that source code satisfies
specification
 Expands with plug-ins
 ACSL is a behavioral
specification language
Bug Hunting
Jessie: verification tools for C programs
 Jessie is a plug-in for the Frama-C
 Functional Checking
 Safety Checking
 Memory Safety
 Integer Overflow
 Checking Termination
Bug Hunting
Jessie: Integer Overflow Safety
#pragma JessieTerminationPolicy(user)
//@ requires n >= 0 && \valid_range(t,0,n−1);
int binary_search(long t[], int n, long v) {
int l = 0, u = n-1;
//@ loop invariant 0 <= l && u <= n−1;
while (l <= u) {
int m = l + (u - l) / 2; //int m = (l + u) / 2;
if (t[m] < v)
l = m + 1;
else if (t[m] > v)
u = m - 1;
else return m;
}
return -1;
}
> frama-c -jessie binary-search.c
Bug Hunting
Immunity Debugger & SMT: Infrastructure
SequenceAnalyzer – Models x86 as operations over a set of SMT primitives.
Solver – Ctypes interface to the CVC3 SMT solver API. Supports a variety of
theories including quantifier free, bit-vector arithmetic, linear arithmetic etc.
CodeGraph/PathGenerator – Purely static CFG building and path generation.
PathWalker – SMT based path traversal. Each conditional jump is checked for
feasibility and the path discarded if not SAT.
BugChecker – Subclasses provide the check_ins method which will be passed the
SMT context representing the current path.
Bug Hunting
Immunity Debugger & SMT: !find_int_overwlow.py
SMT in protection analysis
PROTECTION ANALYSIS
SMT in protection analysis
Using SMT to defeat simple hashing algorithms
def round_hash(a, b, c, d):
out = [ ]
for i, n in enumerate((a, b, c, d)):
nn = 0
for j in range(32):
nn |= (rotl(n, SCRAMBLE_TABLE[(i << 2)+j]) & 1) << j
nn ^= XOR_TABLE[i]
out.append(nn)
out[0] = rotl(out[0], ROT_TABLE[0])
out[1] = rotl(out[1], ROT_TABLE[1])
out[2] = rotl(out[2], ROT_TABLE[2])
out[3] = rotl(out[2], ROT_TABLE[3])
return out
a ^= c
b ^= d
for i in range(128):
a, b, c, d = round_hash(a, b, c, d)
SMT in protection analysis
Automated KeyGen Generation. Kao’s Toy Project
SMT in protection analysis
Automated KeyGen Generation. Kao’s Toy Project
 Lift the checking algorithm to BIL
./toil -binrange ~/toyproject.exe 0x401105 0x401111 -o checkUnlockCode.il
 Convert BIL to single static assignment form (SSA), unroll loop
./iltrans -il checkUnlockCode.il -to-ssa -simp-ssa -to-cfg -unroll 31 -rm-cycles \
-rm-indirect-ast -to-ast -normalize-mem -flatten-mem -pp-ast checkUnlockUnroll.il
egrep -v '^cjmp.*\$' checkUnlockUnroll.il > checkUnlockUnrollOpt.il
 Convert BIL to SMT-formula
./topredicate -il checkUnlockUnrollOpt.il -noopt -solver z3 -stp-out checkLoop.smt
line 18: assert --> define-fun alg () (Array (_ BitVec 32) (_ BitVec 8))
line 921: false --> ?mem_array_83_670
SMT in protection analysis
Create precondition and postcondition
AEG
Automatic Exploit Generation
 Automatically craft an input that redirects control flow
Loosely defined as “Given a program and a vulnerability,
automatically craft an input that redirects control flow to
malicious code”
AEG
Automatically craft an input that hijacks control flow
 Get the trace to vulnerable code
 Convert the trace into set of constraints
Freach
 Generate the set of conditions that make code exploitable
 Solve (Freach U Fexploit)
 SMT-solver defines required input
AEG
Automatically craft an input that hijacks control flow
Freach =
{ t0= eax + ebx, zf ==1 }
Cval =
{ t1= ebp + 4, t1 = ebp +ecx }
AEG
Automatically craft an input that hijacks control flow
Freach =
{t0 = eax + ebx, zf == 1}
Cval =
{t1 = ebp + 4, t1 = ebp + ecx}
ebx = 0x21524111, ecx = 4}
(declare-const t0 (_ BitVec 32))
(declare-const t1 (_ BitVec 32))
(declare-const eax (_ BitVec 32))
(declare-const ebx (_ BitVec 32))
(declare-const ecx (_ BitVec 32))
(declare-const ebp (_ BitVec 32))
; Freach = {zf = 1, t0 = eax + ebx}
(assert (= t0 (bvadd eax ebx)))
; t0 = eax + ebx
(assert (= t0 #x00000000))
; zf = 1
; Cval = { eax = 0xDEADBEEF }
; Caddr = { t1 = ebp + 4, t1 = ebp + ecx}
(assert
(and
(= t1 (bvadd ebp #x00000004)) ; t1 = ebp + 4
; t2 = ebp + ecx
)
sat (model
(define-fun ecx () (_ BitVec 32) #x00000004)
(define-fun eax () (_ BitVec 32) #xdeadbeef)
(define-fun ebx () (_ BitVec 32) #x21524111)
)
http://rise4fun.com/Z3/j2Y
 Data Execution Prevention (DEP)
 Windows 8 ROP mitigation enforces policies on who/where can call VirtualAlloc() or
VirtualProtect() to enable memory executable at run-time
 IOS already totally forbid code injection: Writable pages have NX permission & only signed
pages are executable
 Return Oriented Programming
 fun at first time, then hurt
 hundreds and thousands of ROP-gadgets
 find a suitable gadget can be difficult
 research efforts aimed at solving the problem of automatic generation ROP-chains
An interesting example from 0verckl0ck
Given:
 we can write into eax, but only ASCII printable char
 we have ROP-gadgets like these:
Goal:
 determine the initial value eax
 find the minimum sequence of calls gadgets
An interesting example from 0verckl0ck
source: http://rise4fun.com/Z3Py/OrzP
assert( init_eax + g1*0xc9f4458b +
g3*0x0fcf +
g4*0x13b2 +
g5*0x1337 +
assert(ascii_printable( init_eax ) )
sum (g1,g2,g3,g4,g5,g6) --> min
0x522e707c + 3*0xc9f4458b + 8*0x13b2 = 0xb00bdead
This script looks for a sequence that satisfies
the constraints we specify
OptiROP
OptiROP
ROPC: https://github.com/pakt/ropc
ROPC : Type of gadgets that ROPC find &use
Name
Input
NopG
_
_
nop
OutReg, Value
_
OutReg  Value
MoveRegG
InReg, OutReg
_
OutReg  InReg
ArithmeticG
InReg1, InReg2, OutReg
op
OutReg <- InReg1 op InReg2
StoreMemG
# Bytes, Offset
# Bytes, Offset
ArithmeticStoreG
# Bytes, Offset, op
# Bytes, Offset, op