Fuzzing Openssh With AFL
American fuzzy lop(AFL) is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. This substantially improves the functional coverage for the fuzzed code. The compact synthesized corporaproduced by the tool are also useful for seeding other, more labor- or resource-intensive testing regimes down the road
Reference:
http://lcamtuf.coredump.cx/afl/
http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
http://www.vegardno.net/2017/03/fuzzing-openssh-daemon-using-afl.html
https://github.com/mcarpenter/afl
https://github.com/googleprojectzero/winaf
https://github.com/djmdjm/openssh-fuzz-cases
Usage
┌─[✗]─[root@parrot]─[~/openssh-7.9p1/output]
└──╼ #afl-cmin
corpus minimization tool for afl-fuzz by <lcamtuf@google.com>
Usage: /usr/local/bin/afl-cmin [ options ] -- /path/to/target_app [ ... ]
Required parameters:
-i dir - input directory with the starting corpus
-o dir - output directory for minimized files
Execution control settings:
-f file - location read by the fuzzed program (stdin)
-m megs - memory limit for child process (100 MB)
-t msec - run time limit for child process (none)
-Q - use binary-only instrumentation (QEMU mode)
-U - use unicorn-based instrumentation (Unicorn mode)
Minimization settings:
-C - keep crashing inputs, reject everything else
-e - solve for edge coverage only, ignore hit counts
For additional tips, please consult docs/README.
┌─[✗]─[root@parrot]─[~/openssh-7.9p1/output]
└──╼ #afl-fuzz
afl-fuzz 2.52b by <lcamtuf@google.com>
afl-fuzz [ options ] -- /path/to/fuzzed_app [ ... ]
Required parameters:
-i dir - input directory with test cases
-o dir - output directory for fuzzer findings
Execution control settings:
-f file - location read by the fuzzed program (stdin)
-t msec - timeout for each run (auto-scaled, 50-1000 ms)
-m megs - memory limit for child process (50 MB)
-Q - use binary-only instrumentation (QEMU mode)
-U - use Unicorn-based instrumentation (Unicorn mode)
Fuzzing behavior settings:
-d - quick & dirty mode (skips deterministic steps)
-n - fuzz without instrumentation (dumb mode)
-x dir - optional fuzzer dictionary (see README)
Other stuff:
-T text - text banner to show on the screen
-M / -S id - distributed mode (see parallel_fuzzing.txt)
-C - crash exploration mode (the peruvian rabbit thing)
For additional tips, please consult /usr/local/share/doc/afl/README.
┌─[root@parrot]─[/usr/bin]
└──╼ #afl-gotcpu
afl-gotcpu 2.52b by <lcamtuf@google.com>
[*] Measuring per-core preemption rate (this will take 1.00 sec)...
Core #0: AVAILABLE
>>> PASS: You can run more processes on 1 core. <<<
┌─[root@parrot]─[/usr/bin]
└──╼ #afl-whatsup
status check tool for afl-fuzz by <lcamtuf@google.com>
Usage: /usr/local/bin/afl-whatsup [ -s ] afl_sync_dir
The -s option causes the tool to skip all the per-fuzzer trivia and show
just the summary results. See docs/parallel_fuzzing.txt for additional tips.
┌─[root@parrot]─[/usr/bin]
└──╼ #ls afl-*
afl-analyze afl-clang++ afl-clang-fast++ afl-fuzz afl-gcc afl-plot afl-tmin
afl-clang afl-clang-fast afl-cmin afl-g++ afl-gotcpu afl-showmap afl-whatsupBuild openssh with afl
1.Download sourcecode.
Use github source.
git clone git://anongit.mindrot.org/openssh.git
Use apt command.
apt source openssh2.Configure environment variable
export AFL_HARDEN="1"
export AFL_INST_RATIO="100"
export CC="/usr/bin/afl-clang-fast"
export CXX="/usr/bin/afl-clang-fast++"3.Build
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #mkdir install var-empty
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #./configure CFLAGS="-g -O3" --prefix=$PWD/install --with-privsep-path=$PWD/var-empty --with-sandbox=no --with-privsep-user=vegard
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #make j44.Trace
┌─[root@parrot]─[/]
└──╼ #cd /root/openssh-7.9p1/
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #./sshd -d -e -p 2200 -r -f sshd_config -i
sshd_config line 84: Unsupported option UsePAM
debug1: sshd version OpenSSH_7.9, OpenSSL 1.1.1b 26 Feb 2019
Could not load host key: /root/openssh-7.9p1/install/etc/ssh_host_rsa_key
Could not load host key: /root/openssh-7.9p1/install/etc/ssh_host_ecdsa_key
Could not load host key: /root/openssh-7.9p1/install/etc/ssh_host_ed25519_key
sshd: no hostkeys available -- exiting.
-----------------------------------------------
Use the command below to solve the problem.
-----------------------------------------------
./ssh-keygen -t rsa -f /root/openssh-7.9p1/install/etc/ssh_host_rsa_key
./ssh-keygen -t ecdsa -f /root/openssh-7.9p1/install/etc/ssh_host_ecdsa_key
./ssh-keygen -t ed25519 -f /root/openssh-7.9p1/install/etc/ssh_host_ed25519_key
┌─[✗]─[root@parrot]─[~/openssh-7.9p1]
└──╼ #./sshd -d -e -p 2200 -r -f sshd_config
debug1: sshd version OpenSSH_7.9, OpenSSL 1.1.1b 26 Feb 2019
debug1: private host key #0: ssh-rsa SHA256:3oEQ2gs1voesjOZNhlUaL/wa2YUxiBUCI11NxVdUxpM
debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:iYkpesuAtQT83iTgwFJ3zwbBAluuVuZes3iyo+fH0rA
debug1: private host key #2: ssh-ed25519 SHA256:kTEW7wSlcSt9V9JUFRB2OPv030V3piaNxiIAlkcqcMA
debug1: Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 2200 on 0.0.0.0.
Server listening on 0.0.0.0 port 2200.
debug1: Bind to port 2200 on ::.
Server listening on :: port 2200.
----------------------------------------------------------------
In another terminal,run the command below to generate a trace log file.
----------------------------------------------------------------
┌─[✗]─[root@parrot]─[~/openssh-7.9p1]
└──╼ #strace -e trace=write -o strace.log -f -s 8192 ./ssh -p 2200 vegard@localhost
vegard@localhost's password:
Environment:
USER=vegard
LOGNAME=vegard
HOME=/vegard
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/root/openssh-7.9p1/install/bin
MAIL=/var/mail/vegard
SHELL=/bin/sh
TERM=xterm
SSH_CLIENT=::1 49176 2200
SSH_CONNECTION=::1 49176 ::1 2200
SSH_TTY=/dev/pts/3
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #cat strace.log
6107 write(199, "\0\0\0\0", 4) = -1 EBADF (Bad file descriptor)
6107 write(3, "SSH-2.0-OpenSSH_7.9p1\r\n", 23) = 23
6107 write(3, "\0\0\5l\4\24\370\251\t\227\270J\17\"\30\205 Q\n\373\264Y\0\0\1\rcurve25519-sha256.......
-----------------------------------------------------------------
We see here that the client is communicating over file descriptor 3. You will have to delete all the writes happening on other file descriptors. Then take the strings and paste them into a Python script, something like:
-----------------------------------------------------------------
import sys
for x in [
"SSH-2.0-OpenSSH_7.9p1\r\n"
"\0\0\5..."
...
]:Fuzzing with AFL
1.Create a python script(createtestcase.py) include all strings for file descriptor 3
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
for x in [
"SSH-2.0-OpenSSH_7.9p1\r\n"
"\0\0\5l\4\24\370\251\t\227\270J\17\"\30\205 Q\n\373\264Y\0\0\1\rcurve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c\0\0\1fecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa\0\0\0lchacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com\0\0\0lchacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com\0\0\0\325umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\0\0\0\325umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\0\0\0\32none,zlib@openssh.com,zlib\0\0\0\32none,zlib@openssh.com,zlib\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
"\0\0\0,\6\36\0\0\0 \321\376;\243c\376\364w\32\212\333\344i}\237kH\260c\r\264\236\20(Hq\236\320\335g&\32\0\0\0\0\0\0"
"\0\0\0\f\n\25\0\0\0\0\0\0\0\0\0\0"
"ERB$\250\25j\201!\354~\263\31m\3170\t\203`\202\17:\25\212\253$:\352\371\0266\3.\34\326\354%D\331o>wr\327"
"\177f]\271\177j0\"\345W=!\343P\256!\347_\267m\0\366\345\356\337SV\33\323\322\346\213\0j6\257\263\36\347\250\247\274\2147\343\354\21c\24\301~\2724\203\271\f\261\366\203\334\10\315\5=\276Vi\233"
"8P3\306\235\374_\345\f\302\204\264\320\317\342`\303\304\21\224bn\317\206i\237\214%\222\313^\356\351T\200\202\274]Ln\300.tx\357)\207\242\30\303\261\342R\373M\330v\267\32\256\352\2464\304\252\365f\257\347\332p\3662R\264\311#\370\314.9\236\337C\367\270WN(\360\t\35\377\277\342)\2359\325zlv?\16\220\347j,\253\231\5\354\233\r\355\341\31l\6$;4\372\247\223O\22\254\300\253\377\207\217\0Y*C\371\10=\2041g\353\203\276\277\372I\205\33\217\274x\351\305A)S\252\265>\266\227\\+\252\24<\242[\353\242\304\256\35\370w\271\232^j\n\25\301\223\256Z\327)\202\305q~4_\336\267\353-\261;\f}\333t\214BB\214\331\22l\0032\202\276\21=\3\267V_\306\320\351\315w\366W\366\366\220\351!\321\226:\21\331\241\f\303\354\10\234U\201\234tN\370\216\244^\224\2g\306h\355\5\314\356~1\323\25\252\312\31@\227\r\370\311%\"\246\376VOP\232a\352:Be/\346t\324 \214vs\251\30l\331m~}\\j\345\260\23a\304&wl\311fx,\322\24\213\311}\372\6\22\366Abl\333\3(!\212\213\332\305\334\240o\4\16foI\367\261\364\231\34\364\30\330\222Z\3A\226\331\2532\215z\36\317\2505X?5\237"
"\227\25\314\216\365\372>\237\35\177\177\243\235\250 a\313P\206%/c\341-\26f\217G=\253]\274\"\360\277|{\333GE\265\204L\244\365\353s\346\323\36\34\310\272\347 \310\343\343\240\313\372\254z~\226\331\262\34\227\351\25\1R\273\241B@\236x\310\212K\236\22\300\7\300\355K(g\2\237&\355\365!@\367\201\311\20\206\211\t\312\243\322\36M\376\20\312S\211\230\273\316 =v\213\312\365<S\3549\333\215E\333\6\321\24\363g\330\f\266\307\336\216eC\330\1\23"
"\301\247\313A\223\206l\313-5\243h\v\2527+x\2709\276\227\304\307xLGr\352\233\332\261g\266\26I\360\234\23[\266\211\361\\\231\375\345S\320\360\304\222Z\3b$\243Z'\355BQ\1b\232K\276\321\331bl\320<\"\244\330\250\237\24\t\221\332\365S\231\236\\\224\334M\20\336\357N\r\330P\204\245\27\343\261\231\312\203c\255Q*6c~\31"
"\215+\2067\350\3508%-e\220\340\370\351\f\273\327}c\22\226sbx\353s\362m:\217EQ/\251{\320\321\344\207\344\f\373%\17\2167\240y #p\223\3758T\tB\17C\372\2101z\250\223\7)h\266\17\233\334_Sl\10\263J\241\227o\26\350\330\3\357\213X\"(\251\24\270\t_9\240^Pz\354|\212\210\365\r\223\300${\357\257\230\245\216Q\330@W\226\37\205\250.\217\257q`0\313(\305\363\253`E\245~x83\225+\255\306[\37\370\241\231\314}\261+\307V\243\233\310\36\253^\342\263[\373\360\240\0_^|a\347s\277\323\24U\356\201b\230\250\311V\331f\207d\306\205\32\10\1\215 \211\0251\3772\351\273\212\25F\212\10\211\36\351K;\232\212\r\321\24\260\356\\0\20Qd\17\227\344\377My\25$\304\265MV8\331Q{\355-\335\2604\304u\250\34 tI\26\242i\212b\304\36\202\320\271\23\250\252s\216\232\267&5\220\2\340\211\34\345\0\327\335s\311u\276\313\10\336\375\362\t4^S\373/y\272\233\225_Q\6\327g\327\305\336\303v\\\231\234\224B1h\331Hm\213\32\37K_#\322\301\266\364\253\271#\323R\276)\360:\312Q\310\326\233M\315\245*\311+|Wnz\3D\17p\31b\237o3\222\224V\261\313\251 ,\n\211\361\250"
]:
sys.stdout.write(x)2.Use the script to create testcase
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #python3 createtestcase.py > input/testcase
You could test run without afl:
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #./sshd -d -e -p 2200 -r -f sshd_config -i < input/testcase 2>&1 > /dev/null3.Creating an OpenSSH dictionary(sshd.dict)
s0="3des-cbc"
s1="aes128-cbc"
s2="aes128-ctr"
s3="aes128-gcm@openssh.com"
s4="aes192-cbc"
s5="aes192-ctr"
s6="aes256-cbc"
s7="aes256-ctr"
s8="aes256-gcm@openssh.com"
s9="arcfour"
s10="arcfour128"
s11="arcfour256"
s12="blowfish-cbc"
s13="cast128-cbc"
s14="chacha20-poly1305@openssh.com"
s15="curve25519-sha256@libssh.org"
s16="diffie-hellman-group14-sha1"
s17="diffie-hellman-group1-sha1"
s18="diffie-hellman-group-exchange-sha1"
s19="diffie-hellman-group-exchange-sha256"
s20="ecdh-sha2-nistp256"
s21="ecdh-sha2-nistp384"
s22="ecdh-sha2-nistp521"
s23="ecdsa-sha2-nistp256"
s24="ecdsa-sha2-nistp256-cert-v01@openssh.com"
s25="ecdsa-sha2-nistp384"
s26="ecdsa-sha2-nistp384-cert-v01@openssh.com"
s27="ecdsa-sha2-nistp521"
s28="ecdsa-sha2-nistp521-cert-v01@openssh.com"
s29="hmac-md5"
s30="hmac-md5-96"
s31="hmac-md5-96-etm@openssh.com"
s32="hmac-md5-etm@openssh.com"
s33="hmac-ripemd160"
s34="hmac-ripemd160-etm@openssh.com"
s35="hmac-ripemd160@openssh.com"
s36="hmac-sha1"
s37="hmac-sha1-96"
s38="hmac-sha1-96-etm@openssh.com"
s39="hmac-sha1-etm@openssh.com"
s40="hmac-sha2-256"
s41="hmac-sha2-256-etm@openssh.com"
s42="hmac-sha2-512"
s43="hmac-sha2-512-etm@openssh.com"
s44="rijndael-cbc@lysator.liu.se"
s45="ssh-dss"
s46="ssh-dss-cert-v01@openssh.com"
s47="ssh-ed25519"
s48="ssh-ed25519-cert-v01@openssh.com"
s49="ssh-rsa"
s50="ssh-rsa-cert-v01@openssh.com"
s51="umac-128-etm@openssh.com"
s52="umac-128@openssh.com"
s53="umac-64-etm@openssh.com"
s54="umac-64@openssh.com"4.Optimize Testcase
┌─[✗]─[root@parrot]─[~/openssh-7.9p1]
└──╼ #afl-cmin -i input/ -o input_optimize -- ./sshd -d -e -p 2100 -r -f sshd_config -i @@
corpus minimization tool for afl-fuzz by <lcamtuf@google.com>
[*] Testing the target binary...
[+] OK, 638 tuples recorded.
[*] Obtaining traces for input files in 'input/'...
Processing file 1/1...
[*] Sorting trace sets (this may take a while)...
[+] Found 640 unique tuples across 1 files.
[*] Finding best candidates for each tuple...
Processing file 1/1...
[*] Sorting candidate list (be patient)...
[*] Processing candidates and writing output files...
Processing tuple 640/640...
[!] WARNING: All test cases had the same traces, check syntax!
[+] Narrowed down to 1 files, saved in 'input_optimize'.5.Running AFL
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #afl-fuzz -x sshd.dict -i input_optimize -o output -M 0 -- ./sshd -d -e -p 2100 -r -f sshd_config -i
afl-fuzz 2.52b by <lcamtuf@google.com>
[+] You have 1 CPU core and 1 runnable tasks (utilization: 100%).
[*] Checking core_pattern...
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning 'input'...
Find Bugs In Your Sleep
1.Run Fuzzing Background
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #screen -dmS fuzzer1 /bin/bash -c "afl-fuzz -x sshd.dict -i input -o output -M fuzzer1 -- ./sshd -d -e -p 2100 -r -f sshd_config -i @@"
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #screen -dmS fuzzer2 /bin/bash -c "afl-fuzz -x sshd.dict -i input -o output -S fuzzer2 -- ./sshd -d -e -p 2100 -r -f sshd_config -i @@"
┌─[root@parrot]─[~/openssh-7.9p1]
└──╼ #screen -dmS fuzzer3 /bin/bash -c "afl-fuzz -x sshd.dict -i input -o output -S fuzzer3 -- ./sshd -d -e -p 2100 -r -f sshd_config -i @@"2.Check Screen
┌─[✗]─[root@parrot]─[~/openssh-7.9p1]
└──╼ #screen -rd fuzzer1
and so on...3.Check Output Directory
┌─[✗]─[root@parrot]─[~/openssh-7.9p1]
└──╼ #cd output
┌─[root@parrot]─[~/openssh-7.9p1/output]
└──╼ #tree -L 2
.
├── 0
│ ├── crashes
│ ├── fuzz_bitmap
│ ├── fuzzer_stats
│ ├── hangs
│ ├── plot_data
│ └── queue
├── fuzzer1
│ ├── crashes
│ ├── fuzz_bitmap
│ ├── fuzzer_stats
│ ├── hangs
│ ├── plot_data
│ └── queue
├── fuzzer2
│ ├── crashes
│ ├── hangs
│ ├── plot_data
│ └── queue
└── fuzzer3
├── crashes
├── hangs
├── plot_data
└── queue
16 directories, 8 filesSummary:
Fuzz testing is one of the most powerful tools in the bug hunter’s toolset. At a basic level, fuzzing is the art of repeatedly processing crafted test inputs while checking for ill-effects, such as memory corruptions or information disclosures.
One of the main advantages of fuzz testing is that it works 24×7 without a break and with no need for overtime pay.
In recent months, I have successfully employed fuzz testing to identify security bugs in four major web browsers, a popular server side scripting language, various graphics/media libraries, a cross-platform compression utility, and more.
In my upcoming BSides SF presentation, I will be discussing how I make use of American Fuzzy Lop (AFL) in my research.
American Fuzzy Lop, or AFL for short, is a framework for performing coverage-based fuzzing.
At a high level, this means that AFL refines the test cases it chooses based on feedback of which parts of a target program are exercised. In contrast, other popular fuzzing techniques commonly involve random file mutations or derive test cases from target-specific templates.
The concept of coverage-based fuzzing is not new and, in fact, inspiration for AFL came from Google’s Tavis Ormandy, whose research conducted over 10 years ago showed how to evaluate test cases based on GCov coverage reports.
In the case of AFL, feedback is provided through instrumentation, which records path transitions within a programs execution flow.
Ideally, the instrumentation trampolines are added to a programs assembly during a build process but AFL also supports a much slower mode in which QEMU user emulation is used to trace execution. (A port of AFL also exists to enable runtime instrumentation through the Intel PIN tool but this is again much slower than compiled instrumentation.)
While there are very few “knobs and dials” to tune for successful fuzzing with AFL, there are quite a few experimental features to take advantage of, too, as nuances that can drastically improve or handicap the process.
This will be the focus of my upcoming BSidesSF presentation “Fuzz Smarter Not Harder.”
Here are just a few of the topics for discussion:
Selecting a good fuzz target
Identifying ideal test cases
Using persistent mode to increase execution rate
Finding cross-platform bugs with AFL chaining
Dealing with checksums and other blockers
Crash triage with ASAN, GDB, and Peruvian Were Rabbit
