152 lines
5.9 KiB
Plaintext
152 lines
5.9 KiB
Plaintext
DexFuzz
|
|
=======
|
|
|
|
DexFuzz is primarily a tool for fuzzing DEX files. Fuzzing is the introduction of
|
|
subtle changes ("mutations") to a file to produce a new test case. These test cases
|
|
can be used to test the various modes of execution available to ART (Interpreter,
|
|
Optimizing compiler) to check for bugs in these modes of execution.
|
|
This is done by differential testing - each test file is executed with each mode of
|
|
execution, and any differences between the resulting outputs may be an indication of
|
|
a bug in one of the modes.
|
|
|
|
For a wider overview of DexFuzz, see:
|
|
|
|
http://community.arm.com/groups/android-community/blog/2014/11/26/the-art-of-fuzz-testing
|
|
|
|
In typical operation, you provide DexFuzz with a set of DEX files that are the "seeds"
|
|
for mutation - e.g. some tests taken from the ART test suite - and point it at an
|
|
ADB-connected Android device, and it will fuzz these seed files, and execute the
|
|
resulting new tests on the Android device.
|
|
|
|
How to run DexFuzz
|
|
==================
|
|
|
|
DexFuzz can run its test programs on either an ADB-connected device, or a host-build of
|
|
ART locally.
|
|
|
|
Execution on an ADB-connected device
|
|
------------------------------------
|
|
|
|
1. Build dexfuzz with mmma tools/dexfuzz from within art/.
|
|
2. Make sure you have an Android device connected via ADB, that is capable of
|
|
having DEX files pushed to it and executed with the dalvikvm command.
|
|
3. Make sure you're in the Android build environment!
|
|
(That is, . build/envsetup.sh && lunch)
|
|
4. Create a new directory, and place some DEX files in here. These are the seed files
|
|
that are mutated to form new tests.
|
|
5. Create a directory on your device that mutated test files can be pushed to and
|
|
executed in, using dalvikvm. For example, /data/art-test/
|
|
6. If you currently have multiple devices connected via ADB, find out the name of
|
|
your device using "adb devices -l".
|
|
7. Run this command:
|
|
|
|
dexfuzz --inputs=<seeds dir> --execute --repeat=<attempts> \
|
|
--dump-output <combination of ISA(s) and and backend(s)>
|
|
|
|
You MUST specify one of the following ISAs:
|
|
--arm
|
|
--arm64
|
|
--x86
|
|
--x86_64
|
|
|
|
And also at least two of the following backends:
|
|
--interpreter
|
|
--optimizing
|
|
|
|
Note that if you wanted to test both ARM and ARM64 on an ARM64 device, you can use
|
|
--allarm. Also in this case only one backend is needed, if i.e., you wanted to test
|
|
ARM Optimizing Backend vs. ARM64 Optimizing Backend.
|
|
|
|
Some legal examples:
|
|
--arm --optimizing --interpreter
|
|
--x86 --optimizing --interpreter
|
|
--allarm --optimizing
|
|
|
|
Add in --device=<device name, e.g. device:generic> if you want to specify a device.
|
|
Add in --execute-dir=<dir on device> if you want to specify an execution directory.
|
|
(The default is /data/art-test/)
|
|
|
|
Host Execution
|
|
--------------
|
|
|
|
DexFuzz now supports execution on your host machine.
|
|
Follow steps 1, 3, 4, and 7 as above, but also observe the following:
|
|
- instead of specifying an ISA, use --host
|
|
- ANDROID_DATA must be set, pointing to a location where dex2oat will place
|
|
OAT files after compilation.
|
|
- Files will always be executed in the same directory where you are executing DexFuzz.
|
|
|
|
Fuzzer Operation
|
|
----------------
|
|
|
|
As the fuzzer works, you'll see output like:
|
|
|
|
|-----------------------------------------------------------------|
|
|
|Iterations|VerifyFail|MutateFail|Timed Out |Successful|Divergence|
|
|
|-----------------------------------------------------------------|
|
|
| 48 | 37 | 4 | 0 | 6 | 1 |
|
|
|
|
Iterations - number of attempts we've made to mutate DEX files.
|
|
VerifyFail - the number of mutated files that ended up failing to verify, either
|
|
on the host, or the target.
|
|
MutateFail - because mutation is a random process, and has attempt thresholds to
|
|
avoid attempting to mutate a file indefinitely, it is possible that
|
|
an attempt to mutate a file doesn't actually mutate it. This counts
|
|
those occurrences.
|
|
Timed Out - mutated files that timed out for one or more backends.
|
|
Current timeouts are:
|
|
Optimizing - 5 seconds
|
|
Interpreter - 30 seconds
|
|
(use --short-timeouts to set all backends to 2 seconds.)
|
|
Successful - mutated files that executed and all backends agreed on the resulting
|
|
output. NB: if all backends crashed with the same output, this would
|
|
be considered a success - proper detection of crashes is still to come.
|
|
Divergence - mutated files that executed and some backend disagreed about the
|
|
resulting output. Divergent programs are run multiple times with a
|
|
single backend, to check if they diverge from themselves, and these are
|
|
not included in the count. If multiple architectures are being used
|
|
(ARM/ARM64), and the divergences align with different architectures,
|
|
these are also not included in the count.
|
|
|
|
8. Check report.log for the full report, including input file and RNG seed for each
|
|
test program. This allows you to recreate a bad program with, e.g.:
|
|
|
|
dexfuzz --input=<input file> --seed=<seed value>
|
|
|
|
Check dexfuzz --help for the full list of options.
|
|
|
|
NOTE: DEX files with unicode strings are not fully supported yet, and DEX files with
|
|
JNI elements are not supported at all currently.
|
|
|
|
Mutation Likelihoods
|
|
====================
|
|
|
|
Each bytecode mutation has a chance out of 100% of firing. Following is the listing
|
|
of each mutation's probability. If you wish to easily adjust these values, copy
|
|
these values into a file called likelihoods.txt, and run dexfuzz with
|
|
--likelihoods=likelihoods.txt.
|
|
|
|
ArithOpChanger 75
|
|
BranchShifter 30
|
|
CmpBiasChanger 30
|
|
ConstantValueChanger 70
|
|
ConversionRepeater 50
|
|
FieldFlagChanger 40
|
|
InstructionDeleter 40
|
|
InstructionDuplicator 80
|
|
InstructionSwapper 80
|
|
InvokeChanger 30
|
|
NewArrayLengthChanger 50
|
|
NewInstanceChanger 10
|
|
NewMethodCaller 10
|
|
NonsenseStringPrinter 10
|
|
OppositeBranchChanger 40
|
|
PoolIndexChanger 30
|
|
RandomBranchChanger 30
|
|
RandomInstructionGenerator 30
|
|
RegisterClobber 10
|
|
SwitchBranchShifter 30
|
|
TryBlockShifter 40
|
|
ValuePrinter 40
|
|
VRegChanger 60
|