Skip to content

Commit ed2c96f

Browse files
DavidKorczynskiDavidKorczynski
authored
Doc: update readme structure (#413)
* doc: refine and simplify Removes a lot of the data on the front README to increase attention on OSS-fuzz list of projects and modularises the documentation.
1 parent 46fa3bb commit ed2c96f

File tree

4 files changed

+222
-236
lines changed

4 files changed

+222
-236
lines changed

‎README.md

+10-236
Original file line numberDiff line numberDiff line change
@@ -5,252 +5,26 @@ and identify any potential blockers. Fuzz introspector aggregates the fuzzers’
55
hit frequency, entry points, etc to give the developer a birds eye view of their fuzzer. This helps with
66
identifying fuzz bottlenecks and blockers and eventually helps in developing better fuzzers.
77

8-
Fuzz-introspector can on a high-level guide on how to improve fuzzing of a project by guiding on whether you should:
8+
Fuzz-introspector aims to improve fuzzing experience of a project by guiding on whether you should:
99
- introduce new fuzzers to a fuzz harness
1010
- modify existing fuzzers to improve the quality of your harness.
1111

12-
By and large these capabilities will remain the goals of fuzz-introspector. The focus is on improving these.
13-
14-
**Video demonstration:** A video demonstration of fuzz-introspector is given [here](https://www.youtube.com/watch?v=cheo-liJhuE)
15-
16-
**Sample OSS-Fuzz reports:** OSS-Fuzz supports fuzz introspector and maintains a list of fuzz introspector reports for the OSS-Fuzz projects. This list is available [here](https://oss-fuzz-introspector.storage.googleapis.com/index.html) and is a good reference if you're interested in seeing fuzz introspector in action.
17-
18-
**Try yourself:**
19-
- [Use with OSS-Fuzz](#testing-with-oss-fuzz)
20-
- [Use without OSS-Fuzz](#testing-without-oss-fuzz-integration)
12+
## Documentation and samples
13+
- [Sample OSS-Fuzz reports](https://oss-fuzz-introspector.storage.googleapis.com/index.html). [OSS-Fuzz](https://github.com/google/oss-fuzz) supports Fuzz Introspector and maintains a list of reports.
14+
- [Video demonstration](https://www.youtube.com/watch?v=cheo-liJhuE)
15+
- [List of Case studies](doc/CaseStudies.md)
16+
- [Screenshots](doc/ExampleOutput.md)
17+
- [Feature list](doc/Features.md)
18+
- Try yourself:
19+
- [Use with OSS-Fuzz](oss_fuzz_integration#build-fuzz-introspector-with-oss-fuzz) (Recommended)
20+
- [Use without OSS-Fuzz](doc/LocalBuild.md)
2121

2222
## Architecture
2323
The workflow of fuzz-introspector can be visualised as follows:
2424
![Functions table](/doc/img/fuzz-introspector-architecture.png)
2525

2626
A more detailed description is available in [doc/Architecture](/doc/Architecture.md)
2727

28-
## Features
29-
**High-level features**
30-
31-
- Show fuzzing-relevant data about each function in a given project
32-
- Show reachability of fuzzer(s)
33-
- Integrate seamlessly with OSS-Fuzz
34-
- Show visualisations to enable fuzzer debugging
35-
- Give suggestions for how to improve fuzzing
36-
37-
**Concrete features**
38-
39-
For each function in a project and a fuzzing harness for the project:
40-
- show the number of fuzzers (fuzz drivers) that reach this function
41-
- show cyclomatic complexity of the function
42-
- show the amount of functions reached by the function
43-
- show the sum of cyclomatic complexity of all functions reachable by the function
44-
- show the number of (LLVM IR) basic blocks in the function
45-
- show the function call-depth of the function
46-
- show the total unreached complexity of this function, including the complexity from all unreached functions reached by this function.
47-
48-
Given a fuzz harness for a project show:
49-
- which functions in the project are not reachable by the harness
50-
- which functions in the project are reachable by harness
51-
- identify which functions are the best to target (based on which unreached function reaches most code)
52-
53-
Given a fuzz harness statically analyse the code and merge it with run-time coverage information to:
54-
- visualise statically-extracted calltree of each fuzzer and overlay this calltree with run-time coverage information
55-
- identify nodes in the statically-extracted calltree where fuzzers are blocked based on run-time coverage information
56-
- automatically highlight fuzz-blockers, namely locations in the code where fuzzers are not able to continue execution at run-time despite the code being reachable by the fuzzer based on static analysis.
57-
58-
## Testing with OSS-Fuzz
59-
The recommended way of testing this project is by way of OSS-Fuzz. Please see
60-
[OSS-Fuzz instructions](oss_fuzz_integration/) on how to do this. It is the
61-
recommended way because it's by way of OSS-Fuzz that we currently support combining
62-
runtime coverage data with the compiler plugin.
63-
64-
65-
## Testing without OSS-Fuzz integration
66-
You can build and run fuzz introspector outside the OSS-Fuzz environment.
67-
We use this mainly to develop the LLVM LTO pass as compilation of clang goes
68-
faster (recompilation in particular). However, for the full experience we
69-
recommend working in the OSS-Fuzz environment as described above.
70-
71-
A complication with testing locally is that the full end-to-end process of
72-
both (1) building fuzzers; (2) running them; (3) building with coverage; and
73-
(4) building with introspector analysis, is better supported
74-
in the OSS-Fuzz environment.
75-
76-
77-
### Build locally
78-
79-
#### TLDR:
80-
```bash
81-
git clone https://github.com/ossf/fuzz-introspector
82-
cd fuzz-introspector
83-
84-
# Get python dependencies
85-
python3 -m venv .venv
86-
. .venv/bin/activate
87-
pip install -r requirements.txt
88-
89-
# Build custom clang with Fuzz introspector LLVM pass
90-
./build_all.sh
91-
92-
cd tests
93-
./build_simple_example.sh
94-
cd simple-example-0/web
95-
python3 -m http.server 8008
96-
```
97-
98-
#### Use Docker
99-
100-
Will use sources cloned to /your/path/to/source
101-
102-
```
103-
docker build -t "fuzz-introspector:Dockerfile" .
104-
docker run --rm -it -v /your/path/to/source:/src fuzz-introspector:Dockerfile
105-
106-
```
107-
108-
#### Full process
109-
110-
111-
##### step 1: Start a python venv
112-
```bash
113-
git clone https://github.com/ossf/fuzz-introspector
114-
115-
# create virtual environment
116-
python3 -m venv .venv
117-
. .venv/bin/activate
118-
pip install -r requirements.txt
119-
```
120-
121-
##### step 2: Build custom clang
122-
Fuzz-introspector relies on an LTO LLVM pass and this requires us to build a custom Clang where the LTO pass is part of the compiler tool chain (see https://github.com/ossf/fuzz-introspector/issues/57 for more details on why this is needed).
123-
124-
To build the custom clang from the root of this repository:
125-
126-
```bash
127-
mkdir build
128-
cd build
129-
130-
# Build binutils
131-
apt install texinfo
132-
git clone --depth 1 git://sourceware.org/git/binutils-gdb.git binutils
133-
mkdir build
134-
cd ./build
135-
../binutils/configure --enable-gold --enable-plugins --disable-werror
136-
make all-gold
137-
cd ../
138-
139-
# Build LLVM and Clang
140-
git clone https://github.com/llvm/llvm-project/
141-
cd llvm-project/
142-
143-
# Patch Clang to run fuzz introspector
144-
../../sed_cmds.sh
145-
cp -rf ../../llvm/include/llvm/Transforms/FuzzIntrospector/ ./llvm/include/llvm/Transforms/FuzzIntrospector
146-
cp -rf ../../llvm/lib/Transforms/FuzzIntrospector ./llvm/lib/Transforms/FuzzIntrospector
147-
cd ../
148-
149-
# Build LLVM and clang
150-
mkdir llvm-build
151-
cd llvm-build
152-
cmake -G "Unix Makefiles" -DLLVM_ENABLE_PROJECTS="clang;compiler-rt" \
153-
-DLLVM_BINUTILS_INCDIR=../binutils/include \
154-
-DLLVM_TARGETS_TO_BUILD="X86" ../llvm-project/llvm/
155-
make llvm-headers
156-
make -j5
157-
```
158-
159-
##### step 3: Run local example
160-
161-
Now we have two options, to run the fuzz introspector tools without collecting
162-
runtime coverage and doing it with collecting coverage. We go through each of the two options:
163-
164-
##### step 3, option 1, only static analysis
165-
After having built the custom clang above, you build a test case:
166-
```
167-
# From the root of the fuzz-introspector repository
168-
cd tests/simple-example-0
169-
170-
# Run compiler pass to generate *.data and *.data.yaml files
171-
mkdir work
172-
cd work
173-
FUZZ_INTROSPECTOR=1 ../../../build/llvm-build/bin/clang -fsanitize=fuzzer -flto -g ../fuzzer.c -o fuzzer
174-
175-
# Run post-processing to analyse data files and generate HTML report
176-
python3 ../../../post-processing/main.py correlate --binaries_dir=.
177-
python3 ../../../post-processing/main.py report --target_dir=. --correlation_file=./exe_to_fuzz_introspector_logs.yaml
178-
179-
# The post-processing will have generated various .html, .js, .css and .png fies,
180-
# and these are accessible in the current folder. Simply start a webserver and
181-
# navigate to the report in your local browser (localhost:8008):
182-
python3 -m http.server 8008
183-
```
184-
185-
186-
##### step 3, option 2, include runtime coverage analysis
187-
```
188-
# From the root of the fuzz-introspector repository
189-
cd tests/simple-example-0
190-
191-
# Run compiler pass to generate *.data and *.data.yaml files
192-
mkdir work
193-
cd work
194-
195-
# Run script that will build fuzzer with coverage instrumentation and extract .profraw files
196-
# and convert those to .covreport files with "llvm-cov show"
197-
../build_cov.sh
198-
199-
# Build fuzz-introspector normally
200-
FUZZ_INTROSPECTOR=1 ../../../build/llvm-build/bin/clang -fsanitize=fuzzer -flto -g ../fuzzer.c -o fuzzer
201-
202-
# Run post-processing to analyse data files and generate HTML report
203-
python3 ../../../post-processing/main.py correlate --binaries_dir=.
204-
python3 ../../../post-processing/main.py report --target_dir=. --correlation_file=./exe_to_fuzz_introspector_logs.yaml
205-
206-
# The post-processing will have generated various .html, .js, .css and .png fies,
207-
# and these are accessible in the current folder. Simply start a webserver and
208-
# navigate to the report in your local browser (localhost:8008):
209-
python3 -m http.server 8008
210-
```
211-
212-
You can also use the `build_all_projects.sh` and `build_all_web_only.sh` scripts to control
213-
which examples you want to build as well as whether you want to only build the web data.
214-
215-
216-
## Output
217-
218-
The output of the introspector is a HTML report that gives data about your fuzzer. This includes:
219-
220-
- An overview of reachability by all fuzzers in the repository
221-
- A table with detailed information about each fuzzer in the repository, e.g. number of functions reached, complexity covered and more.
222-
- A table with overview of all functions in the project. With information such as
223-
- Number of fuzzers that reaches this function
224-
- Cyclomatic complexity of this function and all functions reachable by this function
225-
- Number of functions reached by this function
226-
- The amount of undiscovered complexity in this function. Undiscovered complexity is the complexity *not* covered by any fuzzers.
227-
- A call reachability tree for each fuzzer in the project. The reachability tree shows the potential control-flow of a given fuzzer
228-
- An overlay of the reachability tree with coverage collected from a fuzzer run.
229-
- A table giving summary information about which targets are optimal targets to analyse for a fuzzer of the functions that are not being reached by any fuzzer.
230-
- A list of suggestions for new fuzzers (this is super naive at the moment).
231-
232-
### Example output
233-
234-
Here we show a few images from the output report:
235-
236-
Project overview:
237-
238-
![project overview](/doc/img/project_overview.png)
239-
240-
241-
Table with data of all functions in a project. The table is sortable to make enhance the process of understanding the fuzzer-infrastructure of a given project:
242-
243-
![Functions table](/doc/img/functions_overview.png)
244-
245-
Reachability tree with coverage overlay
246-
247-
![Overlay 1](/doc/img/overlay-1.png)
248-
249-
250-
Reachability tree with coverage overlay, showing where a fuzz-blocker is occurring
251-
![Overlay 2](/doc/img/overlay-2.png)
252-
253-
25428
## Contribute
25529
### Code of Conduct
25630
Before contributing, please follow our [Code of Conduct](CODE_OF_CONDUCT.md).

‎doc/ExampleOutput.md

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Example output
2+
Here we maintain a list of images and descriptions of the output
3+
from Fuzz Introspector. This is mainly used to give potential users
4+
a quick insight of what you can get from Fuzz Introspector. For
5+
full examples we recommend inspecting OSS-Fuzz reports [here](https://oss-fuzz-introspector.storage.googleapis.com/index.html)
6+
7+
Here we show a few images from the output report:
8+
9+
Project overview:
10+
11+
![project overview](/doc/img/project_overview.png)
12+
13+
14+
Table with data of all functions in a project. The table is sortable to make enhance the process of understanding the fuzzer-infrastructure of a given project:
15+
16+
![Functions table](/doc/img/functions_overview.png)
17+
18+
Reachability tree with coverage overlay
19+
20+
![Overlay 1](/doc/img/overlay-1.png)
21+
22+
23+
Reachability tree with coverage overlay, showing where a fuzz-blocker is occurring
24+
![Overlay 2](/doc/img/overlay-2.png)
25+

‎doc/Features.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Features
2+
**High-level features**
3+
4+
- Show fuzzing-relevant data about each function in a given project
5+
- Show reachability of fuzzer(s)
6+
- Integrate seamlessly with OSS-Fuzz
7+
- Show visualisations to enable fuzzer debugging
8+
- Give suggestions for how to improve fuzzing
9+
10+
**Concrete features**
11+
12+
For each function in a project and a fuzzing harness for the project:
13+
- show the number of fuzzers (fuzz drivers) that reach this function
14+
- show cyclomatic complexity of the function
15+
- show the amount of functions reached by the function
16+
- show the sum of cyclomatic complexity of all functions reachable by the function
17+
- show the number of (LLVM IR) basic blocks in the function
18+
- show the function call-depth of the function
19+
- show the total unreached complexity of this function, including the complexity from all unreached functions reached by this function.
20+
21+
Given a fuzz harness for a project show:
22+
- which functions in the project are not reachable by the harness
23+
- which functions in the project are reachable by harness
24+
- identify which functions are the best to target (based on which unreached function reaches most code)
25+
26+
Given a fuzz harness statically analyse the code and merge it with run-time coverage information to:
27+
- visualise statically-extracted calltree of each fuzzer and overlay this calltree with run-time coverage information
28+
- identify nodes in the statically-extracted calltree where fuzzers are blocked based on run-time coverage information
29+
- automatically highlight fuzz-blockers, namely locations in the code where fuzzers are not able to continue execution at run-time despite the code being reachable by the fuzzer based on static analysis.
30+
31+
# Output
32+
Screenshots from Fuzz Introspector is is available [here](doc/ExampleOutput.md)
33+
34+
The output of the introspector is a HTML report that gives data about your fuzzer. This includes:
35+
36+
- An overview of reachability by all fuzzers in the repository
37+
- A table with detailed information about each fuzzer in the repository, e.g. number of functions reached, complexity covered and more.
38+
- A table with overview of all functions in the project. With information such as
39+
- Number of fuzzers that reaches this function
40+
- Cyclomatic complexity of this function and all functions reachable by this function
41+
- Number of functions reached by this function
42+
- The amount of undiscovered complexity in this function. Undiscovered complexity is the complexity *not* covered by any fuzzers.
43+
- A call reachability tree for each fuzzer in the project. The reachability tree shows the potential control-flow of a given fuzzer
44+
- An overlay of the reachability tree with coverage collected from a fuzzer run.
45+
- A table giving summary information about which targets are optimal targets to analyse for a fuzzer of the functions that are not being reached by any fuzzer.
46+
- A list of suggestions for new fuzzers (this is super naive at the moment).
47+

0 commit comments

Comments
 (0)