github.com/google/syzkaller@v0.0.0-20240517125934-c0f1611a36d6/docs/coverage.md (about)

     1  # Coverage
     2  
     3  `syzkaller` uses [sanitizer coverage (tracing mode)](https://clang.llvm.org/docs/SanitizerCoverage.html#tracing-pcs)
     4  and [KCOV](https://www.kernel.org/doc/html/latest/dev-tools/kcov.html) for coverage collection.
     5  Sanitizer coverage is also supported by `gcc` and `KCOV` is supported by some other OSes.
     6  Note: `gVisor` coverage is completely different.
     7  
     8  Coverage is based on tracing `coverage points` inserted into the object code by the compiler.
     9  A coverage point generally refers to a [basic block](https://en.wikipedia.org/wiki/Basic_block) of code
    10  or a [CFG edge](https://en.wikipedia.org/wiki/Control-flow_graph)
    11  (this depends on the compiler and instrumentation mode used during build,
    12  e.g. for `Linux` and `clang` the default mode is CFG edges, while for `gcc` the default mode is basic blocks).
    13  Note that coverage points are inserted by the compiler in the middle-end after a significant number
    14  of transformation and optimization passes. As the result coverage may poorly relate to the source code.
    15  For example, you may see a covered line after a non-covered line, or you may not see a coverage point
    16  where you would expect to see it, or vice versa (this may happen if the compiler splits basic blocks,
    17  or turns control flow constructs into conditional moves without control flow, etc).
    18  Assessing coverage is still generally very useful and allows to understand overall fuzzing progress,
    19  but treat it with a grain of salt.
    20  
    21  See [this](linux/coverage.md) for Linux kernel specific coverage information.
    22  
    23  ## Web Interface
    24  
    25  When clicking on `cover` link you get view showing each directory located in your kernel build directory. It's showing either percentage number `X% of N` or `---`. `X% of N` means that `X%` of `N` coverage points are covered so far, . `---` indicates there is no coverage in that directory.
    26  
    27  Directory can be clicked and you get view on files and possible subdirectories. On each source code file there is again either `---` or coverage percentage.
    28  
    29  If you click on any C files you will get source code view. There is certain coloring used in the source code view. Color definitions can be found in [coverTemplate](/pkg/cover/report.go#L504). Coloring is described below.
    30  
    31  If you click on percentage number of any listed source file you will get cover percentage for each function in that source file.
    32  
    33  ### Covered: black (#000000)
    34  
    35  All PC values associated to that line are covered. There is number on the left side indicating how many programs have triggered executing the PC values associated to this line. You can click on that number and it will open last executed program. Example below shows how single line which is fully covered is shown.
    36  
    37  ![Code line is fully covered](coverage_covered.png?raw=true)
    38  
    39  ### Both: orange (#c86400)
    40  
    41  There are several PC values associated to the line and not all of these are executed. Again there is number left to the source code line that can clicked to open last program triggering associated PC values. Example below shows single line which has both executed and non-executed PC values associated to it.
    42  
    43  ![Code line has executed and not executed PC values](coverage_both.png?raw=true)
    44  
    45  ###  Weak-uncovered: crimson red (#c80000)
    46  
    47  Function (symbol) this line is in doesn't have any coverage. I.e. the function is not executed at all. Please note that if compiler have optimized certain symbol out and made the code inline instead symbol associated with this line is the one where the code is compiled into. This makes it sometimes real hard to figure out meaning of coloring. Example below shows how single line which is uncovered and PC values associated to it are in function(s) that are not executed either is shown.
    48  
    49  ![PC values associated to the line are not executed and these PC values are in functions that are not executed either](coverage_weak-uncovered.png?raw=true)
    50  
    51  ### Uncovered: red (#ff0000)
    52  
    53  Line is uncovered. Function (symbol) this line is in is executed and one of the PC values associated to this line. Example below shows how single line which is not covered is shown.
    54  
    55  ![Code line has no executed PC values associated. Function it is in is executed](coverage_uncovered.png?raw=true)
    56  
    57  ### Not instrumented: grey (#505050)
    58  
    59  PC values associated to the line are not instrumented or source line doesn't generate code at all. Example below shows how all not instrumented code is shown.
    60  
    61  ![Not instrumented code lines](coverage_not_instrumented.png?raw=true)
    62  
    63  ## syz-cover
    64  
    65  There is small utility in syzkaller repository to generate coverage report based on raw coverage data. This is available in [syz-cover](/tools/syz-cover) and can be built by:
    66  
    67  ``` bash
    68  GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w" -o ./bin/syz-cover github.com/google/syzkaller/tools/syz-cover
    69  ```
    70  
    71  Raw coverage data can be obtained from running `syz-manager` by:
    72  
    73  ``` bash
    74  wget http://localhost:<your syz-manager port>/rawcover
    75  ```
    76  
    77  Now this raw cover data can be fed to `syz-cover` to generate coverage report:
    78  
    79  ``` bash
    80  ./bin/syz-cover --config <location of your syzkaller config> rawcover
    81  ```
    82  
    83  You can also export CSV file containing function coverage by:
    84  
    85  ``` bash
    86  ./bin/syz-cover --config <location of your syzkaller config> --csv <filename where to export>  rawcover
    87  ```
    88  
    89  You can export a JSON file containing line coverage info by:
    90  
    91  ```bash
    92  ./bin/syz-cover --config <location of your syzkaller config> --json <filename where to export>  rawcover
    93  ```