github.com/google/syzkaller@v0.0.0-20251211124644-a066d2bc4b02/docs/syz_verifier.md (about)

     1  **`syz-verifier` is currently broken and cannot be compiled/used, see
     2  [the backlog issue](https://github.com/google/syzkaller/issues/5976).**
     3  
     4  # syz-verifier
     5  
     6  Many bugs are easy to detect: they might cause assertions failures, crash our
     7  system, or cause other forms of undefined behaviour detectable by various
     8  dynamic analysis tools. However, certain classes of bugs, referred to as
     9  *semantic bugs*, cause none of these while still resulting in a misbehaving
    10  faulty system.
    11  
    12  To find semantic bugs, one needs to establish a specification of the system's
    13  *intended behaviour*. Depending on the complexity of the system, creating and
    14  centralising such specifications can be difficult. For example, the
    15  "specification" of the Linux kernel is not found in one place, but is rather a
    16  collection of documentation, man pages, and the implied expectations of a vast
    17  collection of user space programs. As such, detecting semantic bugs in the
    18  Linux kernel is significantly harder than other classes of bugs. Indeed, many
    19  test suites are meant to detect regressions, but creating and maintaining test
    20  cases, as well as covering new features requires significant amounts of
    21  engineering effort.
    22  
    23  *Differential fuzzing* is a way to automate detection of semantic bugs by
    24  providing the same input to different implementations of the same systems and
    25  then cross-comparing the resulting behaviour to determine whether it is
    26  identical. In case the systems disagree, at least one of them is assumed to be
    27  wrong.
    28  
    29  `syz-verifier` is a differential fuzzing tool that cross-compares the execution
    30  of programs on different versions of the Linux kernel to detect semantic bugs.
    31  
    32  The architecture of `syz-verifier` is shown in the following diagram.
    33  
    34  ![Architecture overview](syz_verifier_structure.png)
    35  
    36  The `syz-verifier` process starts and manages VM instances with the kernels to
    37  be cross-compared. It also starts the `syz-runner` process on the VMs.
    38  Communication between the host and the guest is done via RPCs.
    39  
    40  `syz-verifier` generates and sends a continuous stream of programs to
    41  `syz-runner` via RPCs while `syz-runner` is responsible for starting
    42  `syz-executor` processes and turning the program into input for those.
    43  `syz-executor` processes the input, which triggers a sequence of syscalls in
    44  the kernel. Then, `syz-runner` collects the results and sends them back to the
    45  host.
    46  
    47  At the moment, the results contain the errnos returned by each system call.
    48  When `syz-verifier` has received results from all the kernels for a specific
    49  program, it verifies them to ensure they are identical. If a mismatch is found,
    50  the program is rerun on all the kernels to ensure the mismatch is not flaky
    51  (i.e. it didn't occur because of some background activity or external state).
    52  If the mismatch occurs in all reruns, `syz-verifier` creates a report for the
    53  program and write it to persistent storage.
    54  
    55  # How to use `syz-verifier`
    56  
    57  After cloning the repository (see how
    58  [here](/docs/linux/setup.md#go-and-syzkaller)), build the tool as:
    59  
    60  ```
    61  make verifier runner executor
    62  ```
    63  
    64  To start using the tool, separate configuration files need to be created for
    65  each kernel you want to include in the verification. An example of Linux
    66  configs can be found [here](/docs/linux/setup_ubuntu-host_qemu-vm_x86-64-kernel.md#syzkaller). The configuration files
    67  are identical to those used by `syz-manager`.
    68  
    69  If you want to generate programs from a specific set of system calls, these can
    70  be listed in the kernel config files using the `enable_syscalls` option. If you
    71  want to disable some system calls, use the `disable_syscalls` option.
    72  
    73  Start `syz-verifier` as:
    74  ```
    75  ./bin/syz-verifier -configs=kernel0.cfg,kernel1.cfg
    76  ```
    77  
    78  `syz-verifier` will also gather statistics throughout execution. They will be
    79  printed to `stdout` by default, but an alternative file can be specified using
    80  the `stat` flag.
    81  
    82  # How to interpret the results
    83  
    84  Results can be found in `workdir/results`.
    85  
    86  When `syz-verifier` finds a mismatch in a program, it will create a report for
    87  that program. The report lists the results returned for each system call, by
    88  each of the cross-compared kernels, highlighting the ones were a mismatch was
    89  found. The system calls are listed in the order they appear in the program.
    90  
    91  An extract of such a report is shown below:
    92  
    93  ```
    94  ERRNO mismatches found for program:
    95  
    96  [=] io_uring_register$IORING_REGISTER_PERSONALITY(0xffffffffffffffff, 0x9, 0x0, 0x0)
    97          ↳ Pool: 0, Flags: 3, Errno: 9 (bad file descriptor)
    98          ↳ Pool: 1, Flags: 3, Errno: 9 (bad file descriptor)
    99  
   100  [=] syz_genetlink_get_family_id$devlink(&(0x7f0000000000), 0xffffffffffffffff)
   101          ↳ Pool: 0, Flags: 3, Errno: 2 (no such file or directory)
   102          ↳ Pool: 1, Flags: 3, Errno: 2 (no such file or directory)
   103  
   104  [!] r1 = io_uring_setup(0x238e, &(0x7f0000000240)={0x0, 0xf39a, 0x20, 0x0, 0x146})
   105          ↳ Pool: 0, Flags: 3, Errno: 6 (no such device or address)
   106          ↳ Pool: 1, Flags: 3, Errno: 9 (bad file descriptor)
   107  ...
   108  ```
   109  
   110  The order of the results is given by the order in which configuration files
   111  were passed so `Pool: 0 ` reports results for the kernel created using
   112  `kernel0.cfg` and so on.