github.com/google/syzkaller@v0.0.0-20240517125934-c0f1611a36d6/docs/reproducing_crashes.md (about)

     1  # How to reproduce crashes
     2  
     3  The process of creating reproducer programs for syzkaller bugs is automated,
     4  however it's not perfect, so syzkaller provides a few tools for executing and
     5  reproducing programs manually.
     6  
     7  Crash logs created in manager `workdir/crashes` dir contain programs executed
     8  just before a crash. In parallel execution mode (when `procs` parameter in
     9  manager config is set to value larger than 1), program that caused the crash
    10  does not necessary immediately precedes it; the guilty program can be somewhere
    11  before.  There are two tools that can help you identify and minimize the program
    12  that causes a crash: `tools/syz-execprog` and `tools/syz-prog2c`.
    13  
    14  `tools/syz-execprog` executes a single syzkaller program or a set of programs in
    15  various modes (once or loop indefinitely; in threaded/collide mode (see below),
    16  with or without coverage collection). You can start by running all programs in
    17  the crash log in a loop to check that at least one of them indeed crashes
    18  kernel: `./syz-execprog -executor=./syz-executor -repeat=0 -procs=16 -cover=0
    19  crash-log`. Then try to identify the single program that causes the crash, you
    20  can test programs with `./syz-execprog -executor=./syz-executor -repeat=0
    21  -procs=16 -cover=0 file-with-a-single-program`.
    22  
    23  Note: `syz-execprog` executes programs locally. So you need to copy
    24  `syz-execprog` and `syz-executor` into a VM with the test kernel and run it
    25  there.
    26  
    27  Once you have a single program that causes the crash, try to minimize it by
    28  removing individual syscalls from the program (you can comment out single lines
    29  with `#` at the beginning of line), and by removing unnecessary data
    30  (e.g. replacing `&(0x7f0000001000)="73656c6600"` syscall argument with
    31  `&(0x7f0000001000)=nil`). You can also try to coalesce all mmap calls into a
    32  single mmap call that maps whole required area. Again, test minimization with
    33  `syz-execprog` tool.
    34  
    35  Now that you have a minimized program, check if the crash still reproduces with
    36  `./syz-execprog -threaded=0 -collide=0` flags. If not, then you will need to do
    37  some additional work later.
    38  
    39  Now, run `syz-prog2c` tool on the program. It will give you executable C
    40  source. If the crash reproduces with `-threaded/collide=0` flags, then this C
    41  program should cause the crash as well.
    42  
    43  If the crash is not reproducible with `-threaded/collide=0` flags, then you need
    44  this last step. You can think of threaded mode as if each syscall is
    45  executed in its own thread. To model such execution mode, move individual
    46  syscalls into separate threads. You can see an example here:
    47  https://groups.google.com/d/msg/syzkaller/fHZ42YrQM-Y/Z4Xf-BbUDgAJ.
    48  
    49  This process is automated to some degree in the `syz-repro` utility. You need to
    50  give it your manager config and a crash report file. And you can refer to the
    51  [example config file](/pkg/mgrconfig/testdata/qemu.cfg).
    52  ```
    53  ./syz-repro -config my.cfg crash-qemu-1-1455745459265726910
    54  ```
    55  It will try to find the offending program and minimize it. But since there are
    56  lots of factors that can affect reproducibility, it does not always work.