github.com/u-root/u-root@v7.0.1-0.20200915234505-ad7babab0a8e+incompatible/integration/README.md (about)

     1  # Integration Tests
     2  
     3  These are VM based tests for core u-root functionality such as:
     4  
     5  * retrieving and kexec'ing a Linux kernel,
     6  * DHCP client tests,
     7  * uinit (user init), and
     8  * running unit tests requiring root privileges.
     9  
    10  All tests are in the integration/ directory. Within that, there are a
    11  few subdirectories:
    12  
    13  * generic-tests/ : most tests can be put under this.
    14  * gotests/ : this is for Go unit tests that can be run inside the VM.
    15  * testcmd/ : this contains custom uinits for tests.
    16  * testdata/ : this contains any extra files for tests.
    17  
    18  To learn more about how these tests work under the hood, see the next section,
    19  otherwise jump ahead to the sections on how to write and run these tests.
    20  
    21  ## VM Testing Infrastructure
    22  
    23  Our VM testing infrastructure starts a QEMU virtual machine that boots with
    24  our given kernel and initramfs, and runs the uinit or commands that we want to
    25  test.
    26  
    27  Testing mainly relies on 2 packages: [pkg/vmtest](/pkg/vmtest) and
    28  [pkg/qemu](/pkg/qemu).
    29  
    30  pkg/vmtest takes in integration test options, and given those and the
    31  environment variables, uses pkg/qemu to start a QEMU VM with the correct command
    32  line and configuration.
    33  
    34  Files that need to be shared with the VM are written to a temp dir which is
    35  exposed as a Plan 9 (9p) filesystem in the VM. This includes the kernel and
    36  initramfs being used for the VM.
    37  
    38  The test architecture, kernel and QEMU binary are set using environment
    39  variables.
    40  
    41  The initramfs can come from the following sources:
    42  * User overridden: when the `UROOT_INITRAMFS` environment variable is used to
    43    override the initramfs. The user is responsible for ensuring the initramfs
    44    contains the correct binaries.
    45  * Custom u-root opts: define u-root opts in the test itself (eg. custom uinit).
    46    The testing setup will generate an initramfs with those options.
    47  * Default: provide the set of commands to be tested. The commands are written to
    48    an elvish script in the shared dir. The testing setup will generate a generic
    49    initramfs that mounts the shared 9p filesystem as '/testdata', and then finds
    50    and runs the elvish script.
    51  
    52  To check for the correct behavior, we use the go expect package to find
    53  expected output in QEMU's serial output within a given timeout.
    54  
    55  ## Running Tests
    56  
    57  These tests only run on Linux on amd64 and arm.
    58  
    59  1. **Set Environment Variables**
    60  
    61  -   `UROOT_QEMU` points to a QEMU binary and args, e.g.
    62  
    63  ```sh
    64  export UROOT_QEMU="$HOME/bin/qemu-system-x86_64 -enable-kvm"
    65  ```
    66  
    67  -   `UROOT_KERNEL` points to a Linux kernel binary, e.g.
    68  
    69  ```sh
    70  export UROOT_KERNEL="$HOME/linux/arch/x86/boot/bzImage"
    71  ```
    72  -   (optional) `UROOT_INITRAMFS` is a custom initramfs to use for all tests.
    73      This will override all other initramfs options defined by the tests.
    74  
    75  -   (optional) `UROOT_TESTARCH` (defaults to host architecture) is the
    76      architecture to test. Only `arm` and `amd64` are supported.
    77  
    78  -   (optional) `UROOT_QEMU_TIMEOUT_X` (defaults to 1.0) can be used to multiply
    79      the timeouts for each test in case QEMU on your machine is slower. For
    80      example, if you cannot turn on `-enable-kvm`, use `UROOT_QEMU_TIMEOUT_X=2`
    81      as our test automation does.
    82  
    83  
    84  Our automated CI uses Dockerfiles to build a kernel and QEMU and set these
    85  environment variables. You can see the Dockerfile and the config file used to
    86  build the kernel for each supported architecture [here](/.circleci/images).
    87  
    88  If you don't want to deal with version differences in QEMU and the kernel, you
    89  can use the docker image get both. Inside /.circleci/images/test-image-amd64 (or
    90  whatever arch you have), run
    91  
    92  ```
    93  cd test-image-$GOARCH
    94  docker build . -t uroottest/test-image-$GOARCH:$VERSION
    95  docker run uroottest/test-image-$GOARCH:$VERSION
    96  docker container list -a
    97  ```
    98  
    99  Then look for the container id for your newly built container, and
   100  
   101  ```
   102  docker cp $CONTAINER_ID:bzImage <target>
   103  docker cp $CONTAINER_ID:qemu-system-x86_64 <target>
   104  docker cp $CONTAINER_ID:pc-bios <target>
   105  ```
   106  
   107  The pc bios needs to be passed into qemu with the -L flag for this built version
   108  of qemu.
   109  
   110  
   111  2. **Run Tests**
   112  
   113  Recall that there are 2 subdirectories with tests, generic-tests/ and gotests/.
   114  To run tests in both directories, run:
   115  
   116  ```sh
   117  go test [-v] ./...
   118  ```
   119  
   120  The verbose flag is useful to see the QEMU command line being used and the full
   121  serial output. It is also useful to see which tests are being skipped and why
   122  (particularly for ARM, where many tests are currently skipped).
   123  
   124  Unless you want to wait a long time for all tests to complete, run just the
   125  specific test you want from inside the correct directory e.g.
   126  
   127  ```sh
   128  cd generic-tests/
   129  go test [-v] -test.run=TestDhclient
   130  ```
   131  
   132  *To avoid having to do this every time, check the instructions for the RUNLOCAL
   133  script.*
   134  
   135  ## Writing a New Test
   136  
   137  To write a new test, first decide which of the options from the previous
   138  section best fit your case (custom initramfs, custom uinit, test commands).
   139  
   140  `vmtest.QEMUTest` is the function that starts the QEMU VM and returns the VM
   141  struct. There, provide the test options for your use case.
   142  
   143  The VM struct returned by `vmtest.QEMUTest` represents a running QEMU virtual
   144  machine. Use its family of Expect methods to check for the correct result.
   145