github.com/nya3jp/tast@v0.0.0-20230601000426-85c8e4d83a9b/docs/modifying_tast.md (about)

     1  # Modifying Tast
     2  
     3  This document describes how to make changes to the Tast framework itself. For
     4  information about writing Tast tests, see [Writing Tests].
     5  
     6  [Writing Tests]: writing_tests.md
     7  
     8  [TOC]
     9  
    10  ## Components
    11  
    12  See the [Overview] document for a high-level description of the key components
    13  and terminology in Tast.
    14  
    15  [Overview]: overview.md
    16  
    17  ### Executables
    18  
    19  *   The [tast executable] is run by developers and by builders. It's used to
    20      compile, deploy, and run tests and to collect results.
    21  *   [local_test_runner] is run on the DUT by the `tast` process over an SSH
    22      connection. It collects system information and initiates running local
    23      tests.
    24  *   [remote_test_runner] is run on the host system by the `tast` process to
    25      initiate running remote tests.
    26  
    27  [tast executable]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/cmd/tast/
    28  [local_test_runner]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/cmd/local_test_runner/
    29  [remote_test_runner]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/cmd/remote_test_runner/
    30  
    31  ### Libraries
    32  
    33  Shared library packages are located under the [tast directory]. Several packages
    34  are particularly important:
    35  
    36  *   [bundle] contains code used to implement test bundles, which contain
    37      compiled tests and are executed by test runners.
    38  *   [protocol] defines protocol buffer messages that are used for communication
    39      between the `tast` process, test runners, and test bundles.
    40  *   [runner] contains code shared between `local_test_runner` and
    41      `remote_test_runner`.
    42  *   [ssh] opens SSH connections.
    43  *   [testing] contains code used to define and run tests.
    44  
    45  [tast directory]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/
    46  [bundle]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/bundle/
    47  [protocol]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/internal/protocol/
    48  [runner]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/internal/runner/
    49  [ssh]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/ssh/
    50  [testing]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/testing/
    51  
    52  ## Making changes
    53  
    54  ### IPC
    55  
    56  JSON-marshaled structs are used for communication between processes:
    57  
    58  *   The [tast executable]'s [run] package marshals and passes [runner.Args] to
    59      test runners.
    60  *   [local_test_runner] and [remote_test_runner] use the [runner] package to
    61      unmarshal [runner.Args] and marshal and pass [bundle.Args] structs to test
    62      bundles.
    63  *   Test bundles use the [bundle] package to unmarshal [bundle.Args].
    64  
    65  The [runner.Args] and [bundle.Args] structs contain other `Args`-suffixed
    66  structs that may or may not be set depending on the type of request that is
    67  being made. Note also that [runner.RunTestsArgs] includes [bundle.RunTestsArgs]
    68  and that [runner.ListTestsArgs] includes [bundle.ListTestsArgs].
    69  
    70  When running in the ChromeOS hardware lab or on VM builders, matching versions
    71  of `tast`, test runners, and test bundles are used, so there are no
    72  compatibility concerns.
    73  
    74  More combinations are possible when running in a developer's chroot:
    75  
    76  *   The `tast` binary in the chroot can be updated by emerging `tast-cmd` or
    77      running `fast_build.sh`.
    78  *   The `remote_test_runner` binary in the chroot is typically updated by
    79      emerging `tast-cmd`.
    80  *   The `local_test_runner` binary on the DUT can either be installed by the
    81      `tast-local-test-runner` package or updated by running `tast run
    82      -build=true`.
    83  *   The default test bundle on the DUT can be installed by the
    84      `tast-local-tests-cros` package or updated by running `tast run
    85      -build=true`.
    86  
    87  As a result, developers may end up running the `tast` executable in their chroot
    88  against DUTs that contain either older or newer versions of `local_test_runner`
    89  and test bundles. To maintain compatibility, please ensure that the following
    90  conditions hold across both the `tast`-to-runner and runner-to-bundle
    91  boundaries:
    92  
    93  *   Older receivers correctly handle structs marshaled by newer senders.
    94  *   Newer receivers correctly handle structs marshaled by older senders.
    95  
    96  Go's [json package] does best-effort unmarshaling, ignoring unknown fields by
    97  default. When renaming or moving a field, it's advisable to preserve the old
    98  field for at least two months. Add a `Deprecated` suffix to the old field's name
    99  while preserving its original marshaled name in its `json` tag.
   100  
   101  `tast` and the [runner] package should set both the old and new fields, and the
   102  [runner] and [bundle] packages should copy the old fields to the new fields when
   103  the former are provided. See [change 1474620] for an example.
   104  
   105  [run]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/src/go.chromium.org/tast/core/cmd/tast/internal/run/
   106  [runner.Args]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/internal/runner#Args
   107  [bundle.Args]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/bundle#Args
   108  [runner.RunTestsArgs]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/internal/runner#RunTestsArgs
   109  [bundle.RunTestsArgs]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/bundle#RunTestsArgs
   110  [runner.ListTestsArgs]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/internal/runner#ListTestsArgs
   111  [bundle.ListTestsArgs]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/bundle#ListTestsArgs
   112  [json package]: https://golang.org/pkg/encoding/json/
   113  [change 1474620]: https://crrev.com/c/1474620
   114  
   115  ## Compiling changes
   116  
   117  ### fast_build.sh
   118  
   119  The quickest way to rebuild the `tast` executable after modifying its code is by
   120  running the [fast_build.sh] script located at the top of the `src/platform/tast`
   121  repository within the ChromeOS chroot. This script bypasses Portage and runs
   122  `go build` directly, allowing it to take advantage of [Go's build cache]. Since
   123  dependency checks are skipped, there's no guarantee that the resulting
   124  executable is correct – before uploading a change, you should verify it that it
   125  builds when you run `FEATURES=test sudo emerge tast-cmd` (after running
   126  `cros_workon --host start tast-cmd`).
   127  
   128  *   Without any arguments, `fast_build.sh` compiles the `tast` executable to
   129      `$HOME/go/bin/tast`.
   130  *   `fast_build.sh -t go.chromium.org/tast/core/testing` runs the unit tests for the
   131      `go.chromium.org/tast/core/testing` package.
   132  *   `fast_build.sh -T` runs all unit tests (including ones in the `tast-tests`
   133      repository).
   134  *   `fast_build.sh -c go.chromium.org/tast/core/testing` runs `go vet` against the
   135      `go.chromium.org/tast/core/testing` package.
   136  *   `fast_build.sh -C` vets all packages.
   137  
   138  Run `fast_build.sh -h` to see all available options.
   139  
   140  [fast_build.sh]: https://chromium.googlesource.com/chromiumos/platform/tast/+/main/fast_build.sh
   141  [Go's build cache]: https://golang.org/cmd/go/#hdr-Build_and_test_caching
   142  
   143  ## Testing changes
   144  
   145  ### Unit tests
   146  
   147  The different components of the framework are extensively covered by unit tests.
   148  Please ensure that any changes that you make are also covered by tests.
   149  
   150  The [testutil package] provides utility functions intended to reduce repetitive
   151  code within unit tests.
   152  
   153  [testutil package]: https://godoc.org/chromium.googlesource.com/chromiumos/platform/tast.git/src/go.chromium.org/tast/core/testutil
   154  
   155  ### Meta tests
   156  
   157  There are several [meta tests]. These are remote Tast tests that run a nested
   158  instance of the `tast` executable to perform end-to-end verification of
   159  interactions between `tast`, test runners, and test bundles. They're executed
   160  the same way as other Tast tests, i.e. via `tast run`.
   161  
   162  [meta tests]: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/HEAD/src/go.chromium.org/tast-tests/cros/remote/bundles/cros/meta/