github.com/nya3jp/tast@v0.0.0-20230601000426-85c8e4d83a9b/docs/overview.md (about)

     1  # Tast Overview (go/tast-overview)
     2  
     3  [TOC]
     4  
     5  ## Terminology
     6  
     7  ### Local tests
     8  
     9  Local tests run directly on devices-under-test (DUTs) and are similar in
    10  function to Autotest's "client" tests. They can do anything that any other code
    11  running locally as root can do. Most tests should be local tests.
    12  
    13  ### Remote tests
    14  
    15  Remote tests run on the machine initiating testing (e.g. a developer's
    16  workstation, a.k.a. the "host system") and run shell commands on the DUT
    17  remotely, allowing the DUT to be rebooted in the course of a single test. They
    18  are similar in function to Autotest's "server" tests. They run more slowly than
    19  local tests, are harder to write, and are more susceptible to infrastructure and
    20  hardware issues.
    21  
    22  ### `tast` command
    23  
    24  The `tast` executable is built for the host system by the
    25  `chromeos-base/tast-cmd` Portage package from code in the [tast repository]. The
    26  developer runs it within their chroot to:
    27  
    28  *   compile test bundles for a given target architecture (the DUT's architecture
    29      for local tests or the initiating machine's architecture for remote tests),
    30  *   push test bundles and data files to DUTs (only in the case of local tests)
    31      and run tests, and
    32  *   collect, process, and display test results.
    33  
    34  The [Running Tests] document contains more information about using the `tast`
    35  command.
    36  
    37  ### Test bundles
    38  
    39  Test bundles consist of executables and their associated data files (internal
    40  data files + external data link files). Test bundle executables are composed of
    41  compiled tests along with code for running them and reporting results back to
    42  a test runner. Bundles exist so that test code can be checked into other
    43  (probably non-public) repositories.
    44  
    45  The default local and remote bundles, consisting of ChromeOS tests, are named
    46  `cros` and are checked into the [tast-tests repository].
    47  
    48  Local tests can be included in a system image via Portage packages with names of
    49  the form `chromeos-base/tast-local-tests-<bundle>`, while remote tests can be
    50  installed onto the host system via packages named
    51  `chromeos-base/tast-remote-tests-<bundle>`. Either type of bundle can also be
    52  compiled (and deployed, in the case of local tests) on-the-fly by `tast`.
    53  
    54  Tests don't (currently) require anything special from the OS — `tast` just needs
    55  SSH access to the DUT and (when deploying local test bundles that aren't built
    56  into the system image) a writable partition that isn't mounted `noexec`.
    57  
    58  See the [Writing Tests] document for more information.
    59  
    60  ### Test runners
    61  
    62  Test runners are executables that enumerate and execute test bundles before
    63  aggregating results for the `tast` command. There are two test runners,
    64  `local_test_runner` (which executes local test bundles on-device) and
    65  `remote_test_runner` (which runs remote test bundles on the host system).
    66  Runners are installed by the `chromeos-base/tast-local-test-runner` and
    67  `chromeos-base/tast-cmd` Portage packages, respectively, and are built from code
    68  in the [tast repository].
    69  
    70  ## Process
    71  
    72  Test bundles can be cross-compiled within the chroot by the `tast` executable
    73  using the Go toolchain. Object files are cached and reused where possible —
    74  caching happens at the package level, and the tests within each bundle are
    75  grouped into packages by category, e.g. `ui`, `power`, `security`, etc.
    76  
    77  For local tests, `tast` establishes a single SSH connection to the DUT for all
    78  communication. The connection is used to copy a local test bundle and associated
    79  test data files to the DUT (if not built into the system image), execute
    80  `local_test_runner`, and receive the results (including system logs, crash
    81  dumps, and output data written by the tests).
    82  
    83  For remote tests, tast executes `remote_test_runner` directly on the host system
    84  and receives results provided by it. Each remote test bundle establishes an SSH
    85  connection to the DUT and passes it to tests.
    86  
    87  The [Design Principles] document describes the thinking that guided Tast's
    88  design.
    89  
    90  [tast repository]: https://chromium.googlesource.com/chromiumos/platform/tast/
    91  [Running Tests]: running_tests.md
    92  [tast-tests repository]: https://chromium.googlesource.com/chromiumos/platform/tast-tests/
    93  [Writing Tests]: writing_tests.md
    94  [Design Principles]: design_principles.md