k8s.io/registry.k8s.io@v0.3.1/cmd/archeio/docs/testing.md (about)

     1  # Testing
     2  
     3  ## Unit Tests
     4  
     5  Archeio has unit tests covering 100% of application code (`cmd/archeio/app/...`) 
     6  and general packages (`pkg/...`). The only code without 100% coverage is in main.
     7  
     8  These are standard Go unit tests. In addition to typical unit tests with granular
     9  methods, we also have unit tests covering the HTTP Handlers and full 
    10  [request handling flow](./request-handling.md).
    11  
    12  These tests run on every pull request and must pass before merge.
    13  
    14  **This level of coverage must be maintained**, it is imperative that we have robust
    15  testing in this project that may soon serve all Kubernetes project image downloads.
    16  We automatically enforce 100% code coverage for archeio sources.
    17  
    18  Coverage results are visible by clicking the `pull-registry-test` context link
    19  at the bottom of the pull request.
    20  
    21  Coverage results can be viewed locally by `make test` + open `bin/all-filtered.html`.
    22  
    23  ## Integration Tests
    24  
    25  Package `main` code not covered by unit tests is covered by integration tests.
    26  
    27  Integration tests are written as a go unit test named with prefix `TestIntegration`
    28  in the `main` package which allows `make unit` to skip them.
    29  
    30  The integration tests run the actual application `main()`, and pull image(s)
    31  through a running instance using [crane].
    32  
    33  `make integration` runs only integration tests.
    34  
    35  The integration tests are able to exploit running against a local instance without
    36  a loadbalancer etc in front and fake the client IP address to test provider-IP
    37  codepaths.
    38  
    39  These tests run on every pull request in `pull-registry-test` and must pass before merge.
    40  
    41  ## E2E Testing
    42  
    43  Changes to archeio are auto-deployed to the registry-sandbox.k8s.io staging intance
    44  and NOT to registry.k8s.io. registry.k8s.io serves stable releases.
    45  
    46  ### e2e tests
    47  
    48  We have quick and cheap e2e results using real clients in `make e2e-test`
    49  and `make e2e-test-local`. We run `make e2e-test` against the staging instance.
    50  
    51  These are limited to clients we can run locally and in containerize CI
    52  without privilege escalation (e.g. [crane] again).
    53  
    54  These run immediately in the staging deploy jobs and
    55  continuously against the staging instance here:
    56  
    57  https://testgrid.k8s.io/sig-k8s-infra-registry#registry-sandbox-e2e-gcp
    58  https://testgrid.k8s.io/sig-k8s-infra-registry#registry-sandbox-e2e-aws
    59  
    60  `make e2e-test-local` runs against PRs to ensure the e2e tests themselves work
    61  and must pass before merge.
    62  
    63  ### Cluster e2e Testing
    64  
    65  The instance at registry-sandbox.k8s.io has [kops] cluster CI running in AWS 
    66  pointed at it to validate pulling from AWS. The kops clusters run in random
    67  regions.
    68  
    69  This E2E CI should be consulted before promoting code to stable release + registry.k8s.io.
    70  
    71  Results are visible in [testgrid] at: https://testgrid.k8s.io/sig-k8s-infra-registry#Summary
    72  
    73  The Kubernetes project itself has substantial assorted CI usage of the production instance
    74  and many CI jobs that primarily exist for other purposes will alert us if pulling from it fails.
    75  This includes many variations on "real" clusters and image build clients.
    76  
    77  [crane]: https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md
    78  [kops]: https://github.com/kubernetes/kops
    79  [testgrid]: https://github.com/GoogleCloudPlatform/testgrid