github.com/abayer/test-infra@v0.0.5/prow/pod-utilities.md (about) 1 # Pod Utilities 2 3 Pod utilities are small, focused Go programs used by `plank` to decorate user-provided `PodSpec`s 4 in order to increase the ease of integration for new jobs into the entire CI infrastructure. The 5 utilities today wrap the execution of the test code to ensure that the tests run against correct 6 versions of the source code, that test commands run in the appropriate environment and that output 7 from the test (in the form of status, logs and artifacts) is correctly uploaded to the cloud. 8 9 These utilities are integrated into a test run by adding `InitContainer`s and sidecar `Container`s 10 to the user-provided `PodSpec`, as well as by overwriting the `Container` entrypoint for the test 11 `Container` provided by the user. The following utilities exist today: 12 13 - [`clonerefs`](./cmd/clonerefs/README.md): clones source code under test 14 - [`initupload`](./cmd/initupload/README.md): records the beginning of a test in cloud storage 15 and reports the status of the clone operations 16 - [`entrypoint`](./cmd/entrypoint/README.md): is injected into the test `Container`, wraps the 17 test code to capture logs and exit status 18 - [`sidecar`](./cmd/sidecar/README.md): runs alongside the test `Container`, uploads status, logs 19 and test artifacts to cloud storage once the test is finished 20 21 ## Writing a ProwJob that uses Pod Utilities 22 23 ### What the test container can expect 24 25 Example test container script: 26 ```bash 27 pwd # my repo root 28 ls path/to/file/in/my/repo.txt # access repo file 29 ls ../other-repo # access repo file in another repo 30 echo results.txt > $ARTIFACTS # result info that will be uploaded to GCS. 31 # logs, and job metadata are automatically uploaded. 32 ``` 33 34 More specifically, a ProwJob using the Pod Utilities can expect the following: 35 - **Source Code** - Jobs can expect to begin execution with their working 36 directory set as the root of the checked out repo. The commit that is checked 37 out depends on the type of job: 38 - `presubmit` jobs will have the relevant PR checked out and merged with the base branch. 39 - `postsubmit` jobs will have the upstream commit that triggered the job checked out. 40 - `periodic` jobs will have the working directory set to the root of the repo specified by the first ref in `extra_refs` (if specified). 41 See the `extra_refs` field if you need to clone more than one repo. 42 - **Metadata and Logs** - Jobs can expect metadata about the job to be uploaded 43 before the job starts, and additional metadata and logs to be uploaded when the 44 job completes. 45 - **Artifact Directory** - Jobs can expect an `$ARTIFACTS` environment variable 46 to be specified. It indicates an existent directory where job artifacts can be 47 dumped for automatic upload to GCS upon job completion. 48 49 ### How to configure 50 51 ProwJobs may request Pod Utility decoration by setting `decorate: true` in their config. 52 Example ProwJob configuration: 53 ```yaml 54 55 - name: pull-job 56 agent: kubernetes 57 context: pull-job 58 always_run: true 59 rerun_command: "/test pull-job" 60 trigger: "(?m)^/test (all|pull-job)\\s*" 61 decorate: true 62 spec: 63 containers: 64 - image: alpine 65 command: 66 - "echo" 67 args: 68 - "The artifacts dir is $(ARTIFACTS)" 69 ``` 70 71 In addition to normal ProwJob configuration, ProwJobs using the Pod Utilities 72 must specify the `command` field in the container specification instead of using 73 the Dockerfile's ENTRYPOINT directive. Note that the `command` field is a string 74 array not just a string. It should point to the test binary location in the container. 75 76 Additional fields may be required for some use cases: 77 - Private repos need to do two things: 78 - Add an ssh secret that gives the bot access to the repo to the build cluster 79 and specify the secret name in the `ssh_key_secrets` field of the job spec. 80 - Set the `clone_uri` field of the job spec to `git@github.com:{{.Org}}/{{.Repo}}.git`. 81 - Repos requiring a non-standard clone path can use the `path_alias` field 82 to clone the repo to a path different than the default of `/go/src/github.com/org/repo/` (e.g. `/go/src/k8s.io/kubernetes/kubernetes`). 83 - Jobs that require additional repos to be checked out can arrange for that with 84 the `exta_refs` field. 85 86 ```yaml 87 - name: post-job 88 agent: kubernetes 89 decorate: true 90 ssh_key_secrets: 91 - ssh-secret 92 clone_uri: "git@github.com:{{.Org}}/{{.Repo}}.git" 93 extra_refs: 94 - org: kubernetes 95 repo: other-repo 96 base_ref: master 97 spec: 98 containers: 99 - image: alpine 100 command: 101 - "echo" 102 args: 103 - "The artifacts dir is $(ARTIFACTS)" 104 105 ``` 106 107 ### Why use Pod Utilities? 108 109 Writing a ProwJob that uses the Pod Utilities is much easier than writing one 110 that doesn't because the Pod Utilities will transparently handle many of the 111 tasks the job would otherwise need to do in order to prepare its environment 112 and output more than pass/fail. Historically, this was achieved by wrapping 113 every job with a [bootstrap.py](jenkins/bootstrap.py) script that handled cloning 114 source code, preparing the test environment, and uploading job metadata, logs, 115 and artifacts. This was cumbersome to configure and required every job to be 116 wrapped with the script in the job image. The pod utilities achieve the same goals 117 with less configuration and much simpler job images that are easier to develop 118 and less coupled to Prow.