github.com/jonsyu1/godel@v0.0.0-20171017211503-64567a0cf169/docs/CI-setup.md (about) 1 Summary 2 ------- 3 gödel tasks can be configured to run in a CI environment to verify, build and publish products. 4 5 Tutorial start state 6 -------------------- 7 8 * `$GOPATH/src/github.com/nmiyake/echgo` exists and is the working directory 9 * Project contains `godel` and `godelw` 10 * Project contains `main.go` 11 * Project contains `.gitignore` that ignores IDEA files 12 * Project contains `echo/echo.go`, `echo/echo_test.go` and `echo/echoer.go` 13 * `godel/config/dist.yml` is configured to build `echgo` 14 * Project is tagged as 0.0.1 15 * `godel/config/dist.yml` is configured to create distributions for `echgo` 16 * Project is tagged as 0.0.2 17 * Go files have license headers 18 * `godel/config/generate.yml` is configured to generate string function 19 * `godel/config/exclude.yml` is configured to ignore all `.+_string.go` files 20 * `integration_test` contains integration tests 21 * `godel/config/test.yml` is configured to specify the "integration" tag 22 * `docs` contains documentation 23 24 ([Link](https://github.com/nmiyake/echgo/tree/17c7406291096306e92c6f82da2df09388766693)) 25 26 CI setup 27 -------- 28 29 Now that we have set up a project and a repository, we will configure CI (continuous integration) to verify that all of 30 the PRs for our project properly pass verification and so that artifacts are published for releases. 31 32 We will use CircleCI to set up CI for this project. Run the following to create a CircleCI configuration file: 33 34 ``` 35 ➜ mkdir -p .circleci 36 ➜ echo 'defaults: &defaults 37 working_directory: /go/src/github.com/nmiyake/echgo 38 docker: 39 - image: golang:1.9.1 40 41 version: 2 42 jobs: 43 build: 44 <<: *defaults 45 steps: 46 - type: checkout 47 - type: cache-restore 48 key: godel-{{ checksum "godelw" }} 49 - type: run 50 name: "Verify godel version" 51 command: ./godelw version 52 - type: cache-save 53 key: godel-{{ checksum "godelw" }} 54 paths: 55 - /root/.godel 56 - type: run 57 name: "Verify Go version" 58 command: go version 59 - type: run 60 name: "Install project packages" 61 command: go install $(./godelw packages) 62 - type: run 63 name: "Create test output directory" 64 command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}" 65 - type: run 66 name: "Run godel verification" 67 command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml" 68 - type: test-results-store 69 path: /tmp/test-results 70 - type: artifacts-store 71 path: /tmp/test-results 72 destination: test-results 73 - type: run 74 name: "Create distribution" 75 command: ./godelw dist 76 - type: artifacts-store 77 path: /go/src/github.com/nmiyake/echgo/dist 78 wiki: 79 <<: *defaults 80 steps: 81 - type: checkout 82 - type: cache-restore 83 key: godel-{{ checksum "godelw" }} 84 - type: run 85 name: "Verify godel version" 86 command: ./godelw version 87 - type: run 88 name: "Update GitHub Wiki on master branch" 89 command: ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git 90 publish: 91 <<: *defaults 92 steps: 93 - type: checkout 94 - type: cache-restore 95 key: godel-{{ checksum "godelw" }} 96 - type: run 97 name: "Verify godel version" 98 command: ./godelw version 99 - type: run 100 name: "Publish" 101 command: ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo 102 103 workflows: 104 version: 2 105 build-deploy: 106 jobs: 107 - build: 108 filters: 109 tags: 110 only: /.*/ 111 - wiki: 112 requires: 113 - build 114 filters: 115 branches: 116 only: master 117 - publish: 118 requires: 119 - build 120 filters: 121 tags: 122 only: /^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$/ 123 branches: 124 ignore: /.*/' > .circleci/config.yml 125 ``` 126 127 The primary tasks performed by this CI are the following: 128 129 * Runs `./godelw version` to ensure that the gödel distribution is downloaded and configured in the CI environment 130 * This configuration caches the distribution so that it is only downloaded when the version changes 131 * Runs `./godelw verify` with the `--apply=false` and `--junit-output=<path>` flags 132 * This ensures that the code passes all of the required checks, runs all tests and saves the test output as a JUnit XML file 133 * Runs `./godelw dist` to create the distribution 134 * Runs `./godelw github-wiki` on the "master" branch to update documentation 135 * Runs only on the "master" branch to ensure that only one branch is publishing documentation 136 * Runs `./godelw publish` on release tags 137 138 Commit the changes to the repository by running the following: 139 140 ``` 141 ➜ git add .circleci 142 ➜ git commit -m "Add CircleCI configuration" 143 [master 25d27eb] Add CircleCI configuration 144 1 file changed, 89 insertions(+) 145 create mode 100644 .circleci/config.yml 146 ``` 147 148 You can now configure the GitHub project to be run using CircleCI and it will run the CI process. 149 150 Verify that everything is working as expected by tagging a 1.0.0, pushing the tag and verifying that the tag kicks off a 151 build and publishes the artifacts: 152 153 ``` 154 ➜ git push origin master 155 Counting objects: 4, done. 156 Delta compression using up to 8 threads. 157 Compressing objects: 100% (3/3), done. 158 Writing objects: 100% (4/4), 1.02 KiB | 0 bytes/s, done. 159 Total 4 (delta 1), reused 0 (delta 0) 160 remote: Resolving deltas: 100% (1/1), completed with 1 local object. 161 To git@github.com:nmiyake/echgo.git 162 17c7406..25d27eb master -> master 163 ➜ git tag 1.0.0 164 ➜ git push origin --tags 165 Total 0 (delta 0), reused 0 (delta 0) 166 To git@github.com:nmiyake/echgo.git 167 * [new tag] 1.0.0 -> 1.0.0 168 ``` 169 170 Although this example was for CircleCI 2.0, the general principles/steps should be applicable in any CI system. 171 172 Tutorial end state 173 ------------------ 174 175 * `$GOPATH/src/github.com/nmiyake/echgo` exists and is the working directory 176 * Project contains `godel` and `godelw` 177 * Project contains `main.go` 178 * Project contains `.gitignore` that ignores IDEA files 179 * Project contains `echo/echo.go`, `echo/echo_test.go` and `echo/echoer.go` 180 * `godel/config/dist.yml` is configured to build `echgo` 181 * Project is tagged as 0.0.1 182 * `godel/config/dist.yml` is configured to create distributions for `echgo` 183 * Project is tagged as 0.0.2 184 * Go files have license headers 185 * `godel/config/generate.yml` is configured to generate string function 186 * `godel/config/exclude.yml` is configured to ignore all `.+_string.go` files 187 * `integration_test` contains integration tests 188 * `godel/config/test.yml` is configured to specify the "integration" tag 189 * `docs` contains documentation 190 * `.circleci/config.yml` exists 191 * Project is tagged as 1.0.0 192 193 ([Link](https://github.com/nmiyake/echgo/tree/25d27eb1763e55f228282594691798ca0c2bbe28)) 194 195 Tutorial next step 196 ------------------ 197 [Update gödel](https://github.com/palantir/godel/wiki/Update-g%C3%B6del) 198 199 More 200 ---- 201 202 ### CircleCI 2.0 without workflows 203 204 ```yaml 205 jobs: 206 build: 207 working_directory: /go/src/github.com/nmiyake/echgo 208 docker: 209 - image: golang:1.9.1 210 steps: 211 - type: checkout 212 - type: cache-restore 213 key: godel-{{ checksum "godelw" }} 214 - type: shell 215 name: "Verify godel version" 216 command: ./godelw version 217 - type: cache-save 218 key: godel-{{ checksum "godelw" }} 219 paths: 220 - /root/.godel 221 - type: shell 222 name: "Verify Go version" 223 command: go version 224 - type: shell 225 name: "Install project packages" 226 command: go install $(./godelw packages) 227 - type: shell 228 name: "Create test output directory" 229 command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}" 230 - type: shell 231 name: "Run godel verification" 232 command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml" 233 - type: test-results-store 234 path: /tmp/test-results 235 - type: artifacts-store 236 path: /tmp/test-results 237 destination: test-results 238 - type: shell 239 name: "Create distribution" 240 command: ./godelw dist 241 - type: artifacts-store 242 path: /go/src/github.com/nmiyake/echgo/dist 243 - type: deploy 244 name: "Update GitHub Wiki on master branch" 245 command: | 246 set -eu 247 if [ "${CIRCLE_BRANCH}" == "master" ]; then 248 ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git 249 else 250 echo "Not master branch: skipping wiki publish" 251 fi 252 - type: deploy 253 name: "Publish on release tags" 254 command: | 255 set -eu 256 TAG=$(./godelw project-version) 257 if [[ $TAG =~ ^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$ ]]; then 258 ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo 259 else 260 echo "Not a release tag: skipping publish" 261 fi 262 ``` 263 264 ### CircleCI 2.0 with workflows 265 266 ```yaml 267 defaults: &defaults 268 working_directory: /go/src/github.com/nmiyake/echgo 269 docker: 270 - image: golang:1.9.1 271 272 version: 2 273 jobs: 274 build: 275 <<: *defaults 276 steps: 277 - type: checkout 278 - type: cache-restore 279 key: godel-{{ checksum "godelw" }} 280 - type: run 281 name: "Verify godel version" 282 command: ./godelw version 283 - type: cache-save 284 key: godel-{{ checksum "godelw" }} 285 paths: 286 - /root/.godel 287 - type: run 288 name: "Verify Go version" 289 command: go version 290 - type: run 291 name: "Install project packages" 292 command: go install $(./godelw packages) 293 - type: run 294 name: "Create test output directory" 295 command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}" 296 - type: run 297 name: "Run godel verification" 298 command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml" 299 - type: test-results-store 300 path: /tmp/test-results 301 - type: artifacts-store 302 path: /tmp/test-results 303 destination: test-results 304 - type: run 305 name: "Create distribution" 306 command: ./godelw dist 307 - type: artifacts-store 308 path: /go/src/github.com/nmiyake/echgo/dist 309 wiki: 310 <<: *defaults 311 steps: 312 - type: checkout 313 - type: cache-restore 314 key: godel-{{ checksum "godelw" }} 315 - type: run 316 name: "Verify godel version" 317 command: ./godelw version 318 - type: run 319 name: "Update GitHub Wiki on master branch" 320 command: ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git 321 publish: 322 <<: *defaults 323 steps: 324 - type: checkout 325 - type: cache-restore 326 key: godel-{{ checksum "godelw" }} 327 - type: run 328 name: "Verify godel version" 329 command: ./godelw version 330 - type: run 331 name: "Publish" 332 command: ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo 333 334 workflows: 335 version: 2 336 build-deploy: 337 jobs: 338 - build: 339 filters: 340 tags: 341 only: /.*/ 342 - wiki: 343 requires: 344 - build 345 filters: 346 branches: 347 only: master 348 - publish: 349 requires: 350 - build 351 filters: 352 tags: 353 only: /^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$/ 354 branches: 355 ignore: /.*/ 356 ``` 357 358 ### CircleCI 1.0 359 360 ```yaml 361 machine: 362 environment: 363 GODIST: "go1.9.1.linux-amd64.tar.gz" 364 GOPATH: "$HOME/.go_workspace" 365 IMPORT_PATH: "github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME" 366 GO_PROJECT_SRC_PATH: "$GOPATH/src/$IMPORT_PATH" 367 post: 368 - mkdir -p download 369 - test -e download/$GODIST || wget -O "download/$GODIST" "https://storage.googleapis.com/golang/$GODIST" 370 # create custom Go distribution with packages built for darwin-amd64 if it is not present 371 - | 372 if [ ! -e download/$GODIST-custom.tgz ]; then 373 sudo rm -rf /usr/local/go && \ 374 sudo tar -C /usr/local -xzf download/$GODIST && \ 375 sudo env GOOS=darwin GOARCH=amd64 /usr/local/go/bin/go install std && \ 376 tar -C /usr/local -czf download/$GODIST-custom.tgz go 377 fi 378 - sudo rm -rf /usr/local/go 379 - sudo tar -C /usr/local -xzf download/$GODIST-custom.tgz 380 381 checkout: 382 post: 383 # ensure all tags are fetched and up-to-date 384 - git tag -l | xargs git tag -d && git fetch -t 385 386 dependencies: 387 override: 388 - mkdir -p "$GOPATH/src/$IMPORT_PATH" 389 - rsync -az --delete ./ "$GOPATH/src/$IMPORT_PATH/" 390 - cd "$GO_PROJECT_SRC_PATH" && ./godelw version 391 cache_directories: 392 - ~/.godel 393 - ~/download 394 395 test: 396 pre: 397 - go version 398 - go get golang.org/x/tools/cmd/stringer 399 override: 400 - cd "$GO_PROJECT_SRC_PATH" && go install $(./godelw packages) 401 - cd "$GO_PROJECT_SRC_PATH" && mkdir -p "$CIRCLE_TEST_REPORTS/$CIRCLE_PROJECT_REPONAME" 402 - cd "$GO_PROJECT_SRC_PATH" && ./godelw verify --apply=false --junit-output="$CIRCLE_TEST_REPORTS/$CIRCLE_PROJECT_REPONAME/$CIRCLE_PROJECT_REPONAME-tests.xml" 403 - cd "$GO_PROJECT_SRC_PATH" && ./godelw dist 404 405 deployment: 406 master: 407 branch: master 408 commands: 409 - cd "$GO_PROJECT_SRC_PATH" && ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git 410 release: 411 tag: /[0-9]+(\.[0-9]+)+(-rc[0-9]+)?/ 412 commands: 413 - cd "$GO_PROJECT_SRC_PATH" && ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo 414 ```