github.com/jonsyu1/godel@v0.0.0-20171017211503-64567a0cf169/docs/CI-setup.md (about)

     1  Summary
     2  -------
     3  gödel tasks can be configured to run in a CI environment to verify, build and publish products.
     4  
     5  Tutorial start state
     6  --------------------
     7  
     8  * `$GOPATH/src/github.com/nmiyake/echgo` exists and is the working directory
     9  * Project contains `godel` and `godelw`
    10  * Project contains `main.go`
    11  * Project contains `.gitignore` that ignores IDEA files
    12  * Project contains `echo/echo.go`, `echo/echo_test.go` and `echo/echoer.go`
    13  * `godel/config/dist.yml` is configured to build `echgo`
    14  * Project is tagged as 0.0.1
    15  * `godel/config/dist.yml` is configured to create distributions for `echgo`
    16  * Project is tagged as 0.0.2
    17  * Go files have license headers
    18  * `godel/config/generate.yml` is configured to generate string function
    19  * `godel/config/exclude.yml` is configured to ignore all `.+_string.go` files
    20  * `integration_test` contains integration tests
    21  * `godel/config/test.yml` is configured to specify the "integration" tag
    22  * `docs` contains documentation
    23  
    24  ([Link](https://github.com/nmiyake/echgo/tree/17c7406291096306e92c6f82da2df09388766693))
    25  
    26  CI setup
    27  --------
    28  
    29  Now that we have set up a project and a repository, we will configure CI (continuous integration) to verify that all of
    30  the PRs for our project properly pass verification and so that artifacts are published for releases.
    31  
    32  We will use CircleCI to set up CI for this project. Run the following to create a CircleCI configuration file:
    33  
    34  ```
    35  ➜ mkdir -p .circleci
    36  ➜ echo 'defaults: &defaults
    37    working_directory: /go/src/github.com/nmiyake/echgo
    38    docker:
    39      - image: golang:1.9.1
    40  
    41  version: 2
    42  jobs:
    43    build:
    44      <<: *defaults
    45      steps:
    46        - type: checkout
    47        - type: cache-restore
    48          key: godel-{{ checksum "godelw" }}
    49        - type: run
    50          name: "Verify godel version"
    51          command: ./godelw version
    52        - type: cache-save
    53          key: godel-{{ checksum "godelw" }}
    54          paths:
    55            - /root/.godel
    56        - type: run
    57          name: "Verify Go version"
    58          command: go version
    59        - type: run
    60          name: "Install project packages"
    61          command: go install $(./godelw packages)
    62        - type: run
    63          name: "Create test output directory"
    64          command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}"
    65        - type: run
    66          name: "Run godel verification"
    67          command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml"
    68        - type: test-results-store
    69          path: /tmp/test-results
    70        - type: artifacts-store
    71          path: /tmp/test-results
    72          destination: test-results
    73        - type: run
    74          name: "Create distribution"
    75          command: ./godelw dist
    76        - type: artifacts-store
    77          path: /go/src/github.com/nmiyake/echgo/dist
    78    wiki:
    79      <<: *defaults
    80      steps:
    81        - type: checkout
    82        - type: cache-restore
    83          key: godel-{{ checksum "godelw" }}
    84        - type: run
    85          name: "Verify godel version"
    86          command: ./godelw version
    87        - type: run
    88          name: "Update GitHub Wiki on master branch"
    89          command: ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git
    90    publish:
    91      <<: *defaults
    92      steps:
    93        - type: checkout
    94        - type: cache-restore
    95          key: godel-{{ checksum "godelw" }}
    96        - type: run
    97          name: "Verify godel version"
    98          command: ./godelw version
    99        - type: run
   100          name: "Publish"
   101          command: ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo
   102  
   103  workflows:
   104    version: 2
   105    build-deploy:
   106      jobs:
   107        - build:
   108            filters:
   109              tags:
   110                only: /.*/
   111        - wiki:
   112            requires:
   113              - build
   114            filters:
   115              branches:
   116                only: master
   117        - publish:
   118            requires:
   119              - build
   120            filters:
   121              tags:
   122                only: /^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$/
   123              branches:
   124                ignore: /.*/' > .circleci/config.yml
   125  ```
   126  
   127  The primary tasks performed by this CI are the following:
   128  
   129  * Runs `./godelw version` to ensure that the gödel distribution is downloaded and configured in the CI environment
   130    * This configuration caches the distribution so that it is only downloaded when the version changes
   131  * Runs `./godelw verify` with the `--apply=false` and `--junit-output=<path>` flags
   132    * This ensures that the code passes all of the required checks, runs all tests and saves the test output as a JUnit XML file
   133  * Runs `./godelw dist` to create the distribution
   134  * Runs `./godelw github-wiki` on the "master" branch to update documentation
   135    * Runs only on the "master" branch to ensure that only one branch is publishing documentation
   136  * Runs `./godelw publish` on release tags
   137  
   138  Commit the changes to the repository by running the following:
   139  
   140  ```
   141  ➜ git add .circleci
   142  ➜ git commit -m "Add CircleCI configuration"
   143  [master 25d27eb] Add CircleCI configuration
   144   1 file changed, 89 insertions(+)
   145   create mode 100644 .circleci/config.yml
   146  ```
   147  
   148  You can now configure the GitHub project to be run using CircleCI and it will run the CI process.
   149  
   150  Verify that everything is working as expected by tagging a 1.0.0, pushing the tag and verifying that the tag kicks off a
   151  build and publishes the artifacts:
   152  
   153  ```
   154  ➜ git push origin master
   155  Counting objects: 4, done.
   156  Delta compression using up to 8 threads.
   157  Compressing objects: 100% (3/3), done.
   158  Writing objects: 100% (4/4), 1.02 KiB | 0 bytes/s, done.
   159  Total 4 (delta 1), reused 0 (delta 0)
   160  remote: Resolving deltas: 100% (1/1), completed with 1 local object.
   161  To git@github.com:nmiyake/echgo.git
   162     17c7406..25d27eb  master -> master
   163  ➜ git tag 1.0.0
   164  ➜ git push origin --tags
   165  Total 0 (delta 0), reused 0 (delta 0)
   166  To git@github.com:nmiyake/echgo.git
   167   * [new tag]         1.0.0 -> 1.0.0
   168  ```
   169  
   170  Although this example was for CircleCI 2.0, the general principles/steps should be applicable in any CI system.
   171  
   172  Tutorial end state
   173  ------------------
   174  
   175  * `$GOPATH/src/github.com/nmiyake/echgo` exists and is the working directory
   176  * Project contains `godel` and `godelw`
   177  * Project contains `main.go`
   178  * Project contains `.gitignore` that ignores IDEA files
   179  * Project contains `echo/echo.go`, `echo/echo_test.go` and `echo/echoer.go`
   180  * `godel/config/dist.yml` is configured to build `echgo`
   181  * Project is tagged as 0.0.1
   182  * `godel/config/dist.yml` is configured to create distributions for `echgo`
   183  * Project is tagged as 0.0.2
   184  * Go files have license headers
   185  * `godel/config/generate.yml` is configured to generate string function
   186  * `godel/config/exclude.yml` is configured to ignore all `.+_string.go` files
   187  * `integration_test` contains integration tests
   188  * `godel/config/test.yml` is configured to specify the "integration" tag
   189  * `docs` contains documentation
   190  * `.circleci/config.yml` exists
   191  * Project is tagged as 1.0.0
   192  
   193  ([Link](https://github.com/nmiyake/echgo/tree/25d27eb1763e55f228282594691798ca0c2bbe28))
   194  
   195  Tutorial next step
   196  ------------------
   197  [Update gödel](https://github.com/palantir/godel/wiki/Update-g%C3%B6del)
   198  
   199  More
   200  ----
   201  
   202  ### CircleCI 2.0 without workflows
   203  
   204  ```yaml
   205  jobs:
   206    build:
   207      working_directory: /go/src/github.com/nmiyake/echgo
   208      docker:
   209        - image: golang:1.9.1
   210      steps:
   211        - type: checkout
   212        - type: cache-restore
   213          key: godel-{{ checksum "godelw" }}
   214        - type: shell
   215          name: "Verify godel version"
   216          command: ./godelw version
   217        - type: cache-save
   218          key: godel-{{ checksum "godelw" }}
   219          paths:
   220            - /root/.godel
   221        - type: shell
   222          name: "Verify Go version"
   223          command: go version
   224        - type: shell
   225          name: "Install project packages"
   226          command: go install $(./godelw packages)
   227        - type: shell
   228          name: "Create test output directory"
   229          command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}"
   230        - type: shell
   231          name: "Run godel verification"
   232          command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml"
   233        - type: test-results-store
   234          path: /tmp/test-results
   235        - type: artifacts-store
   236          path: /tmp/test-results
   237          destination: test-results
   238        - type: shell
   239          name: "Create distribution"
   240          command: ./godelw dist
   241        - type: artifacts-store
   242          path: /go/src/github.com/nmiyake/echgo/dist
   243        - type: deploy
   244          name: "Update GitHub Wiki on master branch"
   245          command: |
   246            set -eu
   247            if [ "${CIRCLE_BRANCH}" == "master" ]; then
   248              ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git
   249            else
   250              echo "Not master branch: skipping wiki publish"
   251            fi
   252        - type: deploy
   253          name: "Publish on release tags"
   254          command: |
   255            set -eu
   256            TAG=$(./godelw project-version)
   257            if [[ $TAG =~ ^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$ ]]; then
   258              ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo
   259            else
   260              echo "Not a release tag: skipping publish"
   261            fi
   262  ```
   263  
   264  ### CircleCI 2.0 with workflows
   265  
   266  ```yaml
   267  defaults: &defaults
   268    working_directory: /go/src/github.com/nmiyake/echgo
   269    docker:
   270      - image: golang:1.9.1
   271  
   272  version: 2
   273  jobs:
   274    build:
   275      <<: *defaults
   276      steps:
   277        - type: checkout
   278        - type: cache-restore
   279          key: godel-{{ checksum "godelw" }}
   280        - type: run
   281          name: "Verify godel version"
   282          command: ./godelw version
   283        - type: cache-save
   284          key: godel-{{ checksum "godelw" }}
   285          paths:
   286            - /root/.godel
   287        - type: run
   288          name: "Verify Go version"
   289          command: go version
   290        - type: run
   291          name: "Install project packages"
   292          command: go install $(./godelw packages)
   293        - type: run
   294          name: "Create test output directory"
   295          command: mkdir -p /tmp/test-results/"${CIRCLE_PROJECT_REPONAME}"
   296        - type: run
   297          name: "Run godel verification"
   298          command: ./godelw verify --apply=false --junit-output="/tmp/test-results/${CIRCLE_PROJECT_REPONAME}-tests.xml"
   299        - type: test-results-store
   300          path: /tmp/test-results
   301        - type: artifacts-store
   302          path: /tmp/test-results
   303          destination: test-results
   304        - type: run
   305          name: "Create distribution"
   306          command: ./godelw dist
   307        - type: artifacts-store
   308          path: /go/src/github.com/nmiyake/echgo/dist
   309    wiki:
   310      <<: *defaults
   311      steps:
   312        - type: checkout
   313        - type: cache-restore
   314          key: godel-{{ checksum "godelw" }}
   315        - type: run
   316          name: "Verify godel version"
   317          command: ./godelw version
   318        - type: run
   319          name: "Update GitHub Wiki on master branch"
   320          command: ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git
   321    publish:
   322      <<: *defaults
   323      steps:
   324        - type: checkout
   325        - type: cache-restore
   326          key: godel-{{ checksum "godelw" }}
   327        - type: run
   328          name: "Verify godel version"
   329          command: ./godelw version
   330        - type: run
   331          name: "Publish"
   332          command: ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo
   333  
   334  workflows:
   335    version: 2
   336    build-deploy:
   337      jobs:
   338        - build:
   339            filters:
   340              tags:
   341                only: /.*/
   342        - wiki:
   343            requires:
   344              - build
   345            filters:
   346              branches:
   347                only: master
   348        - publish:
   349            requires:
   350              - build
   351            filters:
   352              tags:
   353                only: /^[0-9]+(\.[0-9]+)+(-rc[0-9]+)?$/
   354              branches:
   355                ignore: /.*/
   356  ```
   357  
   358  ### CircleCI 1.0
   359  
   360  ```yaml
   361  machine:
   362    environment:
   363      GODIST: "go1.9.1.linux-amd64.tar.gz"
   364      GOPATH: "$HOME/.go_workspace"
   365      IMPORT_PATH: "github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME"
   366      GO_PROJECT_SRC_PATH: "$GOPATH/src/$IMPORT_PATH"
   367    post:
   368      - mkdir -p download
   369      - test -e download/$GODIST || wget -O "download/$GODIST" "https://storage.googleapis.com/golang/$GODIST"
   370      # create custom Go distribution with packages built for darwin-amd64 if it is not present
   371      - |
   372        if [ ! -e download/$GODIST-custom.tgz ]; then
   373            sudo rm -rf /usr/local/go && \
   374            sudo tar -C /usr/local -xzf download/$GODIST && \
   375            sudo env GOOS=darwin GOARCH=amd64 /usr/local/go/bin/go install std && \
   376            tar -C /usr/local -czf download/$GODIST-custom.tgz go
   377        fi
   378      - sudo rm -rf /usr/local/go
   379      - sudo tar -C /usr/local -xzf download/$GODIST-custom.tgz
   380  
   381  checkout:
   382    post:
   383      # ensure all tags are fetched and up-to-date
   384      - git tag -l | xargs git tag -d && git fetch -t
   385  
   386  dependencies:
   387    override:
   388      - mkdir -p "$GOPATH/src/$IMPORT_PATH"
   389      - rsync -az --delete ./ "$GOPATH/src/$IMPORT_PATH/"
   390      - cd "$GO_PROJECT_SRC_PATH" && ./godelw version
   391    cache_directories:
   392      - ~/.godel
   393      - ~/download
   394  
   395  test:
   396    pre:
   397      - go version
   398      - go get golang.org/x/tools/cmd/stringer
   399    override:
   400      - cd "$GO_PROJECT_SRC_PATH" && go install $(./godelw packages)
   401      - cd "$GO_PROJECT_SRC_PATH" && mkdir -p "$CIRCLE_TEST_REPORTS/$CIRCLE_PROJECT_REPONAME"
   402      - cd "$GO_PROJECT_SRC_PATH" && ./godelw verify --apply=false --junit-output="$CIRCLE_TEST_REPORTS/$CIRCLE_PROJECT_REPONAME/$CIRCLE_PROJECT_REPONAME-tests.xml"
   403      - cd "$GO_PROJECT_SRC_PATH" && ./godelw dist
   404  
   405  deployment:
   406    master:
   407      branch: master
   408      commands:
   409        - cd "$GO_PROJECT_SRC_PATH" && ./godelw github-wiki --docs-dir docs --repository=git@github.com:nmiyake/echgo.wiki.git
   410    release:
   411      tag: /[0-9]+(\.[0-9]+)+(-rc[0-9]+)?/
   412      commands:
   413        - cd "$GO_PROJECT_SRC_PATH" && ./godelw publish github --url https://api.github.com --user nmiyake --password $GITHUB_TOKEN --owner nmiyake --repository echgo
   414  ```