github.com/cockroachdb/cockroach@v20.2.0-alpha.1+incompatible/build/README.md (about)

     1  # Docker Deploy
     2  
     3  Installing docker is a prerequisite. The instructions differ depending on the
     4  environment. Docker is comprised of two parts: the daemon server which runs on
     5  Linux and accepts commands, and the client which is a Go program capable of
     6  running on MacOS, all Unix variants and Windows.
     7  
     8  ## Docker Installation
     9  
    10  Follow the [Docker install
    11  instructions](https://docs.docker.com/engine/installation/).
    12  
    13  ## Available images
    14  
    15  There are development and deploy images available.
    16  
    17  ### Development
    18  
    19  The development image is a bulky image containing a complete build toolchain.
    20  It is well suited to hacking around and running the tests (including the
    21  acceptance tests). To fetch this image, run `./builder.sh pull`. The image can
    22  be run conveniently via `./builder.sh`.
    23  
    24  Note that if you use the builder image, you should ensure that your
    25  Docker installation grants 4GB or more of RAM to containers. On some
    26  systems, the default configuration limits containers to 2GB memory
    27  usage and this can be insufficient to build/link a CockroachDB
    28  executable.
    29  
    30  ### Deployment
    31  
    32  The deploy image is a downsized image containing a minimal environment for
    33  running CockroachDB. It is based on Debian Jessie and contains only the main
    34  CockroachDB binary. To fetch this image, run `docker pull
    35  cockroachdb/cockroach` in the usual fashion.
    36  
    37  To build the image yourself, use the Dockerfile in the `deploy` directory after
    38  building a release version of the binary with the development image described in
    39  the previous section. The CockroachDB binary will be built inside of that
    40  development container, then placed into the minimal deployment container. The
    41  resulting image `cockroachdb/cockroach` can be run via `docker run` in the
    42  usual fashion. To be more specific, the steps to do this are:
    43  
    44  ```
    45  go/src/github.com/cockroachdb/cockroach $ ./build/builder.sh mkrelease linux-gnu
    46  go/src/github.com/cockroachdb/cockroach $ cp ./cockroach-linux-2.6.32-gnu-amd64 build/deploy/cockroach
    47  go/src/github.com/cockroachdb/cockroach $ cd build/deploy && docker build -t cockroachdb/cockroach .
    48  ```
    49  
    50  The list of valid/recognized targets is available in the script
    51  `build/build/mkrelease.sh`, for example `amd64-linux-gnu` and
    52  `amd64-darwin`. Note that this script supports experimental targets
    53  which may or may not work (and are not officially supported).
    54  
    55  # Upgrading / extending the Docker image
    56  
    57  ## Basic Process
    58  
    59  - Edit `build/builder/Dockerfile` as desired
    60  - Run `build/builder.sh init` to test -- this will build the image locally. Beware this can take a lot of time. The result of `init` is a docker image version which you can subsequently stick into the `version` variable inside the `builder.sh` script for testing locally.
    61  - Once you are happy with the result, run `build/builder.sh push` which pushes your image towards Docker hub, so that it becomes available to others. The result is again a version number, which you then *must* copy back into `builder.sh`. Then commit the change to both Dockerfile and `builder.sh` and submit a PR.
    62  - Finally, use this version number to update the `builder.dockerImage` configuration parameter in TeamCity under the [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
    63  
    64  ## Updating the golang version
    65  
    66  Please copy this checklist (based on [Basic Process](#basic-process)) into the relevant commit message, with a link
    67  back to this document and perform these steps:
    68  
    69  * [ ] Adjust version in Docker image ([source](./builder/Dockerfile#L199-L200)).
    70  * [ ] Rebuild the Docker image and bump the `version` in `builder.sh` accordingly ([source](./builder.sh#L6)).
    71  * [ ] Bump the version in `go-version-check.sh` ([source](./go-version-check.sh)), unless bumping to a new patch release.
    72  * [ ] Bump the default installed version of Go in `bootstrap-debian.sh` ([source](./bootstrap/bootstrap-debian.sh#L40-42)).
    73  * [ ] Update the `builder.dockerImage` parameter in the TeamCity [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
    74  
    75  You can test the new builder image in TeamCity by using the custom parameters
    76  UI (the "..." icon next to the "Run" button) to verify the image before
    77  committing the change.
    78  
    79  ## Updating the nodejs version
    80  
    81  Please follow the instructions above on updating the golang version, omitting the go-version-check.sh step.
    82  
    83  #  Dependencies
    84  
    85  A snapshot of CockroachDB's dependencies is maintained at
    86  https://github.com/cockroachdb/vendored and checked out as a submodule at
    87  `./vendor`.
    88  
    89  ## Updating Dependencies
    90  
    91  This snapshot was built and is managed using `dep` and we manage `vendor` as a
    92  submodule.
    93  
    94  Use the version of `dep` in `bin` (may need to `make` first): import your new
    95  dependency from the Go source you're working on, then run `./bin/dep ensure -v`.
    96  The tool will add the newly-imported packages into the `vendor` directory; see
    97  instructions below about how to commit the changes in the `vendor` submodule. To
    98  update an existing dependency, use `./bin/dep ensure -v -update <pkg import
    99  path>`.
   100  
   101  ### Working with Submodules
   102  
   103  To keep the bloat of all the changes in all our dependencies out of our main
   104  repository, we embed `vendor` as a git submodule, storing its content and
   105  history in [`vendored`](https://github.com/cockroachdb/vendored) instead.
   106  
   107  This split across two repositories however means that changes involving
   108  changed dependencies require a two step process.
   109  
   110  - After using dep to add or update dependencies and making related code
   111  changes, `git status` in `cockroachdb/cockroach` checkout will report that the
   112  `vendor` submodule has `modified/untracked content`.
   113  
   114  - Switch into `vendor` and commit all changes (or use `git -C vendor`), on a
   115  new named branch.
   116  
   117     + At this point the `git status` in your `cockroachdb/cockroach` checkout
   118  will report `new commits` for `vendor` instead of `modified content`.
   119  
   120  - Commit your code changes and new `vendor` submodule ref.
   121  
   122  - Before this commit can be submitted in a pull request to
   123  `cockroachdb/cockroach`, the submodule commit it references must be available
   124  on `github.com/cockroachdb/vendored`.
   125  
   126  * Organization members can push their named branches there directly.
   127  
   128  * Non-members should fork the `vendored` repo and submit a pull request to
   129  `cockroachdb/vendored`, and need wait for it to merge before they will be able
   130  to use it in a `cockroachdb/cockroach` PR.
   131  
   132  #### `master` Branch Pointer in Vendored Repo
   133  
   134  Since the `cockroachdb/cockroach` submodule references individual commit
   135  hashes in `vendored`, there is little significance to the `master` branch in
   136  `vendored` -- as outlined above, new commits are always authored with the
   137  previously referenced commit as their parent, regardless of what `master`
   138  happens to be.
   139  
   140  That said, it is critical that any ref in `vendored` that is referenced from
   141  `cockroachdb/cockroach` remain available in `vendored` in perpetuity: after a
   142  PR referencing a ref merges, the `vendored` `master` branch should be updated
   143  to point to it before the named feature branch can be deleted, to ensure the
   144  ref remains reachable and thus is never garbage collected.
   145  
   146  #### Conflicting Submodule Changes
   147  
   148  The canonical linearization of history is always the main repo. In the event
   149  of concurrent changes to `vendor`, the first should cause the second to see a
   150  conflict on the `vendor` submodule pointer. When resolving that conflict, it
   151  is important to re-run dep against the fetched, updated `vendor` ref, thus
   152  generating a new commit in the submodule that has as its parent the one from
   153  the earlier change.
   154  
   155  ## Repository Name
   156  
   157  We only want the vendor directory used by builds when it is explicitly checked
   158  out *and managed* as a submodule at `./vendor`.
   159  
   160  If a go build fails to find a dependency in `./vendor`, it will continue
   161  searching anything named "vendor" in parent directories. Thus the vendor
   162  repository is _not_ named "vendor", to minimize the risk of it ending up
   163  somewhere in `GOPATH` with the name `vendor` (e.g. if it is manually cloned),
   164  where it could end up being unintentionally used by builds and causing
   165  confusion.