github.com/demonoid81/moby@v0.0.0-20200517203328-62dd8e17c460/docs/contributing/set-up-dev-env.md (about) 1 ### Work with a development container 2 3 In this section, you learn to develop like the Moby Engine core team. 4 The `moby/moby` repository includes a `Dockerfile` at its root. This file defines 5 Moby's development environment. The `Dockerfile` lists the environment's 6 dependencies: system libraries and binaries, Go environment, Go dependencies, 7 etc. 8 9 Moby's development environment is itself, ultimately a Docker container. 10 You use the `moby/moby` repository and its `Dockerfile` to create a Docker image, 11 run a Docker container, and develop code in the container. 12 13 If you followed the procedures that [set up Git for contributing](./set-up-git.md), you should have a fork of the `moby/moby` 14 repository. You also created a branch called `dry-run-test`. In this section, 15 you continue working with your fork on this branch. 16 17 ## Task 1. Remove images and containers 18 19 Moby developers run the latest stable release of the Docker software. They clean their local hosts of 20 unnecessary Docker artifacts such as stopped containers or unused images. 21 Cleaning unnecessary artifacts isn't strictly necessary, but it is good 22 practice, so it is included here. 23 24 To remove unnecessary artifacts: 25 26 1. Verify that you have no unnecessary containers running on your host. 27 28 ```none 29 $ docker ps -a 30 ``` 31 32 You should see something similar to the following: 33 34 ```none 35 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 36 ``` 37 38 There are no running or stopped containers on this host. A fast way to 39 remove old containers is the following: 40 41 You can now use the `docker system prune` command to achieve this: 42 43 ```none 44 $ docker system prune -a 45 ``` 46 47 Older versions of the Docker Engine should reference the command below: 48 49 ```none 50 $ docker rm $(docker ps -a -q) 51 ``` 52 53 This command uses `docker ps` to list all containers (`-a` flag) by numeric 54 IDs (`-q` flag). Then, the `docker rm` command removes the resulting list. 55 If you have running but unused containers, stop and then remove them with 56 the `docker stop` and `docker rm` commands. 57 58 2. Verify that your host has no dangling images. 59 60 ```none 61 $ docker images 62 ``` 63 64 You should see something similar to the following: 65 66 ```none 67 REPOSITORY TAG IMAGE ID CREATED SIZE 68 ``` 69 70 This host has no images. You may have one or more _dangling_ images. A 71 dangling image is not used by a running container and is not an ancestor of 72 another image on your system. A fast way to remove dangling image is 73 the following: 74 75 ```none 76 $ docker rmi -f $(docker images -q -a -f dangling=true) 77 ``` 78 79 This command uses `docker images` to list all images (`-a` flag) by numeric 80 IDs (`-q` flag) and filter them to find dangling images (`-f dangling=true`). 81 Then, the `docker rmi` command forcibly (`-f` flag) removes 82 the resulting list. If you get a "docker: "rmi" requires a minimum of 1 argument." 83 message, that means there were no dangling images. To remove just one image, use the 84 `docker rmi ID` command. 85 86 ## Task 2. Start a development container 87 88 If you followed the last procedure, your host is clean of unnecessary images and 89 containers. In this section, you build an image from the Engine development 90 environment and run it in the container. Both steps are automated for you by the 91 Makefile in the Engine code repository. The first time you build an image, it 92 can take over 15 minutes to complete. 93 94 1. Open a terminal. 95 96 For [Docker Toolbox](https://github.com/docker/toolbox) users, use `docker-machine status your_vm_name` to make sure your VM is running. You 97 may need to run `eval "$(docker-machine env your_vm_name)"` to initialize your 98 shell environment. If you use Docker for Mac or Docker for Windows, you do not need 99 to use Docker Machine. 100 101 2. Change into the root of the `moby-fork` repository. 102 103 ```none 104 $ cd ~/repos/moby-fork 105 ``` 106 107 If you are following along with this guide, you created a `dry-run-test` 108 branch when you [set up Git for contributing](./set-up-git.md). 109 110 3. Ensure you are on your `dry-run-test` branch. 111 112 ```none 113 $ git checkout dry-run-test 114 ``` 115 116 If you get a message that the branch doesn't exist, add the `-b` flag (`git checkout -b dry-run-test`) so the 117 command both creates the branch and checks it out. 118 119 4. Use `make` to build a development environment image and run it in a container. 120 121 ```none 122 $ make BIND_DIR=. shell 123 ``` 124 125 Using the instructions in the 126 `Dockerfile`, the build may need to download and / or configure source and other images. On first build this process may take between 5 - 15 minutes to create an image. The command returns informational messages as it runs. A 127 successful build returns a final message and opens a Bash shell into the 128 container. 129 130 ```none 131 Successfully built 3d872560918e 132 Successfully tagged docker-dev:dry-run-test 133 docker run --rm -i --privileged -e BUILDFLAGS -e KEEPBUNDLE -e DOCKER_BUILD_GOGC -e DOCKER_BUILD_PKGS -e DOCKER_CLIENTONLY -e DOCKER_DEBUG -e DOCKER_EXPERIMENTAL -e DOCKER_GITCOMMIT -e DOCKER_GRAPHDRIVER=devicemapper -e DOCKER_REMAP_ROOT -e DOCKER_STORAGE_OPTS -e DOCKER_USERLANDPROXY -e TESTDIRS -e TESTFLAGS -e TIMEOUT -v "home/ubuntu/repos/docker/bundles:/go/src/github.com/demonoid81/moby/bundles" -t "docker-dev:dry-run-test" bash 134 # 135 ``` 136 137 At this point, your prompt reflects the container's BASH shell. 138 139 5. List the contents of the current directory (`/go/src/github.com/demonoid81/moby`). 140 141 You should see the image's source from the `/go/src/github.com/demonoid81/moby` 142 directory. 143 144 ![List example](images/list_example.png) 145 146 6. Make a `dockerd` binary. 147 148 ```none 149 # hack/make.sh binary 150 Removing bundles/ 151 152 ---> Making bundle: binary (in bundles/binary) 153 Building: bundles/binary-daemon/dockerd-17.06.0-dev 154 Created binary: bundles/binary-daemon/dockerd-17.06.0-dev 155 Copying nested executables into bundles/binary-daemon 156 157 ``` 158 159 7. Run `make install`, which copies the binary to the container's 160 `/usr/local/bin/` directory. 161 162 ```none 163 # make install 164 ``` 165 166 8. Start the Engine daemon running in the background. 167 168 ```none 169 # dockerd -D & 170 ...output snipped... 171 DEBU[0001] Registering POST, /networks/{id:.*}/connect 172 DEBU[0001] Registering POST, /networks/{id:.*}/disconnect 173 DEBU[0001] Registering DELETE, /networks/{id:.*} 174 INFO[0001] API listen on /var/run/docker.sock 175 DEBU[0003] containerd connection state change: READY 176 ``` 177 178 The `-D` flag starts the daemon in debug mode. The `&` starts it as a 179 background process. You'll find these options useful when debugging code 180 development. You will need to hit `return` in order to get back to your shell prompt. 181 182 > **Note**: The following command automates the `build`, 183 > `install`, and `run` steps above. Once the command below completes, hit `ctrl-z` to suspend the process, then run `bg 1` and hit `enter` to resume the daemon process in the background and get back to your shell prompt. 184 185 ```none 186 hack/make.sh binary install-binary run 187 ``` 188 189 9. Inside your container, check your Docker versions: 190 191 ```none 192 # docker version 193 Client: 194 Version: 17.06.0-ce 195 API version: 1.30 196 Go version: go1.8.3 197 Git commit: 02c1d87 198 Built: Fri Jun 23 21:15:15 2017 199 OS/Arch: linux/amd64 200 201 Server: 202 Version: dev 203 API version: 1.35 (minimum version 1.12) 204 Go version: go1.9.2 205 Git commit: 4aa6362da 206 Built: Sat Dec 2 05:22:42 2017 207 OS/Arch: linux/amd64 208 Experimental: false 209 ``` 210 211 Notice the split versions between client and server, which might be 212 unexpected. In more recent times the Docker CLI component (which provides the 213 `docker` command) has split out from the Moby project and is now maintained in: 214 215 * [docker/cli](https://github.com/docker/cli) - The Docker CLI source-code; 216 * [docker/docker-ce](https://github.com/demonoid81/moby-ce) - The Docker CE 217 edition project, which assembles engine, CLI and other components. 218 219 The Moby project now defaults to a [fixed 220 version](https://github.com/demonoid81/moby-ce/commits/v17.06.0-ce) of the 221 `docker` CLI for integration tests. 222 223 You may have noticed the following message when starting the container with the `shell` command: 224 225 ```none 226 Makefile:123: The docker client CLI has moved to github.com/docker/cli. For a dev-test cycle involving the CLI, run: 227 DOCKER_CLI_PATH=/host/path/to/cli/binary make shell 228 then change the cli and compile into a binary at the same location. 229 ``` 230 231 By setting `DOCKER_CLI_PATH` you can supply a newer `docker` CLI to the 232 server development container for testing and for `integration-cli` 233 test-execution: 234 235 ```none 236 make DOCKER_CLI_PATH=/home/ubuntu/git/docker-ce/components/packaging/static/build/linux/docker/docker BIND_DIR=. shell 237 ... 238 # which docker 239 /usr/local/cli/docker 240 # docker --version 241 Docker version 17.09.0-dev, build 242 ``` 243 244 This Docker CLI should be built from the [docker-ce 245 project](https://github.com/demonoid81/moby-ce) and needs to be a Linux 246 binary. 247 248 Inside the container you are running a development version. This is the version 249 on the current branch. It reflects the value of the `VERSION` file at the 250 root of your `docker-fork` repository. 251 252 10. Run the `hello-world` image. 253 254 ```none 255 # docker run hello-world 256 ``` 257 258 11. List the image you just downloaded. 259 260 ```none 261 # docker images 262 REPOSITORY TAG IMAGE ID CREATED SIZE 263 hello-world latest c54a2cc56cbb 3 months ago 1.85 kB 264 ``` 265 266 12. Open another terminal on your local host. 267 268 13. List the container running your development container. 269 270 ```none 271 ubuntu@ubuntu1404:~$ docker ps 272 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 273 a8b2885ab900 docker-dev:dry-run-test "hack/dind bash" 43 minutes ago Up 43 minutes hungry_payne 274 ``` 275 276 Notice that the tag on the container is marked with the `dry-run-test` branch name. 277 278 279 ## Task 3. Make a code change 280 281 At this point, you have experienced the "Moby inception" technique. That is, 282 you have: 283 284 * forked and cloned the Moby Engine code repository 285 * created a feature branch for development 286 * created and started an Engine development container from your branch 287 * built a binary inside of your development container 288 * launched a `docker` daemon using your newly compiled binary 289 * called the `docker` client to run a `hello-world` container inside 290 your development container 291 292 Running the `make BIND_DIR=. shell` command mounted your local Docker repository source into 293 your Docker container. 294 295 > **Note**: Inspecting the `Dockerfile` shows a `COPY . /go/src/github.com/demonoid81/moby` instruction, suggesting that dynamic code changes will _not_ be reflected in the container. However inspecting the `Makefile` shows that the current working directory _will_ be mounted via a `-v` volume mount. 296 297 When you start to develop code though, you'll 298 want to iterate code changes and builds inside the container. If you have 299 followed this guide exactly, you have a bash shell running a development 300 container. 301 302 Try a simple code change and see it reflected in your container. For this 303 example, you'll edit the help for the `attach` subcommand. 304 305 1. If you don't have one, open a terminal in your local host. 306 307 2. Make sure you are in your `moby-fork` repository. 308 309 ```none 310 $ pwd 311 /Users/mary/go/src/github.com/moxiegirl/moby-fork 312 ``` 313 314 Your location should be different because, at least, your username is 315 different. 316 317 3. Open the `cmd/dockerd/docker.go` file. 318 319 4. Edit the command's help message. 320 321 For example, you can edit this line: 322 323 ```go 324 Short: "A self-sufficient runtime for containers.", 325 ``` 326 327 And change it to this: 328 329 ```go 330 Short: "A self-sufficient and really fun runtime for containers.", 331 ``` 332 333 5. Save and close the `cmd/dockerd/docker.go` file. 334 335 6. Go to your running docker development container shell. 336 337 7. Rebuild the binary by using the command `hack/make.sh binary` in the docker development container shell. 338 339 8. Stop Docker if it is running. 340 341 9. Copy the binaries to **/usr/bin** by entering the following commands in the docker development container shell. 342 343 ``` 344 hack/make.sh binary install-binary 345 ``` 346 347 10. To view your change, run the `dockerd --help` command in the docker development container shell. 348 349 ```bash 350 # dockerd --help 351 352 Usage: dockerd COMMAND 353 354 A self-sufficient and really fun runtime for containers. 355 356 Options: 357 ... 358 359 ``` 360 361 You've just done the basic workflow for changing the Engine code base. You made 362 your code changes in your feature branch. Then, you updated the binary in your 363 development container and tried your change out. If you were making a bigger 364 change, you might repeat or iterate through this flow several times. 365 366 ## Where to go next 367 368 Congratulations, you have successfully achieved Docker inception. You've had a 369 small experience of the development process. You've set up your development 370 environment and verified almost all the essential processes you need to 371 contribute. Of course, before you start contributing, [you'll need to learn one 372 more piece of the development process, the test framework](test.md).