github.com/docker/docker@v299999999.0.0-20200612211812-aaf470eca7b5+incompatible/docs/contributing/set-up-dev-env.md (about)

     1  ### Work with a development container
     2  
     3  In this section, you learn to develop like the Moby Engine core team.
     4  The `moby/moby` repository includes a `Dockerfile` at its root. This file defines
     5  Moby's development environment. The `Dockerfile` lists the environment's
     6  dependencies: system libraries and binaries, Go environment, Go dependencies,
     7  etc.
     8  
     9  Moby's development environment is itself, ultimately a Docker container.
    10  You use the `moby/moby` repository and its `Dockerfile` to create a Docker image,
    11  run a Docker container, and develop code in the container.
    12  
    13  If you followed the procedures that [set up Git for contributing](./set-up-git.md), you should have a fork of the `moby/moby`
    14  repository. You also created a branch called `dry-run-test`. In this section,
    15  you continue working with your fork on this branch.
    16  
    17  ##  Task 1. Remove images and containers
    18  
    19  Moby developers run the latest stable release of the Docker software. They clean their local hosts of
    20  unnecessary Docker artifacts such as stopped containers or unused images.
    21  Cleaning unnecessary artifacts isn't strictly necessary, but it is good
    22  practice, so it is included here.
    23  
    24  To remove unnecessary artifacts:
    25  
    26  1. Verify that you have no unnecessary containers running on your host.
    27  
    28     ```none
    29     $ docker ps -a
    30     ```
    31  
    32     You should see something similar to the following:
    33  
    34     ```none
    35     CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    36     ```
    37  
    38     There are no running or stopped containers on this host. A fast way to
    39     remove old containers is the following:
    40  
    41     You can now use the `docker system prune` command to achieve this:
    42  
    43     ```none
    44     $ docker system prune -a
    45     ```
    46  
    47     Older versions of the Docker Engine should reference the command below:
    48  
    49     ```none
    50     $ docker rm $(docker ps -a -q)
    51     ```
    52  
    53     This command uses `docker ps` to list all containers (`-a` flag) by numeric
    54     IDs (`-q` flag). Then, the `docker rm` command removes the resulting list.
    55     If you have running but unused containers, stop and then remove them with
    56     the `docker stop` and `docker rm` commands.
    57  
    58  2. Verify that your host has no dangling images.
    59  
    60     ```none
    61     $ docker images
    62     ```
    63  
    64     You should see something similar to the following:
    65  
    66     ```none
    67     REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    68     ```
    69  
    70     This host has no images. You may have one or more _dangling_ images. A
    71     dangling image is not used by a running container and is not an ancestor of
    72     another image on your system. A fast way to remove dangling image is
    73     the following:
    74  
    75     ```none
    76     $ docker rmi -f $(docker images -q -a -f dangling=true)
    77     ```
    78  
    79     This command uses `docker images` to list all images (`-a` flag) by numeric
    80     IDs (`-q` flag) and filter them to find dangling images (`-f dangling=true`).
    81     Then, the `docker rmi` command forcibly (`-f` flag) removes
    82     the resulting list. If you get a "docker: "rmi" requires a minimum of 1 argument."
    83     message, that means there were no dangling images. To remove just one image, use the
    84     `docker rmi ID` command.
    85  
    86  ## Task 2. Start a development container
    87  
    88  If you followed the last procedure, your host is clean of unnecessary images and
    89  containers. In this section, you build an image from the Engine development
    90  environment and run it in the container. Both steps are automated for you by the
    91  Makefile in the Engine code repository. The first time you build an image, it
    92  can take over 15 minutes to complete.
    93  
    94  1. Open a terminal.
    95  
    96     For [Docker Toolbox](https://github.com/docker/toolbox) users, use `docker-machine status your_vm_name` to make sure your VM is running. You
    97     may need to run `eval "$(docker-machine env your_vm_name)"` to initialize your
    98     shell environment. If you use Docker for Mac or Docker for Windows, you do not need
    99     to use Docker Machine.
   100  
   101  2. Change into the root of the `moby-fork` repository.
   102  
   103     ```none
   104     $ cd ~/repos/moby-fork
   105     ```
   106  
   107     If you are following along with this guide, you created a `dry-run-test`
   108     branch when you [set up Git for contributing](./set-up-git.md).
   109  
   110  3. Ensure you are on your `dry-run-test` branch.
   111  
   112     ```none
   113     $ git checkout dry-run-test
   114     ```
   115  
   116     If you get a message that the branch doesn't exist, add the `-b` flag (`git checkout -b dry-run-test`) so the
   117     command both creates the branch and checks it out.
   118  
   119  4. Use `make` to build a development environment image and run it in a container.
   120  
   121     ```none
   122     $ make BIND_DIR=. shell
   123     ```
   124  
   125     Using the instructions in the
   126     `Dockerfile`, the build may need to download and / or configure source and other images. On first build this process may take between 5 - 15 minutes to create an image. The command returns informational messages as it runs.  A
   127     successful build returns a final message and opens a Bash shell into the
   128     container.
   129  
   130     ```none
   131     Successfully built 3d872560918e
   132     Successfully tagged docker-dev:dry-run-test
   133     docker run --rm -i --privileged -e BUILDFLAGS -e KEEPBUNDLE -e DOCKER_BUILD_GOGC -e DOCKER_BUILD_PKGS -e DOCKER_CLIENTONLY -e DOCKER_DEBUG -e DOCKER_EXPERIMENTAL -e DOCKER_GITCOMMIT -e DOCKER_GRAPHDRIVER=devicemapper -e DOCKER_REMAP_ROOT -e DOCKER_STORAGE_OPTS -e DOCKER_USERLANDPROXY -e TESTDIRS -e TESTFLAGS -e TIMEOUT -v "home/ubuntu/repos/docker/bundles:/go/src/github.com/docker/docker/bundles" -t "docker-dev:dry-run-test" bash
   134     #
   135     ```
   136  
   137     At this point, your prompt reflects the container's BASH shell.
   138  
   139  5. List the contents of the current directory (`/go/src/github.com/docker/docker`).
   140  
   141     You should see the image's source from the  `/go/src/github.com/docker/docker`
   142     directory.
   143  
   144     ![List example](images/list_example.png)
   145  
   146  6. Make a `dockerd` binary.
   147  
   148     ```none
   149     # hack/make.sh binary
   150     Removing bundles/
   151  
   152     ---> Making bundle: binary (in bundles/binary)
   153     Building: bundles/binary-daemon/dockerd-17.06.0-dev
   154     Created binary: bundles/binary-daemon/dockerd-17.06.0-dev
   155     Copying nested executables into bundles/binary-daemon
   156  
   157     ```
   158  
   159  7. Run `make install`, which copies the binary to the container's
   160     `/usr/local/bin/` directory.
   161  
   162     ```none
   163     # make install
   164     ```
   165  
   166  8. Start the Engine daemon running in the background.
   167  
   168     ```none
   169     # dockerd -D &
   170     ...output snipped...
   171     DEBU[0001] Registering POST, /networks/{id:.*}/connect
   172     DEBU[0001] Registering POST, /networks/{id:.*}/disconnect
   173     DEBU[0001] Registering DELETE, /networks/{id:.*}
   174     INFO[0001] API listen on /var/run/docker.sock
   175     DEBU[0003] containerd connection state change: READY
   176     ```
   177  
   178     The `-D` flag starts the daemon in debug mode. The `&` starts it as a
   179     background process. You'll find these options useful when debugging code
   180     development. You will need to hit `return` in order to get back to your shell prompt.
   181  
   182     > **Note**: The following command automates the `build`,
   183     > `install`, and `run` steps above. Once the command below completes, hit `ctrl-z` to suspend the process, then run `bg 1` and hit `enter` to resume the daemon process in the background and get back to your shell prompt.
   184  
   185     ```none
   186     hack/make.sh binary install-binary run
   187     ```
   188  
   189  9. Inside your container, check your Docker versions:
   190  
   191     ```none
   192     # docker version
   193     Client:
   194      Version:      17.06.0-ce
   195      API version:  1.30
   196      Go version:   go1.8.3
   197      Git commit:   02c1d87
   198      Built:        Fri Jun 23 21:15:15 2017
   199      OS/Arch:      linux/amd64
   200  
   201     Server:
   202      Version:      dev
   203      API version:  1.35 (minimum version 1.12)
   204      Go version:   go1.9.2
   205      Git commit:   4aa6362da
   206      Built:        Sat Dec  2 05:22:42 2017
   207      OS/Arch:      linux/amd64
   208      Experimental: false
   209     ```
   210  
   211     Notice the split versions between client and server, which might be
   212     unexpected. In more recent times the Docker CLI component (which provides the
   213     `docker` command) has split out from the Moby project and is now maintained in:
   214     
   215     * [docker/cli](https://github.com/docker/cli) - The Docker CLI source-code;
   216     * [docker/docker-ce](https://github.com/docker/docker-ce) - The Docker CE
   217       edition project, which assembles engine, CLI and other components.
   218     
   219     The Moby project now defaults to a [fixed
   220     version](https://github.com/docker/docker-ce/commits/v17.06.0-ce) of the
   221     `docker` CLI for integration tests.
   222  
   223     You may have noticed the following message when starting the container with the `shell` command:
   224     
   225     ```none
   226     Makefile:123: The docker client CLI has moved to github.com/docker/cli. For a dev-test cycle involving the CLI, run:
   227     DOCKER_CLI_PATH=/host/path/to/cli/binary make shell
   228     then change the cli and compile into a binary at the same location.
   229     ```
   230  
   231     By setting `DOCKER_CLI_PATH` you can supply a newer `docker` CLI to the
   232     server development container for testing and for `integration-cli`
   233     test-execution:
   234  
   235     ```none
   236     make DOCKER_CLI_PATH=/home/ubuntu/git/docker-ce/components/packaging/static/build/linux/docker/docker BIND_DIR=. shell
   237     ...
   238     # which docker
   239     /usr/local/cli/docker
   240     # docker --version
   241     Docker version 17.09.0-dev, build 
   242     ```
   243  
   244      This Docker CLI should be built from the [docker-ce
   245      project](https://github.com/docker/docker-ce) and needs to be a Linux
   246      binary.
   247  
   248     Inside the container you are running a development version. This is the version
   249     on the current branch. It reflects the value of the `VERSION` file at the
   250     root of your `docker-fork` repository.
   251  
   252  10. Run the `hello-world` image.
   253  
   254      ```none
   255      # docker run hello-world
   256      ```
   257  
   258  11. List the image you just downloaded.
   259  
   260      ```none
   261      # docker images
   262  	REPOSITORY   TAG     IMAGE ID      CREATED        SIZE
   263  	hello-world  latest  c54a2cc56cbb  3 months ago   1.85 kB
   264      ```
   265  
   266  12. Open another terminal on your local host.
   267  
   268  13. List the container running your development container.
   269  
   270      ```none
   271      ubuntu@ubuntu1404:~$ docker ps
   272      CONTAINER ID        IMAGE                     COMMAND             CREATED             STATUS              PORTS               NAMES
   273      a8b2885ab900        docker-dev:dry-run-test   "hack/dind bash"    43 minutes ago      Up 43 minutes                           hungry_payne
   274      ```
   275  
   276      Notice that the tag on the container is marked with the `dry-run-test` branch name.
   277  
   278  
   279  ## Task 3. Make a code change
   280  
   281  At this point, you have experienced the "Moby inception" technique. That is,
   282  you have:
   283  
   284  * forked and cloned the Moby Engine code repository
   285  * created a feature branch for development
   286  * created and started an Engine development container from your branch
   287  * built a binary inside of your development container
   288  * launched a `docker` daemon using your newly compiled binary
   289  * called the `docker` client to run a `hello-world` container inside
   290    your development container
   291  
   292  Running the `make BIND_DIR=. shell` command mounted your local Docker repository source into
   293  your Docker container.
   294  
   295     > **Note**: Inspecting the `Dockerfile` shows a `COPY . /go/src/github.com/docker/docker` instruction, suggesting that dynamic code changes will _not_ be reflected in the container. However inspecting the `Makefile` shows that the current working directory _will_ be mounted via a `-v` volume mount.
   296  
   297  When you start to develop code though, you'll
   298  want to iterate code changes and builds inside the container. If you have
   299  followed this guide exactly, you have a bash shell running a development
   300  container.
   301  
   302  Try a simple code change and see it reflected in your container. For this
   303  example, you'll edit the help for the `attach` subcommand.
   304  
   305  1. If you don't have one, open a terminal in your local host.
   306  
   307  2. Make sure you are in your `moby-fork` repository.
   308  
   309     ```none
   310     $ pwd
   311     /Users/mary/go/src/github.com/moxiegirl/moby-fork
   312     ```
   313  
   314     Your location should be different because, at least, your username is
   315     different.
   316  
   317  3. Open the `cmd/dockerd/docker.go` file.
   318  
   319  4. Edit the command's help message.
   320  
   321     For example, you can edit this line:
   322  
   323     ```go
   324     Short:         "A self-sufficient runtime for containers.",
   325     ```
   326  
   327     And change it to this:
   328  
   329     ```go
   330     Short:         "A self-sufficient and really fun runtime for containers.",
   331     ```
   332  
   333  5. Save and close the `cmd/dockerd/docker.go` file.
   334  
   335  6. Go to your running docker development container shell.
   336  
   337  7. Rebuild the binary by using the command `hack/make.sh binary` in the docker development container shell.
   338  
   339  8. Stop Docker if it is running.
   340  
   341  9. Copy the binaries to **/usr/bin** by entering the following commands in the docker development container shell.
   342  
   343     ```
   344     hack/make.sh binary install-binary
   345     ```
   346  
   347  10. To view your change, run the `dockerd --help` command in the docker development container shell.
   348  
   349     ```bash
   350     # dockerd --help
   351  
   352     Usage:        dockerd COMMAND
   353  
   354     A self-sufficient and really fun runtime for containers.
   355  
   356     Options:
   357     ...
   358  
   359     ```
   360  
   361  You've just done the basic workflow for changing the Engine code base. You made
   362  your code changes in your feature branch. Then, you updated the binary in your
   363  development container and tried your change out. If you were making a bigger
   364  change, you might repeat or iterate through this flow several times.
   365  
   366  ## Where to go next
   367  
   368  Congratulations, you have successfully achieved Docker inception. You've had a
   369  small experience of the development process. You've set up your development
   370  environment and verified almost all the essential processes you need to
   371  contribute. Of course, before you start contributing, [you'll need to learn one
   372  more piece of the development process, the test framework](test.md).