go.etcd.io/etcd@v3.3.27+incompatible/Documentation/op-guide/container.md (about)

     1  ---
     2  title: Run etcd clusters inside containers
     3  ---
     4  
     5  The following guide shows how to run etcd with rkt and Docker using the [static bootstrap process](clustering.md#static).
     6  
     7  ## rkt
     8  
     9  ### Running a single node etcd
    10  
    11  The following rkt run command will expose the etcd client API on port 2379 and expose the peer API on port 2380.
    12  
    13  Use the host IP address when configuring etcd.
    14  
    15  ```
    16  export NODE1=192.168.1.21
    17  ```
    18  
    19  Trust the CoreOS [App Signing Key](https://coreos.com/security/app-signing-key/).
    20  
    21  ```
    22  sudo rkt trust --prefix quay.io/coreos/etcd
    23  # gpg key fingerprint is: 18AD 5014 C99E F7E3 BA5F  6CE9 50BD D3E0 FC8A 365E
    24  ```
    25  
    26  Run the `v3.2` version of etcd or specify another release version.
    27  
    28  ```
    29  sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380
    30  ```
    31  
    32  List the cluster member.
    33  
    34  ```
    35  etcdctl --endpoints=http://192.168.1.21:2379 member list
    36  ```
    37  
    38  ### Running a 3 node etcd cluster
    39  
    40  Setup a 3 node cluster with rkt locally, using the `-initial-cluster` flag.
    41  
    42  ```sh
    43  export NODE1=172.16.28.21
    44  export NODE2=172.16.28.22
    45  export NODE3=172.16.28.23
    46  ```
    47  
    48  ```
    49  # node 1
    50  sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380
    51  
    52  # node 2
    53  sudo rkt run --net=default:IP=${NODE2} quay.io/coreos/etcd:v3.2 -- -name=node2 -advertise-client-urls=http://${NODE2}:2379 -initial-advertise-peer-urls=http://${NODE2}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE2}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380
    54  
    55  # node 3
    56  sudo rkt run --net=default:IP=${NODE3} quay.io/coreos/etcd:v3.2 -- -name=node3 -advertise-client-urls=http://${NODE3}:2379 -initial-advertise-peer-urls=http://${NODE3}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE3}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380
    57  ```
    58  
    59  Verify the cluster is healthy and can be reached.
    60  
    61  ```
    62  ETCDCTL_API=3 etcdctl --endpoints=http://172.16.28.21:2379,http://172.16.28.22:2379,http://172.16.28.23:2379 endpoint health
    63  ```
    64  
    65  ### DNS
    66  
    67  Production clusters which refer to peers by DNS name known to the local resolver must mount the [host's DNS configuration](https://coreos.com/kubernetes/docs/latest/kubelet-wrapper.html#customizing-rkt-options).
    68  
    69  ## Docker
    70  
    71  In order to expose the etcd API to clients outside of Docker host, use the host IP address of the container. Please see [`docker inspect`](https://docs.docker.com/engine/reference/commandline/inspect) for more detail on how to get the IP address. Alternatively, specify `--net=host` flag to `docker run` command to skip placing the container inside of a separate network stack.
    72  
    73  ### Running a single node etcd
    74  
    75  Use the host IP address when configuring etcd:
    76  
    77  ```
    78  export NODE1=192.168.1.21
    79  ```
    80  
    81  Configure a Docker volume to store etcd data:
    82  
    83  ```
    84  docker volume create --name etcd-data
    85  export DATA_DIR="etcd-data"
    86  ```
    87  
    88  Run the latest version of etcd:
    89  
    90  ```
    91  REGISTRY=quay.io/coreos/etcd
    92  # available from v3.2.5
    93  REGISTRY=gcr.io/etcd-development/etcd
    94  
    95  docker run \
    96    -p 2379:2379 \
    97    -p 2380:2380 \
    98    --volume=${DATA_DIR}:/etcd-data \
    99    --name etcd ${REGISTRY}:latest \
   100    /usr/local/bin/etcd \
   101    --data-dir=/etcd-data --name node1 \
   102    --initial-advertise-peer-urls http://${NODE1}:2380 --listen-peer-urls http://0.0.0.0:2380 \
   103    --advertise-client-urls http://${NODE1}:2379 --listen-client-urls http://0.0.0.0:2379 \
   104    --initial-cluster node1=http://${NODE1}:2380
   105  ```
   106  
   107  List the cluster member:
   108  
   109  ```
   110  etcdctl --endpoints=http://${NODE1}:2379 member list
   111  ```
   112  
   113  ### Running a 3 node etcd cluster
   114  
   115  ```
   116  REGISTRY=quay.io/coreos/etcd
   117  # available from v3.2.5
   118  REGISTRY=gcr.io/etcd-development/etcd
   119  
   120  # For each machine
   121  ETCD_VERSION=latest
   122  TOKEN=my-etcd-token
   123  CLUSTER_STATE=new
   124  NAME_1=etcd-node-0
   125  NAME_2=etcd-node-1
   126  NAME_3=etcd-node-2
   127  HOST_1=10.20.30.1
   128  HOST_2=10.20.30.2
   129  HOST_3=10.20.30.3
   130  CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
   131  DATA_DIR=/var/lib/etcd
   132  
   133  # For node 1
   134  THIS_NAME=${NAME_1}
   135  THIS_IP=${HOST_1}
   136  docker run \
   137    -p 2379:2379 \
   138    -p 2380:2380 \
   139    --volume=${DATA_DIR}:/etcd-data \
   140    --name etcd ${REGISTRY}:${ETCD_VERSION} \
   141    /usr/local/bin/etcd \
   142    --data-dir=/etcd-data --name ${THIS_NAME} \
   143    --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
   144    --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
   145    --initial-cluster ${CLUSTER} \
   146    --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
   147  
   148  # For node 2
   149  THIS_NAME=${NAME_2}
   150  THIS_IP=${HOST_2}
   151  docker run \
   152    -p 2379:2379 \
   153    -p 2380:2380 \
   154    --volume=${DATA_DIR}:/etcd-data \
   155    --name etcd ${REGISTRY}:${ETCD_VERSION} \
   156    /usr/local/bin/etcd \
   157    --data-dir=/etcd-data --name ${THIS_NAME} \
   158    --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
   159    --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
   160    --initial-cluster ${CLUSTER} \
   161    --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
   162  
   163  # For node 3
   164  THIS_NAME=${NAME_3}
   165  THIS_IP=${HOST_3}
   166  docker run \
   167    -p 2379:2379 \
   168    -p 2380:2380 \
   169    --volume=${DATA_DIR}:/etcd-data \
   170    --name etcd ${REGISTRY}:${ETCD_VERSION} \
   171    /usr/local/bin/etcd \
   172    --data-dir=/etcd-data --name ${THIS_NAME} \
   173    --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
   174    --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
   175    --initial-cluster ${CLUSTER} \
   176    --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
   177  ```
   178  
   179  To run `etcdctl` using API version 3:
   180  
   181  ```
   182  docker exec etcd /bin/sh -c "export ETCDCTL_API=3 && /usr/local/bin/etcdctl put foo bar"
   183  ```
   184  
   185  ## Bare Metal
   186  
   187  To provision a 3 node etcd cluster on bare-metal, the examples in the [baremetal repo](https://github.com/coreos/coreos-baremetal/tree/master/examples) may be useful.
   188  
   189  ## Mounting a certificate volume
   190  
   191  The etcd release container does not include default root certificates. To use HTTPS with certificates trusted by a root authority (e.g., for discovery), mount a certificate directory into the etcd container:
   192  
   193  ```
   194  REGISTRY=quay.io/coreos/etcd
   195  # available from v3.2.5
   196  REGISTRY=docker://gcr.io/etcd-development/etcd
   197  
   198  rkt run \
   199    --insecure-options=image \
   200    --volume etcd-ssl-certs-bundle,kind=host,source=/etc/ssl/certs/ca-certificates.crt \
   201    --mount volume=etcd-ssl-certs-bundle,target=/etc/ssl/certs/ca-certificates.crt \
   202    ${REGISTRY}:latest -- --name my-name \
   203    --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380 \
   204    --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379 \
   205    --discovery https://discovery.etcd.io/c11fbcdc16972e45253491a24fcf45e1
   206  ```
   207  
   208  ```
   209  REGISTRY=quay.io/coreos/etcd
   210  # available from v3.2.5
   211  REGISTRY=gcr.io/etcd-development/etcd
   212  
   213  docker run \
   214    -p 2379:2379 \
   215    -p 2380:2380 \
   216    --volume=/etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt \
   217    ${REGISTRY}:latest \
   218    /usr/local/bin/etcd --name my-name \
   219    --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380 \
   220    --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379 \
   221    --discovery https://discovery.etcd.io/86a9ff6c8cb8b4c4544c1a2f88f8b801
   222  ```