go.etcd.io/etcd@v3.3.27+incompatible/Documentation/op-guide/container.md (about) 1 --- 2 title: Run etcd clusters inside containers 3 --- 4 5 The following guide shows how to run etcd with rkt and Docker using the [static bootstrap process](clustering.md#static). 6 7 ## rkt 8 9 ### Running a single node etcd 10 11 The following rkt run command will expose the etcd client API on port 2379 and expose the peer API on port 2380. 12 13 Use the host IP address when configuring etcd. 14 15 ``` 16 export NODE1=192.168.1.21 17 ``` 18 19 Trust the CoreOS [App Signing Key](https://coreos.com/security/app-signing-key/). 20 21 ``` 22 sudo rkt trust --prefix quay.io/coreos/etcd 23 # gpg key fingerprint is: 18AD 5014 C99E F7E3 BA5F 6CE9 50BD D3E0 FC8A 365E 24 ``` 25 26 Run the `v3.2` version of etcd or specify another release version. 27 28 ``` 29 sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380 30 ``` 31 32 List the cluster member. 33 34 ``` 35 etcdctl --endpoints=http://192.168.1.21:2379 member list 36 ``` 37 38 ### Running a 3 node etcd cluster 39 40 Setup a 3 node cluster with rkt locally, using the `-initial-cluster` flag. 41 42 ```sh 43 export NODE1=172.16.28.21 44 export NODE2=172.16.28.22 45 export NODE3=172.16.28.23 46 ``` 47 48 ``` 49 # node 1 50 sudo rkt run --net=default:IP=${NODE1} quay.io/coreos/etcd:v3.2 -- -name=node1 -advertise-client-urls=http://${NODE1}:2379 -initial-advertise-peer-urls=http://${NODE1}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE1}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380 51 52 # node 2 53 sudo rkt run --net=default:IP=${NODE2} quay.io/coreos/etcd:v3.2 -- -name=node2 -advertise-client-urls=http://${NODE2}:2379 -initial-advertise-peer-urls=http://${NODE2}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE2}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380 54 55 # node 3 56 sudo rkt run --net=default:IP=${NODE3} quay.io/coreos/etcd:v3.2 -- -name=node3 -advertise-client-urls=http://${NODE3}:2379 -initial-advertise-peer-urls=http://${NODE3}:2380 -listen-client-urls=http://0.0.0.0:2379 -listen-peer-urls=http://${NODE3}:2380 -initial-cluster=node1=http://${NODE1}:2380,node2=http://${NODE2}:2380,node3=http://${NODE3}:2380 57 ``` 58 59 Verify the cluster is healthy and can be reached. 60 61 ``` 62 ETCDCTL_API=3 etcdctl --endpoints=http://172.16.28.21:2379,http://172.16.28.22:2379,http://172.16.28.23:2379 endpoint health 63 ``` 64 65 ### DNS 66 67 Production clusters which refer to peers by DNS name known to the local resolver must mount the [host's DNS configuration](https://coreos.com/kubernetes/docs/latest/kubelet-wrapper.html#customizing-rkt-options). 68 69 ## Docker 70 71 In order to expose the etcd API to clients outside of Docker host, use the host IP address of the container. Please see [`docker inspect`](https://docs.docker.com/engine/reference/commandline/inspect) for more detail on how to get the IP address. Alternatively, specify `--net=host` flag to `docker run` command to skip placing the container inside of a separate network stack. 72 73 ### Running a single node etcd 74 75 Use the host IP address when configuring etcd: 76 77 ``` 78 export NODE1=192.168.1.21 79 ``` 80 81 Configure a Docker volume to store etcd data: 82 83 ``` 84 docker volume create --name etcd-data 85 export DATA_DIR="etcd-data" 86 ``` 87 88 Run the latest version of etcd: 89 90 ``` 91 REGISTRY=quay.io/coreos/etcd 92 # available from v3.2.5 93 REGISTRY=gcr.io/etcd-development/etcd 94 95 docker run \ 96 -p 2379:2379 \ 97 -p 2380:2380 \ 98 --volume=${DATA_DIR}:/etcd-data \ 99 --name etcd ${REGISTRY}:latest \ 100 /usr/local/bin/etcd \ 101 --data-dir=/etcd-data --name node1 \ 102 --initial-advertise-peer-urls http://${NODE1}:2380 --listen-peer-urls http://0.0.0.0:2380 \ 103 --advertise-client-urls http://${NODE1}:2379 --listen-client-urls http://0.0.0.0:2379 \ 104 --initial-cluster node1=http://${NODE1}:2380 105 ``` 106 107 List the cluster member: 108 109 ``` 110 etcdctl --endpoints=http://${NODE1}:2379 member list 111 ``` 112 113 ### Running a 3 node etcd cluster 114 115 ``` 116 REGISTRY=quay.io/coreos/etcd 117 # available from v3.2.5 118 REGISTRY=gcr.io/etcd-development/etcd 119 120 # For each machine 121 ETCD_VERSION=latest 122 TOKEN=my-etcd-token 123 CLUSTER_STATE=new 124 NAME_1=etcd-node-0 125 NAME_2=etcd-node-1 126 NAME_3=etcd-node-2 127 HOST_1=10.20.30.1 128 HOST_2=10.20.30.2 129 HOST_3=10.20.30.3 130 CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380 131 DATA_DIR=/var/lib/etcd 132 133 # For node 1 134 THIS_NAME=${NAME_1} 135 THIS_IP=${HOST_1} 136 docker run \ 137 -p 2379:2379 \ 138 -p 2380:2380 \ 139 --volume=${DATA_DIR}:/etcd-data \ 140 --name etcd ${REGISTRY}:${ETCD_VERSION} \ 141 /usr/local/bin/etcd \ 142 --data-dir=/etcd-data --name ${THIS_NAME} \ 143 --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \ 144 --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \ 145 --initial-cluster ${CLUSTER} \ 146 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} 147 148 # For node 2 149 THIS_NAME=${NAME_2} 150 THIS_IP=${HOST_2} 151 docker run \ 152 -p 2379:2379 \ 153 -p 2380:2380 \ 154 --volume=${DATA_DIR}:/etcd-data \ 155 --name etcd ${REGISTRY}:${ETCD_VERSION} \ 156 /usr/local/bin/etcd \ 157 --data-dir=/etcd-data --name ${THIS_NAME} \ 158 --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \ 159 --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \ 160 --initial-cluster ${CLUSTER} \ 161 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} 162 163 # For node 3 164 THIS_NAME=${NAME_3} 165 THIS_IP=${HOST_3} 166 docker run \ 167 -p 2379:2379 \ 168 -p 2380:2380 \ 169 --volume=${DATA_DIR}:/etcd-data \ 170 --name etcd ${REGISTRY}:${ETCD_VERSION} \ 171 /usr/local/bin/etcd \ 172 --data-dir=/etcd-data --name ${THIS_NAME} \ 173 --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \ 174 --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \ 175 --initial-cluster ${CLUSTER} \ 176 --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} 177 ``` 178 179 To run `etcdctl` using API version 3: 180 181 ``` 182 docker exec etcd /bin/sh -c "export ETCDCTL_API=3 && /usr/local/bin/etcdctl put foo bar" 183 ``` 184 185 ## Bare Metal 186 187 To provision a 3 node etcd cluster on bare-metal, the examples in the [baremetal repo](https://github.com/coreos/coreos-baremetal/tree/master/examples) may be useful. 188 189 ## Mounting a certificate volume 190 191 The etcd release container does not include default root certificates. To use HTTPS with certificates trusted by a root authority (e.g., for discovery), mount a certificate directory into the etcd container: 192 193 ``` 194 REGISTRY=quay.io/coreos/etcd 195 # available from v3.2.5 196 REGISTRY=docker://gcr.io/etcd-development/etcd 197 198 rkt run \ 199 --insecure-options=image \ 200 --volume etcd-ssl-certs-bundle,kind=host,source=/etc/ssl/certs/ca-certificates.crt \ 201 --mount volume=etcd-ssl-certs-bundle,target=/etc/ssl/certs/ca-certificates.crt \ 202 ${REGISTRY}:latest -- --name my-name \ 203 --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380 \ 204 --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379 \ 205 --discovery https://discovery.etcd.io/c11fbcdc16972e45253491a24fcf45e1 206 ``` 207 208 ``` 209 REGISTRY=quay.io/coreos/etcd 210 # available from v3.2.5 211 REGISTRY=gcr.io/etcd-development/etcd 212 213 docker run \ 214 -p 2379:2379 \ 215 -p 2380:2380 \ 216 --volume=/etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt \ 217 ${REGISTRY}:latest \ 218 /usr/local/bin/etcd --name my-name \ 219 --initial-advertise-peer-urls http://localhost:2380 --listen-peer-urls http://localhost:2380 \ 220 --advertise-client-urls http://localhost:2379 --listen-client-urls http://localhost:2379 \ 221 --discovery https://discovery.etcd.io/86a9ff6c8cb8b4c4544c1a2f88f8b801 222 ```