github.com/enmand/kubernetes@v1.2.0-alpha.0/docs/getting-started-guides/ubuntu-calico.md (about) 1 <!-- BEGIN MUNGE: UNVERSIONED_WARNING --> 2 3 <!-- BEGIN STRIP_FOR_RELEASE --> 4 5 <img src="http://kubernetes.io/img/warning.png" alt="WARNING" 6 width="25" height="25"> 7 <img src="http://kubernetes.io/img/warning.png" alt="WARNING" 8 width="25" height="25"> 9 <img src="http://kubernetes.io/img/warning.png" alt="WARNING" 10 width="25" height="25"> 11 <img src="http://kubernetes.io/img/warning.png" alt="WARNING" 12 width="25" height="25"> 13 <img src="http://kubernetes.io/img/warning.png" alt="WARNING" 14 width="25" height="25"> 15 16 <h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2> 17 18 If you are using a released version of Kubernetes, you should 19 refer to the docs that go with that version. 20 21 <strong> 22 The latest 1.0.x release of this document can be found 23 [here](http://releases.k8s.io/release-1.0/docs/getting-started-guides/ubuntu-calico.md). 24 25 Documentation for other releases can be found at 26 [releases.k8s.io](http://releases.k8s.io). 27 </strong> 28 -- 29 30 <!-- END STRIP_FOR_RELEASE --> 31 32 <!-- END MUNGE: UNVERSIONED_WARNING --> 33 Kubernetes Deployment On Bare-metal Ubuntu Nodes with Calico Networking 34 ------------------------------------------------ 35 36 ## Introduction 37 38 This document describes how to deploy Kubernetes on Ubuntu bare metal nodes with Calico Networking plugin. See [projectcalico.org](http://projectcalico.org) for more information on what Calico is, and [the calicoctl github](https://github.com/projectcalico/calico-docker) for more information on the command-line tool, `calicoctl`. 39 40 This guide will set up a simple Kubernetes cluster with a master and two nodes. We will start the following processes with systemd: 41 42 On the Master: 43 - `etcd` 44 - `kube-apiserver` 45 - `kube-controller-manager` 46 - `kube-scheduler` 47 - `calico-node` 48 49 On each Node: 50 - `kube-proxy` 51 - `kube-kubelet` 52 - `calico-node` 53 54 ## Prerequisites 55 56 1. This guide uses `systemd` and thus uses Ubuntu 15.04 which supports systemd natively. 57 2. All machines should have the latest docker stable version installed. At the time of writing, that is Docker 1.7.0. 58 - To install docker, follow [these instructions](https://docs.docker.com/installation/ubuntulinux/) 59 3. All hosts should be able to communicate with each other, as well as the internet, to download the necessary files. 60 4. This demo assumes that none of the hosts have been configured with any Kubernetes or Calico software yet. 61 62 ## Setup Master 63 64 First, get the sample configurations for this tutorial 65 66 ``` 67 wget https://github.com/Metaswitch/calico-kubernetes-ubuntu-demo/archive/master.tar.gz 68 tar -xvf master.tar.gz 69 ``` 70 71 ### Setup environment variables for systemd services on Master 72 73 Many of the sample systemd services provided rely on environment variables on a per-node basis. Here we'll edit those environment variables and move them into place. 74 75 1.) Copy the network-environment-template from the `master` directory for editing. 76 77 ``` 78 cp calico-kubernetes-ubuntu-demo-master/master/network-environment-template network-environment 79 ``` 80 81 2.) Edit `network-environment` to represent your current host's settings. 82 83 3.) Move the `network-environment` into `/etc` 84 85 ``` 86 sudo mv -f network-environment /etc 87 ``` 88 89 ### Install Kubernetes on Master 90 91 1.) Build & Install Kubernetes binaries 92 93 ``` 94 # Get the Kubernetes Source 95 wget https://github.com/kubernetes/kubernetes/releases/download/v1.0.3/kubernetes.tar.gz 96 97 # Untar it 98 tar -xf kubernetes.tar.gz 99 tar -xf kubernetes/server/kubernetes-server-linux-amd64.tar.gz 100 kubernetes/cluster/ubuntu/build.sh 101 102 # Add binaries to /usr/bin 103 sudo cp -f binaries/master/* /usr/bin 104 sudo cp -f binaries/kubectl /usr/bin 105 ``` 106 107 2.) Install the sample systemd processes settings for launching kubernetes services 108 109 ``` 110 sudo cp -f calico-kubernetes-ubuntu-demo-master/master/*.service /etc/systemd 111 sudo systemctl enable /etc/systemd/etcd.service 112 sudo systemctl enable /etc/systemd/kube-apiserver.service 113 sudo systemctl enable /etc/systemd/kube-controller-manager.service 114 sudo systemctl enable /etc/systemd/kube-scheduler.service 115 ``` 116 117 3.) Launch the processes. 118 119 ``` 120 sudo systemctl start etcd.service 121 sudo systemctl start kube-apiserver.service 122 sudo systemctl start kube-controller-manager.service 123 sudo systemctl start kube-scheduler.service 124 ``` 125 126 ### Install Calico on Master 127 128 In order to allow the master to route to pods on our nodes, we will launch the calico-node daemon on our master. This will allow it to learn routes over BGP from the other calico-node daemons in the cluster. The docker daemon should already be running before calico is started. 129 130 ``` 131 # Install the calicoctl binary, which will be used to launch calico 132 wget https://github.com/projectcalico/calico-docker/releases/download/v0.5.5/calicoctl 133 chmod +x calicoctl 134 sudo cp -f calicoctl /usr/bin 135 136 # Install and start the calico service 137 sudo cp -f calico-kubernetes-ubuntu-demo-master/master/calico-node.service /etc/systemd 138 sudo systemctl enable /etc/systemd/calico-node.service 139 sudo systemctl start calico-node.service 140 ``` 141 142 >Note: calico-node may take a few minutes on first boot while it downloads the calico-node docker image. 143 144 ## Setup Nodes 145 146 Perform these steps **once on each node**, ensuring you appropriately set the environment variables on each node 147 148 ### Setup environment variables for systemd services on the Node 149 150 1.) Get the sample configurations for this tutorial 151 152 ``` 153 wget https://github.com/Metaswitch/calico-kubernetes-ubuntu-demo/archive/master.tar.gz 154 tar -xvf master.tar.gz 155 ``` 156 157 2.) Copy the network-environment-template from the `node` directory 158 159 ``` 160 cp calico-kubernetes-ubuntu-demo-master/node/network-environment-template network-environment 161 ``` 162 163 3.) Edit `network-environment` to represent your current host's settings. 164 165 4.) Move `network-environment` into `/etc` 166 167 ``` 168 sudo mv -f network-environment /etc 169 ``` 170 171 ### Configure Docker on the Node 172 173 #### Create the veth 174 175 Instead of using docker's default interface (docker0), we will configure a new one to use desired IP ranges 176 177 ``` 178 sudo apt-get install -y bridge-utils 179 sudo brctl addbr cbr0 180 sudo ifconfig cbr0 up 181 sudo ifconfig cbr0 <IP>/24 182 ``` 183 184 > Replace \<IP\> with the subnet for this host's containers. Example topology: 185 186 Node | cbr0 IP 187 -------- | ------------- 188 node-1 | 192.168.1.1/24 189 node-2 | 192.168.2.1/24 190 node-X | 192.168.X.1/24 191 192 #### Start docker on cbr0 193 194 The Docker daemon must be started and told to use the already configured cbr0 instead of using the usual docker0, as well as disabling ip-masquerading and modification of the ip-tables. 195 196 1.) Edit the ubuntu-15.04 docker.service for systemd at: `/lib/systemd/system/docker.service` 197 198 2.) Find the line that reads `ExecStart=/usr/bin/docker -d -H fd://` and append the following flags: `--bridge=cbr0 --iptables=false --ip-masq=false` 199 200 3.) Reload systemctl and restart docker. 201 202 ``` 203 sudo systemctl daemon-reload 204 sudo systemctl restart docker 205 ``` 206 207 ### Install Calico on the Node 208 209 1.) Install Calico 210 211 ``` 212 # Get the calicoctl binary 213 wget https://github.com/projectcalico/calico-docker/releases/download/v0.5.5/calicoctl 214 chmod +x calicoctl 215 sudo cp -f calicoctl /usr/bin 216 217 # Start calico on this node 218 sudo cp calico-kubernetes-ubuntu-demo-master/node/calico-node.service /etc/systemd 219 sudo systemctl enable /etc/systemd/calico-node.service 220 sudo systemctl start calico-node.service 221 ``` 222 223 >The calico-node service will automatically get the kubernetes-calico plugin binary and install it on the host system. 224 225 2.) Use calicoctl to add an IP pool. We must specify the IP and port that the master's etcd is listening on. 226 **NOTE: This step only needs to be performed once per Kubernetes deployment, as it covers all the node's IP ranges.** 227 228 ``` 229 ETCD_AUTHORITY=<MASTER_IP>:4001 calicoctl pool add 192.168.0.0/16 230 ``` 231 232 ### Install Kubernetes on the Node 233 234 1.) Build & Install Kubernetes binaries 235 236 ``` 237 # Get the Kubernetes Source 238 wget https://github.com/kubernetes/kubernetes/releases/download/v1.0.3/kubernetes.tar.gz 239 240 # Untar it 241 tar -xf kubernetes.tar.gz 242 tar -xf kubernetes/server/kubernetes-server-linux-amd64.tar.gz 243 kubernetes/cluster/ubuntu/build.sh 244 245 # Add binaries to /usr/bin 246 sudo cp -f binaries/minion/* /usr/bin 247 248 # Get the iptables based kube-proxy reccomended for this demo 249 wget https://github.com/projectcalico/calico-kubernetes/releases/download/v0.1.1/kube-proxy 250 sudo cp kube-proxy /usr/bin/ 251 sudo chmod +x /usr/bin/kube-proxy 252 ``` 253 254 2.) Install and launch the sample systemd processes settings for launching Kubernetes services 255 256 ``` 257 sudo cp calico-kubernetes-ubuntu-demo-master/node/kube-proxy.service /etc/systemd/ 258 sudo cp calico-kubernetes-ubuntu-demo-master/node/kube-kubelet.service /etc/systemd/ 259 sudo systemctl enable /etc/systemd/kube-proxy.service 260 sudo systemctl enable /etc/systemd/kube-kubelet.service 261 sudo systemctl start kube-proxy.service 262 sudo systemctl start kube-kubelet.service 263 ``` 264 265 >*You may want to consider checking their status after to ensure everything is running* 266 267 ## Install the DNS Addon 268 269 Most Kubernetes deployments will require the DNS addon for service discovery. For more on DNS service discovery, check [here](../../cluster/addons/dns/). 270 271 The config repository for this guide comes with manifest files to start the DNS addon. To install DNS, do the following on your Master node. 272 273 Replace `<MASTER_IP>` in `calico-kubernetes-ubuntu-demo-master/dns/skydns-rc.yaml` with your Master's IP address. Then, create `skydns-rc.yaml` and `skydns-svc.yaml` using `kubectl create -f <FILE>`. 274 275 ## Launch other Services With Calico-Kubernetes 276 277 At this point, you have a fully functioning cluster running on kubernetes with a master and 2 nodes networked with Calico. You can now follow any of the [standard documentation](../../examples/) to set up other services on your cluster. 278 279 ## Connectivity to outside the cluster 280 281 With this sample configuration, because the containers have private `192.168.0.0/16` IPs, you will need NAT to allow connectivity between containers and the internet. However, in a full datacenter deployment, NAT is not always necessary, since Calico can peer with the border routers over BGP. 282 283 ### NAT on the nodes 284 285 The simplest method for enabling connectivity from containers to the internet is to use an iptables masquerade rule. This is the standard mechanism [recommended](../../docs/admin/networking.md#google-compute-engine-gce) in the Kubernetes GCE environment. 286 287 We need to NAT traffic that has a destination outside of the cluster. Internal traffic includes the master/nodes, and the container IP pools. A suitable masquerade chain would follow the pattern below, replacing the following variables: 288 - `CONTAINER_SUBNET`: The cluster-wide subnet from which container IPs are chosen. All cbr0 bridge subnets fall within this range. The above example uses `192.168.0.0/16`. 289 - `KUBERNETES_HOST_SUBNET`: The subnet from which Kubernetes node / master IP addresses have been chosen. 290 - `HOST_INTERFACE`: The interface on the Kubernetes node which is used for external connectivity. The above example uses `eth0` 291 292 ``` 293 sudo iptables -t nat -N KUBE-OUTBOUND-NAT 294 sudo iptables -t nat -A KUBE-OUTBOUND-NAT -d <CONTAINER_SUBNET> -o <HOST_INTERFACE> -j RETURN 295 sudo iptables -t nat -A KUBE-OUTBOUND-NAT -d <KUBERNETES_HOST_SUBNET> -o <HOST_INTERFACE> -j RETURN 296 sudo iptables -t nat -A KUBE-OUTBOUND-NAT -j MASQUERADE 297 sudo iptables -t nat -A POSTROUTING -j KUBE-OUTBOUND-NAT 298 ``` 299 300 This chain should be applied on the master and all nodes. In production, these rules should be persisted, e.g. with `iptables-persistent`. 301 302 ### NAT at the border router 303 304 In a datacenter environment, it is recommended to configure Calico to peer with the border routers over BGP. This means that the container IPs will be routable anywhere in the datacenter, and so NAT is not needed on the nodes (though it may be enabled at the datacenter edge to allow outbound-only internet connectivity). 305 306 307 <!-- BEGIN MUNGE: GENERATED_ANALYTICS --> 308 [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/getting-started-guides/ubuntu-calico.md?pixel)]() 309 <!-- END MUNGE: GENERATED_ANALYTICS -->