github.com/alibaba/sealer@v0.8.6-0.20220430115802-37a2bdaa8173/docs/site/src/zh/getting-started/build-cloudimage.md (about) 1 # 构建集群镜像 2 3 集群镜像的构建过程与构建Docker镜像很类似，通过定义一个Kubefile文件，文件中定义一些指令，build结束就可以把集群启动依赖的所有文件和docker镜像打包成集群镜像了。 4 5 ## 构建模式 6 7 目前sealer支持以下构建模式： 8 9 * 默认模式，lite build, 这种方式构建过程中不需要启动一个k8s集群，通过解析用户提供的yaml文件或者helm chart或者用户自定义的imageList来拉取集群中依赖的容易镜像 10 11 ### lite build 模式 12 13 这是默认的构建模式，比较轻量快速，会解析用户的yaml文件或者helm chart 分析出里面包含的容器镜像，并拉取下来存储到集群镜像自带的registry中 14 15 Kubefile: 16 17 ```shell 18 FROM kubernetes:v1.19.8 19 COPY imageList manifests 20 COPY apollo charts 21 COPY helm /bin 22 CMD helm install charts/apollo 23 COPY recommended.yaml manifests 24 CMD kubectl apply -f manifests/recommended.yaml 25 ``` 26 27 * `manifests` 目录: 这是一个特殊的目录，sealer build的时候会解析这个目录下面的所有yaml文件并把里面的容器镜像地址提取出来，然后拉取。用户标准的kubernetes yaml不放在这个目录的话不会处理。 28 * `charts` 目录: 这也是一个特殊目录，sealer会执行helm template的能力，然后提取chart中的容器镜像地址，拉取并存储到集群镜像中。chart不拷贝到这个目录下不处理。 29 * `manifests/imageList`: 这是个特殊的文件里面是其它需要拉取的镜像地址列表，比如镜像地址在CRD中sealer解析不到，那就需要手动配置到这个文件中。 30 31 imageList 内容示例： 32 33 ``` 34 nginx:latest 35 mysql:5.6 36 ``` 37 38 Build集群镜像： 39 40 ```shell 41 sealer build -t my-cluster:v1.19.9 . 42 ``` 43 44 ## 私有镜像仓库 45 46 集群镜像也可以被推送到docker registry中 47 48 ```shell 49 sealer login registry.cn-qingdao.aliyuncs.com -u username -p password 50 sealer push registry.cn-qingdao.aliyuncs.com/sealer-io/kuberentes:v1.19.8 51 sealer pull registry.cn-qingdao.aliyuncs.com/sealer-io/kuberentes:v1.19.8 52 ``` 53 54 集群镜像中自带一个docker registry, 所有容器镜像会存储在这个registry中，可以自定义该registry的一些配置： 55 56 [registry config](../../../../design/docker-image-cache.md) 57 58 编辑好 `registry_config.yaml`配置文件，然后在Kubefile中进行overwrite: 59 60 ```shell 61 FROM kubernetes:v1.19.8 62 COPY registry_config.yaml etc/ 63 ``` 64 65 ## 自定义 kubeadm 配置 66 67 Sealer 会把用户自定义的kubeadm配置文件与默认文件merge, 集群镜像中 $Rootfs/etc/kubeadm.yml 文件为镜像默认的Kubeadm配置， 68 69 用户可以直接在镜像中覆盖它，或者在Clusterfile中定义kubeadm配置文件，执行时会把Clusterfile中的kubeadm配置与镜像中的合并。 70 71 这里合并时只覆盖对应字段，而不是全部替换，比如你只关心 `bindPort` 这一个参数，那只需要在Clusterfile中配置这一个字段即可，不用写全量配置。 72 73 ### 如自定义 Docker Unix socket. 74 75 1. 修改 kubeadm init configuration: 76 77 ```yaml 78 apiVersion: kubeadm.k8s.io/v1beta2 79 kind: InitConfiguration 80 localAPIEndpoint: 81 bindPort: 6443 82 nodeRegistration: 83 criSocket: /var/run/dockershim.sock 84 ``` 85 86 2. 修改 kubeadm join configuration: 87 88 ```yaml 89 apiVersion: kubeadm.k8s.io/v1beta2 90 kind: JoinConfiguration 91 caCertPath: /etc/kubernetes/pki/ca.crt 92 discovery: 93 timeout: 5m0s 94 nodeRegistration: 95 criSocket: /var/run/dockershim.sock 96 controlPlane: 97 localAPIEndpoint: 98 bindPort: 6443 99 ``` 100 101 3. 把该文件命名为kubeadm.yml, 构建时拷贝到etc/下即可 102 103 ```yaml 104 #Kubefile 105 FROM kubernetes-clusterv2:v1.19.8 106 COPY kubeadm.yml etc 107 ``` 108 109 这里注意kubeadm.yml中也不需要全量配置，只需要配置对应关心的字段，sealer会与默认配置合并。 110 111 merge规则：Clusterfile中的配置 > 集群镜像中的配置 > 默认kubeadm配置(硬编码在代码中) 112 113 > sealer build -t user-define-kubeadm-kubernetes:v1.19.8 . 114 115 ## 默认kubeadm 配置文件全内容: 116 117 kubeadm.yml： 118 119 ```yaml 120 apiVersion: kubeadm.k8s.io/v1beta2 121 kind: InitConfiguration 122 localAPIEndpoint: 123 # advertiseAddress: 192.168.2.110 124 bindPort: 6443 125 nodeRegistration: 126 criSocket: /var/run/dockershim.sock 127 128 --- 129 apiVersion: kubeadm.k8s.io/v1beta2 130 kind: ClusterConfiguration 131 kubernetesVersion: v1.19.8 132 #controlPlaneEndpoint: "apiserver.cluster.local:6443" 133 imageRepository: sea.hub:5000/library 134 networking: 135 # dnsDomain: cluster.local 136 podSubnet: 100.64.0.0/10 137 serviceSubnet: 10.96.0.0/22 138 apiServer: 139 # certSANs: 140 # - 127.0.0.1 141 # - apiserver.cluster.local 142 # - aliyun-inc.com 143 # - 10.0.0.2 144 # - 10.103.97.2 145 extraArgs: 146 # etcd-servers: https://192.168.2.110:2379 147 feature-gates: TTLAfterFinished=true,EphemeralContainers=true 148 audit-policy-file: "/etc/kubernetes/audit-policy.yml" 149 audit-log-path: "/var/log/kubernetes/audit.log" 150 audit-log-format: json 151 audit-log-maxbackup: '10' 152 audit-log-maxsize: '100' 153 audit-log-maxage: '7' 154 enable-aggregator-routing: 'true' 155 extraVolumes: 156 - name: "audit" 157 hostPath: "/etc/kubernetes" 158 mountPath: "/etc/kubernetes" 159 pathType: DirectoryOrCreate 160 - name: "audit-log" 161 hostPath: "/var/log/kubernetes" 162 mountPath: "/var/log/kubernetes" 163 pathType: DirectoryOrCreate 164 - name: localtime 165 hostPath: /etc/localtime 166 mountPath: /etc/localtime 167 readOnly: true 168 pathType: File 169 controllerManager: 170 extraArgs: 171 feature-gates: TTLAfterFinished=true,EphemeralContainers=true 172 experimental-cluster-signing-duration: 876000h 173 extraVolumes: 174 - hostPath: /etc/localtime 175 mountPath: /etc/localtime 176 name: localtime 177 readOnly: true 178 pathType: File 179 scheduler: 180 extraArgs: 181 feature-gates: TTLAfterFinished=true,EphemeralContainers=true 182 extraVolumes: 183 - hostPath: /etc/localtime 184 mountPath: /etc/localtime 185 name: localtime 186 readOnly: true 187 pathType: File 188 etcd: 189 local: 190 extraArgs: 191 listen-metrics-urls: http://0.0.0.0:2381 192 --- 193 apiVersion: kubeproxy.config.k8s.io/v1alpha1 194 kind: KubeProxyConfiguration 195 mode: "ipvs" 196 ipvs: 197 excludeCIDRs: 198 - "10.103.97.2/32" 199 200 --- 201 apiVersion: kubelet.config.k8s.io/v1beta1 202 kind: KubeletConfiguration 203 authentication: 204 anonymous: 205 enabled: false 206 webhook: 207 cacheTTL: 2m0s 208 enabled: true 209 x509: 210 clientCAFile: /etc/kubernetes/pki/ca.crt 211 authorization: 212 mode: Webhook 213 webhook: 214 cacheAuthorizedTTL: 5m0s 215 cacheUnauthorizedTTL: 30s 216 cgroupDriver: 217 cgroupsPerQOS: true 218 clusterDomain: cluster.local 219 configMapAndSecretChangeDetectionStrategy: Watch 220 containerLogMaxFiles: 5 221 containerLogMaxSize: 10Mi 222 contentType: application/vnd.kubernetes.protobuf 223 cpuCFSQuota: true 224 cpuCFSQuotaPeriod: 100ms 225 cpuManagerPolicy: none 226 cpuManagerReconcilePeriod: 10s 227 enableControllerAttachDetach: true 228 enableDebuggingHandlers: true 229 enforceNodeAllocatable: 230 - pods 231 eventBurst: 10 232 eventRecordQPS: 5 233 evictionHard: 234 imagefs.available: 15% 235 memory.available: 100Mi 236 nodefs.available: 10% 237 nodefs.inodesFree: 5% 238 evictionPressureTransitionPeriod: 5m0s 239 failSwapOn: true 240 fileCheckFrequency: 20s 241 hairpinMode: promiscuous-bridge 242 healthzBindAddress: 127.0.0.1 243 healthzPort: 10248 244 httpCheckFrequency: 20s 245 imageGCHighThresholdPercent: 85 246 imageGCLowThresholdPercent: 80 247 imageMinimumGCAge: 2m0s 248 iptablesDropBit: 15 249 iptablesMasqueradeBit: 14 250 kubeAPIBurst: 10 251 kubeAPIQPS: 5 252 makeIPTablesUtilChains: true 253 maxOpenFiles: 1000000 254 maxPods: 110 255 nodeLeaseDurationSeconds: 40 256 nodeStatusReportFrequency: 10s 257 nodeStatusUpdateFrequency: 10s 258 oomScoreAdj: -999 259 podPidsLimit: -1 260 port: 10250 261 registryBurst: 10 262 registryPullQPS: 5 263 rotateCertificates: true 264 runtimeRequestTimeout: 2m0s 265 serializeImagePulls: true 266 staticPodPath: /etc/kubernetes/manifests 267 streamingConnectionIdleTimeout: 4h0m0s 268 syncFrequency: 1m0s 269 volumeStatsAggPeriod: 1m0s 270 --- 271 apiVersion: kubeadm.k8s.io/v1beta2 272 kind: JoinConfiguration 273 caCertPath: /etc/kubernetes/pki/ca.crt 274 discovery: 275 timeout: 5m0s 276 nodeRegistration: 277 criSocket: /var/run/dockershim.sock 278 controlPlane: 279 localAPIEndpoint: 280 bindPort: 6443 281 ```