github.com/jingruilea/kubeedge@v1.2.0-beta.0.0.20200410162146-4bb8902b3879/docs/proposals/cri.md (about) 1 --- 2 title: CRI Design for edged 3 authors: 4 - "@gpinaik" 5 approvers: 6 - "@qizha" 7 - "@CindyXing" 8 - "@Baoqiang-Zhang" 9 - "@m1093782566" 10 creation-date: 2019-04-19 11 last-updated: 2019-04-21 12 status: implementable 13 --- 14 15 # CRI Support in edged 16 17 * [CRI Support in edged](#CRI-Support-in-edged) 18 * [Motivation](#motivation) 19 * [Goals](#goals) 20 * [Non\-goals](#non-goals) 21 * [Proposal](#proposal) 22 * [Use Cases](#use-cases) 23 * [High Level Design](#high-level-design) 24 * [Edged with CRI support](#edged-with-cri-support) 25 * [Low Level Design](#low-level-design) 26 * [Configuration parameters](#configuration-parameters) 27 * [Data structure modifications](#data-structure-modifications) 28 * [Edged object creation modifications](#edged-object-creation-modifications) 29 * [Runtime dependent module modifications](#runtime-dependent-module-modifications) 30 * [Runtime dependent functional modifications](#runtime-dependent-functional-modifications) 31 * [Open Questions](#open-questions) 32 33 34 ## Motivation 35 This proposal addresses the Container Runtime Interface support in edged to enable the following 36 1. Support light weight container runtimes on resource constrained edge node which are unable to run the existing docker runtime 37 2. Support multiple container runtimes like docker, containerd, cri-o etc on the edge node. 38 39 ### Goals 40 CRI support in edged must: 41 * support multiple runtimes like docker, contianerd, cri-o etc. 42 * Support for corresponding CNI with pause container and IP will be considered later 43 44 ### Non-goals 45 46 * Automatic detection of container runtimes and its selection. 47 48 ## Proposal 49 50 Currently Kubernetes kubelet CRI supports container runtimes like containerd, cri-o etc and support for docker runtime is 51 provided using dockershim as well. However going forward even docker runtime will be supported through only CRI. However 52 currently kubeedge edged supports only docker runtime using the legacy dockertools. Hence we propose to support multiple 53 container runtime in kubeedge edged as follows 54 1. Include CRI support as in kubernetes kubelet to support contianerd, cri-o etc 55 2. Continue with docker runtime support using legacy dockertools until CRI support for the same is available i.e. support 56 for docker runtime using dockershim is not considered in edged 57 58 59 ### Use Cases 60 61 * Customer can run light weight container runtime on resource constrained edge node that cannot run the existing docker runtime 62 * Customer has the option to choose from multiple container runtimes on his edge platform 63 64 65 ## High Level Design 66 67 ### Edged with CRI support 68 <img src="../images/edged/docker-cri.png"> 69 70 ## Low Level Design 71 72 ### Configuration parameters 73 74 The following configuration parameters need to be added 75 76 No | Parameter | Type | Values | Description 77 ---|---|---|---|--- 78 1 | runtimeType | string | docker/remote | Runtime name 79 2 | remoteRuntimeEndpoint | string | /var/run/*.sock | Endpoint of remote runtime service 80 3 | remoteImageEndpoint | string | same as remoteRuntimeEndpoint if not specified | Endpoint of remote image service 81 4 | RuntimeRequestTimeout | Duration | time value | timeout for all runtime request 82 5 | PodSandboxImage | string | image name | Image used for pause container in sandbox 83 84 ```go 85 type Config struct { 86 .... 87 runtimeType string 88 remoteRuntimeEndpoint string 89 remoteImageEndpoint string 90 RuntimeRequestTimeout metav1.Duration 91 PodSandboxImage string 92 .... 93 } 94 ``` 95 ### Data structure modifications 96 97 The edged data strcuture needs to include the remote runtime and runtime name. Also need to add os interface, pod cache and container life cycle manager parameters required for initializing and executing remote runtime. 98 99 ```go 100 //Define edged 101 type edged struct { 102 .... 103 containerRuntimeName string 104 // Container runtime 105 containerRuntime kubecontainer.Runtime 106 podCache kubecontainer.Cache 107 os kubecontainer.OSInterface 108 clcm clcm.ContainerLifecycleManager 109 .... 110 } 111 112 ``` 113 114 ### Edged object creation modifications 115 116 The existing newEdged() function needs to modified include creating CRI runtime object based on the runtime type including 117 creations of objects for runtime and image services. However the existing edged does not provide the support for all the 118 parameters required to create the CRI runtime object and default parameters need to be considered for the same like Image GC manager, Container GC manager, Volume manager and container lifecycle manager (clcm) 119 120 ```go 121 122 //newEdged creates new edged object and initialises it 123 func newEdged() (*edged, error) { 124 conf := getConfig() 125 ...... 126 127 switch based on runtimeType { 128 case DockerContainerRuntime: 129 Create runtime based on docker tools 130 Set containerRuntimeName to DockerContainerRuntime 131 Initialize Container GC, Image GC and Volume Plugin Manager accordingly 132 133 case RemoteContainerRuntime: 134 Set remoteImageEndpoint same as remoteRuntimeEndpoint if not explicitly specified 135 Initialize the following required for initializing remote runtime 136 containerRefManager 137 httpClient 138 runtimeService 139 imageService 140 clcm 141 machineInfo with only memory capacity 142 Initialize Generic Runtime Manager 143 Set ContainerRuntimeName to RemoteContainerRuntime 144 Set runtimeService 145 Initialize Container GC, Image GC and Volume Plugin Manager accordingly 146 147 default: 148 unsupported CRI runtime 149 } 150 ..... 151 ..... 152 } 153 154 //Function to get CRI runtime and image service 155 func getRuntimeAndImageServices(remoteRuntimeEndpoint string, remoteImageEndpoint string, runtimeRequestTimeout metav1.Duration) (internalapi.RuntimeService, internalapi.ImageManagerService, error) { 156 Initialize Remote Runtime Service 157 Initialize Remote Image Service 158 } 159 ``` 160 The function to read the configuration needs to be modified to include the parameters required for remote runtime. By default 161 docker runtime shall be used and also default values for the parameters need to be provided if the parameters are not provided 162 in the configuration file. 163 164 ``` 165 // Get Config 166 func getConfig() *Config { 167 .... 168 Check for rumtime type and if not provided set to docker 169 if runtimeType is remote 170 Get edged memory capacity and set to default of 2G if not provided 171 Get remote runtime endpoint and set to /var/run/containerd/container.sock if not provided 172 Get remote image endpoint and set same as remote runtime endpoint of not provided 173 Get runtime Request Timeout and set to default of 2 min if not provided 174 Get PodSandboxImage and set to k8s.gcr.io/pause 175 .... 176 } 177 178 ``` 179 ### Runtime dependent module modifications 180 The following modules which are dependent on the runtime needs to be handled during edged start based on docker runtime or 181 remote CRI runtime. 182 1. Volume Manager 183 2. PLEG 184 185 ```go 186 func (e *edged) Start(c *context.Context) { 187 .... 188 switch based on runtime type { 189 case DockerContainerRuntime: 190 Initialize volume manager based on dockertools 191 Initialize PLEG based on dockertools 192 193 case RemoteContainerRuntime: 194 Initialize volume manager based on remote runtime 195 Initialize PLEG based on remote runtime 196 } 197 .... 198 199 } 200 ``` 201 202 ### Runtime dependent functional modifications 203 204 The following functionalaties which are based on the docker runtime need to be modified to handle the CRI runtimes as well 205 206 ```go 207 func (e *edged) initializeModules() error { 208 .... 209 switch based on runtime type { 210 case DockerContainerRuntime: 211 Start with docker runtime 212 213 case RemoteContainerRuntime: 214 Start with remote runtime 215 .... 216 } 217 func (e *edged) consumePodAddition(namespacedName *types.NamespacedName) error { 218 .... 219 switch based on runtime type { 220 case DockerContainerRuntime: 221 Ensure iamge exists for docker runtime 222 Start pod with docker runtime 223 224 case RemoteContainerRuntime: 225 Get current status from pod cache 226 Sync pod with remote runtime 227 .... 228 } 229 230 func (e *edged) consumePodDeletion(namespacedName *types.NamespacedName) error { 231 .... 232 switch based on runtime type { 233 case DockerContainerRuntime: 234 TerminatePod with docker runtime 235 236 case RemoteContainerRuntime: 237 KillPod with remote runtime 238 } 239 240 .... 241 } 242 243 244 func (e *edged) addPod(obj interface{}) { 245 .... 246 UpdatePluginResources for only docker runtime 247 .... 248 } 249 250 func (e *edged) HandlePodCleanups() error { 251 .... 252 switch switch based on runtime type { 253 case DockerContainerRuntime: 254 GetPods for docker runtime 255 256 case RemoteContainerRuntime: 257 GetPods for remote runtime 258 } 259 .... 260 .... 261 } 262 263 ``` 264 265 Existing PLEG wrapper needs to be modified to handle remote runtime. Addition new function needs to be provided to initialize 266 PLEG and update pod status 267 268 ``` 269 func NewGenericLifecycleRemote(runtime kubecontainer.Runtime, probeManager prober.Manager, channelCapacity int, 270 relistPeriod time.Duration, podManager podmanager.Manager, statusManager status.Manager, podCache kubecontainer.Cache, clock clock.Clock, iface string) pleg.PodLifecycleEventGenerator { 271 Intialize with additional paramaters 272 } 273 274 275 func (gl *GenericLifecycle) updatePodStatus(pod *v1.Pod) error { 276 .... 277 Get pod status based on remote/docker runtime 278 Convert to API pod status for remote runtime 279 Set pod status phase for remote runtime 280 281 .... 282 } 283 ``` 284