github.com/containerd/containerd@v22.0.0-20200918172823-438c87b8e050+incompatible/PLUGINS.md (about) 1 # containerd Plugins 2 3 containerd supports extending its functionality using most of its defined 4 interfaces. This includes using a customized runtime, snapshotter, content 5 store, and even adding gRPC interfaces. 6 7 ## Smart Client Model 8 9 containerd has a smart client architecture, meaning any functionality which is 10 not required by the daemon is done by the client. This includes most high 11 level interactions such as creating a container's specification, interacting 12 with an image registry, or loading an image from tar. containerd's Go client 13 gives a user access to many points of extensions from creating their own 14 options on container creation to resolving image registry names. 15 16 See [containerd's Go documentation](https://godoc.org/github.com/containerd/containerd) 17 18 ## External Plugins 19 20 External plugins allow extending containerd's functionality using an officially 21 released version of containerd without needing to recompile the daemon to add a 22 plugin. 23 24 containerd allows extensions through two method: 25 - via a binary available in containerd's PATH 26 - by configuring containerd to proxy to another gRPC service 27 28 ### V2 Runtimes 29 30 The runtime v2 interface allows resolving runtimes to binaries on the system. 31 These binaries are used to start the shim process for containerd and allows 32 containerd to manage those containers using the runtime shim api returned by 33 the binary. 34 35 See [runtime v2 documentation](runtime/v2/README.md) 36 37 ### Proxy Plugins 38 39 A proxy plugin is configured using containerd's config file and will be loaded 40 alongside the internal plugins when containerd is started. These plugins are 41 connected to containerd using a local socket serving one of containerd's gRPC 42 API services. Each plugin is configured with a type and name just as internal 43 plugins are. 44 45 #### Configuration 46 47 Update the containerd config file, which by default is at 48 `/etc/containerd/config.toml`. Add a `[proxy_plugins]` section along with a 49 section for your given plugin `[proxy_plugins.myplugin]`. The `address` must 50 refer to a local socket file which the containerd process has access to. The 51 currently supported types are `snapshot` and `content`. 52 53 ``` 54 [proxy_plugins] 55 [proxy_plugins.customsnapshot] 56 type = "snapshot" 57 address = "/var/run/mysnapshotter.sock" 58 ``` 59 60 #### Implementation 61 62 Implementing a proxy plugin is as easy as implementing the gRPC API for a 63 service. For implementing a proxy plugin in Go, look at the go doc for 64 [content store service](https://godoc.org/github.com/containerd/containerd/api/services/content/v1#ContentServer) 65 and [snapshotter service](https://godoc.org/github.com/containerd/containerd/api/services/snapshots/v1#SnapshotsServer). 66 67 The following example creates a snapshot plugin binary which can be used 68 with any implementation of 69 [containerd's Snapshotter interface](https://godoc.org/github.com/containerd/containerd/snapshots#Snapshotter) 70 ```go 71 package main 72 73 import ( 74 "fmt" 75 "net" 76 "os" 77 78 "google.golang.org/grpc" 79 80 snapshotsapi "github.com/containerd/containerd/api/services/snapshots/v1" 81 "github.com/containerd/containerd/contrib/snapshotservice" 82 "github.com/containerd/containerd/snapshots/native" 83 ) 84 85 func main() { 86 // Provide a unix address to listen to, this will be the `address` 87 // in the `proxy_plugin` configuration. 88 // The root will be used to store the snapshots. 89 if len(os.Args) < 3 { 90 fmt.Printf("invalid args: usage: %s <unix addr> <root>\n", os.Args[0]) 91 os.Exit(1) 92 } 93 94 // Create a gRPC server 95 rpc := grpc.NewServer() 96 97 // Configure your custom snapshotter, this example uses the native 98 // snapshotter and a root directory. Your custom snapshotter will be 99 // much more useful than using a snapshotter which is already included. 100 // https://godoc.org/github.com/containerd/containerd/snapshots#Snapshotter 101 sn, err := native.NewSnapshotter(os.Args[2]) 102 if err != nil { 103 fmt.Printf("error: %v\n", err) 104 os.Exit(1) 105 } 106 107 // Convert the snapshotter to a gRPC service, 108 // example in github.com/containerd/containerd/contrib/snapshotservice 109 service := snapshotservice.FromSnapshotter(sn) 110 111 // Register the service with the gRPC server 112 snapshotsapi.RegisterSnapshotsServer(rpc, service) 113 114 // Listen and serve 115 l, err := net.Listen("unix", os.Args[1]) 116 if err != nil { 117 fmt.Printf("error: %v\n", err) 118 os.Exit(1) 119 } 120 if err := rpc.Serve(l); err != nil { 121 fmt.Printf("error: %v\n", err) 122 os.Exit(1) 123 } 124 } 125 ``` 126 127 Using the previous configuration and example, you could run a snapshot plugin 128 with 129 ``` 130 # Start plugin in one terminal 131 $ go run ./main.go /var/run/mysnapshotter.sock /tmp/snapshots 132 133 # Use ctr in another 134 $ CONTAINERD_SNAPSHOTTER=customsnapshot ctr images pull docker.io/library/alpine:latest 135 $ tree -L 3 /tmp/snapshots 136 /tmp/snapshots 137 |-- metadata.db 138 `-- snapshots 139 `-- 1 140 |-- bin 141 |-- dev 142 |-- etc 143 |-- home 144 |-- lib 145 |-- media 146 |-- mnt 147 |-- proc 148 |-- root 149 |-- run 150 |-- sbin 151 |-- srv 152 |-- sys 153 |-- tmp 154 |-- usr 155 `-- var 156 157 18 directories, 1 file 158 ``` 159 160 ## Built-in Plugins 161 162 containerd uses plugins internally to ensure that internal implementations are 163 decoupled, stable, and treated equally with external plugins. To see all the 164 plugins containerd has, use `ctr plugins ls` 165 166 ``` 167 $ ctr plugins ls 168 TYPE ID PLATFORMS STATUS 169 io.containerd.content.v1 content - ok 170 io.containerd.snapshotter.v1 btrfs linux/amd64 ok 171 io.containerd.snapshotter.v1 aufs linux/amd64 error 172 io.containerd.snapshotter.v1 native linux/amd64 ok 173 io.containerd.snapshotter.v1 overlayfs linux/amd64 ok 174 io.containerd.snapshotter.v1 zfs linux/amd64 error 175 io.containerd.metadata.v1 bolt - ok 176 io.containerd.differ.v1 walking linux/amd64 ok 177 io.containerd.gc.v1 scheduler - ok 178 io.containerd.service.v1 containers-service - ok 179 io.containerd.service.v1 content-service - ok 180 io.containerd.service.v1 diff-service - ok 181 io.containerd.service.v1 images-service - ok 182 io.containerd.service.v1 leases-service - ok 183 io.containerd.service.v1 namespaces-service - ok 184 io.containerd.service.v1 snapshots-service - ok 185 io.containerd.runtime.v1 linux linux/amd64 ok 186 io.containerd.runtime.v2 task linux/amd64 ok 187 io.containerd.monitor.v1 cgroups linux/amd64 ok 188 io.containerd.service.v1 tasks-service - ok 189 io.containerd.internal.v1 restart - ok 190 io.containerd.grpc.v1 containers - ok 191 io.containerd.grpc.v1 content - ok 192 io.containerd.grpc.v1 diff - ok 193 io.containerd.grpc.v1 events - ok 194 io.containerd.grpc.v1 healthcheck - ok 195 io.containerd.grpc.v1 images - ok 196 io.containerd.grpc.v1 leases - ok 197 io.containerd.grpc.v1 namespaces - ok 198 io.containerd.grpc.v1 snapshots - ok 199 io.containerd.grpc.v1 tasks - ok 200 io.containerd.grpc.v1 version - ok 201 io.containerd.grpc.v1 cri linux/amd64 ok 202 ``` 203 204 From the output all the plugins can be seen as well those which did not 205 successfully load. In this case `aufs` and `zfs` are expected not to load 206 since they are not support on the machine. The logs will show why it failed, 207 but you can also get more details using the `-d` option. 208 209 ``` 210 $ ctr plugins ls -d id==aufs id==zfs 211 Type: io.containerd.snapshotter.v1 212 ID: aufs 213 Platforms: linux/amd64 214 Exports: 215 root /var/lib/containerd/io.containerd.snapshotter.v1.aufs 216 Error: 217 Code: Unknown 218 Message: modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.17.2-1-ARCH\n": exit status 1 219 220 Type: io.containerd.snapshotter.v1 221 ID: zfs 222 Platforms: linux/amd64 223 Exports: 224 root /var/lib/containerd/io.containerd.snapshotter.v1.zfs 225 Error: 226 Code: Unknown 227 Message: path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter 228 ``` 229 230 The error message which the plugin returned explains why the plugin was unable 231 to load. 232 233 #### Configuration 234 235 Plugins are configured using the `[plugins]` section of containerd's config. 236 Every plugin can have its own section using the pattern `[plugins.<plugin id>]`. 237 238 example configuration 239 ``` 240 [plugins] 241 [plugins.cgroups] 242 no_prometheus = false 243 [plugins.cri] 244 stream_server_address = "" 245 stream_server_port = "10010" 246 enable_selinux = false 247 sandbox_image = "k8s.gcr.io/pause:3.1" 248 stats_collect_period = 10 249 systemd_cgroup = false 250 [plugins.cri.containerd] 251 snapshotter = "overlayfs" 252 [plugins.cri.containerd.default_runtime] 253 runtime_type = "io.containerd.runtime.v1.linux" 254 runtime_engine = "" 255 runtime_root = "" 256 [plugins.cri.containerd.untrusted_workload_runtime] 257 runtime_type = "" 258 runtime_engine = "" 259 runtime_root = "" 260 [plugins.cri.cni] 261 bin_dir = "/opt/cni/bin" 262 conf_dir = "/etc/cni/net.d" 263 [plugins.cri.registry] 264 [plugins.cri.registry.mirrors] 265 [plugins.cri.registry.mirrors."docker.io"] 266 endpoint = ["https://registry-1.docker.io"] 267 ```