github.com/ilhicas/nomad@v0.11.8/website/pages/docs/internals/plugins/task-drivers.mdx (about) 1 --- 2 layout: docs 3 page_title: Task Driver Plugins 4 sidebar_title: Task Drivers 5 description: Learn how to author a Nomad task driver plugin. 6 --- 7 8 # Task Drivers 9 10 Task drivers in Nomad are the runtime components that execute workloads. For 11 a real world example of a Nomad task driver plugin implementation, see the [LXC 12 driver source][lxcdriver]. 13 14 ## Authoring Task Driver Plugins 15 16 Authoring a task driver (shortened to driver in this documentation) in Nomad 17 consists of implementing the [DriverPlugin][driverplugin] interface and adding 18 a main package to launch the plugin. A driver plugin is long-lived and its 19 lifetime is not bound to the Nomad client. This means that the Nomad client can 20 be restarted without restarting the driver. Nomad will ensure that one 21 instance of the driver is running, meaning if the driver crashes or otherwise 22 terminates, Nomad will launch another instance of it. 23 24 Drivers should maintain as little state as possible. State for a task is stored 25 by the Nomad client on task creation. This enables a pattern where the driver 26 can maintain an in-memory state of the running tasks, and if necessary the 27 Nomad client can recover tasks into the driver state. 28 29 The [driver plugin skeleton project][skeletonproject] exists to help bootstrap 30 the development of new driver plugins. It provides most of the boilerplate 31 necessary for a driver plugin, along with detailed comments. 32 33 ## Task Driver Plugin API 34 35 The [base plugin][baseplugin] must be implemented in addition to the following 36 functions. 37 38 ### `TaskConfigSchema() (*hclspec.Spec, error)` 39 40 This function returns the schema for the driver configuration of the task. For 41 more information on `hclspec.Spec` see the HCL section in the [base 42 plugin][baseplugin] documentation. 43 44 ### `Capabilities() (*Capabilities, error)` 45 46 Capabilities define what features the driver implements. Example: 47 48 ```go 49 Capabilities { 50 // Does the driver support sending OS signals to the task? 51 SendSignals: true, 52 // Does the driver support executing a command within the task execution 53 // environment? 54 Exec: true, 55 // What filesystem isolation is supported by the driver. Options include 56 // FSIsolationImage, FSIsolationChroot, and FSIsolationNone 57 FSIsolation: FSIsolationImage, 58 59 // NetIsolationModes lists the set of isolation modes supported by the driver. 60 // Options include NetIsolationModeHost, NetIsolationModeGroup, 61 // NetIsolationModeTask, and NetIsolationModeNone. 62 NetIsolationModes []NetIsolationMode 63 64 // MustInitiateNetwork tells Nomad that the driver must create the network 65 // namespace and that the CreateNetwork and DestroyNetwork RPCs are implemented. 66 MustInitiateNetwork bool 67 68 // MountConfigs tells Nomad which mounting config options the driver 69 // supports. This is used to check whether mounting host volumes or CSI 70 // volumes is allowed. Options include MountConfigSupportAll (default), 71 // or MountConfigSupportNone. 72 MountConfigs MountConfigSupport 73 } 74 ``` 75 76 ### `Fingerprint(context.Context) (<-chan *Fingerprint, error)` 77 78 This function is called by the client when the plugin is started. It allows the 79 driver to indicate its health to the client. The channel returned should 80 immediately send an initial Fingerprint, then send periodic updates at an 81 interval that is appropriate for the driver until the context is canceled. 82 83 The fingerprint consists of a `HealthState` and `HealthDescription` to inform 84 the client about its health. Additionally an `Attributes` field is available 85 for the driver to add additional attributes to the client node. The fingerprint 86 `HealthState` can be one of three states. 87 88 - `HealthStateUndetected`: Indicates that the necessary dependencies for the 89 driver are not detected on the system. Ex. java runtime for the java driver 90 - `HealthStateUnhealthy`: Indicates that something is wrong with the driver 91 runtime. Ex. docker daemon stopped for the Docker driver 92 - `HealthStateHealthy`: All systems go 93 94 ### `StartTask(*TaskConfig) (*TaskHandle, *DriverNetwork, error)` 95 96 This function takes a [`TaskConfig`][taskconfig] which includes all of the configuration 97 needed to launch the task. Additionally the driver configuration can be decoded 98 from the `TaskConfig` by calling `*TaskConfig.DecodeDriverConfig(t interface{})` 99 passing in a pointer to the driver specific configuration struct. The 100 `TaskConfig` includes an `ID` field which future operations on the task will be 101 referenced by. 102 103 Drivers return a [`*TaskHandle`][taskhandle] which contains 104 the required information for the driver to reattach to the running task in the 105 case of plugin crashes or restarts. Some of this required state 106 will be specific to the driver implementation, thus a `DriverState` field 107 exists to allow the driver to encode custom state into the struct. Helper 108 fields exist on the `TaskHandle` to `GetDriverState` and `SetDriverState` 109 removing the need for the driver to handle serialization. 110 111 A `*DriverNetwork` can optionally be returned to describe the network of the 112 task if it is modified by the driver. An example of this is in the Docker 113 driver where tasks can be attached to a specific Docker network. 114 115 If an error occurs, it is expected that the driver will cleanup any created 116 resources prior to returning the error. 117 118 #### Logging 119 120 Nomad handles all rotation and plumbing of task logs. In order for task stdout 121 and stderr to be received by Nomad, they must be written to the correct 122 location. Prior to starting the task through the driver, the Nomad client 123 creates FIFOs for stdout and stderr. These paths are given to the driver in the 124 `TaskConfig`. The [`fifo` package][fifopackage] can be used to support 125 cross platform writing to these paths. 126 127 #### TaskHandle Schema Versioning 128 129 A `Version` field is available on the TaskHandle struct to facilitate backwards 130 compatible recovery of tasks. This field is opaque to Nomad, but allows the 131 driver to handle recover tasks that were created by an older version of the 132 plugin. 133 134 ### `RecoverTask(*TaskHandle) error` 135 136 When a driver is restarted it is not expected to persist any internal state to 137 disk. To support this, Nomad will attempt to recover a task that was 138 previously started if the driver does not recognize the task ID. During task 139 recovery, Nomad calls `RecoverTask` passing the `TaskHandle` that was 140 returned by the `StartTask` function. If no error was returned, it is 141 expected that the driver can now operate on the task by referencing the task 142 ID. If an error occurs, the Nomad client will mark the task as `lost`. 143 144 ### `WaitTask(context.Context, id string) (<-chan *ExitResult, error)` 145 146 The `WaitTask` function is expected to return a channel that will send an 147 `*ExitResult` when the task exits or close the channel when the context is 148 canceled. It is also expected that calling `WaitTask` on an exited task will 149 immediately send an `*ExitResult` on the returned channel. 150 151 ### `StopTask(taskID string, timeout time.Duration, signal string) error` 152 153 The `StopTask` function is expected to stop a running task by sending the given 154 signal to it. If the task does not stop during the given timeout, the driver 155 must forcefully kill the task. 156 157 `StopTask` does not clean up resources of the task or remove it from the 158 driver's internal state. A call to `WaitTask` after `StopTask` is valid and 159 should be handled. 160 161 ### `DestroyTask(taskID string, force bool) error` 162 163 The `DestroyTask` function cleans up and removes a task that has terminated. If 164 force is set to true, the driver must destroy the task even if it is still 165 running. If `WaitTask` is called after `DestroyTask`, it should return 166 `drivers.ErrTaskNotFound` as no task state should exist after `DestroyTask` is 167 called. 168 169 ### `InspectTask(taskID string) (*TaskStatus, error)` 170 171 The `InspectTask` function returns detailed status information for the 172 referenced `taskID`. 173 174 ### `TaskStats(context.Context, id string, time.Duration) (<-chan *cstructs.TaskResourceUsage, error)` 175 176 The `TaskStats` function returns a channel which the driver should send stats 177 to at the given interval. The driver must send stats at the given interval 178 until the given context is canceled or the task terminates. 179 180 ### `TaskEvents(context.Context) (<-chan *TaskEvent, error)` 181 182 The Nomad client publishes events associated with an allocation. The 183 `TaskEvents` function allows the driver to publish driver specific events about 184 tasks and the Nomad client will associate them with the correct allocation. 185 186 An `Eventer` utility is available in the 187 `github.com/hashicorp/nomad/drivers/shared/eventer` package implements an 188 event loop and publishing mechanism for use in the `TaskEvents` function. 189 190 ### `SignalTask(taskID string, signal string) error` 191 192 > Optional - can be skipped by embedding `drivers.DriverSignalTaskNotSupported` 193 194 The `SignalTask` function is used by drivers which support sending OS signals 195 (`SIGHUP`, `SIGKILL`, `SIGUSR1` etc.) to the task. It is an optional function 196 and is listed as a capability in the driver `Capabilities` struct. 197 198 ### `ExecTask(taskID string, cmd []string, timeout time.Duration) (*ExecTaskResult, error)` 199 200 > Optional - can be skipped by embedding `drivers.DriverExecTaskNotSupported` 201 202 The `ExecTask` function is used by the Nomad client to execute commands inside 203 the task execution context. For example, the Docker driver executes commands 204 inside the running container. `ExecTask` is called for Consul script checks. 205 206 [lxcdriver]: https://github.com/hashicorp/nomad-driver-lxc 207 [driverplugin]: https://github.com/hashicorp/nomad/blob/v0.9.0/plugins/drivers/driver.go#L39-L57 208 [skeletonproject]: https://github.com/hashicorp/nomad-skeleton-driver-plugin 209 [baseplugin]: /docs/internals/plugins/base 210 [taskconfig]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskConfig 211 [taskhandle]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskHandle 212 [fifopackage]: https://godoc.org/github.com/hashicorp/nomad/client/lib/fifo