github.com/iqoqo/nomad@v0.11.3-0.20200911112621-d7021c74d101/website/pages/docs/internals/plugins/task-drivers.mdx (about) 1 --- 2 layout: docs 3 page_title: Task Driver Plugins 4 sidebar_title: Task Drivers 5 description: Learn how to author a Nomad task driver plugin. 6 --- 7 8 # Task Drivers 9 10 Task drivers in Nomad are the runtime components that execute workloads. For 11 a real world example of a Nomad task driver plugin implementation, see the [LXC 12 driver source][lxcdriver]. 13 14 ## Authoring Task Driver Plugins 15 16 Authoring a task driver (shortened to driver in this documentation) in Nomad 17 consists of implementing the [DriverPlugin][driverplugin] interface and adding 18 a main package to launch the plugin. A driver plugin is long-lived and its 19 lifetime is not bound to the Nomad client. This means that the Nomad client can 20 be restarted without restarting the driver. Nomad will ensure that one 21 instance of the driver is running, meaning if the driver crashes or otherwise 22 terminates, Nomad will launch another instance of it. 23 24 Drivers should maintain as little state as possible. State for a task is stored 25 by the Nomad client on task creation. This enables a pattern where the driver 26 can maintain an in-memory state of the running tasks, and if necessary the 27 Nomad client can recover tasks into the driver state. 28 29 The [driver plugin skeleton project][skeletonproject] exists to help bootstrap 30 the development of new driver plugins. It provides most of the boilerplate 31 necessary for a driver plugin, along with detailed comments. 32 33 ## Task Driver Plugin API 34 35 The [base plugin][baseplugin] must be implemented in addition to the following 36 functions. 37 38 ### `TaskConfigSchema() (*hclspec.Spec, error)` 39 40 This function returns the schema for the driver configuration of the task. For 41 more information on `hclspec.Spec` see the HCL section in the [base 42 plugin][baseplugin] documentation. 43 44 ### `Capabilities() (*Capabilities, error)` 45 46 Capabilities define what features the driver implements. Example: 47 48 ```go 49 Capabilities { 50 // Does the driver support sending OS signals to the task? 51 SendSignals: true, 52 // Does the driver support executing a command within the task execution 53 // environment? 54 Exec: true, 55 // What filesystem isolation is supported by the driver. Options include 56 // FSIsolationImage, FSIsolationChroot, and FSIsolationNone 57 FSIsolation: FSIsolationImage, 58 } 59 ``` 60 61 ### `Fingerprint(context.Context) (<-chan *Fingerprint, error)` 62 63 This function is called by the client when the plugin is started. It allows the 64 driver to indicate its health to the client. The channel returned should 65 immediately send an initial Fingerprint, then send periodic updates at an 66 interval that is appropriate for the driver until the context is canceled. 67 68 The fingerprint consists of a `HealthState` and `HealthDescription` to inform 69 the client about its health. Additionally an `Attributes` field is available 70 for the driver to add additional attributes to the client node. The fingerprint 71 `HealthState` can be one of three states. 72 73 - `HealthStateUndetected`: Indicates that the necessary dependencies for the 74 driver are not detected on the system. Ex. java runtime for the java driver 75 - `HealthStateUnhealthy`: Indicates that something is wrong with the driver 76 runtime. Ex. docker daemon stopped for the Docker driver 77 - `HealthStateHealthy`: All systems go 78 79 ### `StartTask(*TaskConfig) (*TaskHandle, *DriverNetwork, error)` 80 81 This function takes a [`TaskConfig`][taskconfig] which includes all of the configuration 82 needed to launch the task. Additionally the driver configuration can be decoded 83 from the `TaskConfig` by calling `*TaskConfig.DecodeDriverConfig(t interface{})` 84 passing in a pointer to the driver specific configuration struct. The 85 `TaskConfig` includes an `ID` field which future operations on the task will be 86 referenced by. 87 88 Drivers return a [`*TaskHandle`][taskhandle] which contains 89 the required information for the driver to reattach to the running task in the 90 case of plugin crashes or restarts. Some of this required state 91 will be specific to the driver implementation, thus a `DriverState` field 92 exists to allow the driver to encode custom state into the struct. Helper 93 fields exist on the `TaskHandle` to `GetDriverState` and `SetDriverState` 94 removing the need for the driver to handle serialization. 95 96 A `*DriverNetwork` can optionally be returned to describe the network of the 97 task if it is modified by the driver. An example of this is in the Docker 98 driver where tasks can be attached to a specific Docker network. 99 100 If an error occurs, it is expected that the driver will cleanup any created 101 resources prior to returning the error. 102 103 #### Logging 104 105 Nomad handles all rotation and plumbing of task logs. In order for task stdout 106 and stderr to be received by Nomad, they must be written to the correct 107 location. Prior to starting the task through the driver, the Nomad client 108 creates FIFOs for stdout and stderr. These paths are given to the driver in the 109 `TaskConfig`. The [`fifo` package][fifopackage] can be used to support 110 cross platform writing to these paths. 111 112 #### TaskHandle Schema Versioning 113 114 A `Version` field is available on the TaskHandle struct to facilitate backwards 115 compatible recovery of tasks. This field is opaque to Nomad, but allows the 116 driver to handle recover tasks that were created by an older version of the 117 plugin. 118 119 ### `RecoverTask(*TaskHandle) error` 120 121 When a driver is restarted it is not expected to persist any internal state to 122 disk. To support this, Nomad will attempt to recover a task that was 123 previously started if the driver does not recognize the task ID. During task 124 recovery, Nomad calls `RecoverTask` passing the `TaskHandle` that was 125 returned by the `StartTask` function. If no error was returned, it is 126 expected that the driver can now operate on the task by referencing the task 127 ID. If an error occurs, the Nomad client will mark the task as `lost`. 128 129 ### `WaitTask(context.Context, id string) (<-chan *ExitResult, error)` 130 131 The `WaitTask` function is expected to return a channel that will send an 132 `*ExitResult` when the task exits or close the channel when the context is 133 canceled. It is also expected that calling `WaitTask` on an exited task will 134 immediately send an `*ExitResult` on the returned channel. 135 136 ### `StopTask(taskID string, timeout time.Duration, signal string) error` 137 138 The `StopTask` function is expected to stop a running task by sending the given 139 signal to it. If the task does not stop during the given timeout, the driver 140 must forcefully kill the task. 141 142 `StopTask` does not clean up resources of the task or remove it from the 143 driver's internal state. A call to `WaitTask` after `StopTask` is valid and 144 should be handled. 145 146 ### `DestroyTask(taskID string, force bool) error` 147 148 The `DestroyTask` function cleans up and removes a task that has terminated. If 149 force is set to true, the driver must destroy the task even if it is still 150 running. If `WaitTask` is called after `DestroyTask`, it should return 151 `drivers.ErrTaskNotFound` as no task state should exist after `DestroyTask` is 152 called. 153 154 ### `InspectTask(taskID string) (*TaskStatus, error)` 155 156 The `InspectTask` function returns detailed status information for the 157 referenced `taskID`. 158 159 ### `TaskStats(context.Context, id string, time.Duration) (<-chan *cstructs.TaskResourceUsage, error)` 160 161 The `TaskStats` function returns a channel which the driver should send stats 162 to at the given interval. The driver must send stats at the given interval 163 until the given context is canceled or the task terminates. 164 165 ### `TaskEvents(context.Context) (<-chan *TaskEvent, error)` 166 167 The Nomad client publishes events associated with an allocation. The 168 `TaskEvents` function allows the driver to publish driver specific events about 169 tasks and the Nomad client will associate them with the correct allocation. 170 171 An `Eventer` utility is available in the 172 `github.com/hashicorp/nomad/drivers/shared/eventer` package implements an 173 event loop and publishing mechanism for use in the `TaskEvents` function. 174 175 ### `SignalTask(taskID string, signal string) error` 176 177 > Optional - can be skipped by embedding `drivers.DriverSignalTaskNotSupported` 178 179 The `SignalTask` function is used by drivers which support sending OS signals 180 (`SIGHUP`, `SIGKILL`, `SIGUSR1` etc.) to the task. It is an optional function 181 and is listed as a capability in the driver `Capabilities` struct. 182 183 ### `ExecTask(taskID string, cmd []string, timeout time.Duration) (*ExecTaskResult, error)` 184 185 > Optional - can be skipped by embedding `drivers.DriverExecTaskNotSupported` 186 187 The `ExecTask` function is used by the Nomad client to execute commands inside 188 the task execution context. For example, the Docker driver executes commands 189 inside the running container. `ExecTask` is called for Consul script checks. 190 191 [lxcdriver]: https://github.com/hashicorp/nomad-driver-lxc 192 [driverplugin]: https://github.com/hashicorp/nomad/blob/v0.9.0/plugins/drivers/driver.go#L39-L57 193 [skeletonproject]: https://github.com/hashicorp/nomad-skeleton-driver-plugin 194 [baseplugin]: /docs/internals/plugins/base 195 [taskconfig]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskConfig 196 [taskhandle]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskHandle 197 [fifopackage]: https://godoc.org/github.com/hashicorp/nomad/client/lib/fifo