github.com/ilhicas/nomad@v0.11.8/website/pages/docs/internals/plugins/task-drivers.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: Task Driver Plugins
     4  sidebar_title: Task Drivers
     5  description: Learn how to author a Nomad task driver plugin.
     6  ---
     7  
     8  # Task Drivers
     9  
    10  Task drivers in Nomad are the runtime components that execute workloads. For
    11  a real world example of a Nomad task driver plugin implementation, see the [LXC
    12  driver source][lxcdriver].
    13  
    14  ## Authoring Task Driver Plugins
    15  
    16  Authoring a task driver (shortened to driver in this documentation) in Nomad
    17  consists of implementing the [DriverPlugin][driverplugin] interface and adding
    18  a main package to launch the plugin. A driver plugin is long-lived and its
    19  lifetime is not bound to the Nomad client. This means that the Nomad client can
    20  be restarted without restarting the driver. Nomad will ensure that one
    21  instance of the driver is running, meaning if the driver crashes or otherwise
    22  terminates, Nomad will launch another instance of it.
    23  
    24  Drivers should maintain as little state as possible. State for a task is stored
    25  by the Nomad client on task creation. This enables a pattern where the driver
    26  can maintain an in-memory state of the running tasks, and if necessary the
    27  Nomad client can recover tasks into the driver state.
    28  
    29  The [driver plugin skeleton project][skeletonproject] exists to help bootstrap
    30  the development of new driver plugins. It provides most of the boilerplate
    31  necessary for a driver plugin, along with detailed comments.
    32  
    33  ## Task Driver Plugin API
    34  
    35  The [base plugin][baseplugin] must be implemented in addition to the following
    36  functions.
    37  
    38  ### `TaskConfigSchema() (*hclspec.Spec, error)`
    39  
    40  This function returns the schema for the driver configuration of the task. For
    41  more information on `hclspec.Spec` see the HCL section in the [base
    42  plugin][baseplugin] documentation.
    43  
    44  ### `Capabilities() (*Capabilities, error)`
    45  
    46  Capabilities define what features the driver implements. Example:
    47  
    48  ```go
    49  Capabilities {
    50      // Does the driver support sending OS signals to the task?
    51      SendSignals: true,
    52      // Does the driver support executing a command within the task execution
    53      // environment?
    54      Exec:        true,
    55      // What filesystem isolation is supported by the driver. Options include
    56      // FSIsolationImage, FSIsolationChroot, and FSIsolationNone
    57      FSIsolation: FSIsolationImage,
    58  
    59      // NetIsolationModes lists the set of isolation modes supported by the driver.
    60      // Options include NetIsolationModeHost, NetIsolationModeGroup,
    61      // NetIsolationModeTask, and NetIsolationModeNone.
    62      NetIsolationModes []NetIsolationMode
    63  
    64      // MustInitiateNetwork tells Nomad that the driver must create the network
    65      // namespace and that the CreateNetwork and DestroyNetwork RPCs are implemented.
    66      MustInitiateNetwork bool
    67  
    68      // MountConfigs tells Nomad which mounting config options the driver
    69      // supports. This is used to check whether mounting host volumes or CSI
    70      // volumes is allowed. Options include MountConfigSupportAll (default),
    71      // or MountConfigSupportNone.
    72      MountConfigs MountConfigSupport
    73  }
    74  ```
    75  
    76  ### `Fingerprint(context.Context) (<-chan *Fingerprint, error)`
    77  
    78  This function is called by the client when the plugin is started. It allows the
    79  driver to indicate its health to the client. The channel returned should
    80  immediately send an initial Fingerprint, then send periodic updates at an
    81  interval that is appropriate for the driver until the context is canceled.
    82  
    83  The fingerprint consists of a `HealthState` and `HealthDescription` to inform
    84  the client about its health. Additionally an `Attributes` field is available
    85  for the driver to add additional attributes to the client node. The fingerprint
    86  `HealthState` can be one of three states.
    87  
    88  - `HealthStateUndetected`: Indicates that the necessary dependencies for the
    89    driver are not detected on the system. Ex. java runtime for the java driver
    90  - `HealthStateUnhealthy`: Indicates that something is wrong with the driver
    91    runtime. Ex. docker daemon stopped for the Docker driver
    92  - `HealthStateHealthy`: All systems go
    93  
    94  ### `StartTask(*TaskConfig) (*TaskHandle, *DriverNetwork, error)`
    95  
    96  This function takes a [`TaskConfig`][taskconfig] which includes all of the configuration
    97  needed to launch the task. Additionally the driver configuration can be decoded
    98  from the `TaskConfig` by calling `*TaskConfig.DecodeDriverConfig(t interface{})`
    99  passing in a pointer to the driver specific configuration struct. The
   100  `TaskConfig` includes an `ID` field which future operations on the task will be
   101  referenced by.
   102  
   103  Drivers return a [`*TaskHandle`][taskhandle] which contains
   104  the required information for the driver to reattach to the running task in the
   105  case of plugin crashes or restarts. Some of this required state
   106  will be specific to the driver implementation, thus a `DriverState` field
   107  exists to allow the driver to encode custom state into the struct. Helper
   108  fields exist on the `TaskHandle` to `GetDriverState` and `SetDriverState`
   109  removing the need for the driver to handle serialization.
   110  
   111  A `*DriverNetwork` can optionally be returned to describe the network of the
   112  task if it is modified by the driver. An example of this is in the Docker
   113  driver where tasks can be attached to a specific Docker network.
   114  
   115  If an error occurs, it is expected that the driver will cleanup any created
   116  resources prior to returning the error.
   117  
   118  #### Logging
   119  
   120  Nomad handles all rotation and plumbing of task logs. In order for task stdout
   121  and stderr to be received by Nomad, they must be written to the correct
   122  location. Prior to starting the task through the driver, the Nomad client
   123  creates FIFOs for stdout and stderr. These paths are given to the driver in the
   124  `TaskConfig`. The [`fifo` package][fifopackage] can be used to support
   125  cross platform writing to these paths.
   126  
   127  #### TaskHandle Schema Versioning
   128  
   129  A `Version` field is available on the TaskHandle struct to facilitate backwards
   130  compatible recovery of tasks. This field is opaque to Nomad, but allows the
   131  driver to handle recover tasks that were created by an older version of the
   132  plugin.
   133  
   134  ### `RecoverTask(*TaskHandle) error`
   135  
   136  When a driver is restarted it is not expected to persist any internal state to
   137  disk. To support this, Nomad will attempt to recover a task that was
   138  previously started if the driver does not recognize the task ID. During task
   139  recovery, Nomad calls `RecoverTask` passing the `TaskHandle` that was
   140  returned by the `StartTask` function. If no error was returned, it is
   141  expected that the driver can now operate on the task by referencing the task
   142  ID. If an error occurs, the Nomad client will mark the task as `lost`.
   143  
   144  ### `WaitTask(context.Context, id string) (<-chan *ExitResult, error)`
   145  
   146  The `WaitTask` function is expected to return a channel that will send an
   147  `*ExitResult` when the task exits or close the channel when the context is
   148  canceled. It is also expected that calling `WaitTask` on an exited task will
   149  immediately send an `*ExitResult` on the returned channel.
   150  
   151  ### `StopTask(taskID string, timeout time.Duration, signal string) error`
   152  
   153  The `StopTask` function is expected to stop a running task by sending the given
   154  signal to it. If the task does not stop during the given timeout, the driver
   155  must forcefully kill the task.
   156  
   157  `StopTask` does not clean up resources of the task or remove it from the
   158  driver's internal state. A call to `WaitTask` after `StopTask` is valid and
   159  should be handled.
   160  
   161  ### `DestroyTask(taskID string, force bool) error`
   162  
   163  The `DestroyTask` function cleans up and removes a task that has terminated. If
   164  force is set to true, the driver must destroy the task even if it is still
   165  running. If `WaitTask` is called after `DestroyTask`, it should return
   166  `drivers.ErrTaskNotFound` as no task state should exist after `DestroyTask` is
   167  called.
   168  
   169  ### `InspectTask(taskID string) (*TaskStatus, error)`
   170  
   171  The `InspectTask` function returns detailed status information for the
   172  referenced `taskID`.
   173  
   174  ### `TaskStats(context.Context, id string, time.Duration) (<-chan *cstructs.TaskResourceUsage, error)`
   175  
   176  The `TaskStats` function returns a channel which the driver should send stats
   177  to at the given interval. The driver must send stats at the given interval
   178  until the given context is canceled or the task terminates.
   179  
   180  ### `TaskEvents(context.Context) (<-chan *TaskEvent, error)`
   181  
   182  The Nomad client publishes events associated with an allocation. The
   183  `TaskEvents` function allows the driver to publish driver specific events about
   184  tasks and the Nomad client will associate them with the correct allocation.
   185  
   186  An `Eventer` utility is available in the
   187  `github.com/hashicorp/nomad/drivers/shared/eventer` package implements an
   188  event loop and publishing mechanism for use in the `TaskEvents` function.
   189  
   190  ### `SignalTask(taskID string, signal string) error`
   191  
   192  > Optional - can be skipped by embedding `drivers.DriverSignalTaskNotSupported`
   193  
   194  The `SignalTask` function is used by drivers which support sending OS signals
   195  (`SIGHUP`, `SIGKILL`, `SIGUSR1` etc.) to the task. It is an optional function
   196  and is listed as a capability in the driver `Capabilities` struct.
   197  
   198  ### `ExecTask(taskID string, cmd []string, timeout time.Duration) (*ExecTaskResult, error)`
   199  
   200  > Optional - can be skipped by embedding `drivers.DriverExecTaskNotSupported`
   201  
   202  The `ExecTask` function is used by the Nomad client to execute commands inside
   203  the task execution context. For example, the Docker driver executes commands
   204  inside the running container. `ExecTask` is called for Consul script checks.
   205  
   206  [lxcdriver]: https://github.com/hashicorp/nomad-driver-lxc
   207  [driverplugin]: https://github.com/hashicorp/nomad/blob/v0.9.0/plugins/drivers/driver.go#L39-L57
   208  [skeletonproject]: https://github.com/hashicorp/nomad-skeleton-driver-plugin
   209  [baseplugin]: /docs/internals/plugins/base
   210  [taskconfig]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskConfig
   211  [taskhandle]: https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskHandle
   212  [fifopackage]: https://godoc.org/github.com/hashicorp/nomad/client/lib/fifo