github.com/anth0d/nomad@v0.0.0-20221214183521-ae3a0a2cad06/website/content/docs/commands/node/drain.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: 'Commands: node drain'
     4  description: |
     5    The node drain command is used to configure a node's drain strategy.
     6  ---
     7  
     8  # Command: node drain
     9  
    10  The `node drain` command is used to toggle drain mode on a given node. Drain
    11  mode prevents any new tasks from being allocated to the node, and begins
    12  migrating all existing allocations away. Allocations will be migrated according
    13  to their [`migrate`][migrate] stanza until the drain's deadline is reached.
    14  
    15  By default the `node drain` command blocks until a node is done draining and
    16  all allocations have terminated. Canceling the `node drain` command _will not_
    17  cancel the drain. Drains may be canceled by using the `-disable` parameter
    18  below.
    19  
    20  When draining more than one node at a time, it is recommended you first
    21  disable [scheduling eligibility][eligibility] on all nodes that will be
    22  drained. For example if you are decommissioning an entire class of nodes,
    23  first run `node eligibility -disable` on all of their node IDs, and then run
    24  `node drain -enable`. This will ensure allocations drained from the first node
    25  are not placed on another node about to be drained.
    26  
    27  The [node status] command compliments this nicely by providing the current drain
    28  status of a given node.
    29  
    30  See the [Workload Migration guide] for detailed examples of node draining.
    31  
    32  ## Usage
    33  
    34  ```plaintext
    35  nomad node drain [options] <node>
    36  ```
    37  
    38  A `-self` flag can be used to drain the local node. If this is not supplied, a
    39  node ID or prefix must be provided. If there is an exact match, the drain mode
    40  will be adjusted for that node. Otherwise, a list of matching nodes and
    41  information will be displayed.
    42  
    43  It is also required to pass one of `-enable` or `-disable`, depending on which
    44  operation is desired.
    45  
    46  If ACLs are enabled, this option requires a token with the 'node:write'
    47  capability.
    48  
    49  ## General Options
    50  
    51  @include 'general_options_no_namespace.mdx'
    52  
    53  ## Drain Options
    54  
    55  - `-enable`: Enable node drain mode.
    56  
    57  - `-disable`: Disable node drain mode.
    58  
    59  - `-deadline`: Set the deadline by which all allocations must be moved off the
    60    node. Remaining allocations after the deadline are force removed from the
    61    node. Defaults to 1 hour.
    62  
    63  - `-detach`: Return immediately instead of entering monitor mode.
    64  
    65  - `-monitor`: Enter monitor mode directly without modifying the drain status.
    66  
    67  - `-force`: Force remove allocations off the node immediately.
    68  
    69  - `-no-deadline`: No deadline allows the allocations to drain off the node
    70    without being force stopped after a certain deadline.
    71  
    72  - `-ignore-system`: Ignore system allows the drain to complete without
    73    stopping system job allocations. By default system jobs (and CSI
    74    plugins) are stopped last.
    75  
    76  - `-keep-ineligible`: Keep ineligible will maintain the node's scheduling
    77    ineligibility even if the drain is being disabled. This is useful when an
    78    existing drain is being cancelled but additional scheduling on the node is not
    79    desired.
    80  
    81  - `-m`: Message for the drain update operation. Registered in drain metadata as
    82    `"message"` during drain enable and `"cancel_message"` during drain disable.
    83  
    84  - `-meta <key>=<value>`: Custom metadata to store on the drain operation, can be
    85    used multiple times.
    86  
    87  - `-self`: Drain the local node.
    88  
    89  - `-yes`: Automatic yes to prompts.
    90  
    91  ## Examples
    92  
    93  Enable drain mode on node with ID prefix "4d2ba53b":
    94  
    95  ```shell-session
    96  $ nomad node drain -enable f4e8a9e5 -m "node maintenance"
    97  Are you sure you want to enable drain mode for node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e"? [y/N] y
    98  2018-03-30T23:13:16Z: Ctrl-C to stop monitoring: will not cancel the node drain
    99  2018-03-30T23:13:16Z: Node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" drain strategy set
   100  2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" marked for migration
   101  2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" draining
   102  2018-03-30T23:13:17Z: Alloc "1877230b-64d3-a7dd-9c31-dc5ad3c93e9a" status running -> complete
   103  2018-03-30T23:13:29Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" marked for migration
   104  2018-03-30T23:13:29Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" draining
   105  2018-03-30T23:13:30Z: Alloc "3fce5308-818c-369e-0bb7-f61f0a1be9ed" status running -> complete
   106  2018-03-30T23:13:41Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" marked for migration
   107  2018-03-30T23:13:41Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" draining
   108  2018-03-30T23:13:41Z: Node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" has marked all allocations for migration
   109  2018-03-30T23:13:42Z: Alloc "9a98c5aa-a719-2f34-ecfc-0e6268b5d537" status running -> complete
   110  2018-03-30T23:13:42Z: All allocations on node "f4e8a9e5-30d8-3536-1e6f-cda5c869c35e" have stopped.
   111  ```
   112  
   113  Enable drain mode on the local node:
   114  
   115  ```shell-session
   116  $ nomad node drain -enable -self
   117  ...
   118  ```
   119  
   120  Enable drain mode but do not stop system jobs:
   121  
   122  ```shell-session
   123  $ nomad node drain -enable -ignore-system 4d2ba53b
   124  ...
   125  ```
   126  
   127  Disable drain mode but keep the node ineligible for scheduling. Useful for
   128  inspecting the current state of a misbehaving node without Nomad trying to
   129  start or migrate allocations:
   130  
   131  ```shell-session
   132  $ nomad node drain -disable -keep-ineligible 4d2ba53b
   133  ...
   134  ```
   135  
   136  Enable drain mode and detach from monitoring, then reattach later:
   137  
   138  ```shell-session
   139  $ nomad node drain -enable -detach -self
   140  ...
   141  $ nomad node drain -self -monitor
   142  ...
   143  ```
   144  
   145  [eligibility]: /docs/commands/node/eligibility
   146  [migrate]: /docs/job-specification/migrate
   147  [node status]: /docs/commands/node/status
   148  [workload migration guide]: https://learn.hashicorp.com/tutorials/nomad/node-drain
   149  [internals-csi]: /docs/concepts/plugins/csi