github.com/iqoqo/nomad@v0.11.3-0.20200911112621-d7021c74d101/website/pages/docs/upgrade/index.mdx (about)

     1  ---
     2  layout: docs
     3  page_title: 'Upgrading'
     4  sidebar_current: 'guides-upgrade'
     5  description: |-
     6    Learn how to upgrade Nomad.
     7  ---
     8  
     9  # Upgrading
    10  
    11  Nomad is designed to be flexible and resilient when upgrading from one Nomad
    12  version to the next. Upgrades should cause neither a Nomad nor a service
    13  outage. However, there are some restrictions to be aware of before upgrading:
    14  
    15  - Nomad strives to be backward compatible for at least 1 point release, so
    16    Nomad v0.10 hosts work with v0.9 hosts. Upgrading 2 point releases (eg v0.8
    17    to v0.10) may work but is untested and unsupported.
    18  
    19    - Nomad does _not_ support downgrading at this time. Downgrading clients
    20      requires draining allocations and removing the [data directory][data_dir].
    21      Downgrading servers safely requires re-provisioning the cluster.
    22  
    23    - New features are unlikely to work correctly until all nodes have been
    24      upgraded.
    25  
    26    - Check the [version upgrade details page][upgrade-specific] for important
    27      changes and backward incompatibilities.
    28  
    29  - When upgrading a Nomad Client, if it takes longer than the
    30    [`heartbeat_grace`][heartbeat_grace] (10s by default) period to restart, all
    31    allocations on that node may be rescheduled.
    32  
    33  Nomad supports upgrading in place or by rolling in new servers:
    34  
    35  - In Place: The Nomad binary can be updated on existing hosts. Running
    36    allocations will continue running uninterrupted.
    37  
    38  - Rolling: New hosts containing the new Nomad version may be added followed by
    39    the removal of old hosts. The old nodes must be drained to migrate running
    40    allocations to the new nodes.
    41  
    42  This guide describes both approaches.
    43  
    44  ## Upgrade Process
    45  
    46  Once you have checked the [upgrade details for the new
    47  version][upgrade-specific], the upgrade process is as simple as updating the
    48  binary on each host and restarting the Nomad service.
    49  
    50  At a high level we complete the following steps to upgrade Nomad:
    51  
    52  - **Add the new version**
    53  - **Check cluster health**
    54  - **Remove the old version**
    55  - **Check cluster health**
    56  - **Upgrade clients**
    57  
    58  ### 1. Add the new version to the existing cluster
    59  
    60  While it is possible to upgrade Nomad client nodes before servers, this guide
    61  recommends upgrading servers first as many new client features will not work
    62  until servers are upgraded.
    63  
    64  Whether you are replacing Nomad in place on existing systems or bringing up new
    65  servers you should make changes incrementally, verifying cluster health at each
    66  step of the upgrade.
    67  
    68  On a single server, install the new version of Nomad. You can do this by
    69  joining a new server to the cluster or by replacing or upgrading the binary
    70  locally and restarting the Nomad service.
    71  
    72  ### 2. Check cluster health
    73  
    74  [Monitor the Nomad logs][monitor] on the remaining servers to check that the
    75  new server has joined the cluster correctly.
    76  
    77  Run `nomad agent-info` on the new servers and check that the `last_log_index`
    78  is of a similar value to the other servers. This step ensures that changes have
    79  been replicated to the new server.
    80  
    81  ```shell
    82  ubuntu@nomad-server-10-1-1-4:~$ nomad agent-info
    83  nomad
    84    bootstrap = false
    85    known_regions = 1
    86    leader = false
    87    server = true
    88  raft
    89    applied_index = 53460
    90    commit_index = 53460
    91    fsm_pending = 0
    92    last_contact = 54.512216ms
    93    last_log_index = 53460
    94    last_log_term = 1
    95    last_snapshot_index = 49511
    96    last_snapshot_term = 1
    97    num_peers = 2
    98  ...
    99  ```
   100  
   101  Continue with the upgrades across the servers making sure to do a single Nomad
   102  server at a time. You can check state of the servers with [`nomad server members`][server-members], and the state of the client nodes with [`nomad node status`][node-status].
   103  
   104  ### 3. Remove the old versions from servers
   105  
   106  If you are doing an in place upgrade on existing servers this step is not
   107  necessary as the version was changed in place.
   108  
   109  If you are doing an upgrade by adding new servers and removing old servers
   110  from the fleet you need to ensure that the server has left the fleet safely.
   111  
   112  1. Stop the service on the existing host
   113  2. On another server issue a `nomad server members` and check the status, if
   114     the server is now in a left state you are safe to continue.
   115  3. If the server is not in a left state, issue a `nomad server force-leave <server id>`
   116     to remove the server from the cluster.
   117  
   118  Monitor the logs of the other hosts in the Nomad cluster over this period.
   119  
   120  ### 4. Check cluster health
   121  
   122  Use the same actions in step #2 above to confirm cluster health.
   123  
   124  ### 5. Upgrade clients
   125  
   126  Following the successful upgrade of the servers you can now update your
   127  clients using a similar process as the servers. You may either upgrade clients
   128  in-place or start new nodes on the new version. See the [Workload Migration
   129  Guide](https://learn.hashicorp.com/nomad/operating-nomad/node-draining) for instructions on how to migrate running
   130  allocations from the old nodes to the new nodes with the [`nomad node drain`](/docs/commands/node/drain) command.
   131  
   132  ## Done!
   133  
   134  You are now running the latest Nomad version. You can verify all
   135  Clients joined by running `nomad node status` and checking all the clients
   136  are in a `ready` state.
   137  
   138  ## Upgrading to Nomad Enterprise
   139  
   140  The process of upgrading to a Nomad Enterprise version is identical to upgrading
   141  between versions of open source Nomad. The same guidance above should be
   142  followed and as always, prior to starting the upgrade please check the [specific
   143  version details](/docs/upgrade/upgrade-specific) page as some version
   144  differences may require specific steps.
   145  
   146  [data_dir]: /docs/configuration#data_dir
   147  [heartbeat_grace]: /docs/configuration/server#heartbeat_grace
   148  [monitor]: /docs/commands/monitor
   149  [node-status]: /docs/commands/node/status
   150  [server-members]: /docs/commands/server/members
   151  [upgrade-specific]: /docs/upgrade/upgrade-specific