github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/deploy-manage/manage/upgrades.md (about)

     1  # Upgrade Pachyderm
     2  
     3  If you need to upgrade Pachyderm from one major version
     4  to another, such as from `1.10.x` to `1.11.x`, follow the
     5  instructions in the [Migrate between major versions](./migrations.md).
     6  
     7  Upgrades from one minor version to another, such as from version `1.11.0` to
     8  version `{{ config.pach_latest_version }}` do not introduce breaking changes. Therefore, the upgrade
     9  procedure is simple and requires little to no downtime.
    10  
    11  !!! warning
    12      Do not use these steps to upgrade between major versions because
    13      it might result in data corruption.
    14  
    15  To upgrade Pachyderm to a minor version, complete the following steps:
    16  
    17  1. Back up your cluster as described in the [Backup and Restore](../backup_restore/#general-backup-procedure)
    18  section.
    19  
    20  1. Destroy your Pachyderm cluster:
    21  
    22     ```
    23     pachctl undeploy
    24     ```
    25  
    26  1. Upgrade `pachctl` by using `brew` for macOS or `apt` for Linux:
    27  
    28     **Example:**
    29  
    30     ```shell
    31     brew upgrade pachyderm/tap/pachctl@1.11
    32     ```
    33  
    34     **System response:**
    35  
    36     ```shell
    37     ==> Upgrading 1 outdated package:
    38     pachyderm/tap/pachctl@1.11
    39     ==> Upgrading pachyderm/tap/pachctl@1.11
    40     ...
    41     ```
    42  
    43     **Note:** You need to specify the version of `pachctl` to which
    44     you want to upgrade. For example, if you want to upgrade `1.11.0` to
    45     `{{ config.pach_latest_version }}`, add `@1.9` at the end of the upgrade path.
    46  
    47  1. Confirm that the new version has been successfully installed by running
    48  the following command:
    49  
    50     ```shell
    51     pachctl version --client-only
    52     ```
    53  
    54     **System response:**
    55  
    56     ```shell
    57     COMPONENT           VERSION
    58     pachctl             {{ config.pach_latest_version }}
    59     ```
    60  
    61  1. Redeploy Pachyderm by running the `pachctl deploy` command
    62  with the same arguments, fields, and storage resources
    63  that you specified when you deployed the previous version
    64  of Pachyderm:
    65  
    66     ```shell
    67     pachctl deploy <args>
    68     ```
    69  
    70     **System response:**
    71  
    72     ```shell
    73     serviceaccount "pachyderm" created
    74     storageclass "etcd-storage-class" created
    75     service "etcd-headless" created
    76     statefulset "etcd" created
    77     service "etcd" created
    78     service "pachd" created
    79     deployment "pachd" created
    80     service "dash" created
    81     deployment "dash" created
    82     secret "pachyderm-storage-secret" created
    83  
    84     Pachyderm is launching. Check its status with "kubectl get all"
    85     Once launched, access the dashboard by running "pachctl port-forward"
    86     ```
    87  
    88     The deployment takes some time. You can run `kubectl get pods` periodically
    89     to check the status of the deployment. When Pachyderm is deployed, the command
    90     shows all pods as `READY`:
    91  
    92  
    93     ```shell
    94     kubectl get pods
    95     ```
    96  
    97     **System response:**
    98  
    99     ```shell
   100     NAME                     READY     STATUS    RESTARTS   AGE
   101     dash-482120938-np8cc     2/2       Running   0          4m
   102     etcd-0                   1/1       Running   0          4m
   103     pachd-3677268306-9sqm0   1/1       Running   0          4m
   104     ```
   105  
   106  1. Verify that the new version has been deployed:
   107  
   108     ```shell
   109     pachctl version
   110     ```
   111  
   112     **System response:**
   113  
   114     ```shell
   115     COMPONENT           VERSION
   116     pachctl             {{ config.pach_latest_version }}
   117     pachd               {{ config.pach_latest_version }}
   118     ```
   119  
   120     The `pachd` and `pachctl` versions must both match the new version.
   121  
   122  ## Troubleshooting Minor Upgrades
   123  
   124  <!-- We might want to move this section to Troubleshooting -->
   125  
   126  This section describes issues that you might run into when
   127  upgrading Pachyderm and provides guidelines on how to resolve
   128  them.
   129  
   130  ### StatefulSets vs static persistent volumes
   131  
   132  [StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) are a mechanism provided in Kubernetes 1.9 and newer to manage the deployment and scaling of applications. 
   133  It can use Persistent Volume Provisioning or pre-provisioned PV’s,
   134  both of which are dynamically allocated from Pachyderm's point of view.
   135  Thus, the `--dynamic-etcd-nodes` flag to `pachctl deploy` is used to deploy Pachyderm using StatefulSets.
   136  
   137  It is recommended that you deploy Pachyderm using StatefulSets when possible. 
   138  All of the instructions for cloud provider deployments do this by default.
   139  We also provide [instructions for on-premises deployments using StatefulSets](../../deploy/on_premises/#statefulsets).
   140  
   141  If you have deployed Pachyderm using StatefulSets, 
   142  you can still use the *same* deploy command to re-deploy Pachyderm. 
   143  Kubernetes is smart enough to see the previously utilized volumes and re-use them.
   144  
   145  ### `etcd` re-deploy problems
   146  
   147  Depending on the cloud you are deploying to and the previous deployment configuration, 
   148  we have seen certain cases in which volumes don't get attached to the right nodes on re-deploy (especially when using AWS). 
   149  In these scenarios, you may see the `etcd` pod stuck in a `Pending`, `CrashLoopBackoff`, or other failed state. 
   150  Most often, deleting the corresponding `etcd` pod(s) or nodes to redeploy them 
   151  or re-deploying all of Pachyderm again will fix the issue. 
   152  
   153  ### `AlreadyExists` errors on re-deploy
   154  
   155  Occasionally, you might see errors similar to the following:
   156  
   157  ```
   158  Error from server (AlreadyExists): error when creating "STDIN": secrets "pachyderm-storage-secret" already exists
   159  ```
   160  
   161  This might happen when re-deploying the enterprise dashboard, for example. These warning are benign.
   162  
   163  ### `pachctl` connnection problems
   164  
   165  When you upgrade Pachyderm versions, you may lose your local `port-forward` to connect `pachctl` to your cluster. 
   166  If you are not using `port-forward` and you are instead setting pachd address config value to connect `pachctl` to your cluster, 
   167  the IP address for Pachyderm may have changed. 
   168  
   169  To fix problems with connections to `pachd` after upgrading, you can perform the appropriate remedy for your situation:
   170  
   171  - Re-run `pachctl port-forward &`, or
   172  - Set the pachd address config value to the updated value, e.g.: ```pachctl config update context `pachctl config get active-context` --pachd-address=<cluster ip>:30650```
   173  
   174  
   175  
   176  
   177  
   178  
   179  
   180  
   181