github.com/argoproj/argo-cd/v3@v3.2.1/docs/proposals/resource-deletion-with-approval.md (about)

     1  ---
     2  title: Neat-enhancement-idea
     3  authors:
     4  - "@alexmt"
     5    sponsors:
     6  - TBD
     7    reviewers:
     8  - "@jessesuen"
     9  - TBD
    10    approvers:
    11  - "@jessesuen"
    12  - TBD
    13  
    14  creation-date: 2020-04-19
    15  last-updated: 2020-04-19
    16  
    17  ---
    18  # Neat Enhancement Idea
    19  
    20  Support manual approval for pruning and deleting Kubernetes resources during application syncing/deletion.
    21  
    22  ## Summary
    23  
    24  Introduce Kubernetes resource-level annotations that require manual user approval using Argo CD UI/CLI/API before the
    25  resource is pruned or deleted. The annotations should be respected while Argo CD attempts to synchronize or delete the
    26  application.
    27  
    28  ## Motivation
    29  
    30  We’ve seen cases where Argo CD deleted Kubernetes resources due to a bug or misconfiguration.​ Examples include [corrupted
    31  data](https://github.com/argoproj/argo-cd/issues/4423) in Redis, user errors
    32  ([1](https://github.com/argoproj/argo-cd/issues/9093), [2](https://github.com/argoproj/argo-cd/issues/4844))
    33  and [bug](https://github.com/argoproj/argo-cd/issues/3473) in the automation on top of Argo CD. These examples don’t
    34  mean Argo CD is not reliable; however, there are cases where misbehavior is catastrophic, and erroneous deletion is not
    35  acceptable. Examples include the app-of-apps pattern where Argo CD is used to manage itself, or namespaces in production
    36  clusters.
    37  
    38  ### Goals
    39  
    40  The goals of a proposal ares:
    41  
    42  #### Allow developers to mark resources that require manual approval before application deletion.
    43  
    44  Developers should be able to add an annotation to resources that require manual approval before deletion. The annotation
    45  should be respected by Argo CD when it attempts to delete the application.
    46  
    47  #### Allow developers to mark resources that require manual approval before pruning
    48  
    49  Developers should be able to add an annotation to resources that require manual approval before pruning. The annotation
    50  should be respected by Argo CD when it attempts to prune extra resources while syncing the application.
    51  
    52  ### Non-Goals
    53  
    54  #### Implement automatic self check while deleting resources
    55  
    56  We've made our best effort to implement corrected behavior, and as of now, we are not aware of any bugs that cause
    57  erroneous deletion. The goal of this proposal is to provide a safety net for cases where deletion is not acceptable.
    58  
    59  ## Proposal
    60  
    61  It is proposed to introduce two new sync options for Argo CD applications: `Prune=confirm` and `Delete=confirm`. Options would
    62  protect resources from accidental deletion during cascading application deletion as well as during sync operations.
    63  
    64  ### Introduce `confirm` option for Prune sync option.
    65  
    66  Argo CD already supports `argocd.argoproj.io/sync-options: Prune=false` sync option that prevents resource deletion while syncing
    67  the application. This, however, is not ideal since it prevents implementing fully automated workflows that include resource deletion.
    68  
    69  In order to improve the situation, we propose to introduce `confirm` option for Prune sync option. When `confirm` option is set, Argo CD should pause the sync operation
    70  **before deleting any app resources** and wait for the user to confirm the deletion. The confirmation can be done in a very friendly way using Argo CD UI, CLI or API.
    71  
    72  * **Sync Operation status**. I suggest not to introduce new sync operation states to avoid disturbing the existing automation around syncing (CI pipelines, scripts etc). 
    73    If Argo CD is waiting for the operation state should remain `Progressing`. Once the user confirms the deletion, the operation should resume.
    74  * **Sync Waves**. The sync wave should be "paused" while Argo CD is waiting for the user to confirm the deletion. No difference from waiting for the resource to became healthy.
    75  
    76  ### Introduce `confirm` option for Delete sync option.
    77  
    78  Similarly to `Prune` sync option we need to introduce `confirm` value for `Delete` sync option: `argocd.argoproj.io/sync-options: Delete=confirm`. The `confirm` option
    79  should pause the sync operation **before deleting any app resources** and wait for the user to confirm the deletion. The confirmation can be done in a very friendly way
    80  using Argo CD UI, CLI or API.
    81  
    82  
    83  ### Friendly prunning/deletion manual approval
    84  
    85  Since we know Argo CD is often used to implement fully automated developer workflows that include resource deletion, the
    86  deletion approval process should be as painless as possible. This way, platform administrators can instruct end users to
    87  apply the new prune/delete option to resources that require special care without significantly disturbing the developer
    88  experience.
    89  
    90  In both cases where Argo CD requires manual approval, the user should be able to approve the deletion using Argo CD UI,
    91  CLI, or API. The approval process should be as simple as possible and should not require the user to understand the
    92  internals of Argo CD.
    93  
    94  #### New `requiresDeletionApproval` resource field in application status
    95  
    96  A new field `requiresDeletionApproval` should be added to the `status.resources` list items. The field should be set to `true` when the resource deletion approval is required.
    97  
    98  ```yaml
    99    - health:
   100        status: Healthy
   101      kind: Service
   102      name: guestbook-ui
   103      namespace: default
   104      status: OutOfSync
   105      version: v1
   106      requiresPruning: true
   107      requiresDeletionApproval: true # new field that indicates that deletion approval is required
   108  ```
   109  
   110  The Argo CD UI, CLI should visualize the `requiresDeletionApproval` field so that the user can easily discover which resources require manual approval.
   111  
   112  #### Approve deletion resource action
   113  
   114  The Argo CD UI, CLI should bundle the `Approve Deletion` [resource action](https://argo-cd.readthedocs.io/en/stable/operator-manual/resource_actions/)
   115  that would allow the user to approve the deletion. The action should patch the resource with the `argocd.argoproj.io/deletion-approved: true` annotation.
   116  Once annotation is applied the Argo CD should proceed with the deletion.
   117  
   118  The main reason to use the action is that we can reuse existing [RBAC](https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/) to control who can approve the deletion.
   119  
   120  #### UI/CLI Convenience to approve all resources
   121  
   122  The Argo CD UI should provide a convenient way to approve resources that require manual approval. The existing user interface will provide a button that allows end user
   123  execute the `Approve Deletion` action and approve resources one by one. In addition to the single resource approval, the UI should provide a way to approve all resources
   124  that require manual approval. The new button should execute the `Approve Deletion` action for all resources that require manual approval.
   125  
   126  Argo CD CLI would no need changes since existing `argocd app actions run` command allows to execute an action against multiple resources.
   127  
   128  #### Require deletion approval notification
   129  
   130  The default Argo CD notification catalog should include a trigger and notification template that notifies the user when
   131  deletion approval is required. The notification template should include a list of resources that require approval.
   132  
   133  
   134  #### Declarative approval
   135  
   136  The user should be able to approve resource deletion without using the UI or CLI by manually adding the `argocd.argoproj.io/deletion-approved: true` annotation to the resource.
   137  
   138  ### Use cases
   139  
   140  Add a list of detailed use cases this enhancement intends to take care of.
   141  
   142  ## Use case 1:
   143  
   144  As a developer, I would like to mark resources that require manual pruning approval so I can prevent the accidental deletion of critical resources.
   145  
   146  ## Use case 2:
   147  
   148  As a developer, I would like to mark resources that require manual deletion approval so I can prevent the accidental deletion of critical resources.
   149  
   150  
   151  ### Security Considerations
   152  
   153  The resource approval would require a mechanism to control who can approve the deletion. The proposal to use
   154  resource-level actions solves this problem and allows us to reuse the existing RBAC model.
   155  
   156  ### Risks and Mitigations
   157  
   158  None.
   159  
   160  ### Upgrade / Downgrade Strategy
   161  
   162  In case of rollback to the previous version the sync option would be ignored and the resources would be deleted as before.
   163  
   164  ## Open Issues
   165  
   166  The proposal would require end users to learn about the new behavior and adjust their workflows. It includes a set of
   167  enhancements aimed at minimizing the impact on end users.
   168  
   169  ## Alternatives
   170  
   171  None.