github.com/argoproj/argo-cd/v2@v2.10.5/docs/proposals/server-side-apply.md (about)

     1  ---
     2  title: Server-Side Apply
     3  
     4  authors:
     5  - "@leoluz"
     6  
     7  sponsors:
     8  - TBD
     9  
    10  reviewers:
    11  - TBD
    12  
    13  approvers:
    14  - TBD
    15  
    16  creation-date: 2022-03-17
    17  
    18  ---
    19  
    20  # Server-Side Apply support for ArgoCD
    21  
    22  [Server-Side Apply (SSA)][1] allows calculating the final patch to update
    23  resources in Kubernetes in the server instead of the client. This proposal
    24  describes how ArgoCD can leverage SSA during syncs.
    25  
    26  * [Open Questions](#open-questions)
    27      * [[Q-1] How to handle conflicts?](#q-1-how-to-handle-conflicts)
    28      * [[Q-2] Should we support multiple managers?](#q-2-should-we-support-multiple-managers)
    29  * [Summary](#summary)
    30  * [Motivation](#motivation)
    31      * [Better interoperability with Admission Controllers](#better-interoperability-with-admission-controllers)
    32      * [Better resource conflict management](#better-resource-conflict-management)
    33      * [Better CRD support](#better-crd-support)
    34  * [Goals](#goals)
    35      * [[G-1] Fine grained configuration](#g-1-fine-grained-configuration)
    36      * [[G-2] Strategic merge patch while diffing](#g-2-strategic-merge-patch-while-diffing)
    37      * [[G-3] Admission Controllers compatibility](#g-3-admission-controllers-compatibility)
    38      * [[G-4] Conflict management](#g-4-conflict-management)
    39      * [[G-5] Register a proper manager](#g-5-register-a-proper-manager)
    40  * [Non-Goals](#non-goals)
    41  * [Proposal](#proposal)
    42      * [Non-Functional Requirements](#non-functional-requirements)
    43      * [Use cases](#use-cases)
    44          * [[UC-1]: enable SSA at the controller level](#uc-1-as-a-user-i-would-like-enable-ssa-at-the-controller-level-so-all-application-are-applied-server-side)
    45          * [[UC-2]: enable SSA at the Application level](#uc-2-as-a-user-i-would-like-enable-ssa-at-the-application-level-so-all-resources-are-applied-server-side)
    46          * [[UC-3]: enable SSA at the resource level](#uc-3-as-a-user-i-would-like-enable-ssa-at-the-resource-level-so-only-a-single-manifest-is-applied-server-side)
    47      * [Security Considerations](#security-considerations)
    48      * [Risks and Mitigations](#risks-and-mitigations)
    49          * [[R-1] Supported K8s version check](#r-1-supported-k8s-version-check)
    50          * [[R-2] Alternating Server-Side Client-Side syncs](#r-2-alternating-server-side-client-side-syncs)
    51      * [Upgrade / Downgrade](#upgrade--downgrade)
    52  * [Drawbacks](#drawbacks)
    53  
    54  ---
    55  
    56  ## Open Questions
    57  
    58  ### [Q-1] How to handle conflicts?
    59  When SSA is enabled, the server may return field conflicts with other managers.
    60  What ArgoCD controller should do in case of conflict? Just force the sync and
    61  log warnings (like some other controllers do?)
    62  
    63  #### Conclusion
    64  The first version should use the force flag and override even if there are
    65  conflicts. We could improve and add other options once there is a use case.
    66  
    67  ### [Q-2] Should we support multiple managers?
    68  Should Server-Side Apply support in ArgoCD be implemented allowing multiple
    69  managers for the same controller? ([more details][10])
    70  
    71  ## Summary
    72  
    73  ArgoCD can benefit from [Server-Side Apply][1] during syncs. A few
    74  improvements to consider:
    75  
    76  - More reliable dry-runs (as admission controller is executed) ([ISSUE-804][5])
    77  - [Syncs always run][2] mutating webhooks (even without diff)
    78  - [Fix big CRD][3] sync issues
    79  - Better interoperability with different controllers
    80  
    81  Kubernetes SSA Proposal ([KEP-555][13]) has more details about how it works.
    82  
    83  ## Motivation
    84  
    85  ArgoCD uses kubectl library while syncing resources in the cluster. Kubectl uses
    86  by default a 3-way-merge logic between the live state (in k8s), desired state
    87  (in git) and the previous state (`last-applied-configuration` annotation) to
    88  calculate diffs and patch resources in the cluster. This logic is executed in
    89  the client (ArgoCD) and once the patch is calculated it is then sent to the
    90  server.
    91  
    92  This strategy works well in the majority of the use cases. However, there are
    93  some scenarios where calculating patches in the client side can cause problems.
    94  
    95  Some of the known problems about using client-side approach:
    96  
    97  ### Better interoperability with Admission Controllers 
    98  
    99  More and more users are adopting and configuring [Admission Controllers][4] in
   100  Kubernetes with different flavors of Validating Webhooks and Mutating Webhooks.
   101  Admission Controllers will only execute in server-side. In cases where users
   102  rely on dry-run executions to decide if they should proceed with a deployment,
   103  having the patch calculated at the client side might provide undesired results.
   104  Having SSA enabled syncs also guaranties that Admission Controllers are always
   105  executed, even when there is no diff in the resource.
   106  
   107  ### Better resource conflict management
   108  
   109  Server-Side Apply will better handle and identify conflicts during syncs by
   110  analyzing the `managedFields` metadata available in all Kubernetes resources
   111  (since kubernetes 1.18). 
   112  
   113  ### Better CRD support
   114  
   115  By not having to rely on the `last-applied-configuration` annotation, SSA would
   116  help with failing syncs caused by exceeded annotation size limit. This is a
   117  common issue when syncing CRDs with large schemas.
   118  
   119  ## Goals
   120  
   121  All following goals should be achieve in order to conclude this proposal:
   122  
   123  #### [G-1] Fine grained configuration
   124  
   125  - Provide the ability for users to define if they want to use SSA during syncs
   126  - Users should be able to enable SSA at the controller level (via binary flag)
   127  (see [UC-1](#uc-1-as-a-user-i-would-like-enable-ssa-at-the-controller-level-so-all-application-are-applied-server-side))
   128  - Users should be able to enable SSA for a given Application (via syncOptions)
   129  (see [UC-2](#uc-2-as-a-user-i-would-like-enable-ssa-at-the-application-level-so-all-resources-are-applied-server-side))
   130  - Users should be able to enable SSA at resource level (via annotation) (see
   131  [UC-3](#uc-3-as-a-user-i-would-like-enable-ssa-at-the-resource-level-so-only-a-single-manifest-is-applied-server-side)
   132  - Relates to [ISSUE-2267][6]
   133  
   134  #### [G-2] Strategic merge patch while diffing
   135  
   136  - Diffing needs to support strategic merge patch (see [ISSUE-2268][7])
   137  - Make sure Services can be patched correctly ([more details][14])
   138  
   139  #### [G-3] Admission Controllers compatibility
   140  
   141  - Allow Admission Controllers to execute even when there is no diff for a
   142    particular resource. (Needs investigation) ([more details][2])
   143  
   144  #### [G-4] Conflict management
   145  
   146  - ArgoCD should respect field ownership and provide a configuration to allow
   147    users to define the behavior in case of conflicts (see
   148    [Q-1](#q-1-how-to-handle-conflicts) outcome)
   149  
   150  #### [G-5] Register a proper manager
   151  
   152  - ArgoCD must register itself with a pre-defined manager (suggestion:
   153    `argocd-controller`). It shouldn't rely on the default value defined in the
   154    kubectl code. ([more details][11])
   155  
   156  ## Non-Goals
   157  
   158  TBD
   159  
   160  ## Proposal
   161  
   162  Change ArgoCD controller to accept new parameter to enable Server-Side Apply
   163  during syncs. Changes are necessary in ArgoCD as well as in
   164  gitops-engine library.
   165  
   166  ### Use cases
   167  
   168  The following use cases should be implemented:
   169  
   170  #### [UC-1]: As a user, I would like enable SSA at the controller level so all Application are applied server-side
   171  
   172  Implement a binary flag to configure ArgoCD to run all syncs using SSA.
   173  (suggestion: `--server-side-apply=true`). Default value should be `false`.
   174  
   175  #### [UC-2]: As a user, I would like enable SSA at the Application level so all resources are applied server-side
   176  
   177  Implement a new syncOption to allow users to enable SSA at the application
   178  level (Suggestion `ServerSideApply=true`). UI needs to be updated to support
   179  this new sync option. If not informed, the controller should keep the current
   180  behaviour (client-side).
   181  
   182  #### [UC-3]: As a user, I would like enable SSA at the resource level so only a single manifest is applied server-side
   183  
   184  Leverage the existing `argocd.argoproj.io/sync-options` annotation allowing the
   185  `ServerSideApply=true` to be informed at the resource level. Must not impact
   186  other sync-options informed in the annotation (make sure this annotation
   187  supports providing multiple options).
   188  
   189  ### Security Considerations
   190  
   191  TBD
   192  
   193  ### Risks and Mitigations
   194  
   195  #### [R-1] Supported K8s version check
   196  
   197  ArgoCD must check if the target Kubernetes cluster has full support for SSA. The
   198  feature turned [GA in Kubernetes 1.22][8]. Full support for managed fields was
   199  introduced as [beta in Kubernetes 1.18][9]. The implementation must check that
   200  the target kubernetes cluster is running at least version 1.18. If SSA is
   201  enabled and target cluster version < 1.18 ArgoCD should log warning and fallback
   202  to client sync.
   203  
   204  #### [R-2] Alternating Server-Side Client-Side syncs
   205  
   206  Kubernetes SSA proposal ([KEP-555][13]) mentions about alternating between
   207  server-side and client-side applies in the [Upgrade/Downgrade Strategy][12]
   208  section. It is stated that Kubernetes will verify the incoming apply request
   209  validating if the user-agent is `kubectl` to decide if the
   210  `last-applied-configuration` annotation should be updated. ArgoCD relies on this
   211  annotation and the implementation must make sure that this agent is correctly
   212  informed when changing to server-side apply and specifying a manager different
   213  than `kubectl`. This is mainly to make sure that
   214  [G-5](#g-5-register-a-proper-manager) isn't impacting the
   215  client-side/server-side compatibility.
   216  
   217  ### Upgrade / Downgrade
   218  
   219  No CRD update necessary as `syncOption` field in Application resource is non-typed
   220  (string array). Upgrade will only require ArgoCD controller update.
   221  
   222  ## Drawbacks
   223  
   224  Slight increase in ArgoCD code base complexity.
   225  
   226  [1]: https://kubernetes.io/docs/reference/using-api/server-side-apply/
   227  [2]: https://github.com/argoproj/argo-cd/issues/2267#issuecomment-920445236
   228  [3]: https://github.com/prometheus-community/helm-charts/issues/1500#issuecomment-1017961377
   229  [4]: https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
   230  [5]: https://github.com/argoproj/argo-cd/issues/804
   231  [6]: https://github.com/argoproj/argo-cd/issues/2267
   232  [7]: https://github.com/argoproj/argo-cd/issues/2268
   233  [8]: https://kubernetes.io/blog/2021/08/06/server-side-apply-ga/
   234  [9]: https://kubernetes.io/blog/2020/04/01/kubernetes-1.18-feature-server-side-apply-beta-2/
   235  [10]: https://github.com/argoproj/gitops-engine/pull/363#issuecomment-1013641708
   236  [11]: https://github.com/argoproj/gitops-engine/pull/363#issuecomment-1013289982
   237  [12]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/555-server-side-apply/README.md#upgrade--downgrade-strategy
   238  [13]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/555-server-side-apply/README.md
   239  [14]: https://github.com/argoproj/argo-cd/pull/8812#discussion_r849140565