github.com/argoproj/argo-cd/v2@v2.10.9/docs/operator-manual/security.md (about)

     1  # Security
     2  
     3  Argo CD has undergone rigorous internal security reviews and penetration testing to satisfy [PCI
     4  compliance](https://www.pcisecuritystandards.org) requirements. The following are some security
     5  topics and implementation details of Argo CD.
     6  
     7  ## Authentication
     8  
     9  Authentication to Argo CD API server is performed exclusively using [JSON Web Tokens](https://jwt.io)
    10  (JWTs). Username/password bearer tokens are not used for authentication. The JWT is obtained/managed
    11  in one of the following ways:
    12  
    13  1. For the local `admin` user, a username/password is exchanged for a JWT using the `/api/v1/session`
    14     endpoint. This token is signed & issued by the Argo CD API server itself and it expires after 24 hours 
    15     (this token used not to expire, see [CVE-2021-26921](https://github.com/argoproj/argo-cd/security/advisories/GHSA-9h6w-j7w4-jr52)).
    16     When the admin password is updated, all existing admin JWT tokens are immediately revoked.
    17     The password is stored as a bcrypt hash in the [`argocd-secret`](https://github.com/argoproj/argo-cd/blob/master/manifests/base/config/argocd-secret.yaml) Secret.
    18  
    19  2. For Single Sign-On users, the user completes an OAuth2 login flow to the configured OIDC identity
    20     provider (either delegated through the bundled Dex provider, or directly to a self-managed OIDC
    21     provider). This JWT is signed & issued by the IDP, and expiration and revocation is handled by
    22     the provider. Dex tokens expire after 24 hours.
    23  
    24  3. Automation tokens are generated for a project using the `/api/v1/projects/{project}/roles/{role}/token`
    25     endpoint, and are signed & issued by Argo CD. These tokens are limited in scope and privilege,
    26     and can only be used to manage application resources in the project which it belongs to. Project
    27     JWTs have a configurable expiration and can be immediately revoked by deleting the JWT reference
    28     ID from the project role.
    29  
    30  ## Authorization
    31  
    32  Authorization is performed by iterating the list of group membership in a user's JWT groups claims,
    33  and comparing each group against the roles/rules in the [RBAC](../rbac) policy. Any matched rule
    34  permits access to the API request.
    35  
    36  ## TLS
    37  
    38  All network communication is performed over TLS including service-to-service communication between
    39  the three components (argocd-server, argocd-repo-server, argocd-application-controller). The Argo CD
    40  API server can enforce the use of TLS 1.2 using the flag: `--tlsminversion 1.2`.
    41  Communication with Redis is performed over plain HTTP by default. TLS can be setup with command line arguments.
    42  
    43  ## Git & Helm Repositories
    44  
    45  Git and helm repositories are managed by a stand-alone service, called the repo-server. The
    46  repo-server does not carry any Kubernetes privileges and does not store credentials to any services
    47  (including git). The repo-server is responsible for cloning repositories which have been permitted
    48  and trusted by Argo CD operators, and generating Kubernetes manifests at a given path in the
    49  repository. For performance and bandwidth efficiency, the repo-server maintains local clones of
    50  these repositories so that subsequent commits to the repository are efficiently downloaded.
    51  
    52  There are security considerations when configuring git repositories that Argo CD is permitted to
    53  deploy from. In short, gaining unauthorized write access to a git repository trusted by Argo CD
    54  will have serious security implications outlined below.
    55  
    56  ### Unauthorized Deployments
    57  
    58  Since Argo CD deploys the Kubernetes resources defined in git, an attacker with access to a trusted
    59  git repo would be able to affect the Kubernetes resources which are deployed. For example, an
    60  attacker could update the deployment manifest deploy malicious container images to the environment,
    61  or delete resources in git causing them to be pruned in the live environment.
    62  
    63  ### Tool command invocation
    64  
    65  In addition to raw YAML, Argo CD natively supports two popular Kubernetes config management tools,
    66  helm and kustomize. When rendering manifests, Argo CD executes these config management tools
    67  (i.e. `helm template`, `kustomize build`) to generate the manifests. It is possible that an attacker
    68  with write access to a trusted git repository may construct malicious helm charts or kustomizations
    69  that attempt to read files out-of-tree. This includes adjacent git repos, as well as files on the
    70  repo-server itself. Whether or not this is a risk to your organization depends on if the contents
    71  in the git repos are sensitive in nature. By default, the repo-server itself does not contain
    72  sensitive information, but might be configured with Config Management Plugins which do
    73  (e.g. decryption keys). If such plugins are used, extreme care must be taken to ensure the
    74  repository contents can be trusted at all times.
    75  
    76  Optionally the built-in config management tools might be individually disabled.
    77  If you know that your users will not need a certain config management tool, it's advisable
    78  to disable that tool.
    79  See [Tool Detection](../user-guide/tool_detection.md) for more information.
    80  
    81  ### Remote bases and helm chart dependencies
    82  
    83  Argo CD's repository allow-list only restricts the initial repository which is cloned. However, both
    84  kustomize and helm contain features to reference and follow *additional* repositories
    85  (e.g. kustomize remote bases, helm chart dependencies), of which might not be in the repository
    86  allow-list. Argo CD operators must understand that users with write access to trusted git
    87  repositories could reference other remote git repositories containing Kubernetes resources not
    88  easily searchable or auditable in the configured git repositories.
    89  
    90  ## Sensitive Information
    91  
    92  ### Secrets
    93  
    94  Argo CD never returns sensitive data from its API, and redacts all sensitive data in API payloads
    95  and logs. This includes:
    96  
    97  * cluster credentials
    98  * Git credentials
    99  * OAuth2 client secrets
   100  * Kubernetes Secret values
   101  
   102  ### External Cluster Credentials
   103  
   104  To manage external clusters, Argo CD stores the credentials of the external cluster as a Kubernetes
   105  Secret in the argocd namespace. This secret contains the K8s API bearer token associated with the
   106  `argocd-manager` ServiceAccount created during `argocd cluster add`, along with connection options
   107  to that API server (TLS configuration/certs, AWS role-arn, etc...).
   108  The information is used to reconstruct a REST config and kubeconfig to the cluster used by Argo CD
   109  services.
   110  
   111  To rotate the bearer token used by Argo CD, the token can be deleted (e.g. using kubectl) which
   112  causes Kubernetes to generate a new secret with a new bearer token. The new token can be re-inputted
   113  to Argo CD by re-running `argocd cluster add`. Run the following commands against the *_managed_*
   114  cluster:
   115  
   116  ```bash
   117  # run using a kubeconfig for the externally managed cluster
   118  kubectl delete secret argocd-manager-token-XXXXXX -n kube-system
   119  argocd cluster add CONTEXTNAME
   120  ```
   121  
   122  !!! note
   123      Kubernetes 1.24 [stopped automatically creating tokens for Service Accounts](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#no-really-you-must-read-this-before-you-upgrade).
   124      [Starting in Argo CD 2.4](https://github.com/argoproj/argo-cd/pull/9546), `argocd cluster add` creates a 
   125      ServiceAccount _and_ a non-expiring Service Account token Secret when adding 1.24 clusters. In the future, Argo CD 
   126      will [add support for the Kubernetes TokenRequest API](https://github.com/argoproj/argo-cd/issues/9610) to avoid 
   127      using long-lived tokens.
   128  
   129  To revoke Argo CD's access to a managed cluster, delete the RBAC artifacts against the *_managed_*
   130  cluster, and remove the cluster entry from Argo CD:
   131  
   132  ```bash
   133  # run using a kubeconfig for the externally managed cluster
   134  kubectl delete sa argocd-manager -n kube-system
   135  kubectl delete clusterrole argocd-manager-role
   136  kubectl delete clusterrolebinding argocd-manager-role-binding
   137  argocd cluster rm https://your-kubernetes-cluster-addr
   138  ```
   139  <!-- markdownlint-disable MD027 -->
   140  > NOTE: for AWS EKS clusters, the [get-token](https://docs.aws.amazon.com/cli/latest/reference/eks/get-token.html) command
   141    is used to authenticate to the external cluster, which uses IAM roles in lieu of locally stored
   142    tokens, so token rotation is not needed, and revocation is handled through IAM.
   143  <!-- markdownlint-enable MD027 -->
   144  
   145  ## Cluster RBAC
   146  
   147  By default, Argo CD uses a [clusteradmin level role](https://github.com/argoproj/argo-cd/blob/master/manifests/base/application-controller/argocd-application-controller-role.yaml)
   148  in order to:
   149  
   150  1. watch & operate on cluster state
   151  2. deploy resources to the cluster
   152  
   153  Although Argo CD requires cluster-wide **_read_** privileges to resources in the managed cluster to
   154  function properly, it does not necessarily need full **_write_** privileges to the cluster. The
   155  ClusterRole used by argocd-server and argocd-application-controller can be modified such
   156  that write privileges are limited to only the namespaces and resources that you wish Argo CD to
   157  manage.
   158  
   159  To fine-tune privileges of externally managed clusters, edit the ClusterRole of the `argocd-manager-role`
   160  
   161  ```bash
   162  # run using a kubeconfig for the externally managed cluster
   163  kubectl edit clusterrole argocd-manager-role
   164  ```
   165  
   166  To fine-tune privileges which Argo CD has against its own cluster (i.e. `https://kubernetes.default.svc`),
   167  edit the following cluster roles where Argo CD is running in:
   168  
   169  ```bash
   170  # run using a kubeconfig to the cluster Argo CD is running in
   171  kubectl edit clusterrole argocd-server
   172  kubectl edit clusterrole argocd-application-controller
   173  ```
   174  
   175  !!! tip
   176      If you want to deny Argo CD access to a kind of resource then add it as an [excluded resource](declarative-setup.md#resource-exclusion).
   177  
   178  ## Auditing
   179  
   180  As a GitOps deployment tool, the Git commit history provides a natural audit log of what changes
   181  were made to application configuration, when they were made, and by whom. However, this audit log
   182  only applies to what happened in Git and does not necessarily correlate one-to-one with events
   183  that happen in a cluster. For example, User A could have made multiple commits to application
   184  manifests, but User B could have just only synced those changes to the cluster sometime later.
   185  
   186  To complement the Git revision history, Argo CD emits Kubernetes Events of application activity,
   187  indicating the responsible actor when applicable. For example:
   188  
   189  ```bash
   190  $ kubectl get events
   191  LAST SEEN   FIRST SEEN   COUNT   NAME                         KIND          SUBOBJECT   TYPE      REASON               SOURCE                          MESSAGE
   192  1m          1m           1       guestbook.157f7c5edd33aeac   Application               Normal    ResourceCreated      argocd-server                   admin created application
   193  1m          1m           1       guestbook.157f7c5f0f747acf   Application               Normal    ResourceUpdated      argocd-application-controller   Updated sync status:  -> OutOfSync
   194  1m          1m           1       guestbook.157f7c5f0fbebbff   Application               Normal    ResourceUpdated      argocd-application-controller   Updated health status:  -> Missing
   195  1m          1m           1       guestbook.157f7c6069e14f4d   Application               Normal    OperationStarted     argocd-server                   admin initiated sync to HEAD (8a1cb4a02d3538e54907c827352f66f20c3d7b0d)
   196  1m          1m           1       guestbook.157f7c60a55a81a8   Application               Normal    OperationCompleted   argocd-application-controller   Sync operation to 8a1cb4a02d3538e54907c827352f66f20c3d7b0d succeeded
   197  1m          1m           1       guestbook.157f7c60af1ccae2   Application               Normal    ResourceUpdated      argocd-application-controller   Updated sync status: OutOfSync -> Synced
   198  1m          1m           1       guestbook.157f7c60af5bc4f0   Application               Normal    ResourceUpdated      argocd-application-controller   Updated health status: Missing -> Progressing
   199  1m          1m           1       guestbook.157f7c651990e848   Application               Normal    ResourceUpdated      argocd-application-controller   Updated health status: Progressing -> Healthy
   200  ```
   201  
   202  These events can be then be persisted for longer periods of time using other tools as
   203  [Event Exporter](https://github.com/GoogleCloudPlatform/k8s-stackdriver/tree/master/event-exporter) or
   204  [Event Router](https://github.com/heptiolabs/eventrouter).
   205  
   206  ## WebHook Payloads
   207  
   208  Payloads from webhook events are considered untrusted. Argo CD only examines the payload to infer
   209  the involved applications of the webhook event (e.g. which repo was modified), then refreshes
   210  the related application for reconciliation. This refresh is the same refresh which occurs regularly
   211  at three minute intervals, just fast-tracked by the webhook event.
   212  
   213  ## Logging
   214  
   215  ### Security field
   216  
   217  Security-related logs are tagged with a `security` field to make them easier to find, analyze, and report on.
   218  
   219  | Level | Friendly Level | Description                                                                                       | Example                                     |
   220  |-------|----------------|---------------------------------------------------------------------------------------------------|---------------------------------------------|
   221  | 1     | Low            | Unexceptional, non-malicious events                                                               | Successful access                           |
   222  | 2     | Medium         | Could indicate malicious events, but has a high likelihood of being user/system error             | Access denied                               |
   223  | 3     | High           | Likely malicious events but one that had no side effects or was blocked                           | Out of bounds symlinks in repo              |
   224  | 4     | Critical       | Any malicious or exploitable event that had a side effect                                         | Secrets being left behind on the filesystem |
   225  | 5     | Emergency      | Unmistakably malicious events that should NEVER occur accidentally and indicates an active attack | Brute forcing of accounts                   |
   226  
   227  Where applicable, a `CWE` field is also added specifying the [Common Weakness Enumeration](https://cwe.mitre.org/index.html) number.
   228  
   229  !!! warning
   230      Please be aware that not all security logs are comprehensively tagged yet and these examples are not necessarily implemented.
   231  
   232  ### API Logs
   233  
   234  Argo CD logs payloads of most API requests except request that are considered sensitive, such as
   235  `/cluster.ClusterService/Create`, `/session.SessionService/Create` etc. The full list of method
   236  can be found in [server/server.go](https://github.com/argoproj/argo-cd/blob/abba8dddce8cd897ba23320e3715690f465b4a95/server/server.go#L516).
   237  
   238  Argo CD does not log IP addresses of clients requesting API endpoints, since the API server is typically behind a proxy. Instead, it is recommended
   239  to configure IP addresses logging in the proxy server that sits in front of the API server.
   240  
   241  ## ApplicationSets
   242  
   243  Argo CD's ApplicationSets feature has its own [security considerations](./applicationset/Security.md). Be aware of those
   244  issues before using ApplicationSets.
   245  
   246  ## Limiting Directory App Memory Usage
   247  
   248  > >2.2.10, 2.1.16, >2.3.5
   249  
   250  Directory-type Applications (those whose source is raw JSON or YAML files) can consume significant
   251  [repo-server](architecture.md#repository-server) memory, depending on the size and structure of the YAML files.
   252  
   253  To avoid over-using memory in the repo-server (potentially causing a crash and denial of service), set the
   254  `reposerver.max.combined.directory.manifests.size` config option in [argocd-cmd-params-cm](argocd-cmd-params-cm.yaml).
   255  
   256  This option limits the combined size of all JSON or YAML files in an individual app. Note that the in-memory
   257  representation of a manifest may be as much as 300x the size of the manifest on disk. Also note that the limit is per
   258  Application. If manifests are generated for multiple applications at once, memory usage will be higher.
   259  
   260  **Example:**
   261  
   262  Suppose your repo-server has a 10G memory limit, and you have ten Applications which use raw JSON or YAML files. To
   263  calculate the max safe combined file size per Application, divide 10G by 300 * 10 Apps (300 being the worst-case memory
   264  growth factor for the manifests).
   265  
   266  ```
   267  10G / 300 * 10 = 3M
   268  ```
   269  
   270  So a reasonably safe configuration for this setup would be a 3M limit per app.
   271  
   272  ```yaml
   273  apiVersion: v1
   274  kind: ConfigMap
   275  metadata:
   276    name: argocd-cmd-params-cm
   277  data:
   278    reposerver.max.combined.directory.manifests.size: '3M'
   279  ```
   280  
   281  The 300x ratio assumes a maliciously-crafted manifest file. If you only want to protect against accidental excessive
   282  memory use, it is probably safe to use a smaller ratio.
   283  
   284  Keep in mind that if a malicious user can create additional Applications, they can increase the total memory usage.
   285  Grant [App creation privileges](rbac.md) carefully.