github.com/argoproj/argo-cd/v3@v3.2.1/docs/faq.md (about) 1 # FAQ 2 3 ## I've deleted/corrupted my repo and can't delete my app. 4 5 Argo CD can't delete an app if it cannot generate manifests. You need to either: 6 7 1. Reinstate/fix your repo. 8 1. Delete the app using `--cascade=false` and then manually deleting the resources. 9 10 ## Why is my application still `OutOfSync` immediately after a successful Sync? 11 12 See [Diffing](user-guide/diffing.md) documentation for reasons resources can be OutOfSync, and ways to configure Argo CD 13 to ignore fields when differences are expected. 14 15 ## Why is my application stuck in `Progressing` state? 16 17 Argo CD provides health for several standard Kubernetes types. The `Ingress`, `StatefulSet` and `SealedSecret` types have known issues 18 which might cause health check to return `Progressing` state instead of `Healthy`. 19 20 * `Ingress` is considered healthy if `status.loadBalancer.ingress` list is non-empty, with at least one value 21 for `hostname` or `IP`. Some ingress controllers 22 ([contour](https://github.com/heptio/contour/issues/403) 23 , [traefik](https://github.com/argoproj/argo-cd/issues/968#issuecomment-451082913)) don't update 24 `status.loadBalancer.ingress` field which causes `Ingress` to stuck in `Progressing` state forever. 25 26 * `StatefulSet` is considered healthy if value of `status.updatedReplicas` field matches to `spec.replicas` field. Due 27 to Kubernetes bug 28 [kubernetes/kubernetes#68573](https://github.com/kubernetes/kubernetes/issues/68573) the `status.updatedReplicas` is 29 not populated. So unless you run Kubernetes version which include the 30 fix [kubernetes/kubernetes#67570](https://github.com/kubernetes/kubernetes/pull/67570) `StatefulSet` might stay 31 in `Progressing` state. 32 * Your `StatefulSet` or `DaemonSet` is using `OnDelete` instead of `RollingUpdate` strategy. 33 See [#1881](https://github.com/argoproj/argo-cd/issues/1881). 34 * For `SealedSecret`, see [Why are resources of type `SealedSecret` stuck in the `Progressing` state?](#sealed-secret-stuck-progressing) 35 36 As workaround Argo CD allows providing [health check](operator-manual/health.md) customization which overrides default 37 behavior. 38 39 If you are using Traefik for your Ingress, you can update the Traefik config to publish the loadBalancer IP using [publishedservice](https://doc.traefik.io/traefik/providers/kubernetes-ingress/#publishedservice), which will resolve this issue. 40 41 ```yaml 42 providers: 43 kubernetesIngress: 44 publishedService: 45 enabled: true 46 ``` 47 48 ## I forgot the admin password, how do I reset it? 49 50 For Argo CD v1.8 and earlier, the initial password is set to the name of the server pod, as 51 per [the getting started guide](getting_started.md). For Argo CD v1.9 and later, the initial password is available from 52 a secret named `argocd-initial-admin-secret`. 53 54 To change the password, edit the `argocd-secret` secret and update the `admin.password` field with a new bcrypt hash. 55 56 !!! note "Generating a bcrypt hash" 57 Use the following command to generate a bcrypt hash for `admin.password` 58 59 argocd account bcrypt --password <YOUR-PASSWORD-HERE> 60 61 To apply the new password hash, use the following command (replacing the hash with your own): 62 63 ```bash 64 # bcrypt(password)=$2a$10$rRyBsGSHK6.uc8fntPwVIuLVHgsAhAX7TcdrqW/RADU0uh7CaChLa 65 kubectl -n argocd patch secret argocd-secret \ 66 -p '{"stringData": { 67 "admin.password": "$2a$10$rRyBsGSHK6.uc8fntPwVIuLVHgsAhAX7TcdrqW/RADU0uh7CaChLa", 68 "admin.passwordMtime": "'$(date +%FT%T%Z)'" 69 }}' 70 ``` 71 72 Another option is to delete both the `admin.password` and `admin.passwordMtime` keys and restart argocd-server. This 73 will generate a new password as per [the getting started guide](getting_started.md), so either to the name of the pod 74 (Argo CD 1.8 and earlier) 75 or a randomly generated password stored in a secret (Argo CD 1.9 and later). 76 77 ## How to disable admin user? 78 79 Add `admin.enabled: "false"` to the `argocd-cm` ConfigMap 80 (see [user management](./operator-manual/user-management/index.md)). 81 82 ## Argo CD cannot deploy Helm Chart based applications without internet access, how can I solve it? 83 84 Argo CD might fail to generate Helm chart manifests if the chart has dependencies located in external repositories. To 85 solve the problem you need to make sure that `requirements.yaml` 86 uses only internally available Helm repositories. Even if the chart uses only dependencies from internal repos Helm 87 might decide to refresh `stable` repo. As workaround override 88 `stable` repo URL in `argocd-cm` config map: 89 90 ```yaml 91 data: 92 repositories: | 93 - type: helm 94 url: http://<internal-helm-repo-host>:8080 95 name: stable 96 ``` 97 98 ## After deploying my Helm application with Argo CD I cannot see it with `helm ls` and other Helm commands 99 100 When deploying a Helm application Argo CD is using Helm 101 only as a template mechanism. It runs `helm template` and 102 then deploys the resulting manifests on the cluster instead of doing `helm install`. This means that you cannot use any Helm command 103 to view/verify the application. It is fully managed by Argo CD. 104 Note that Argo CD supports natively some capabilities that you might miss in Helm (such as the history and rollback commands). 105 106 This decision was made so that Argo CD is neutral 107 to all manifest generators. 108 109 110 ## I've configured [cluster secret](./operator-manual/declarative-setup.md#clusters) but it does not show up in CLI/UI, how do I fix it? 111 112 Check if cluster secret has `argocd.argoproj.io/secret-type: cluster` label. If secret has the label but the cluster is 113 still not visible then make sure it might be a permission issue. Try to list clusters using `admin` user 114 (e.g. `argocd login --username admin && argocd cluster list`). 115 116 ## Argo CD is unable to connect to my cluster, how do I troubleshoot it? 117 118 Use the following steps to reconstruct configured cluster config and connect to your cluster manually using kubectl: 119 120 ```bash 121 kubectl exec -it <argocd-pod-name> bash # ssh into any argocd server pod 122 argocd admin cluster kubeconfig https://<cluster-url> /tmp/config --namespace argocd # generate your cluster config 123 KUBECONFIG=/tmp/config kubectl get pods # test connection manually 124 ``` 125 126 Now you can manually verify that cluster is accessible from the Argo CD pod. 127 128 ## How Can I Terminate A Sync? 129 130 To terminate the sync, click on the "synchronization" then "terminate": 131 132   133 134 ## Why Is My App `Out Of Sync` Even After Syncing? 135 136 In some cases, the tool you use may conflict with Argo CD by adding the `app.kubernetes.io/instance` label. E.g. using 137 Kustomize common labels feature. 138 139 Argo CD automatically sets the `app.kubernetes.io/instance` label and uses it to determine which resources form the app. 140 If the tool does this too, this causes confusion. You can change this label by setting 141 the `application.instanceLabelKey` value in the `argocd-cm`. We recommend that you use `argocd.argoproj.io/instance`. 142 143 !!! note 144 When you make this change your applications will become out of sync and will need re-syncing. 145 146 See [#1482](https://github.com/argoproj/argo-cd/issues/1482). 147 148 ## How often does Argo CD check for changes to my Git or Helm repository ? 149 150 The default maximum polling interval is 3 minutes (120 seconds + 60 seconds jitter). 151 You can change the setting by updating the `timeout.reconciliation` value and the `timeout.reconciliation.jitter` in the [argocd-cm](https://github.com/argoproj/argo-cd/blob/2d6ce088acd4fb29271ffb6f6023dbb27594d59b/docs/operator-manual/argocd-cm.yaml#L279-L282) config map. If there are any Git changes, Argo CD will only update applications with the [auto-sync setting](user-guide/auto_sync.md) enabled. If you set it to `0` then Argo CD will stop polling Git repositories automatically and you can only use alternative methods such as [webhooks](operator-manual/webhook.md) and/or manual syncs for deploying applications. 152 153 154 ## Why is my ArgoCD application `Out Of Sync` when there are no actual changes to the resource limits (or other fields with unit values)? 155 156 Kubernetes has normalized your resource limits when they are applied, and then Argo CD has compared the version in 157 your generated manifests from git to the normalized ones in the Kubernetes cluster - they may not match. 158 159 E.g. 160 161 * `'1000m'` normalized to `'1'` 162 * `'0.1'` normalized to `'100m'` 163 * `'3072Mi'` normalized to `'3Gi'` 164 * `3072` normalized to `'3072'` (quotes added) 165 * `8760h` normalized to `8760h0m0s` 166 167 To fix this use [diffing customizations](./user-guide/diffing.md#known-kubernetes-types-in-crds-resource-limits-volume-mounts-etc). 168 169 ## How Do I Fix `invalid cookie, longer than max length 4093`? 170 171 Argo CD uses a JWT as the auth token. You likely are part of many groups and have gone over the 4KB limit which is set 172 for cookies. You can get the list of groups by opening "developer tools -> network" 173 174 * Click log in 175 * Find the call to `<argocd_instance>/auth/callback?code=<random_string>` 176 177 Decode the token at [https://jwt.io/](https://jwt.io/). That will provide the list of teams that you can remove yourself 178 from. 179 180 See [#2165](https://github.com/argoproj/argo-cd/issues/2165). 181 182 ## Why Am I Getting `rpc error: code = Unavailable desc = transport is closing` When Using The CLI? 183 184 Maybe you're behind a proxy that does not support HTTP 2? Try the `--grpc-web` flag: 185 186 ```bash 187 argocd ... --grpc-web 188 ``` 189 190 ## Why Am I Getting `x509: certificate signed by unknown authority` When Using The CLI? 191 192 The certificate created by default by Argo CD is not automatically recognised by the Argo CD CLI, in order 193 to create a secure system you must follow the instructions to [install a certificate](operator-manual/tls.md) 194 and configure your client OS to trust that certificate. 195 196 If you're not running in a production system (e.g. you're testing Argo CD out), try the `--insecure` flag: 197 198 ```bash 199 argocd ... --insecure 200 ``` 201 202 !!! warning "Do not use `--insecure` in production" 203 204 ## I have configured Dex via `dex.config` in `argocd-cm`, it still says Dex is unconfigured. Why? 205 206 Most likely you forgot to set the `url` in `argocd-cm` to point to your Argo CD as well. See also 207 [the docs](./operator-manual/user-management/index.md#2-configure-argo-cd-for-sso). 208 209 ## Why are `SealedSecret` resources reporting a `Status`? 210 211 Versions of `SealedSecret` up to and including `v0.15.0` (especially through helm `1.15.0-r3`) don't include 212 a [modern CRD](https://github.com/bitnami-labs/sealed-secrets/issues/555) and thus the status field will not 213 be exposed (on k8s `1.16+`). If your Kubernetes deployment is [modern]( 214 https://www.openshift.com/blog/a-look-into-the-technical-details-of-kubernetes-1-16), ensure you're using a 215 fixed CRD if you want this feature to work at all. 216 217 ## <a name="sealed-secret-stuck-progressing"></a>Why are resources of type `SealedSecret` stuck in the `Progressing` state? 218 219 The controller of the `SealedSecret` resource may expose the status condition on resource it provisioned. Since 220 version `v2.0.0` Argo CD picks up that status condition to derive a health status for the `SealedSecret`. 221 222 Versions before `v0.15.0` of the `SealedSecret` controller are affected by an issue regarding this status 223 conditions updates, which is why this feature is disabled by default in these versions. Status condition updates may be 224 enabled by starting the `SealedSecret` controller with the `--update-status` command line parameter or by setting 225 the `SEALED_SECRETS_UPDATE_STATUS` environment variable. 226 227 To disable Argo CD from checking the status condition on `SealedSecret` resources, add the following resource 228 customization in your `argocd-cm` ConfigMap via `resource.customizations.health.<group_kind>` key. 229 230 ```yaml 231 resource.customizations.health.bitnami.com_SealedSecret: | 232 hs = {} 233 hs.status = "Healthy" 234 hs.message = "Controller doesn't report resource status" 235 return hs 236 ``` 237 238 ## How do I fix `The order in patch list … doesn't match $setElementOrder list: …`? 239 240 An application may trigger a sync error labeled a `ComparisonError` with a message like: 241 242 > The order in patch list: [map[name:**KEY_BC** value:150] map[name:**KEY_BC** value:500] map[name:**KEY_BD** value:250] map[name:**KEY_BD** value:500] map[name:KEY_BI value:something]] doesn't match $setElementOrder list: [map[name:KEY_AA] map[name:KEY_AB] map[name:KEY_AC] map[name:KEY_AD] map[name:KEY_AE] map[name:KEY_AF] map[name:KEY_AG] map[name:KEY_AH] map[name:KEY_AI] map[name:KEY_AJ] map[name:KEY_AK] map[name:KEY_AL] map[name:KEY_AM] map[name:KEY_AN] map[name:KEY_AO] map[name:KEY_AP] map[name:KEY_AQ] map[name:KEY_AR] map[name:KEY_AS] map[name:KEY_AT] map[name:KEY_AU] map[name:KEY_AV] map[name:KEY_AW] map[name:KEY_AX] map[name:KEY_AY] map[name:KEY_AZ] map[name:KEY_BA] map[name:KEY_BB] map[name:**KEY_BC**] map[name:**KEY_BD**] map[name:KEY_BE] map[name:KEY_BF] map[name:KEY_BG] map[name:KEY_BH] map[name:KEY_BI] map[name:**KEY_BC**] map[name:**KEY_BD**]] 243 244 245 There are two parts to the message: 246 247 1. `The order in patch list: [` 248 249 This identifies values for items, especially items that appear multiple times: 250 251 > map[name:**KEY_BC** value:150] map[name:**KEY_BC** value:500] map[name:**KEY_BD** value:250] map[name:**KEY_BD** value:500] map[name:KEY_BI value:something] 252 253 You'll want to identify the keys that are duplicated -- you can focus on the first part, as each duplicated key will appear, once for each of its value with its value in the first list. The second list is really just 254 255 `]` 256 257 2. `doesn't match $setElementOrder list: [` 258 259 This includes all of the keys. It's included for debugging purposes -- you don't need to pay much attention to it. It will give you a hint about the precise location in the list for the duplicated keys: 260 261 > map[name:KEY_AA] map[name:KEY_AB] map[name:KEY_AC] map[name:KEY_AD] map[name:KEY_AE] map[name:KEY_AF] map[name:KEY_AG] map[name:KEY_AH] map[name:KEY_AI] map[name:KEY_AJ] map[name:KEY_AK] map[name:KEY_AL] map[name:KEY_AM] map[name:KEY_AN] map[name:KEY_AO] map[name:KEY_AP] map[name:KEY_AQ] map[name:KEY_AR] map[name:KEY_AS] map[name:KEY_AT] map[name:KEY_AU] map[name:KEY_AV] map[name:KEY_AW] map[name:KEY_AX] map[name:KEY_AY] map[name:KEY_AZ] map[name:KEY_BA] map[name:KEY_BB] map[name:**KEY_BC**] map[name:**KEY_BD**] map[name:KEY_BE] map[name:KEY_BF] map[name:KEY_BG] map[name:KEY_BH] map[name:KEY_BI] map[name:**KEY_BC**] map[name:**KEY_BD**] 262 263 `]` 264 265 In this case, the duplicated keys have been **emphasized** to help you identify the problematic keys. Many editors have the ability to highlight all instances of a string, using such an editor can help with such problems. 266 267 The most common instance of this error is with `env:` fields for `containers`. 268 269 !!! note "Dynamic applications" 270 It's possible that your application is being generated by a tool in which case the duplication might not be evident within the scope of a single file. If you have trouble debugging this problem, consider filing a ticket to the owner of the generator tool asking them to improve its validation and error reporting. 271 272 ## How to rotate Redis secret? 273 * Delete `argocd-redis` secret in the namespace where Argo CD is installed. 274 ```bash 275 kubectl delete secret argocd-redis -n <argocd namespace> 276 ``` 277 * If you are running Redis in HA mode, restart Redis in HA. 278 ```bash 279 kubectl rollout restart deployment argocd-redis-ha-haproxy 280 kubectl rollout restart statefulset argocd-redis-ha-server 281 ``` 282 * If you are running Redis in non-HA mode, restart Redis. 283 ```bash 284 kubectl rollout restart deployment argocd-redis 285 ``` 286 * Restart other components. 287 ```bash 288 kubectl rollout restart deployment argocd-server argocd-repo-server 289 kubectl rollout restart statefulset argocd-application-controller 290 ``` 291 292 ## How to turn off Redis auth if users really want to? 293 294 Argo CD default installation is now configured to automatically enable Redis authentication. 295 If for some reason authenticated Redis does not work for you and you want to use non-authenticated Redis, here are the steps: 296 297 1. You need to have your own Redis installation. 298 2. Configure Argo CD to use your own Redis instance. See this [doc](https://argo-cd.readthedocs.io/en/stable/operator-manual/argocd-cmd-params-cm-yaml/) for the Argo CD configuration. 299 3. If you already installed Redis shipped with Argo CD, you also need to clean up the existing components: 300 301 * When HA Redis is used: 302 303 - kubectl delete deployment argocd-redis-ha-haproxy 304 - kubectl delete statefulset argocd-redis-ha-server 305 306 * When non-HA Redis is used: 307 308 - kubectl delete deployment argocd-redis 309 310 4. Remove environment variable `REDIS_PASSWORD` from the following manifests: 311 * Deployment: argocd-repo-server 312 * Deployment: argocd-server 313 * StatefulSet: argocd-application-controller 314 315 ## How do I provide my own Redis credentials? 316 The Redis password is stored in Kubernetes secret `argocd-redis` with key `auth` in the namespace where Argo CD is installed. 317 You can config your secret provider to generate Kubernetes secret accordingly. 318 319 ## How do I fix `Manifest generation error (cached)`? 320 321 `Manifest generation error (cached)` means that there was an error when generating manifests and that the error message has been cached to avoid runaway retries. 322 323 Doing a hard refresh (ignoring the cached error) can overcome transient issues. But if there's an ongoing reason manifest generation is failing, a hard refresh will not help. 324 325 Instead, try searching the repo-server logs for the app name in order to identify the error that is causing manifest generation to fail. 326 327 ## How do I fix `field not declared in schema`? 328 329 For certain features, Argo CD relies on a static (hard-coded) set of schemas for built-in Kubernetes resource types. 330 331 If your manifests use fields which are not present in the hard-coded schemas, you may get an error like `field not 332 declared in schema`. 333 334 The schema version is based on the Kubernetes libraries version that Argo CD is built against. To find the Kubernetes 335 version for a given Argo CD version, navigate to this page, where `X.Y.Z` is the Argo CD version: 336 337 ``` 338 https://github.com/argoproj/argo-cd/blob/vX.Y.Z/go.mod 339 ``` 340 341 Then find the Kubernetes version in the `go.mod` file. For example, for Argo CD v2.11.4, the Kubernetes libraries 342 version is v0.26.11 343 344 ``` 345 k8s.io/api => k8s.io/api v0.26.11 346 ``` 347 348 ### How do I fix the issue? 349 350 To completely resolve the issue, upgrade to an Argo CD version which contains a static schema supporting all the needed 351 fields. 352 353 ### How do I work around the issue? 354 355 As mentioned above, only certain Argo CD features rely on the static schema: 1) `ignoreDifferences` with 356 `managedFieldManagers`, 2) server-side apply _without_ server-side diff, and 3) server-side diff _with_ mutation 357 webhooks. 358 359 If you can avoid using these features, you can avoid triggering the error. The options are as follows: 360 361 1. **Disable `ignoreDifferences` which have `managedFieldsManagers`**: see [diffing docs](user-guide/diffing.md) for 362 details about that feature. Removing this config could cause undesired diffing behavior. 363 2. **Disable server-side apply**: see [server-side apply docs](user-guide/sync-options.md#server-side-apply) for details about that 364 feature. Disabling server-side apply may have undesired effects on sync behavior. Note that you can bypass this issue 365 if you use server-side diff and [exclude mutation webhooks from the diff](user-guide/diff-strategies.md#mutation-webhooks). 366 Excluding mutation webhooks from the diff could cause undesired diffing behavior. 367 3. **Disable mutation webhooks when using server-side diff**: see [server-side diff docs](user-guide/diff-strategies.md#mutation-webhooks) 368 for details about that feature. Disabling mutation webhooks may have undesired effects on sync behavior.