github.com/verrazzano/verrazzano@v1.7.1/tools/vz/pkg/internal/test/cluster/testCattleSystempods/cluster-snapshot/cattle-system/rancher-649dc876b-kdrb2/logs.txt (about) 1 # Copyright (c) 2022, 2024, Oracle and/or its affiliates. 2 # Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl. 3 ==== START logs for container rancher of pod cattle-system/rancher-649dc876b-kdrb2 ==== 4 2022/06/16 10:23:02 [INFO] Rancher version v2.6.4-20220504205847-7131173f2 (7131173f2) is starting 5 2022/06/16 10:23:02 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:false AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLevel:0 Features: ClusterRegistry:} 6 2022/06/16 10:23:02 [INFO] Listening on /tmp/log.sock 7 2022/06/16 10:23:02 [INFO] Running in clustered mode with ID REDACTED-IP4-ADDRESS, monitoring endpoint cattle-system/rancher 8 2022/06/16 10:23:05 [INFO] Applying CRD features.management.cattle.io 9 2022/06/16 10:23:12 [INFO] Applying CRD navlinks.ui.cattle.io 10 2022/06/16 10:23:13 [INFO] Applying CRD clusters.management.cattle.io 11 2022/06/16 10:23:13 [INFO] Applying CRD apiservices.management.cattle.io 12 2022/06/16 10:23:14 [INFO] Applying CRD clusterregistrationtokens.management.cattle.io 13 2022/06/16 10:23:14 [INFO] Applying CRD settings.management.cattle.io 14 2022/06/16 10:23:15 [INFO] Applying CRD preferences.management.cattle.io 15 2022/06/16 10:23:16 [INFO] Applying CRD features.management.cattle.io 16 2022/06/16 10:23:17 [INFO] Applying CRD clusterrepos.catalog.cattle.io 17 2022/06/16 10:23:18 [INFO] Applying CRD operations.catalog.cattle.io 18 2022/06/16 10:23:18 [INFO] Applying CRD apps.catalog.cattle.io 19 2022/06/16 10:23:19 [INFO] Applying CRD fleetworkspaces.management.cattle.io 20 2022/06/16 10:23:20 [INFO] Applying CRD bundles.fleet.cattle.io 21 2022/06/16 10:23:20 [INFO] Applying CRD clusters.fleet.cattle.io 22 2022/06/16 10:23:21 [INFO] Applying CRD managedcharts.management.cattle.io 23 2022/06/16 10:23:22 [INFO] Applying CRD clusters.provisioning.cattle.io 24 2022/06/16 10:23:23 [INFO] Applying CRD clusters.provisioning.cattle.io 25 2022/06/16 10:23:23 [INFO] Applying CRD rkeclusters.rke.cattle.io 26 2022/06/16 10:23:24 [INFO] Applying CRD rkecontrolplanes.rke.cattle.io 27 2022/06/16 10:23:25 [INFO] Applying CRD rkebootstraps.rke.cattle.io 28 2022/06/16 10:23:25 [INFO] Applying CRD rkebootstraptemplates.rke.cattle.io 29 2022/06/16 10:23:26 [INFO] Applying CRD rkecontrolplanes.rke.cattle.io 30 2022/06/16 10:23:27 [INFO] Applying CRD custommachines.rke.cattle.io 31 2022/06/16 10:23:28 [INFO] Applying CRD etcdsnapshots.rke.cattle.io 32 2022/06/16 10:23:28 [INFO] Applying CRD clusters.cluster.x-k8s.io 33 2022/06/16 10:23:28 [INFO] Applying CRD machinedeployments.cluster.x-k8s.io 34 2022/06/16 10:23:29 [INFO] Applying CRD machinehealthchecks.cluster.x-k8s.io 35 2022/06/16 10:23:29 [INFO] Applying CRD machines.cluster.x-k8s.io 36 2022/06/16 10:23:30 [INFO] Applying CRD machinesets.cluster.x-k8s.io 37 2022/06/16 10:23:39 [INFO] Creating CRD apps.project.cattle.io 38 2022/06/16 10:23:39 [INFO] Creating CRD apprevisions.project.cattle.io 39 2022/06/16 10:23:39 [INFO] Creating CRD pipelineexecutions.project.cattle.io 40 2022/06/16 10:23:39 [INFO] Creating CRD pipelinesettings.project.cattle.io 41 2022/06/16 10:23:39 [INFO] Creating CRD pipelines.project.cattle.io 42 2022/06/16 10:23:39 [INFO] Creating CRD sourcecodecredentials.project.cattle.io 43 2022/06/16 10:23:39 [INFO] Creating CRD sourcecodeproviderconfigs.project.cattle.io 44 2022/06/16 10:23:39 [INFO] Creating CRD sourcecoderepositories.project.cattle.io 45 2022/06/16 10:23:39 [INFO] Creating CRD clusterroletemplatebindings.management.cattle.io 46 2022/06/16 10:23:39 [INFO] Creating CRD dynamicschemas.management.cattle.io 47 2022/06/16 10:23:39 [INFO] Creating CRD catalogs.management.cattle.io 48 2022/06/16 10:23:39 [INFO] Creating CRD etcdbackups.management.cattle.io 49 2022/06/16 10:23:40 [INFO] Creating CRD globalrolebindings.management.cattle.io 50 2022/06/16 10:23:40 [INFO] Creating CRD catalogtemplates.management.cattle.io 51 2022/06/16 10:23:40 [INFO] Creating CRD globalroles.management.cattle.io 52 2022/06/16 10:23:40 [INFO] Creating CRD catalogtemplateversions.management.cattle.io 53 2022/06/16 10:23:40 [INFO] Creating CRD clusteralerts.management.cattle.io 54 2022/06/16 10:23:40 [INFO] Creating CRD kontainerdrivers.management.cattle.io 55 2022/06/16 10:23:40 [INFO] Creating CRD nodedrivers.management.cattle.io 56 2022/06/16 10:23:40 [INFO] Creating CRD clusteralertgroups.management.cattle.io 57 2022/06/16 10:23:40 [INFO] Creating CRD nodepools.management.cattle.io 58 2022/06/16 10:23:40 [INFO] Creating CRD nodetemplates.management.cattle.io 59 2022/06/16 10:23:40 [INFO] Creating CRD clustercatalogs.management.cattle.io 60 2022/06/16 10:23:40 [INFO] Creating CRD nodes.management.cattle.io 61 2022/06/16 10:23:40 [INFO] Creating CRD podsecuritypolicytemplateprojectbindings.management.cattle.io 62 2022/06/16 10:23:40 [INFO] Creating CRD clusterloggings.management.cattle.io 63 2022/06/16 10:23:40 [INFO] Creating CRD podsecuritypolicytemplates.management.cattle.io 64 2022/06/16 10:23:40 [INFO] Creating CRD clusteralertrules.management.cattle.io 65 2022/06/16 10:23:40 [INFO] Creating CRD projectnetworkpolicies.management.cattle.io 66 2022/06/16 10:23:40 [INFO] Creating CRD projectroletemplatebindings.management.cattle.io 67 2022/06/16 10:23:40 [INFO] Creating CRD clustermonitorgraphs.management.cattle.io 68 2022/06/16 10:23:40 [INFO] Creating CRD projects.management.cattle.io 69 2022/06/16 10:23:40 [INFO] Creating CRD rkek8ssystemimages.management.cattle.io 70 2022/06/16 10:23:40 [INFO] Creating CRD clusterscans.management.cattle.io 71 2022/06/16 10:23:40 [INFO] Creating CRD rkek8sserviceoptions.management.cattle.io 72 2022/06/16 10:23:41 [INFO] Creating CRD composeconfigs.management.cattle.io 73 2022/06/16 10:23:41 [INFO] Creating CRD rkeaddons.management.cattle.io 74 2022/06/16 10:23:41 [INFO] Creating CRD multiclusterapps.management.cattle.io 75 2022/06/16 10:23:41 [INFO] Creating CRD roletemplates.management.cattle.io 76 2022/06/16 10:23:41 [INFO] Creating CRD multiclusterapprevisions.management.cattle.io 77 2022/06/16 10:23:41 [INFO] Waiting for CRD sourcecodecredentials.project.cattle.io to become available 78 2022/06/16 10:23:41 [INFO] Creating CRD samltokens.management.cattle.io 79 2022/06/16 10:23:41 [INFO] Creating CRD monitormetrics.management.cattle.io 80 2022/06/16 10:23:41 [INFO] Creating CRD clustertemplates.management.cattle.io 81 2022/06/16 10:23:41 [INFO] Creating CRD clustertemplaterevisions.management.cattle.io 82 2022/06/16 10:23:41 [INFO] Creating CRD notifiers.management.cattle.io 83 2022/06/16 10:23:41 [INFO] Creating CRD projectalerts.management.cattle.io 84 2022/06/16 10:23:41 [INFO] Creating CRD projectalertgroups.management.cattle.io 85 2022/06/16 10:23:42 [INFO] Done waiting for CRD sourcecodecredentials.project.cattle.io to become available 86 2022/06/16 10:23:42 [INFO] Waiting for CRD sourcecodeproviderconfigs.project.cattle.io to become available 87 2022/06/16 10:23:42 [INFO] Creating CRD projectcatalogs.management.cattle.io 88 2022/06/16 10:23:42 [INFO] Creating CRD projectloggings.management.cattle.io 89 2022/06/16 10:23:42 [INFO] Creating CRD projectalertrules.management.cattle.io 90 2022/06/16 10:23:42 [INFO] Creating CRD projectmonitorgraphs.management.cattle.io 91 2022/06/16 10:23:42 [INFO] Done waiting for CRD sourcecodeproviderconfigs.project.cattle.io to become available 92 2022/06/16 10:23:42 [INFO] Waiting for CRD sourcecoderepositories.project.cattle.io to become available 93 2022/06/16 10:23:42 [INFO] Creating CRD cisconfigs.management.cattle.io 94 2022/06/16 10:23:43 [INFO] Creating CRD cisbenchmarkversions.management.cattle.io 95 2022/06/16 10:23:43 [INFO] Creating CRD templates.management.cattle.io 96 2022/06/16 10:23:43 [INFO] Creating CRD templateversions.management.cattle.io 97 2022/06/16 10:23:43 [INFO] Done waiting for CRD sourcecoderepositories.project.cattle.io to become available 98 2022/06/16 10:23:43 [INFO] Creating CRD templatecontents.management.cattle.io 99 2022/06/16 10:23:43 [INFO] Creating CRD globaldnses.management.cattle.io 100 2022/06/16 10:23:44 [INFO] Creating CRD globaldnsproviders.management.cattle.io 101 2022/06/16 10:23:44 [INFO] Waiting for CRD rkeaddons.management.cattle.io to become available 102 2022/06/16 10:23:44 [INFO] Done waiting for CRD rkeaddons.management.cattle.io to become available 103 2022/06/16 10:23:44 [INFO] Waiting for CRD samltokens.management.cattle.io to become available 104 2022/06/16 10:23:45 [INFO] Done waiting for CRD samltokens.management.cattle.io to become available 105 2022/06/16 10:23:45 [INFO] Waiting for CRD clustertemplaterevisions.management.cattle.io to become available 106 2022/06/16 10:23:46 [INFO] Done waiting for CRD clustertemplaterevisions.management.cattle.io to become available 107 2022/06/16 10:23:46 [INFO] Waiting for CRD podsecuritypolicytemplateprojectbindings.management.cattle.io to become available 108 2022/06/16 10:23:46 [INFO] Waiting for CRD templateversions.management.cattle.io to become available 109 2022/06/16 10:23:46 [INFO] Done waiting for CRD podsecuritypolicytemplateprojectbindings.management.cattle.io to become available 110 2022/06/16 10:23:46 [INFO] Waiting for CRD projectroletemplatebindings.management.cattle.io to become available 111 2022/06/16 10:23:46 [INFO] Done waiting for CRD templateversions.management.cattle.io to become available 112 2022/06/16 10:23:46 [INFO] Waiting for CRD templatecontents.management.cattle.io to become available 113 2022/06/16 10:23:47 [INFO] Done waiting for CRD projectroletemplatebindings.management.cattle.io to become available 114 2022/06/16 10:23:47 [INFO] Waiting for CRD projectnetworkpolicies.management.cattle.io to become available 115 2022/06/16 10:23:47 [INFO] Done waiting for CRD templatecontents.management.cattle.io to become available 116 2022/06/16 10:23:47 [INFO] Waiting for CRD projectalertrules.management.cattle.io to become available 117 2022/06/16 10:23:47 [INFO] Done waiting for CRD projectnetworkpolicies.management.cattle.io to become available 118 2022/06/16 10:23:47 [INFO] Waiting for CRD clustertemplates.management.cattle.io to become available 119 2022/06/16 10:23:48 [INFO] Done waiting for CRD projectalertrules.management.cattle.io to become available 120 2022/06/16 10:23:48 [INFO] Waiting for CRD templates.management.cattle.io to become available 121 2022/06/16 10:23:48 [INFO] Done waiting for CRD clustertemplates.management.cattle.io to become available 122 2022/06/16 10:23:48 [INFO] Waiting for CRD roletemplates.management.cattle.io to become available 123 2022/06/16 10:23:48 [INFO] Done waiting for CRD templates.management.cattle.io to become available 124 2022/06/16 10:23:48 [INFO] Waiting for CRD globaldnses.management.cattle.io to become available 125 2022/06/16 10:23:49 [INFO] Done waiting for CRD roletemplates.management.cattle.io to become available 126 2022/06/16 10:23:49 [INFO] Waiting for CRD podsecuritypolicytemplates.management.cattle.io to become available 127 2022/06/16 10:23:49 [INFO] Done waiting for CRD globaldnses.management.cattle.io to become available 128 2022/06/16 10:23:49 [INFO] Waiting for CRD projectloggings.management.cattle.io to become available 129 2022/06/16 10:23:49 [INFO] Done waiting for CRD podsecuritypolicytemplates.management.cattle.io to become available 130 2022/06/16 10:23:49 [INFO] Waiting for CRD rkek8ssystemimages.management.cattle.io to become available 131 2022/06/16 10:23:49 [INFO] Done waiting for CRD projectloggings.management.cattle.io to become available 132 2022/06/16 10:23:49 [INFO] Waiting for CRD projectmonitorgraphs.management.cattle.io to become available 133 2022/06/16 10:23:50 [INFO] Done waiting for CRD rkek8ssystemimages.management.cattle.io to become available 134 2022/06/16 10:23:50 [INFO] Waiting for CRD rkek8sserviceoptions.management.cattle.io to become available 135 2022/06/16 10:23:50 [INFO] Done waiting for CRD projectmonitorgraphs.management.cattle.io to become available 136 2022/06/16 10:23:50 [INFO] Waiting for CRD globaldnsproviders.management.cattle.io to become available 137 2022/06/16 10:23:50 [INFO] Done waiting for CRD rkek8sserviceoptions.management.cattle.io to become available 138 2022/06/16 10:23:50 [INFO] Waiting for CRD nodes.management.cattle.io to become available 139 2022/06/16 10:23:50 [INFO] Done waiting for CRD globaldnsproviders.management.cattle.io to become available 140 2022/06/16 10:23:50 [INFO] Waiting for CRD cisconfigs.management.cattle.io to become available 141 2022/06/16 10:23:51 [INFO] Done waiting for CRD nodes.management.cattle.io to become available 142 2022/06/16 10:23:51 [INFO] Waiting for CRD projects.management.cattle.io to become available 143 2022/06/16 10:23:51 [INFO] Done waiting for CRD cisconfigs.management.cattle.io to become available 144 2022/06/16 10:23:51 [INFO] Waiting for CRD cisbenchmarkversions.management.cattle.io to become available 145 2022/06/16 10:23:51 [INFO] Done waiting for CRD projects.management.cattle.io to become available 146 2022/06/16 10:23:51 [INFO] Done waiting for CRD cisbenchmarkversions.management.cattle.io to become available 147 E0616 10:24:04.089786 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 148 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=RoleTemplate controller 149 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Node controller 150 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=CatalogTemplate controller 151 2022/06/16 10:24:04 [INFO] Starting provisioning.cattle.io/v1, Kind=Cluster controller 152 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ClusterRoleTemplateBinding controller 153 2022/06/16 10:24:04 [INFO] Starting project.cattle.io/v3, Kind=SourceCodeCredential controller 154 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Preference controller 155 2022/06/16 10:24:04 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=Role controller 156 2022/06/16 10:24:04 [INFO] Starting project.cattle.io/v3, Kind=Pipeline controller 157 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=AuthConfig controller 158 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=NodePool controller 159 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Project controller 160 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=ConfigMap controller 161 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=Endpoints controller 162 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=UserAttribute controller 163 2022/06/16 10:24:04 [INFO] Starting cluster.x-k8s.io/v1beta1, Kind=Machine controller 164 2022/06/16 10:24:04 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller 165 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=User controller 166 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ProjectRoleTemplateBinding controller 167 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Group controller 168 2022/06/16 10:24:04 [INFO] Starting API controllers 169 2022/06/16 10:24:04 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRole controller 170 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=ServiceAccount controller 171 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Feature controller 172 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=GlobalDns controller 173 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=RkeK8sSystemImage controller 174 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=PodSecurityPolicyTemplateProjectBinding controller 175 I0616 10:24:04.110925 9 leaderelection.go:248] attempting to acquire leader lease kube-system/cattle-controllers... 176 2022/06/16 10:24:04 [INFO] Starting rke.cattle.io/v1, Kind=RKEBootstrap controller 177 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ClusterCatalog controller 178 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=KontainerDriver controller 179 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=GlobalRoleBinding controller 180 2022/06/16 10:24:04 [INFO] Starting apiregistration.k8s.io/v1, Kind=APIService controller 181 2022/06/16 10:24:04 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=RoleBinding controller 182 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ProjectCatalog controller 183 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=APIService controller 184 2022/06/16 10:24:04 [INFO] Starting project.cattle.io/v3, Kind=App controller 185 2022/06/16 10:24:04 [INFO] Starting project.cattle.io/v3, Kind=SourceCodeRepository controller 186 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=MultiClusterAppRevision controller 187 2022/06/16 10:24:04 [INFO] Starting project.cattle.io/v3, Kind=PipelineExecution controller 188 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Setting controller 189 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Cluster controller 190 2022/06/16 10:24:04 [INFO] Starting apiextensions.k8s.io/v1, Kind=CustomResourceDefinition controller 191 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=RkeAddon controller 192 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ClusterTemplateRevision controller 193 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ProjectLogging controller 194 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ClusterTemplate controller 195 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=GroupMember controller 196 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=CisBenchmarkVersion controller 197 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Catalog controller 198 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=CatalogTemplateVersion controller 199 2022/06/16 10:24:04 [INFO] Starting catalog.cattle.io/v1, Kind=ClusterRepo controller 200 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=Namespace controller 201 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=PodSecurityPolicyTemplate controller 202 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=CisConfig controller 203 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=NodeTemplate controller 204 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=ClusterRegistrationToken controller 205 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=Secret controller 206 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=RkeK8sServiceOption controller 207 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=MultiClusterApp controller 208 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Token controller 209 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=NodeDriver controller 210 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=DynamicSchema controller 211 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=GlobalRole controller 212 I0616 10:24:04.163148 9 leaderelection.go:258] successfully acquired lease kube-system/cattle-controllers 213 2022/06/16 10:24:04 [INFO] Reconciling GlobalRoles 214 2022/06/16 10:24:04 [INFO] Creating features-manage 215 2022/06/16 10:24:04 [INFO] Creating clustertemplaterevisions-create 216 2022/06/16 10:24:04 [INFO] Creating user-base 217 2022/06/16 10:24:04 [INFO] Creating nodedrivers-manage 218 2022/06/16 10:24:04 [INFO] Creating catalogs-use 219 2022/06/16 10:24:04 [INFO] Creating users-manage 220 2022/06/16 10:24:04 [INFO] Creating authn-manage 221 2022/06/16 10:24:04 [INFO] Creating clusters-create 222 2022/06/16 10:24:04 [INFO] Creating catalogs-manage 223 2022/06/16 10:24:04 [INFO] Creating roles-manage 224 2022/06/16 10:24:04 [INFO] Creating view-rancher-metrics 225 2022/06/16 10:24:04 [INFO] Creating podsecuritypolicytemplates-manage 226 2022/06/16 10:24:04 [INFO] Creating admin 227 2022/06/16 10:24:04 [INFO] Creating restricted-admin 228 2022/06/16 10:24:04 [INFO] Creating user 229 2022/06/16 10:24:04 [INFO] 230 2022/06/16 10:24:04 [INFO] ----------------------------------------- 231 2022/06/16 10:24:04 [INFO] Welcome to Rancher 232 2022/06/16 10:24:04 [INFO] A bootstrap password has been generated for your admin user. 233 2022/06/16 10:24:04 [INFO] 234 2022/06/16 10:24:04 [INFO] Bootstrap Password: zlpkqp8gc9x4njg5tpwts82dh5tkfsjkndjb4vjjkggzz894nzp7p8 235 2022/06/16 10:24:04 [INFO] 236 2022/06/16 10:24:04 [INFO] Use https://REDACTED-IP4-ADDRESS/dashboard/?setup=zlpkqp8gc9x4njg5tpwts82dh5tkfsjkndjb4vjjkggzz894nzp7p8 to complete setup in the UI 237 2022/06/16 10:24:04 [INFO] ----------------------------------------- 238 2022/06/16 10:24:04 [INFO] 239 2022/06/16 10:24:04 [INFO] Creating kontainerdrivers-manage 240 2022/06/16 10:24:04 [INFO] Creating settings-manage 241 2022/06/16 10:24:04 [INFO] Creating clustertemplates-create 242 2022/06/16 10:24:04 [INFO] Created default admin user and binding 243 E0616 10:24:04.442872 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 244 2022/06/16 10:24:04 [INFO] Reconciling RoleTemplates 245 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=User controller 246 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Token controller 247 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=UserAttribute controller 248 2022/06/16 10:24:04 [INFO] Starting /v1, Kind=Secret controller 249 2022/06/16 10:24:04 [INFO] Creating projectroletemplatebindings-view 250 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Cluster controller 251 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=GroupMember controller 252 2022/06/16 10:24:04 [INFO] Starting management.cattle.io/v3, Kind=Group controller 253 2022/06/16 10:24:04 [INFO] Waiting for initial data to be populated 254 2022/06/16 10:24:04 [INFO] Creating projects-create 255 2022/06/16 10:24:04 [INFO] Creating clusterroletemplatebindings-view 256 2022/06/16 10:24:04 [INFO] Creating project-owner 257 2022/06/16 10:24:04 [INFO] Creating ingress-view 258 2022/06/16 10:24:04 [INFO] Creating secrets-manage 259 2022/06/16 10:24:04 [INFO] Creating cluster-admin 260 2022/06/16 10:24:04 [INFO] Creating ingress-manage 261 2022/06/16 10:24:04 [INFO] Creating persistentvolumeclaims-manage 262 2022/06/16 10:24:04 [INFO] Creating monitoring-ui-view 263 2022/06/16 10:24:04 [INFO] Creating cluster-member 264 2022/06/16 10:24:04 [INFO] Steve auth startup complete 265 2022/06/16 10:24:04 [INFO] Creating backups-manage 266 2022/06/16 10:24:04 [INFO] Creating read-only 267 2022/06/16 10:24:04 [INFO] Creating persistentvolumeclaims-view 268 2022/06/16 10:24:04 [INFO] Creating project-monitoring-readonly 269 2022/06/16 10:24:04 [INFO] Creating clustercatalogs-view 270 2022/06/16 10:24:04 [INFO] Creating navlinks-manage 271 2022/06/16 10:24:04 [INFO] Creating services-manage 272 2022/06/16 10:24:04 [INFO] Creating projectroletemplatebindings-manage 273 2022/06/16 10:24:04 [INFO] Creating project-member 274 2022/06/16 10:24:04 [INFO] Creating workloads-view 275 2022/06/16 10:24:04 [INFO] Creating configmaps-manage 276 2022/06/16 10:24:04 [INFO] Creating services-view 277 2022/06/16 10:24:04 [INFO] Creating projectcatalogs-manage 278 2022/06/16 10:24:04 [INFO] Creating projectcatalogs-view 279 2022/06/16 10:24:04 [INFO] Creating edit 280 2022/06/16 10:24:04 [INFO] Creating view 281 2022/06/16 10:24:04 [INFO] Creating projects-view 282 2022/06/16 10:24:04 [INFO] Creating storage-manage 283 2022/06/16 10:24:04 [INFO] Creating clusterroletemplatebindings-manage 284 2022/06/16 10:24:04 [INFO] Creating navlinks-view 285 2022/06/16 10:24:04 [INFO] Creating admin 286 2022/06/16 10:24:04 [INFO] Creating cluster-owner 287 2022/06/16 10:24:04 [INFO] Creating nodes-manage 288 2022/06/16 10:24:04 [INFO] Creating clustercatalogs-manage 289 2022/06/16 10:24:04 [INFO] Creating workloads-manage 290 2022/06/16 10:24:04 [INFO] Creating serviceaccounts-view 291 2022/06/16 10:24:04 [INFO] Creating nodes-view 292 2022/06/16 10:24:04 [INFO] Creating create-ns 293 2022/06/16 10:24:05 [INFO] Creating secrets-view 294 2022/06/16 10:24:05 [INFO] Creating configmaps-view 295 2022/06/16 10:24:05 [INFO] Creating serviceaccounts-manage 296 2022/06/16 10:24:05 [INFO] Registering namespaceHandler for adding labels 297 2022/06/16 10:24:06 [INFO] Waiting for initial data to be populated 298 2022/06/16 10:24:08 [INFO] Waiting for initial data to be populated 299 2022/06/16 10:24:09 [INFO] adding kontainer driver rancherKubernetesEngine 300 2022/06/16 10:24:09 [INFO] adding kontainer driver googleKubernetesEngine 301 2022/06/16 10:24:09 [INFO] adding kontainer driver azureKubernetesService 302 2022/06/16 10:24:09 [INFO] adding kontainer driver amazonElasticContainerService 303 2022/06/16 10:24:09 [INFO] adding kontainer driver baiducloudcontainerengine 304 2022/06/16 10:24:09 [INFO] adding kontainer driver aliyunkubernetescontainerservice 305 2022/06/16 10:24:09 [INFO] adding kontainer driver tencentkubernetesengine 306 2022/06/16 10:24:09 [INFO] adding kontainer driver huaweicontainercloudengine 307 2022/06/16 10:24:09 [INFO] adding kontainer driver oraclecontainerengine 308 2022/06/16 10:24:09 [INFO] adding kontainer driver linodekubernetesengine 309 2022/06/16 10:24:09 [INFO] adding kontainer driver opentelekomcloudcontainerengine 310 2022/06/16 10:24:09 [INFO] Created cattle-global-data namespace 311 2022/06/16 10:24:09 [INFO] Created cattle-global-nt namespace 312 2022/06/16 10:24:09 [INFO] Creating node driver pinganyunecs 313 2022/06/16 10:24:09 [INFO] Creating node driver aliyunecs 314 2022/06/16 10:24:09 [INFO] Creating node driver amazonec2 315 2022/06/16 10:24:09 [INFO] Creating node driver azure 316 2022/06/16 10:24:09 [INFO] Creating node driver cloudca 317 2022/06/16 10:24:09 [INFO] Creating node driver cloudscale 318 2022/06/16 10:24:09 [INFO] Creating node driver digitalocean 319 2022/06/16 10:24:09 [INFO] Creating node driver exoscale 320 2022/06/16 10:24:09 [INFO] Creating node driver google 321 2022/06/16 10:24:09 [INFO] Creating node driver harvester 322 2022/06/16 10:24:09 [INFO] Creating node driver linode 323 2022/06/16 10:24:09 [INFO] Creating node driver oci 324 2022/06/16 10:24:09 [INFO] Creating node driver openstack 325 2022/06/16 10:24:09 [INFO] Creating node driver otc 326 2022/06/16 10:24:09 [INFO] Creating node driver packet 327 2022/06/16 10:24:09 [INFO] Creating node driver pnap 328 2022/06/16 10:24:09 [INFO] Creating node driver rackspace 329 2022/06/16 10:24:09 [INFO] Creating node driver softlayer 330 2022/06/16 10:24:09 [INFO] Creating node driver nutanix 331 2022/06/16 10:24:09 [INFO] Creating node driver vmwarevsphere 332 2022/06/16 10:24:09 [INFO] Starting catalog controller 333 2022/06/16 10:24:09 [INFO] Starting project-level catalog controller 334 2022/06/16 10:24:09 [INFO] Starting cluster-level catalog controller 335 2022/06/16 10:24:10 [ERROR] error syncing 'server-url': handler apiservice-settings: secrets "tls-rancher-internal-ca" not found, requeuing 336 2022/06/16 10:24:10 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found 337 2022/06/16 10:24:11 [INFO] Listening on :443 338 2022/06/16 10:24:11 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:11 +0000 UTC 339 2022/06/16 10:24:11 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:11 +0000 UTC 340 2022/06/16 10:24:11 [INFO] Listening on :80 341 2022/06/16 10:24:11 [ERROR] error syncing 'server-url': handler apiservice-settings: secrets "tls-rancher-internal-ca" not found, requeuing 342 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=GlobalDnsProvider controller 343 2022/06/16 10:24:11 [INFO] Starting networking.k8s.io/v1, Kind=Ingress controller 344 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=ClusterScan controller 345 2022/06/16 10:24:11 [INFO] Starting rke.cattle.io/v1, Kind=RKECluster controller 346 2022/06/16 10:24:11 [INFO] Starting fleet.cattle.io/v1alpha1, Kind=Bundle controller 347 2022/06/16 10:24:11 [INFO] Starting /v1, Kind=Service controller 348 2022/06/16 10:24:11 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:11 +0000 UTC 349 2022/06/16 10:24:11 [INFO] Creating new TLS secret for serving-cert (count: 4): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=9C14593255349B254849B1662DDED1BC79596D3E] 350 2022/06/16 10:24:11 [INFO] Starting apps/v1, Kind=ReplicaSet controller 351 2022/06/16 10:24:11 [INFO] Detected v2.6.0 upgrade, forcing logout for all web users 352 2022/06/16 10:24:11 [INFO] Starting rke.cattle.io/v1, Kind=ETCDSnapshot controller 353 2022/06/16 10:24:11 [INFO] Starting /v1, Kind=Pod controller 354 2022/06/16 10:24:11 [INFO] Starting cluster.x-k8s.io/v1beta1, Kind=MachineDeployment controller 355 2022/06/16 10:24:11 [INFO] Starting fleet.cattle.io/v1alpha1, Kind=Cluster controller 356 2022/06/16 10:24:11 [INFO] Starting /v1, Kind=ReplicationController controller 357 2022/06/16 10:24:11 [INFO] Starting apps/v1, Kind=StatefulSet controller 358 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=EtcdBackup controller 359 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=ComposeConfig controller 360 2022/06/16 10:24:11 [INFO] Starting admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration controller 361 2022/06/16 10:24:11 [INFO] Starting cluster.x-k8s.io/v1beta1, Kind=Cluster controller 362 2022/06/16 10:24:11 [INFO] Starting rke.cattle.io/v1, Kind=RKEControlPlane controller 363 2022/06/16 10:24:11 [INFO] Starting apps/v1, Kind=Deployment controller 364 2022/06/16 10:24:11 [INFO] Active TLS secret serving-cert (ver=7160) (count 4): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=9C14593255349B254849B1662DDED1BC79596D3E] 365 2022/06/16 10:24:11 [ERROR] error syncing 'server-url': handler apiservice-settings: secrets "tls-rancher-internal-ca" not found, requeuing 366 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=Notifier controller 367 2022/06/16 10:24:11 [INFO] Starting catalog.cattle.io/v1, Kind=App controller 368 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=ClusterAlertGroup controller 369 2022/06/16 10:24:11 [INFO] Starting rke.cattle.io/v1, Kind=CustomMachine controller 370 2022/06/16 10:24:11 [INFO] Starting /v1, Kind=Node controller 371 2022/06/16 10:24:11 [INFO] Starting batch/v1beta1, Kind=CronJob controller 372 2022/06/16 10:24:11 [INFO] Starting admissionregistration.k8s.io/v1, Kind=MutatingWebhookConfiguration controller 373 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=FleetWorkspace controller 374 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=ClusterAlertRule controller 375 2022/06/16 10:24:11 [INFO] Starting batch/v1, Kind=Job controller 376 2022/06/16 10:24:11 [INFO] Starting monitoring.coreos.com/v1, Kind=ServiceMonitor controller 377 2022/06/16 10:24:11 [INFO] Starting management.cattle.io/v3, Kind=ManagedChart controller 378 2022/06/16 10:24:11 [INFO] Starting apps/v1, Kind=DaemonSet controller 379 2022/06/16 10:24:11 [INFO] Starting rke.cattle.io/v1, Kind=RKEBootstrapTemplate controller 380 2022/06/16 10:24:11 [INFO] Starting monitoring.coreos.com/v1, Kind=Prometheus controller 381 2022/06/16 10:24:11 [INFO] Starting catalog.cattle.io/v1, Kind=Operation controller 382 2022/06/16 10:24:11 [ERROR] error parsing azure-group-cache-size, skipping update strconv.Atoi: parsing "": invalid syntax 383 2022/06/16 10:24:11 [INFO] Updating workload [monitoring/node-exporter] with public endpoints [[{"nodeName":":REDACTED-IP4-ADDRESS","addresses":["REDACTED-IP4-ADDRESS"],"port":9100,"protocol":"TCP","podName":"monitoring:node-exporter-79v9k","allNodes":false},{"nodeName":":REDACTED-IP4-ADDRESS","addresses":["REDACTED-IP4-ADDRESS"],"port":9100,"protocol":"TCP","podName":"monitoring:node-exporter-9p58d","allNodes":false}]] 384 2022/06/16 10:24:11 [INFO] Refreshing driverMetadata in 1440 minutes 385 2022/06/16 10:24:11 [ERROR] error parsing azure-group-cache-size, skipping update strconv.Atoi: parsing "": invalid syntax 386 2022/06/16 10:24:11 [INFO] Updating workload [verrazzano-monitoring/prometheus-node-exporter] with public endpoints [[{"nodeName":":REDACTED-IP4-ADDRESS","addresses":["REDACTED-IP4-ADDRESS"],"port":9400,"protocol":"TCP","podName":"verrazzano-monitoring:prometheus-node-exporter-7tdvs","allNodes":false},{"nodeName":":REDACTED-IP4-ADDRESS","addresses":["REDACTED-IP4-ADDRESS"],"port":9400,"protocol":"TCP","podName":"verrazzano-monitoring:prometheus-node-exporter-psvls","allNodes":false}]] 387 2022/06/16 10:24:11 [ERROR] error parsing azure-group-cache-size, skipping update strconv.Atoi: parsing "": invalid syntax 388 2022/06/16 10:24:12 [INFO] getReleaseConfig: reloading config for k3s 389 2022/06/16 10:24:12 [INFO] getReleaseConfig: reloading config for rke2 390 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-kontainerdrivers-manage for corresponding GlobalRole 391 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-clustertemplaterevisions-create for corresponding GlobalRole 392 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-roles-manage for corresponding GlobalRole 393 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-podsecuritypolicytemplates-manage for corresponding GlobalRole 394 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-view-rancher-metrics for corresponding GlobalRole 395 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-settings-manage for corresponding GlobalRole 396 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-nodedrivers-manage for corresponding GlobalRole 397 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-clustertemplates-create for corresponding GlobalRole 398 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-authn-manage for corresponding GlobalRole 399 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-catalogs-use for corresponding GlobalRole 400 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-users-manage for corresponding GlobalRole 401 2022/06/16 10:24:12 [INFO] Creating new GlobalRoleBinding for GlobalRoleBinding globalrolebinding-qj499 402 2022/06/16 10:24:12 [INFO] [mgmt-auth-grb-controller] Creating clusterRoleBinding for globalRoleBinding globalrolebinding-qj499 for user user-f8kfd with role cattle-globalrole-admin 403 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-catalogs-manage for corresponding GlobalRole 404 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-clusters-create for corresponding GlobalRole 405 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-admin for corresponding GlobalRole 406 2022/06/16 10:24:12 [ERROR] error syncing 'server-url': handler apiservice-settings: secrets "tls-rancher-internal-ca" not found, requeuing 407 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-restricted-admin for corresponding GlobalRole 408 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-features-manage for corresponding GlobalRole 409 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-user-base for corresponding GlobalRole 410 2022/06/16 10:24:12 [INFO] create kontainerdriver aliyunkubernetescontainerservice 411 2022/06/16 10:24:12 [INFO] create kontainerdriver azurekubernetesservice 412 2022/06/16 10:24:12 [INFO] create kontainerdriver rancherkubernetesengine 413 2022/06/16 10:24:12 [INFO] create kontainerdriver amazonelasticcontainerservice 414 2022/06/16 10:24:12 [INFO] create kontainerdriver opentelekomcloudcontainerengine 415 2022/06/16 10:24:12 [INFO] create kontainerdriver baiducloudcontainerengine 416 2022/06/16 10:24:12 [INFO] [mgmt-auth-gr-controller] Creating clusterRole cattle-globalrole-user for corresponding GlobalRole 417 2022/06/16 10:24:12 [INFO] create kontainerdriver linodekubernetesengine 418 2022/06/16 10:24:12 [INFO] create kontainerdriver googlekubernetesengine 419 2022/06/16 10:24:12 [INFO] create kontainerdriver huaweicontainercloudengine 420 2022/06/16 10:24:12 [INFO] create kontainerdriver oraclecontainerengine 421 2022/06/16 10:24:12 [INFO] create kontainerdriver tencentkubernetesengine 422 2022/06/16 10:24:12 [INFO] Listening on :444 423 2022/06/16 10:24:12 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:12 +0000 UTC notAfter=2023-06-16 10:24:12 +0000 UTC 424 2022/06/16 10:24:12 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:12 +0000 UTC notAfter=2023-06-16 10:24:12 +0000 UTC 425 2022/06/16 10:24:12 [ERROR] error syncing 'server-url': handler apiservice-settings: secrets "tls-rancher-internal-ca" not found, requeuing 426 2022/06/16 10:24:12 [INFO] driverMetadata: refreshing data from upstream https://releases.rancher.com/kontainer-driver-metadata/release-v2.6/data.json 427 2022/06/16 10:24:12 [INFO] Retrieve data.json from local path /var/lib/rancher-data/driver-metadata/data.json 428 2022/06/16 10:24:13 [INFO] create kontainerdriver rancherkubernetesengine 429 2022/06/16 10:24:13 [INFO] Rancher startup complete 430 2022/06/16 10:24:13 [INFO] checking configmap cattle-system/admincreated to determine if orphan bindings cleanup needs to run 431 2022/06/16 10:24:13 [INFO] create kontainerdriver googlekubernetesengine 432 2022/06/16 10:24:13 [INFO] create kontainerdriver amazonelasticcontainerservice 433 2022/06/16 10:24:13 [INFO] checking configmap cattle-system/admincreated to determine if duplicate bindings cleanup needs to run 434 2022/06/16 10:24:13 [INFO] create kontainerdriver azurekubernetesservice 435 2022/06/16 10:24:13 [INFO] create kontainerdriver baiducloudcontainerengine 436 2022/06/16 10:24:13 [INFO] Updating global catalog system-library 437 2022/06/16 10:24:13 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:12 +0000 UTC notAfter=2023-06-16 10:24:13 +0000 UTC 438 2022/06/16 10:24:13 [INFO] Creating new TLS secret for tls-rancher-internal (count: 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=9B8D6CBDAF676752330562BBF745A57E02B7D2DF] 439 2022/06/16 10:24:13 [INFO] update kontainerdriver aliyunkubernetescontainerservice 440 2022/06/16 10:24:13 [INFO] update kontainerdriver linodekubernetesengine 441 2022/06/16 10:24:13 [INFO] update kontainerdriver opentelekomcloudcontainerengine 442 2022/06/16 10:24:13 [INFO] update kontainerdriver tencentkubernetesengine 443 2022/06/16 10:24:13 [INFO] [clean-orphan-bindings] cleaning up orphaned bindings 444 2022/06/16 10:24:13 [INFO] [clean-orphan-bindings] checking for orphaned rolebindings 445 2022/06/16 10:24:13 [INFO] update kontainerdriver googlekubernetesengine 446 2022/06/16 10:24:13 [INFO] create kontainerdriver huaweicontainercloudengine 447 2022/06/16 10:24:13 [INFO] create kontainerdriver oraclecontainerengine 448 2022/06/16 10:24:13 [INFO] update kontainerdriver rancherkubernetesengine 449 2022/06/16 10:24:13 [INFO] update kontainerdriver amazonelasticcontainerservice 450 2022/06/16 10:24:13 [INFO] update kontainerdriver rancherkubernetesengine 451 2022/06/16 10:24:13 [INFO] bindings cleanup needed, waiting 3 minutes before starting 452 2022/06/16 10:24:13 [INFO] Active TLS secret tls-rancher-internal (ver=7435) (count 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=9B8D6CBDAF676752330562BBF745A57E02B7D2DF] 453 2022/06/16 10:24:13 [INFO] update kontainerdriver azurekubernetesservice 454 2022/06/16 10:24:13 [INFO] update kontainerdriver baiducloudcontainerengine 455 2022/06/16 10:24:13 [INFO] update kontainerdriver baiducloudcontainerengine 456 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment 457 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=Machine 458 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineSet 459 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineHealthCheck 460 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=Cluster 461 2022/06/16 10:24:13 [ERROR] failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment 462 2022/06/16 10:24:13 [ERROR] failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=Machine 463 2022/06/16 10:24:13 [ERROR] failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=MachineSet 464 2022/06/16 10:24:13 [ERROR] failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=MachineHealthCheck 465 2022/06/16 10:24:13 [ERROR] failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=Cluster 466 2022/06/16 10:24:13 [INFO] Starting provisioning.cattle.io/v1, Kind=Cluster controller 467 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=RKEControlPlane controller 468 2022/06/16 10:24:13 [ERROR] failed to start controller for cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment: failed to wait for caches to sync 469 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=CustomMachine controller 470 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=ETCDSnapshot controller 471 2022/06/16 10:24:13 [ERROR] failed to start controller for cluster.x-k8s.io/v1alpha3, Kind=Machine: failed to wait for caches to sync 472 2022/06/16 10:24:13 [ERROR] failed to start controller for cluster.x-k8s.io/v1alpha3, Kind=MachineSet: failed to wait for caches to sync 473 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=RKEBootstrapTemplate controller 474 2022/06/16 10:24:13 [ERROR] failed to start controller for cluster.x-k8s.io/v1alpha3, Kind=MachineHealthCheck: failed to wait for caches to sync 475 2022/06/16 10:24:13 [ERROR] failed to start controller for cluster.x-k8s.io/v1alpha3, Kind=Cluster: failed to wait for caches to sync 476 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=RKECluster controller 477 2022/06/16 10:24:13 [INFO] Starting rke.cattle.io/v1, Kind=RKEBootstrap controller 478 E0616 10:24:13.964421 9 gvks.go:69] failed to sync schemas: failed to sync cache for cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment 479 2022/06/16 10:24:13 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=Machine 480 2022/06/16 10:24:13 [INFO] update kontainerdriver huaweicontainercloudengine 481 2022/06/16 10:24:13 [INFO] update kontainerdriver huaweicontainercloudengine 482 2022/06/16 10:24:14 [INFO] update kontainerdriver oraclecontainerengine 483 2022/06/16 10:24:14 [INFO] update kontainerdriver oraclecontainerengine 484 2022/06/16 10:24:14 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment 485 2022/06/16 10:24:14 [INFO] Starting cluster.x-k8s.io/v1alpha3, Kind=Machine controller 486 2022/06/16 10:24:14 [INFO] [CleanupOrphanBindingsDone] successfully cleaned up orphan bindings 487 2022/06/16 10:24:14 [INFO] checking configmap cattle-system/admincreated to determine if orphan bindings cleanup needs to run 488 2022/06/16 10:24:14 [INFO] Starting cluster.x-k8s.io/v1alpha3, Kind=MachineDeployment controller 489 2022/06/16 10:24:14 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineSet 490 2022/06/16 10:24:14 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=MachineHealthCheck 491 2022/06/16 10:24:14 [INFO] Watching metadata for cluster.x-k8s.io/v1alpha3, Kind=Cluster 492 2022/06/16 10:24:14 [INFO] [clean-catalog-orphan-bindings] cleaning up orphaned catalog bindings 493 2022/06/16 10:24:14 [INFO] [clean-catalog-orphan-bindings] Processing 1 rolebindings 494 2022/06/16 10:24:14 [INFO] [clean-catalog-orphan-bindings] Deleting orphaned role global-catalog 495 2022/06/16 10:24:14 [WARNING] [clean-catalog-orphan-bindings] Error when deleting role global-catalog, roles.rbac.authorization.k8s.io "global-catalog" not found 496 2022/06/16 10:24:14 [WARNING] [CleanupOrphanCatalogBindingsDone] error during orphan binding cleanup: roles.rbac.authorization.k8s.io "global-catalog" not found 497 2022/06/16 10:24:14 [ERROR] failed to cleanup orphan catalog bindings 498 2022/06/16 10:24:14 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 499 2022/06/16 10:24:14 [INFO] Loaded configuration from &{} in [0x6f36500] 500 2022/06/16 10:24:14 [INFO] Starting cluster.x-k8s.io/v1alpha3, Kind=MachineSet controller 501 2022/06/16 10:24:14 [INFO] Starting cluster.x-k8s.io/v1alpha3, Kind=MachineHealthCheck controller 502 2022/06/16 10:24:14 [INFO] Starting cluster.x-k8s.io/v1alpha3, Kind=Cluster controller 503 2022/06/16 10:24:15 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 504 2022/06/16 10:24:15 [INFO] Loaded configuration from &{} in [0x6f36500] 505 E0616 10:24:15.277978 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 506 2022/06/16 10:24:15 [INFO] Starting /v1, Kind=Secret controller 507 2022/06/16 10:24:15 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:15 +0000 UTC 508 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating role/clusterRole local-clusterowner 509 2022/06/16 10:24:15 [INFO] Updating TLS secret for serving-cert (count: 5): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=2854FB62908252BCD30E81794EE96D1B1891BA7C] 510 2022/06/16 10:24:15 [INFO] Active TLS secret tls-rancher-internal (ver=7515) (count 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=5D79E4FF2D062BA40C9D7D1AC732E88643748ACD] 511 2022/06/16 10:24:15 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found 512 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating clusterRoleBinding for membership in cluster local for subject u-mo773yttt4 513 2022/06/16 10:24:15 [INFO] Active TLS secret serving-cert (ver=7556) (count 5): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=2854FB62908252BCD30E81794EE96D1B1891BA7C] 514 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace local 515 2022/06/16 10:24:15 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 516 2022/06/16 10:24:15 [ERROR] error syncing 'fleet-local/local': handler workspace-backport-cluster: fleetworkspaces.management.cattle.io "fleet-local" not found, requeuing 517 2022/06/16 10:24:15 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 518 2022/06/16 10:24:15 [INFO] [mgmt-cluster-rbac-delete] Creating Default project for cluster local 519 2022/06/16 10:24:15 [INFO] [mgmt-project-rbac-create] Creating namespace p-bthll 520 2022/06/16 10:24:15 [INFO] [mgmt-cluster-rbac-delete] Creating System project for cluster local 521 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace local 522 2022/06/16 10:24:15 [INFO] [mgmt-project-rbac-create] Creating namespace p-5hdmj 523 2022/06/16 10:24:15 [INFO] [mgmt-cluster-rbac-delete] Updating cluster local 524 2022/06/16 10:24:15 [WARNING] Project p-bthll has no creatorId annotation. Cannot add creator as owner 525 2022/06/16 10:24:15 [INFO] [mgmt-project-rbac-create] Updating project p-bthll 526 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-5hdmj 527 2022/06/16 10:24:15 [WARNING] Cluster local has no creatorId annotation. Cannot add creator as owner 528 2022/06/16 10:24:15 [INFO] [mgmt-cluster-rbac-delete] Updating cluster local 529 2022/06/16 10:24:15 [WARNING] Project p-5hdmj has no creatorId annotation. Cannot add creator as owner 530 2022/06/16 10:24:15 [INFO] [mgmt-project-rbac-create] Updating project p-5hdmj 531 2022/06/16 10:24:15 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 532 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace p-5hdmj 533 2022/06/16 10:24:15 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace p-bthll 534 2022/06/16 10:24:16 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-mo773yttt4 with role cluster-owner in namespace p-bthll 535 2022/06/16 10:24:16 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 536 2022/06/16 10:24:16 [INFO] uploading azureConfig to nodeconfig schema 537 2022/06/16 10:24:16 [INFO] uploading azureConfig to nodetemplateconfig schema 538 2022/06/16 10:24:16 [INFO] uploading digitaloceanConfig to nodeconfig schema 539 2022/06/16 10:24:16 [INFO] uploading digitaloceanConfig to nodetemplateconfig schema 540 2022/06/16 10:24:16 [INFO] uploading amazonec2Config to nodeconfig schema 541 2022/06/16 10:24:16 [INFO] uploading amazonec2Config to nodetemplateconfig schema 542 2022/06/16 10:24:16 [INFO] uploading linodeConfig to nodeconfig schema 543 2022/06/16 10:24:16 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 544 2022/06/16 10:24:16 [INFO] uploading linodeConfig to nodetemplateconfig schema 545 2022/06/16 10:24:16 [INFO] uploading harvesterConfig to nodeconfig schema 546 2022/06/16 10:24:16 [INFO] uploading harvesterConfig to nodetemplateconfig schema 547 2022/06/16 10:24:17 [INFO] uploading vmwarevsphereConfig to nodeconfig schema 548 2022/06/16 10:24:17 [INFO] uploading vmwarevsphereConfig to nodetemplateconfig schema 549 2022/06/16 10:24:17 [INFO] uploading azurecredentialConfig to credentialconfig schema 550 E0616 10:24:17.809197 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 551 2022/06/16 10:24:17 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=AzureConfig 552 2022/06/16 10:24:17 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=DigitaloceanConfig 553 2022/06/16 10:24:17 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachine 554 2022/06/16 10:24:17 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachineTemplate 555 2022/06/16 10:24:17 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachine 556 2022/06/16 10:24:18 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:18 +0000 UTC 557 2022/06/16 10:24:18 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 558 2022/06/16 10:24:18 [INFO] Adding peer wss://REDACTED-IP4-ADDRESS/v3/connect, REDACTED-IP4-ADDRESS 559 2022/06/16 10:24:18 [ERROR] Failed to connect to peer wss://REDACTED-IP4-ADDRESS/v3/connect [local ID=REDACTED-IP4-ADDRESS]: EOF 560 2022/06/16 10:24:18 [INFO] uploading digitaloceancredentialConfig to credentialconfig schema 561 2022/06/16 10:24:18 [INFO] Updating TLS secret for serving-cert (count: 6): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=8BD3DB21263AF6375D218BC92A11BB501875F17D] 562 2022/06/16 10:24:18 [INFO] uploading amazonec2credentialConfig to credentialconfig schema 563 2022/06/16 10:24:18 [INFO] uploading googlecredentialConfig to credentialconfig schema 564 2022/06/16 10:24:18 [INFO] Active TLS secret serving-cert (ver=7789) (count 6): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=8BD3DB21263AF6375D218BC92A11BB501875F17D] 565 2022/06/16 10:24:18 [INFO] Active TLS secret tls-rancher-internal (ver=7784) (count 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=7060DB796373F05108A33C081F1B39BF217F1660] 566 2022/06/16 10:24:18 [INFO] uploading linodecredentialConfig to credentialconfig schema 567 2022/06/16 10:24:18 [INFO] kontainerdriver googlekubernetesengine listening on address REDACTED-IP4-ADDRESS:33963 568 2022/06/16 10:24:18 [INFO] kontainerdriver googlekubernetesengine stopped 569 2022/06/16 10:24:18 [INFO] kontainerdriver azurekubernetesservice listening on address REDACTED-IP4-ADDRESS:39513 570 2022/06/16 10:24:18 [INFO] kontainerdriver amazonelasticcontainerservice listening on address REDACTED-IP4-ADDRESS:35859 571 2022/06/16 10:24:18 [INFO] kontainerdriver azurekubernetesservice stopped 572 2022/06/16 10:24:18 [INFO] kontainerdriver amazonelasticcontainerservice stopped 573 2022/06/16 10:24:19 [INFO] uploading harvestercredentialConfig to credentialconfig schema 574 2022/06/16 10:24:19 [INFO] uploading vmwarevspherecredentialConfig to credentialconfig schema 575 2022/06/16 10:24:19 [INFO] Active TLS secret tls-rancher-internal (ver=7806) (count 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=CDFDDACCB4B7E8FA865290DF7C42A52ABF1C7A61] 576 2022/06/16 10:24:19 [INFO] Handling backend connection request [REDACTED-IP4-ADDRESS] 577 2022/06/16 10:24:19 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:57542: response 400: cluster not found 578 2022/06/16 10:24:19 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:49756: response 400: cluster not found 579 2022/06/16 10:24:19 [INFO] update kontainerdriver amazonelasticcontainerservice 580 2022/06/16 10:24:19 [WARNING] Cluster local has no creatorId annotation. Cannot add creator as owner 581 2022/06/16 10:24:19 [INFO] [mgmt-cluster-rbac-delete] Updating cluster local 582 2022/06/16 10:24:19 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 583 2022/06/16 10:24:19 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=AzureConfig controller 584 2022/06/16 10:24:19 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=DigitaloceanConfig controller 585 2022/06/16 10:24:19 [INFO] Starting rke-machine.cattle.io/v1, Kind=DigitaloceanMachine controller 586 2022/06/16 10:24:19 [INFO] Starting rke-machine.cattle.io/v1, Kind=AzureMachineTemplate controller 587 2022/06/16 10:24:19 [INFO] Starting rke-machine.cattle.io/v1, Kind=AzureMachine controller 588 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=Amazonec2Config 589 2022/06/16 10:24:19 [INFO] uploading googlecredentialConfig to credentialconfig schema 590 2022/06/16 10:24:19 [INFO] update kontainerdriver azurekubernetesservice 591 2022/06/16 10:24:19 [INFO] update kontainerdriver googlekubernetesengine 592 2022/06/16 10:24:19 [INFO] uploading vmwarevspherecredentialConfig to credentialconfig schema 593 2022/06/16 10:24:19 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=Amazonec2Config controller 594 E0616 10:24:19.949824 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 595 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=LinodeConfig 596 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig 597 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig 598 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachine 599 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate 600 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachine 601 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate 602 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 603 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate 604 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate 605 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2Machine 606 2022/06/16 10:24:19 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine 607 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig 608 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig 609 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=LinodeMachine 610 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate 611 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=HarvesterMachine 612 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate 613 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 614 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate 615 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate 616 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=Amazonec2Machine 617 2022/06/16 10:24:20 [ERROR] failed to sync cache for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine 618 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig: failed to wait for caches to sync 619 2022/06/16 10:24:20 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=LinodeConfig controller 620 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig: failed to wait for caches to sync 621 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=LinodeMachine: failed to wait for caches to sync 622 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate: failed to wait for caches to sync 623 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=HarvesterMachine: failed to wait for caches to sync 624 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate: failed to wait for caches to sync 625 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate: failed to wait for caches to sync 626 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate: failed to wait for caches to sync 627 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate: failed to wait for caches to sync 628 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=Amazonec2Machine: failed to wait for caches to sync 629 2022/06/16 10:24:20 [ERROR] failed to start controller for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine: failed to wait for caches to sync 630 2022/06/16 10:24:20 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig 631 E0616 10:24:20.181801 9 gvks.go:69] failed to sync schemas: failed to sync cache for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig 632 2022/06/16 10:24:20 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found 633 2022/06/16 10:24:20 [INFO] Catalog sync done. 7 templates created, 0 templates updated, 0 templates deleted, 0 templates failed 634 2022/06/16 10:24:22 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 635 2022/06/16 10:24:22 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=HarvesterConfig controller 636 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig 637 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachine 638 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate 639 2022/06/16 10:24:22 [INFO] Starting rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig controller 640 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachine 641 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate 642 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 643 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate 644 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate 645 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2Machine 646 2022/06/16 10:24:22 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine 647 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=LinodeMachine controller 648 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate controller 649 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate controller 650 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate controller 651 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate controller 652 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=HarvesterMachine controller 653 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate controller 654 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=Amazonec2Machine controller 655 2022/06/16 10:24:22 [INFO] Starting rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine controller 656 2022/06/16 10:24:24 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:57668: response 400: cluster not found 657 2022/06/16 10:24:24 [INFO] kontainerdriver amazonelasticcontainerservice listening on address REDACTED-IP4-ADDRESS:33277 658 2022/06/16 10:24:24 [INFO] kontainerdriver amazonelasticcontainerservice stopped 659 2022/06/16 10:24:24 [INFO] dynamic schema for kontainerdriver amazonelasticcontainerservice updating 660 2022/06/16 10:24:24 [INFO] kontainerdriver azurekubernetesservice listening on address REDACTED-IP4-ADDRESS:37569 661 2022/06/16 10:24:24 [INFO] kontainerdriver azurekubernetesservice stopped 662 2022/06/16 10:24:24 [INFO] dynamic schema for kontainerdriver azurekubernetesservice updating 663 2022/06/16 10:24:24 [INFO] kontainerdriver googlekubernetesengine listening on address REDACTED-IP4-ADDRESS:42153 664 2022/06/16 10:24:24 [INFO] kontainerdriver googlekubernetesengine stopped 665 2022/06/16 10:24:24 [INFO] dynamic schema for kontainerdriver googlekubernetesengine updating 666 2022/06/16 10:24:25 [ERROR] Failed to find system chart fleet will try again in 5 seconds: configmaps "" not found 667 2022/06/16 10:24:26 [INFO] Starting cluster controllers for local 668 2022/06/16 10:24:27 [ERROR] error syncing 'git-webhook': handler apiservice: failed to create cattle-fleet-system/stv-aggregation /v1, Kind=Secret for apiservice git-webhook: namespaces "cattle-fleet-system" not found, requeuing 669 2022/06/16 10:24:28 [INFO] Starting management.cattle.io/v3, Kind=SamlToken controller 670 E0616 10:24:28.939829 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 671 2022/06/16 10:24:28 [INFO] Starting cluster agent for local [owner=false] 672 2022/06/16 10:24:28 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=RoleBinding controller 673 2022/06/16 10:24:28 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=Role controller 674 2022/06/16 10:24:28 [INFO] Starting /v1, Kind=Namespace controller 675 2022/06/16 10:24:28 [INFO] Starting /v1, Kind=Secret controller 676 2022/06/16 10:24:28 [INFO] Starting /v1, Kind=ServiceAccount controller 677 2022/06/16 10:24:28 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller 678 2022/06/16 10:24:28 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRole controller 679 2022/06/16 10:24:29 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:57768: response 400: cluster not found 680 2022/06/16 10:24:29 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:24:29 +0000 UTC 681 2022/06/16 10:24:29 [INFO] Updating TLS secret for serving-cert (count: 7): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=160606628277A4D81C7526AA88640324AD0E4C95] 682 2022/06/16 10:24:30 [INFO] Active TLS secret serving-cert (ver=8146) (count 7): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=160606628277A4D81C7526AA88640324AD0E4C95] 683 2022/06/16 10:24:30 [INFO] Adding peer wss://REDACTED-IP4-ADDRESS/v3/connect, REDACTED-IP4-ADDRESS 684 2022/06/16 10:24:30 [INFO] Stopping cluster agent for local 685 2022/06/16 10:24:30 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=ClusterRole workers 686 2022/06/16 10:24:30 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding workers 687 2022/06/16 10:24:30 [INFO] Shutting down /v1, Kind=ServiceAccount workers 688 2022/06/16 10:24:30 [INFO] Shutting down /v1, Kind=Secret workers 689 2022/06/16 10:24:30 [INFO] Shutting down /v1, Kind=Namespace workers 690 2022/06/16 10:24:30 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=Role workers 691 2022/06/16 10:24:30 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=RoleBinding workers 692 2022/06/16 10:24:30 [INFO] error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF 693 2022/06/16 10:24:30 [ERROR] Failed to serve peer connection REDACTED-IP4-ADDRESS: websocket: close 1006 (abnormal closure): unexpected EOF 694 2022/06/16 10:24:30 [INFO] Removing peer REDACTED-IP4-ADDRESS 695 E0616 10:24:30.582660 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 696 2022/06/16 10:24:30 [INFO] Starting /v1, Kind=ConfigMap controller 697 2022/06/16 10:24:30 [INFO] Starting /v1, Kind=ConfigMap controller 698 2022/06/16 10:24:30 [INFO] Registering istio for cluster "local" 699 2022/06/16 10:24:30 [INFO] Starting cluster controllers for local 700 2022/06/16 10:24:30 [INFO] Starting management.cattle.io/v3, Kind=ClusterAlert controller 701 2022/06/16 10:24:30 [INFO] Starting management.cattle.io/v3, Kind=ClusterLogging controller 702 E0616 10:24:31.567834 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 703 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=Node controller 704 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=Namespace controller 705 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=ServiceAccount controller 706 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=LimitRange controller 707 2022/06/16 10:24:31 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRole controller 708 2022/06/16 10:24:31 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=Role controller 709 2022/06/16 10:24:31 [INFO] Starting apiregistration.k8s.io/v1, Kind=APIService controller 710 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=Secret controller 711 2022/06/16 10:24:31 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=RoleBinding controller 712 2022/06/16 10:24:31 [INFO] Starting /v1, Kind=ResourceQuota controller 713 2022/06/16 10:24:31 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller 714 2022/06/16 10:24:31 [INFO] Starting cluster agent for local [owner=true] 715 2022/06/16 10:24:31 [INFO] Creating clusterRole for roleTemplate Cluster Owner (cluster-owner). 716 2022/06/16 10:24:31 [ERROR] error parsing azure-group-cache-size, skipping update strconv.Atoi: parsing "": invalid syntax 717 2022/06/16 10:24:31 [INFO] Creating clusterRoleBinding User u-mo773yttt4 Role cluster-owner 718 2022/06/16 10:24:31 [INFO] Creating clusterRole for roleTemplate Cluster Owner (cluster-owner). 719 2022/06/16 10:24:31 [INFO] Watching metadata for networking.k8s.io/v1, Kind=NetworkPolicy 720 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Service 721 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=PodSecurityPolicyTemplate 722 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=EtcdBackup 723 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Template 724 2022/06/16 10:24:31 [INFO] Watching metadata for cluster.x-k8s.io/v1beta1, Kind=Cluster 725 2022/06/16 10:24:31 [INFO] Watching metadata for storage.k8s.io/v1, Kind=StorageClass 726 2022/06/16 10:24:31 [INFO] Watching metadata for cluster.x-k8s.io/v1beta1, Kind=MachineSet 727 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=HealthScope 728 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=IngressTrait 729 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectAlertGroup 730 2022/06/16 10:24:31 [INFO] Watching metadata for rbac.authorization.k8s.io/v1, Kind=ClusterRole 731 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterScan 732 2022/06/16 10:24:31 [INFO] Watching metadata for telemetry.istio.io/v1alpha1, Kind=Telemetry 733 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=Prometheus 734 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=DestinationRule 735 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=RKECluster 736 2022/06/16 10:24:31 [INFO] Watching metadata for storage.k8s.io/v1, Kind=VolumeAttachment 737 2022/06/16 10:24:31 [INFO] Watching metadata for rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding 738 2022/06/16 10:24:31 [INFO] Watching metadata for storage.k8s.io/v1, Kind=CSINode 739 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=GroupMember 740 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterRegistrationToken 741 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=AuthConfig 742 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=RoleTemplate 743 2022/06/16 10:24:31 [INFO] Watching metadata for admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration 744 2022/06/16 10:24:31 [INFO] Watching metadata for policy/v1, Kind=PodDisruptionBudget 745 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=PodMonitor 746 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterAlertRule 747 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Notifier 748 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=VerrazzanoProject 749 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=PipelineExecution 750 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=RKEControlPlane 751 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=UserAttribute 752 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=CatalogTemplate 753 2022/06/16 10:24:31 [INFO] Watching metadata for flowcontrol.apiserver.k8s.io/v1beta1, Kind=PriorityLevelConfiguration 754 2022/06/16 10:24:31 [INFO] Watching metadata for install.verrazzano.io/v1alpha1, Kind=Verrazzano 755 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=GlobalRole 756 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=CisBenchmarkVersion 757 2022/06/16 10:24:31 [INFO] Watching metadata for discovery.k8s.io/v1, Kind=EndpointSlice 758 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Group 759 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Node 760 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=SourceCodeCredential 761 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=VerrazzanoWebLogicWorkload 762 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=NodeTemplate 763 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=CatalogTemplateVersion 764 2022/06/16 10:24:31 [INFO] Watching metadata for catalog.cattle.io/v1, Kind=ClusterRepo 765 2022/06/16 10:24:31 [INFO] Watching metadata for apiregistration.k8s.io/v1, Kind=APIService 766 2022/06/16 10:24:31 [INFO] Watching metadata for apiextensions.k8s.io/v1, Kind=CustomResourceDefinition 767 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=TemplateVersion 768 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterTemplateRevision 769 2022/06/16 10:24:31 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=Bundle 770 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=Probe 771 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=RKEBootstrapTemplate 772 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectRoleTemplateBinding 773 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=VerrazzanoManagedCluster 774 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=PersistentVolume 775 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Event 776 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=ManualScalerTrait 777 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=LimitRange 778 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=CustomMachine 779 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=CisConfig 780 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterAlertGroup 781 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterRoleTemplateBinding 782 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectAlertRule 783 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Pod 784 2022/06/16 10:24:31 [INFO] Watching metadata for batch/v1, Kind=CronJob 785 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=RkeK8sSystemImage 786 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=VirtualService 787 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=FleetWorkspace 788 2022/06/16 10:24:31 [INFO] Watching metadata for catalog.cattle.io/v1, Kind=Operation 789 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=VerrazzanoHelidonWorkload 790 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=MultiClusterAppRevision 791 2022/06/16 10:24:31 [INFO] Watching metadata for flowcontrol.apiserver.k8s.io/v1beta1, Kind=FlowSchema 792 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=SourceCodeRepository 793 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1alpha3, Kind=EnvoyFilter 794 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=PrometheusRule 795 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Preference 796 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Setting 797 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=PodSecurityPolicyTemplateProjectBinding 798 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=RKEBootstrap 799 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=MultiClusterApp 800 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=MultiClusterComponent 801 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=ServiceEntry 802 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=SourceCodeProviderConfig 803 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1alpha1, Kind=AlertmanagerConfig 804 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=GlobalRoleBinding 805 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterLogging 806 2022/06/16 10:24:31 [INFO] Watching metadata for verrazzano.io/v1, Kind=VerrazzanoMonitoringInstance 807 2022/06/16 10:24:31 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=Cluster 808 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=MonitorMetric 809 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Node 810 2022/06/16 10:24:31 [INFO] Watching metadata for storage.k8s.io/v1beta1, Kind=CSIStorageCapacity 811 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ComposeConfig 812 2022/06/16 10:24:31 [INFO] Watching metadata for autoscaling/v2beta2, Kind=HorizontalPodAutoscaler 813 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=Pipeline 814 2022/06/16 10:24:31 [INFO] Watching metadata for apps/v1, Kind=Deployment 815 2022/06/16 10:24:31 [INFO] Watching metadata for cert-manager.io/v1, Kind=CertificateRequest 816 2022/06/16 10:24:31 [INFO] Watching metadata for cert-manager.io/v1, Kind=Certificate 817 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=GlobalDnsProvider 818 2022/06/16 10:24:31 [INFO] Watching metadata for cluster.x-k8s.io/v1beta1, Kind=Machine 819 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=KontainerDriver 820 2022/06/16 10:24:31 [INFO] Watching metadata for provisioning.cattle.io/v1, Kind=Cluster 821 2022/06/16 10:24:31 [INFO] Watching metadata for rbac.authorization.k8s.io/v1, Kind=Role 822 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectAlert 823 2022/06/16 10:24:31 [INFO] Watching metadata for security.istio.io/v1beta1, Kind=PeerAuthentication 824 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=MultiClusterApplicationConfiguration 825 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=ScopeDefinition 826 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=NodePool 827 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=ConfigMap 828 2022/06/16 10:24:31 [INFO] Watching metadata for networking.k8s.io/v1, Kind=IngressClass 829 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=PodTemplate 830 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=RkeAddon 831 2022/06/16 10:24:31 [INFO] Watching metadata for node.k8s.io/v1, Kind=RuntimeClass 832 2022/06/16 10:24:31 [INFO] Watching metadata for weblogic.oracle/v8, Kind=Domain 833 2022/06/16 10:24:31 [INFO] Watching metadata for cluster.x-k8s.io/v1beta1, Kind=MachineDeployment 834 2022/06/16 10:24:31 [INFO] Watching metadata for acme.cert-manager.io/v1, Kind=Order 835 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectCatalog 836 2022/06/16 10:24:31 [INFO] Watching metadata for admissionregistration.k8s.io/v1, Kind=MutatingWebhookConfiguration 837 2022/06/16 10:24:31 [INFO] Watching metadata for batch/v1, Kind=Job 838 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ManagedChart 839 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterTemplate 840 2022/06/16 10:24:31 [INFO] Watching metadata for storage.k8s.io/v1, Kind=CSIDriver 841 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.kiali.io/v1alpha1, Kind=MonitoringDashboard 842 2022/06/16 10:24:31 [INFO] Watching metadata for app.verrazzano.io/v1alpha1, Kind=MetricsTemplate 843 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=ProxyConfig 844 2022/06/16 10:24:31 [INFO] Watching metadata for policy/v1beta1, Kind=PodSecurityPolicy 845 2022/06/16 10:24:31 [INFO] Watching metadata for apps/v1, Kind=ControllerRevision 846 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterCatalog 847 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=GlobalDns 848 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=User 849 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Token 850 2022/06/16 10:24:31 [INFO] Watching metadata for apps/v1, Kind=ReplicaSet 851 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=ResourceQuota 852 2022/06/16 10:24:31 [INFO] Watching metadata for cluster.x-k8s.io/v1beta1, Kind=MachineHealthCheck 853 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=PipelineSetting 854 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=Gateway 855 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectMonitorGraph 856 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Secret 857 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Endpoints 858 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=WorkloadGroup 859 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Project 860 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=ThanosRuler 861 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectNetworkPolicy 862 2022/06/16 10:24:31 [INFO] Watching metadata for scheduling.k8s.io/v1, Kind=PriorityClass 863 2022/06/16 10:24:31 [INFO] Watching metadata for networking.k8s.io/v1, Kind=Ingress 864 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=ServiceMonitor 865 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=TemplateContent 866 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=App 867 2022/06/16 10:24:31 [INFO] Watching metadata for extensions.istio.io/v1alpha1, Kind=WasmPlugin 868 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=MultiClusterConfigMap 869 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=ReplicationController 870 2022/06/16 10:24:31 [INFO] Watching metadata for ui.cattle.io/v1, Kind=NavLink 871 2022/06/16 10:24:31 [INFO] Watching metadata for acme.cert-manager.io/v1, Kind=Challenge 872 2022/06/16 10:24:31 [INFO] Watching metadata for coherence.oracle.com/v1, Kind=Coherence 873 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=Component 874 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterAlert 875 2022/06/16 10:24:31 [INFO] Watching metadata for cert-manager.io/v1, Kind=ClusterIssuer 876 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=ServiceAccount 877 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=MetricsTrait 878 2022/06/16 10:24:31 [INFO] Watching metadata for security.istio.io/v1beta1, Kind=RequestAuthentication 879 2022/06/16 10:24:31 [INFO] Watching metadata for rbac.authorization.k8s.io/v1, Kind=RoleBinding 880 2022/06/16 10:24:31 [INFO] Watching metadata for security.istio.io/v1beta1, Kind=AuthorizationPolicy 881 2022/06/16 10:24:31 [INFO] Watching metadata for catalog.cattle.io/v1, Kind=App 882 2022/06/16 10:24:31 [INFO] Watching metadata for events.k8s.io/v1, Kind=Event 883 2022/06/16 10:24:31 [INFO] Watching metadata for monitoring.coreos.com/v1, Kind=Alertmanager 884 2022/06/16 10:24:31 [INFO] Watching metadata for rke.cattle.io/v1, Kind=ETCDSnapshot 885 2022/06/16 10:24:31 [INFO] Watching metadata for cert-manager.io/v1, Kind=Issuer 886 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=RkeK8sServiceOption 887 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=Namespace 888 2022/06/16 10:24:31 [INFO] Watching metadata for apps/v1, Kind=StatefulSet 889 2022/06/16 10:24:31 [INFO] Watching metadata for apps/v1, Kind=DaemonSet 890 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=TraitDefinition 891 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=DynamicSchema 892 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=WorkloadDefinition 893 2022/06/16 10:24:31 [INFO] Watching metadata for install.istio.io/v1alpha1, Kind=IstioOperator 894 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=LoggingTrait 895 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Cluster 896 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ClusterMonitorGraph 897 2022/06/16 10:24:31 [INFO] Watching metadata for oam.verrazzano.io/v1alpha1, Kind=VerrazzanoCoherenceWorkload 898 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=WorkloadEntry 899 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=ContainerizedWorkload 900 2022/06/16 10:24:31 [INFO] Watching metadata for core.oam.dev/v1alpha2, Kind=ApplicationConfiguration 901 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Catalog 902 2022/06/16 10:24:31 [INFO] Watching metadata for clusters.verrazzano.io/v1alpha1, Kind=MultiClusterSecret 903 2022/06/16 10:24:31 [INFO] Watching metadata for certificates.k8s.io/v1, Kind=CertificateSigningRequest 904 2022/06/16 10:24:31 [INFO] Watching metadata for project.cattle.io/v3, Kind=AppRevision 905 2022/06/16 10:24:31 [INFO] Watching metadata for networking.istio.io/v1beta1, Kind=Sidecar 906 2022/06/16 10:24:31 [INFO] Watching metadata for /v1, Kind=PersistentVolumeClaim 907 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=ProjectLogging 908 2022/06/16 10:24:31 [INFO] Watching metadata for coordination.k8s.io/v1, Kind=Lease 909 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=NodeDriver 910 2022/06/16 10:24:31 [INFO] Watching metadata for app.verrazzano.io/v1alpha1, Kind=MetricsBinding 911 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=SamlToken 912 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=Feature 913 2022/06/16 10:24:31 [INFO] Watching metadata for management.cattle.io/v3, Kind=APIService 914 2022/06/16 10:24:32 [INFO] Created machine for node [REDACTED-IP4-ADDRESS] 915 2022/06/16 10:24:32 [ERROR] error syncing 'local/p-5hdmj': handler project-namespace-auth: clusterroles.rbac.authorization.k8s.io "p-5hdmj-namespaces-edit" already exists, requeuing 916 2022/06/16 10:24:32 [INFO] Created machine for node [REDACTED-IP4-ADDRESS] 917 2022/06/16 10:24:34 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-public 918 2022/06/16 10:24:34 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=fleet-default 919 2022/06/16 10:24:34 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-public": the object has been modified; please apply your changes to the latest version and try again 920 2022/06/16 10:24:34 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "fleet-default": the object has been modified; please apply your changes to the latest version and try again 921 2022/06/16 10:24:34 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-system, err=Operation cannot be fulfilled on namespaces "cattle-fleet-system": the object has been modified; please apply your changes to the latest version and try again 922 2022/06/16 10:24:34 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=p-5hdmj, err=Operation cannot be fulfilled on namespaces "p-5hdmj": the object has been modified; please apply your changes to the latest version and try again 923 2022/06/16 10:24:34 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-system, err=Operation cannot be fulfilled on namespaces "cattle-system": the object has been modified; please apply your changes to the latest version and try again 924 2022/06/16 10:24:34 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=fleet-default 925 2022/06/16 10:24:34 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=fleet-local 926 2022/06/16 10:24:34 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cert-manager, err=Operation cannot be fulfilled on namespaces "cert-manager": the object has been modified; please apply your changes to the latest version and try again 927 2022/06/16 10:24:34 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-bthll to namespace=default 928 2022/06/16 10:24:34 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "fleet-local": the object has been modified; please apply your changes to the latest version and try again 929 2022/06/16 10:24:34 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "default": the object has been modified; please apply your changes to the latest version and try again 930 2022/06/16 10:24:34 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=kube-node-lease, err=Operation cannot be fulfilled on namespaces "kube-node-lease": the object has been modified; please apply your changes to the latest version and try again 931 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-global-data 932 2022/06/16 10:24:35 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-global-nt, err=Operation cannot be fulfilled on namespaces "cattle-global-nt": the object has been modified; please apply your changes to the latest version and try again 933 2022/06/16 10:24:35 [INFO] Creating user for principal system://local 934 2022/06/16 10:24:35 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=p-bthll, err=Operation cannot be fulfilled on namespaces "p-bthll": the object has been modified; please apply your changes to the latest version and try again 935 2022/06/16 10:24:35 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-global-data, err=Operation cannot be fulfilled on namespaces "cattle-global-data": the object has been modified; please apply your changes to the latest version and try again 936 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-node-lease 937 2022/06/16 10:24:35 [INFO] Creating globalRoleBindings for u-b4qkhsnliz 938 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-global-nt 939 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=fleet-local 940 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-bthll to namespace=default 941 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-impersonation-system 942 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-system 943 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-public 944 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-node-lease": the object has been modified; please apply your changes to the latest version and try again 945 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "default": the object has been modified; please apply your changes to the latest version and try again 946 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-global-nt": the object has been modified; please apply your changes to the latest version and try again 947 2022/06/16 10:24:35 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=ingress-nginx 948 2022/06/16 10:24:35 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=fleet-local, err=Operation cannot be fulfilled on namespaces "fleet-local": the object has been modified; please apply your changes to the latest version and try again 949 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-impersonation-system": the object has been modified; please apply your changes to the latest version and try again 950 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "ingress-nginx": the object has been modified; please apply your changes to the latest version and try again 951 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-fleet-system": the object has been modified; please apply your changes to the latest version and try again 952 2022/06/16 10:24:35 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-public": the object has been modified; please apply your changes to the latest version and try again 953 2022/06/16 10:24:35 [INFO] Creating new GlobalRoleBinding for GlobalRoleBinding grb-mktpm 954 2022/06/16 10:24:35 [INFO] [mgmt-auth-grb-controller] Creating clusterRoleBinding for globalRoleBinding grb-mktpm for user u-b4qkhsnliz with role cattle-globalrole-user 955 2022/06/16 10:24:35 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=ingress-nginx, err=Operation cannot be fulfilled on namespaces "ingress-nginx": the object has been modified; please apply your changes to the latest version and try again 956 2022/06/16 10:24:36 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-impersonation-system, err=Operation cannot be fulfilled on namespaces "cattle-impersonation-system": the object has been modified; please apply your changes to the latest version and try again 957 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-system 958 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-public 959 2022/06/16 10:24:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-fleet-system": the object has been modified; please apply your changes to the latest version and try again 960 2022/06/16 10:24:36 [INFO] [mgmt-auth-crtb-controller] Creating clusterRoleBinding for membership in cluster local for subject u-b4qkhsnliz 961 2022/06/16 10:24:36 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-b4qkhsnliz with role cluster-owner in namespace local 962 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-impersonation-system 963 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-system 964 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-global-nt 965 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-node-lease 966 2022/06/16 10:24:36 [INFO] Active TLS secret serving-cert (ver=8460) (count 8): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=160606628277A4D81C7526AA88640324AD0E4C95] 967 2022/06/16 10:24:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-impersonation-system": the object has been modified; please apply your changes to the latest version and try again 968 2022/06/16 10:24:36 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-global-data, err=Operation cannot be fulfilled on namespaces "cattle-global-data": the object has been modified; please apply your changes to the latest version and try again 969 2022/06/16 10:24:36 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-b4qkhsnliz with role cluster-owner in namespace p-bthll 970 2022/06/16 10:24:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-global-nt": the object has been modified; please apply your changes to the latest version and try again 971 2022/06/16 10:24:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-node-lease": the object has been modified; please apply your changes to the latest version and try again 972 2022/06/16 10:24:36 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject u-b4qkhsnliz with role cluster-owner in namespace p-5hdmj 973 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-bthll to namespace=default 974 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=ingress-nginx 975 2022/06/16 10:24:36 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "ingress-nginx": the object has been modified; please apply your changes to the latest version and try again 976 2022/06/16 10:24:36 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cert-manager 977 2022/06/16 10:24:36 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-global-data, err=Operation cannot be fulfilled on namespaces "cattle-global-data": the object has been modified; please apply your changes to the latest version and try again 978 2022/06/16 10:24:37 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cert-manager": the object has been modified; please apply your changes to the latest version and try again 979 2022/06/16 10:24:37 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-node-lease 980 2022/06/16 10:24:37 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-impersonation-system 981 2022/06/16 10:24:37 [INFO] Active TLS secret tls-rancher-internal (ver=8526) (count 2): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=CDFDDACCB4B7E8FA865290DF7C42A52ABF1C7A61] 982 2022/06/16 10:24:37 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-system 983 2022/06/16 10:24:37 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-global-nt 984 2022/06/16 10:24:37 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-system": the object has been modified; please apply your changes to the latest version and try again 985 2022/06/16 10:24:37 [INFO] Creating clusterRoleBinding User u-b4qkhsnliz Role cluster-owner 986 2022/06/16 10:24:38 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=ingress-nginx 987 2022/06/16 10:24:38 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "ingress-nginx": the object has been modified; please apply your changes to the latest version and try again 988 2022/06/16 10:24:38 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=ingress-nginx 989 2022/06/16 10:24:39 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cert-manager 990 2022/06/16 10:24:39 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-system 991 2022/06/16 10:24:39 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-system": the object has been modified; please apply your changes to the latest version and try again 992 2022/06/16 10:24:39 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-system 993 2022/06/16 10:24:39 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "kube-system": the object has been modified; please apply your changes to the latest version and try again 994 2022/06/16 10:24:40 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-system 995 2022/06/16 10:24:44 [INFO] Handling backend connection request [REDACTED-IP4-ADDRESS] 996 2022/06/16 10:24:45 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=kube-system 997 2022/06/16 10:24:47 [INFO] Refreshing driverMetadata in 1440 minutes 998 2022/06/16 10:24:47 [INFO] getReleaseConfig: reloading config for k3s 999 2022/06/16 10:24:47 [INFO] getReleaseConfig: reloading config for rke2 1000 2022/06/16 10:24:48 [INFO] driverMetadata: refreshing data from upstream https://releases.rancher.com/kontainer-driver-metadata/release-v2.6/data.json 1001 2022/06/16 10:24:48 [INFO] Retrieve data.json from local path /var/lib/rancher-data/driver-metadata/data.json 1002 2022/06/16 10:24:48 [INFO] Loaded configuration from &{} in [0x6f36500] 1003 2022/06/16 10:24:48 [INFO] Loaded configuration from &{} in [0x6f36500] 1004 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2Machine 1005 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=Amazonec2MachineTemplate 1006 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=DigitaloceanConfig 1007 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachineTemplate 1008 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachineTemplate 1009 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachine 1010 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachineTemplate 1011 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachineTemplate 1012 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=DigitaloceanMachine 1013 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=LinodeMachine 1014 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=VmwarevsphereMachineTemplate 1015 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=Amazonec2Config 1016 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=LinodeConfig 1017 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=HarvesterConfig 1018 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=VmwarevsphereConfig 1019 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine-config.cattle.io/v1, Kind=AzureConfig 1020 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=AzureMachine 1021 2022/06/16 10:24:54 [INFO] Watching metadata for rke-machine.cattle.io/v1, Kind=HarvesterMachine 1022 2022/06/16 10:25:22 [ERROR] Failed to serve peer connection REDACTED-IP4-ADDRESS: websocket: close 1006 (abnormal closure): unexpected EOF 1023 2022/06/16 10:25:22 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:46800: response 400: cluster not found 1024 2022/06/16 10:25:23 [INFO] certificate CN=dynamic,O=dynamic signed by CN=dynamiclistener-ca,O=dynamiclistener-org: notBefore=2022-06-16 10:24:11 +0000 UTC notAfter=2023-06-16 10:25:23 +0000 UTC 1025 2022/06/16 10:25:23 [INFO] Updating TLS secret for serving-cert (count: 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=302E55AEA811FABD94ECEA62F3180B5CF3DEF64B] 1026 2022/06/16 10:25:23 [INFO] Active TLS secret serving-cert (ver=9074) (count 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=302E55AEA811FABD94ECEA62F3180B5CF3DEF64B] 1027 2022/06/16 10:25:23 [INFO] Updating TLS secret for serving-cert (count: 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=A0E09BA6257822D13E280A8189D890AFA489D992] 1028 2022/06/16 10:25:23 [INFO] Active TLS secret serving-cert (ver=9075) (count 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=A0E09BA6257822D13E280A8189D890AFA489D992] 1029 2022/06/16 10:25:23 [INFO] Updating TLS secret for serving-cert (count: 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=302E55AEA811FABD94ECEA62F3180B5CF3DEF64B] 1030 2022/06/16 10:25:23 [INFO] Active TLS secret tls-rancher-internal (ver=9077) (count 1): map[listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/fingerprint:SHA1=982D78345E1ADCBF49D4B6066632834F0AE75DE9] 1031 2022/06/16 10:25:23 [INFO] Active TLS secret serving-cert (ver=9078) (count 9): map[field.cattle.io/projectId:local:p-5hdmj listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-REDACTED-IP4-ADDRESS:REDACTED-IP4-ADDRESS listener.cattle.io/cn-localhost:localhost listener.cattle.io/cn-rancher.cattle-system:rancher.cattle-system listener.cattle.io/fingerprint:SHA1=302E55AEA811FABD94ECEA62F3180B5CF3DEF64B] 1032 2022/06/16 10:25:27 [INFO] error in remotedialer server [400]: read tcp REDACTED-IP4-ADDRESS:443->REDACTED-IP4-ADDRESS:58186: use of closed network connection 1033 2022/06/16 10:25:32 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:47124: response 400: cluster not found 1034 2022/06/16 10:25:32 [INFO] Handling backend connection request [REDACTED-IP4-ADDRESS] 1035 2022/06/16 10:25:35 [ERROR] Failed to install system chart fleet: pod cattle-system/helm-operation-d2p4w failed, watch closed 1036 2022/06/16 10:25:37 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:47262: response 400: cluster not found 1037 2022/06/16 10:25:38 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-fleet-clusters-system, err=Operation cannot be fulfilled on namespaces "cattle-fleet-clusters-system": the object has been modified; please apply your changes to the latest version and try again 1038 2022/06/16 10:25:38 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-clusters-system 1039 2022/06/16 10:25:38 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-fleet-clusters-system": the object has been modified; please apply your changes to the latest version and try again 1040 2022/06/16 10:25:38 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-clusters-system 1041 2022/06/16 10:25:40 [ERROR] error syncing 'cattle-fleet-system/helm-operation-d2p4w': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-d2p4w), requeuing 1042 2022/06/16 10:25:40 [ERROR] error syncing 'cattle-fleet-system/helm-operation-d2p4w': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-d2p4w), requeuing 1043 2022/06/16 10:25:40 [ERROR] error syncing 'cattle-fleet-system/helm-operation-d2p4w': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-d2p4w), requeuing 1044 2022/06/16 10:25:40 [ERROR] error syncing 'cattle-fleet-system/helm-operation-d2p4w': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-d2p4w), requeuing 1045 2022/06/16 10:25:40 [ERROR] error syncing 'cattle-fleet-system/helm-operation-d2p4w': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-d2p4w), requeuing 1046 2022/06/16 10:25:42 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:47358: response 400: cluster not found 1047 2022/06/16 10:25:47 [ERROR] Failed to handling tunnel request from remote address REDACTED-IP4-ADDRESS:47442: response 400: cluster not found 1048 2022/06/16 10:25:50 [INFO] Adding peer wss://REDACTED-IP4-ADDRESS/v3/connect, REDACTED-IP4-ADDRESS 1049 2022/06/16 10:25:50 [INFO] Stopping cluster agent for local 1050 2022/06/16 10:25:50 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=Role workers 1051 2022/06/16 10:25:50 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding workers 1052 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=ResourceQuota workers 1053 2022/06/16 10:25:50 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=RoleBinding workers 1054 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=Secret workers 1055 2022/06/16 10:25:50 [INFO] Shutting down apiregistration.k8s.io/v1, Kind=APIService workers 1056 2022/06/16 10:25:50 [INFO] Shutting down rbac.authorization.k8s.io/v1, Kind=ClusterRole workers 1057 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=LimitRange workers 1058 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=ServiceAccount workers 1059 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=Namespace workers 1060 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=Node workers 1061 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=ConfigMap workers 1062 2022/06/16 10:25:50 [INFO] Shutting down /v1, Kind=ConfigMap workers 1063 2022/06/16 10:25:50 [INFO] Starting cluster controllers for local 1064 E0616 10:25:51.224648 9 memcache.go:196] couldn't get resource list for custom.metrics.k8s.io/v1beta1: Got empty response for: custom.metrics.k8s.io/v1beta1 1065 2022/06/16 10:25:51 [INFO] Starting /v1, Kind=Secret controller 1066 2022/06/16 10:25:51 [INFO] Starting /v1, Kind=ServiceAccount controller 1067 2022/06/16 10:25:51 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller 1068 2022/06/16 10:25:51 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=RoleBinding controller 1069 2022/06/16 10:25:51 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=ClusterRole controller 1070 2022/06/16 10:25:51 [INFO] Starting rbac.authorization.k8s.io/v1, Kind=Role controller 1071 2022/06/16 10:25:51 [INFO] Starting /v1, Kind=Namespace controller 1072 2022/06/16 10:25:51 [INFO] Starting cluster agent for local [owner=false] 1073 2022/06/16 10:25:52 [INFO] Handling backend connection request [REDACTED-IP4-ADDRESS] 1074 2022/06/16 10:25:55 [ERROR] error syncing 'cattle-fleet-system/helm-operation-gjfls': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-gjfls), requeuing 1075 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=GitRepoRestriction 1076 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=ClusterRegistrationToken 1077 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=BundleNamespaceMapping 1078 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=Content 1079 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=ClusterRegistration 1080 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=BundleDeployment 1081 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=ClusterGroup 1082 2022/06/16 10:25:56 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=GitRepo 1083 2022/06/16 10:25:59 [INFO] Watching metadata for fleet.cattle.io/v1alpha1, Kind=ImageScan 1084 2022/06/16 10:25:59 [INFO] Watching metadata for gitjob.cattle.io/v1, Kind=GitJob 1085 2022/06/16 10:26:00 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cluster-fleet-local-local-1a3d67d0a899 1086 2022/06/16 10:26:00 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cluster-fleet-local-local-1a3d67d0a899": the object has been modified; please apply your changes to the latest version and try again 1087 2022/06/16 10:26:00 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cluster-fleet-local-local-1a3d67d0a899 1088 2022/06/16 10:26:02 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-local-system 1089 2022/06/16 10:26:02 [ERROR] namespaceHandler: Sync: error adding project id label to namespace err=Operation cannot be fulfilled on namespaces "cattle-fleet-local-system": the object has been modified; please apply your changes to the latest version and try again 1090 2022/06/16 10:26:02 [INFO] namespaceHandler: addProjectIDLabelToNamespace: adding label field.cattle.io/projectId=p-5hdmj to namespace=cattle-fleet-local-system 1091 2022/06/16 10:26:04 [ERROR] error syncing 'cattle-fleet-system/helm-operation-xnktw': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-xnktw), requeuing 1092 2022/06/16 10:26:05 [ERROR] error syncing 'cattle-fleet-system/helm-operation-xnktw': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-xnktw), requeuing 1093 2022/06/16 10:27:05 [ERROR] Failed to install system chart rancher-webhook: pod cattle-system/helm-operation-dlhvn failed, watch closed 1094 2022/06/16 10:27:13 [INFO] starting duplicate binding cleanup 1095 2022/06/16 10:27:13 [INFO] [clean-dupe-bindings] starting bindings cleanup 1096 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] total PRTB duplicate clusterRoleBindings 0, roleBindings 0 1097 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] binding with deterministic name not found, will delete all except the oldest binding 1098 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] binding with deterministic name not found, will delete all except the oldest binding 1099 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] binding with deterministic name not found, will delete all except the oldest binding 1100 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] binding with deterministic name not found, will delete all except the oldest binding 1101 2022/06/16 10:27:14 [INFO] [clean-dupe-bindings] total CRTB duplicate clusterRoleBindings 0, roleBindings 0 1102 2022/06/16 10:27:14 [INFO] successfully cleaned up duplicate bindings 1103 2022/06/16 10:27:28 [ERROR] error syncing 'cattle-system/helm-operation-s5stv': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-s5stv), requeuing 1104 2022/06/16 10:27:28 [ERROR] error syncing 'cattle-system/helm-operation-s5stv': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-s5stv), requeuing 1105 2022/06/16 10:27:29 [ERROR] error syncing 'cattle-system/helm-operation-s5stv': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-s5stv), requeuing 1106 2022/06/16 10:27:29 [ERROR] error syncing 'cattle-system/helm-operation-s5stv': handler helm-operation: an error on the server ("container not found (\"proxy\")") has prevented the request from succeeding (get pods helm-operation-s5stv), requeuing 1107 ==== END logs for container rancher of pod cattle-system/rancher-649dc876b-kdrb2 ====