github.com/crossplane/upjet@v1.3.0/pkg/registry/testdata/gcp/r/container_cluster.html.markdown (about) 1 <!-- 2 SPDX-FileCopyrightText: 2023 The Crossplane Authors <https://crossplane.io> 3 4 SPDX-License-Identifier: Apache-2.0 5 --> 6 7 --- 8 subcategory: "Kubernetes (Container) Engine" 9 layout: "google" 10 page_title: "Google: google_container_cluster" 11 sidebar_current: "docs-google-container-cluster" 12 description: |- 13 Creates a Google Kubernetes Engine (GKE) cluster. 14 --- 15 16 # google\_container\_cluster 17 18 -> Visit the [Provision a GKE Cluster (Google Cloud)](https://learn.hashicorp.com/tutorials/terraform/gke?in=terraform/kubernetes&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) Learn tutorial to learn how to provision and interact 19 with a GKE cluster. 20 21 -> See the [Using GKE with Terraform](/docs/providers/google/guides/using_gke_with_terraform.html) 22 guide for more information about using GKE with Terraform. 23 24 Manages a Google Kubernetes Engine (GKE) cluster. For more information see 25 [the official documentation](https://cloud.google.com/container-engine/docs/clusters) 26 and [the API reference](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters). 27 28 ~> **Warning:** All arguments and attributes, including basic auth username and 29 passwords as well as certificate outputs will be stored in the raw state as 30 plaintext. [Read more about sensitive data in state](https://www.terraform.io/language/state/sensitive-data). 31 32 ## Example Usage - with a separately managed node pool (recommended) 33 34 ```hcl 35 resource "google_service_account" "default" { 36 account_id = "service-account-id" 37 display_name = "Service Account" 38 } 39 40 resource "google_container_cluster" "primary" { 41 name = "my-gke-cluster" 42 location = "us-central1" 43 44 # We can't create a cluster with no node pool defined, but we want to only use 45 # separately managed node pools. So we create the smallest possible default 46 # node pool and immediately delete it. 47 remove_default_node_pool = true 48 initial_node_count = 1 49 } 50 51 resource "google_container_node_pool" "primary_preemptible_nodes" { 52 name = "my-node-pool" 53 location = "us-central1" 54 cluster = google_container_cluster.primary.name 55 node_count = 1 56 57 node_config { 58 preemptible = true 59 machine_type = "e2-medium" 60 61 # Google recommends custom service accounts that have cloud-platform scope and permissions granted via IAM Roles. 62 service_account = google_service_account.default.email 63 oauth_scopes = [ 64 "https://www.googleapis.com/auth/cloud-platform" 65 ] 66 } 67 } 68 ``` 69 70 ~> **Note:** It is recommended that node pools be created and managed as separate resources as in the example above. 71 This allows node pools to be added and removed without recreating the cluster. Node pools defined directly in the 72 `google_container_cluster` resource cannot be removed without re-creating the cluster. 73 74 ## Example Usage - with the default node pool 75 76 ```hcl 77 resource "google_service_account" "default" { 78 account_id = "service-account-id" 79 display_name = "Service Account" 80 } 81 82 resource "google_container_cluster" "primary" { 83 name = "marcellus-wallace" 84 location = "us-central1-a" 85 initial_node_count = 3 86 node_config { 87 # Google recommends custom service accounts that have cloud-platform scope and permissions granted via IAM Roles. 88 service_account = google_service_account.default.email 89 oauth_scopes = [ 90 "https://www.googleapis.com/auth/cloud-platform" 91 ] 92 labels = { 93 foo = "bar" 94 } 95 tags = ["foo", "bar"] 96 } 97 timeouts { 98 create = "30m" 99 update = "40m" 100 } 101 } 102 ``` 103 104 ## Argument Reference 105 106 * `name` - (Required) The name of the cluster, unique within the project and 107 location. 108 109 - - - 110 111 * `location` - (Optional) The location (region or zone) in which the cluster 112 master will be created, as well as the default node location. If you specify a 113 zone (such as `us-central1-a`), the cluster will be a zonal cluster with a 114 single cluster master. If you specify a region (such as `us-west1`), the 115 cluster will be a regional cluster with multiple masters spread across zones in 116 the region, and with default node locations in those zones as well 117 118 * `node_locations` - (Optional) The list of zones in which the cluster's nodes 119 are located. Nodes must be in the region of their regional cluster or in the 120 same region as their cluster's zone for zonal clusters. If this is specified for 121 a zonal cluster, omit the cluster's zone. 122 123 -> A "multi-zonal" cluster is a zonal cluster with at least one additional zone 124 defined; in a multi-zonal cluster, the cluster master is only present in a 125 single zone while nodes are present in each of the primary zone and the node 126 locations. In contrast, in a regional cluster, cluster master nodes are present 127 in multiple zones in the region. For that reason, regional clusters should be 128 preferred. 129 130 * `addons_config` - (Optional) The configuration for addons supported by GKE. 131 Structure is [documented below](#nested_addons_config). 132 133 * `cluster_ipv4_cidr` - (Optional) The IP address range of the Kubernetes pods 134 in this cluster in CIDR notation (e.g. `10.96.0.0/14`). Leave blank to have one 135 automatically chosen or specify a `/14` block in `10.0.0.0/8`. This field will 136 only work for routes-based clusters, where `ip_allocation_policy` is not defined. 137 138 * `cluster_autoscaling` - (Optional) 139 Per-cluster configuration of Node Auto-Provisioning with Cluster Autoscaler to 140 automatically adjust the size of the cluster and create/delete node pools based 141 on the current needs of the cluster's workload. See the 142 [guide to using Node Auto-Provisioning](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning) 143 for more details. Structure is [documented below](#nested_cluster_autoscaling). 144 145 * `database_encryption` - (Optional) 146 Structure is [documented below](#nested_database_encryption). 147 148 * `description` - (Optional) Description of the cluster. 149 150 * `default_max_pods_per_node` - (Optional) The default maximum number of pods 151 per node in this cluster. This doesn't work on "routes-based" clusters, clusters 152 that don't have IP Aliasing enabled. See the [official documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr) 153 for more information. 154 155 * `enable_binary_authorization` - (Optional) Enable Binary Authorization for this cluster. 156 If enabled, all container images will be validated by Google Binary Authorization. 157 158 * `enable_kubernetes_alpha` - (Optional) Whether to enable Kubernetes Alpha features for 159 this cluster. Note that when this option is enabled, the cluster cannot be upgraded 160 and will be automatically deleted after 30 days. 161 162 * `enable_tpu` - (Optional) Whether to enable Cloud TPU resources in this cluster. 163 See the [official documentation](https://cloud.google.com/tpu/docs/kubernetes-engine-setup). 164 165 * `enable_legacy_abac` - (Optional) Whether the ABAC authorizer is enabled for this cluster. 166 When enabled, identities in the system, including service accounts, nodes, and controllers, 167 will have statically granted permissions beyond those provided by the RBAC configuration or IAM. 168 Defaults to `false` 169 170 * `enable_shielded_nodes` - (Optional) Enable Shielded Nodes features on all nodes in this cluster. Defaults to `true`. 171 172 * `enable_autopilot` - (Optional) Enable Autopilot for this cluster. Defaults to `false`. 173 Note that when this option is enabled, certain features of Standard GKE are not available. 174 See the [official documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#comparison) 175 for available features. 176 177 * `initial_node_count` - (Optional) The number of nodes to create in this 178 cluster's default node pool. In regional or multi-zonal clusters, this is the 179 number of nodes per zone. Must be set if `node_pool` is not set. If you're using 180 `google_container_node_pool` objects with no default node pool, you'll need to 181 set this to a value of at least `1`, alongside setting 182 `remove_default_node_pool` to `true`. 183 184 * `ip_allocation_policy` - (Optional) Configuration of cluster IP allocation for 185 VPC-native clusters. Adding this block enables [IP aliasing](https://cloud.google.com/kubernetes-engine/docs/how-to/ip-aliases), 186 making the cluster VPC-native instead of routes-based. Structure is [documented 187 below](#nested_ip_allocation_policy). 188 189 * `networking_mode` - (Optional) Determines whether alias IPs or routes will be used for pod IPs in the cluster. 190 Options are `VPC_NATIVE` or `ROUTES`. `VPC_NATIVE` enables [IP aliasing](https://cloud.google.com/kubernetes-engine/docs/how-to/ip-aliases), 191 and requires the `ip_allocation_policy` block to be defined. By default, when this field is unspecified and no `ip_allocation_policy` blocks are set, GKE will create a `ROUTES`-based cluster. 192 193 * `logging_config` - (Optional) Logging configuration for the cluster. 194 Structure is [documented below](#nested_logging_config). 195 196 * `logging_service` - (Optional) The logging service that the cluster should 197 write logs to. Available options include `logging.googleapis.com`(Legacy Stackdriver), 198 `logging.googleapis.com/kubernetes`(Stackdriver Kubernetes Engine Logging), and `none`. Defaults to `logging.googleapis.com/kubernetes` 199 200 * `maintenance_policy` - (Optional) The maintenance policy to use for the cluster. Structure is 201 [documented below](#nested_maintenance_policy). 202 203 * `master_auth` - (Optional) The authentication information for accessing the 204 Kubernetes master. Some values in this block are only returned by the API if 205 your service account has permission to get credentials for your GKE cluster. If 206 you see an unexpected diff unsetting your client cert, ensure you have the 207 `container.clusters.getCredentials` permission. 208 Structure is [documented below](#nested_master_auth). 209 210 * `master_authorized_networks_config` - (Optional) The desired 211 configuration options for master authorized networks. Omit the 212 nested `cidr_blocks` attribute to disallow external access (except 213 the cluster node IPs, which GKE automatically whitelists). 214 Structure is [documented below](#nested_master_authorized_networks_config). 215 216 * `min_master_version` - (Optional) The minimum version of the master. GKE 217 will auto-update the master to new versions, so this does not guarantee the 218 current master version--use the read-only `master_version` field to obtain that. 219 If unset, the cluster's version will be set by GKE to the version of the most recent 220 official release (which is not necessarily the latest version). Most users will find 221 the `google_container_engine_versions` data source useful - it indicates which versions 222 are available, and can be use to approximate fuzzy versions in a 223 Terraform-compatible way. If you intend to specify versions manually, 224 [the docs](https://cloud.google.com/kubernetes-engine/versioning-and-upgrades#specifying_cluster_version) 225 describe the various acceptable formats for this field. 226 227 -> If you are using the `google_container_engine_versions` datasource with a regional cluster, ensure that you have provided a `location` 228 to the datasource. A region can have a different set of supported versions than its corresponding zones, and not all zones in a 229 region are guaranteed to support the same version. 230 231 * `monitoring_config` - (Optional) Monitoring configuration for the cluster. 232 Structure is [documented below](#nested_monitoring_config). 233 234 * `monitoring_service` - (Optional) The monitoring service that the cluster 235 should write metrics to. 236 Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. 237 VM metrics will be collected by Google Compute Engine regardless of this setting 238 Available options include 239 `monitoring.googleapis.com`(Legacy Stackdriver), `monitoring.googleapis.com/kubernetes`(Stackdriver Kubernetes Engine Monitoring), and `none`. 240 Defaults to `monitoring.googleapis.com/kubernetes` 241 242 * `network` - (Optional) The name or self_link of the Google Compute Engine 243 network to which the cluster is connected. For Shared VPC, set this to the self link of the 244 shared network. 245 246 * `network_policy` - (Optional) Configuration options for the 247 [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/networkpolicies/) 248 feature. Structure is [documented below](#nested_network_policy). 249 250 * `node_config` - (Optional) Parameters used in creating the default node pool. 251 Generally, this field should not be used at the same time as a 252 `google_container_node_pool` or a `node_pool` block; this configuration 253 manages the default node pool, which isn't recommended to be used with 254 Terraform. Structure is [documented below](#nested_node_config). 255 256 * `network_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) Configuration for 257 [Adding Pod IP address ranges](https://cloud.google.com/kubernetes-engine/docs/how-to/multi-pod-cidr)) to the node pool. Structure is [documented below](#nested_network_config) 258 259 * `node_pool` - (Optional) List of node pools associated with this cluster. 260 See [google_container_node_pool](container_node_pool.html) for schema. 261 **Warning:** node pools defined inside a cluster can't be changed (or added/removed) after 262 cluster creation without deleting and recreating the entire cluster. Unless you absolutely need the ability 263 to say "these are the _only_ node pools associated with this cluster", use the 264 [google_container_node_pool](container_node_pool.html) resource instead of this property. 265 266 * `node_version` - (Optional) The Kubernetes version on the nodes. Must either be unset 267 or set to the same value as `min_master_version` on create. Defaults to the default 268 version set by GKE which is not necessarily the latest version. This only affects 269 nodes in the default node pool. While a fuzzy version can be specified, it's 270 recommended that you specify explicit versions as Terraform will see spurious diffs 271 when fuzzy versions are used. See the `google_container_engine_versions` data source's 272 `version_prefix` field to approximate fuzzy versions in a Terraform-compatible way. 273 To update nodes in other node pools, use the `version` attribute on the node pool. 274 275 * `notification_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) Configuration for the [cluster upgrade notifications](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-upgrade-notifications) feature. Structure is [documented below](#nested_notification_config). 276 277 * `confidential_nodes` - Configuration for [Confidential Nodes](https://cloud.google.com/kubernetes-engine/docs/how-to/confidential-gke-nodes) feature. Structure is documented below [documented below](#nested_confidential_nodes). 278 279 * `pod_security_policy_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) Configuration for the 280 [PodSecurityPolicy](https://cloud.google.com/kubernetes-engine/docs/how-to/pod-security-policies) feature. 281 Structure is [documented below](#nested_pod_security_policy_config). 282 283 * `authenticator_groups_config` - (Optional) Configuration for the 284 [Google Groups for GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#groups-setup-gsuite) feature. 285 Structure is [documented below](#nested_authenticator_groups_config). 286 287 * `private_cluster_config` - (Optional) Configuration for [private clusters](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters), 288 clusters with private nodes. Structure is [documented below](#nested_private_cluster_config). 289 290 * `cluster_telemetry` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) Configuration for 291 [ClusterTelemetry](https://cloud.google.com/monitoring/kubernetes-engine/installing#controlling_the_collection_of_application_logs) feature, 292 Structure is [documented below](#nested_cluster_telemetry). 293 294 * `project` - (Optional) The ID of the project in which the resource belongs. If it 295 is not provided, the provider project is used. 296 297 * `release_channel` - (Optional) 298 Configuration options for the [Release channel](https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels) 299 feature, which provide more control over automatic upgrades of your GKE clusters. 300 When updating this field, GKE imposes specific version requirements. See 301 [Selecting a new release channel](https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels#selecting_a_new_release_channel) 302 for more details; the `google_container_engine_versions` datasource can provide 303 the default version for a channel. Note that removing the `release_channel` 304 field from your config will cause Terraform to stop managing your cluster's 305 release channel, but will not unenroll it. Instead, use the `"UNSPECIFIED"` 306 channel. Structure is [documented below](#nested_release_channel). 307 308 * `remove_default_node_pool` - (Optional) If `true`, deletes the default node 309 pool upon cluster creation. If you're using `google_container_node_pool` 310 resources with no default node pool, this should be set to `true`, alongside 311 setting `initial_node_count` to at least `1`. 312 313 * `resource_labels` - (Optional) The GCE resource labels (a map of key/value pairs) to be applied to the cluster. 314 315 * `resource_usage_export_config` - (Optional) Configuration for the 316 [ResourceUsageExportConfig](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-usage-metering) feature. 317 Structure is [documented below](#nested_resource_usage_export_config). 318 319 * `subnetwork` - (Optional) The name or self_link of the Google Compute Engine 320 subnetwork in which the cluster's instances are launched. 321 322 * `vertical_pod_autoscaling` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) 323 Vertical Pod Autoscaling automatically adjusts the resources of pods controlled by it. 324 Structure is [documented below](#nested_vertical_pod_autoscaling). 325 326 * `workload_identity_config` - (Optional) 327 Workload Identity allows Kubernetes service accounts to act as a user-managed 328 [Google IAM Service Account](https://cloud.google.com/iam/docs/service-accounts#user-managed_service_accounts). 329 Structure is [documented below](#nested_workload_identity_config). 330 331 * `enable_intranode_visibility` - (Optional) 332 Whether Intra-node visibility is enabled for this cluster. This makes same node pod to pod traffic visible for VPC network. 333 334 * `enable_l4_ilb_subsetting` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) 335 Whether L4ILB Subsetting is enabled for this cluster. 336 337 * `private_ipv6_google_access` - (Optional) 338 The desired state of IPv6 connectivity to Google Services. By default, no private IPv6 access to or from Google Services (all access will be via IPv4). 339 340 * `datapath_provider` - (Optional) 341 The desired datapath provider for this cluster. By default, uses the IPTables-based kube-proxy implementation. 342 343 * `default_snat_status` - (Optional) 344 [GKE SNAT](https://cloud.google.com/kubernetes-engine/docs/how-to/ip-masquerade-agent#how_ipmasq_works) DefaultSnatStatus contains the desired state of whether default sNAT should be disabled on the cluster, [API doc](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#networkconfig). Structure is [documented below](#nested_default_snat_status) 345 346 * `dns_config` - (Optional) 347 Configuration for [Using Cloud DNS for GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns). Structure is [documented below](#nested_dns_config). 348 349 <a name="nested_default_snat_status"></a>The `default_snat_status` block supports 350 351 * `disabled` - (Required) Whether the cluster disables default in-node sNAT rules. In-node sNAT rules will be disabled when defaultSnatStatus is disabled.When disabled is set to false, default IP masquerade rules will be applied to the nodes to prevent sNAT on cluster internal traffic 352 353 <a name="nested_cluster_telemetry"></a>The `cluster_telemetry` block supports 354 * `type` - Telemetry integration for the cluster. Supported values (`ENABLED, DISABLED, SYSTEM_ONLY`); 355 `SYSTEM_ONLY` (Only system components are monitored and logged) is only available in GKE versions 1.15 and later. 356 357 <a name="nested_addons_config"></a>The `addons_config` block supports: 358 359 * `horizontal_pod_autoscaling` - (Optional) The status of the Horizontal Pod Autoscaling 360 addon, which increases or decreases the number of replica pods a replication controller 361 has based on the resource usage of the existing pods. 362 It is enabled by default; 363 set `disabled = true` to disable. 364 365 * `http_load_balancing` - (Optional) The status of the HTTP (L7) load balancing 366 controller addon, which makes it easy to set up HTTP load balancers for services in a 367 cluster. It is enabled by default; set `disabled = true` to disable. 368 369 * `network_policy_config` - (Optional) Whether we should enable the network policy addon 370 for the master. This must be enabled in order to enable network policy for the nodes. 371 To enable this, you must also define a [`network_policy`](#network_policy) block, 372 otherwise nothing will happen. 373 It can only be disabled if the nodes already do not have network policies enabled. 374 Defaults to disabled; set `disabled = false` to enable. 375 376 * `gcp_filestore_csi_driver_config` - (Optional) The status of the Filestore CSI driver addon, 377 which allows the usage of filestore instance as volumes. 378 It is disabled by default; set `enabled = true` to enable. 379 380 * `cloudrun_config` - (Optional). Structure is [documented below](#nested_cloudrun_config). 381 382 * `istio_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 383 Structure is [documented below](#nested_istio_config). 384 385 * `identity_service_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). Structure is [documented below](#nested_identity_service_config). 386 387 * `dns_cache_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 388 The status of the NodeLocal DNSCache addon. It is disabled by default. 389 Set `enabled = true` to enable. 390 391 **Enabling/Disabling NodeLocal DNSCache in an existing cluster is a disruptive operation. 392 All cluster nodes running GKE 1.15 and higher are recreated.** 393 394 * `gce_persistent_disk_csi_driver_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 395 Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver. Defaults to disabled; set `enabled = true` to enable. 396 397 * `kalm_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 398 Configuration for the KALM addon, which manages the lifecycle of k8s. It is disabled by default; Set `enabled = true` to enable. 399 400 * `config_connector_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 401 The status of the ConfigConnector addon. It is disabled by default; Set `enabled = true` to enable. 402 403 * `gke_backup_agent_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)). 404 The status of the Backup for GKE agent addon. It is disabled by default; Set `enabled = true` to enable. 405 406 This example `addons_config` disables two addons: 407 408 ```hcl 409 addons_config { 410 http_load_balancing { 411 disabled = true 412 } 413 414 horizontal_pod_autoscaling { 415 disabled = true 416 } 417 } 418 ``` 419 420 <a name="nested_database_encryption"></a>The `database_encryption` block supports: 421 422 * `state` - (Required) `ENCRYPTED` or `DECRYPTED` 423 424 * `key_name` - (Required) the key to use to encrypt/decrypt secrets. See the [DatabaseEncryption definition](https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#Cluster.DatabaseEncryption) for more information. 425 426 <a name="nested_cloudrun_config"></a>The `cloudrun_config` block supports: 427 428 * `disabled` - (Optional) The status of the CloudRun addon. It is disabled by default. Set `disabled=false` to enable. 429 430 * `load_balancer_type` - (Optional) The load balancer type of CloudRun ingress service. It is external load balancer by default. 431 Set `load_balancer_type=LOAD_BALANCER_TYPE_INTERNAL` to configure it as internal load balancer. 432 433 <a name="nested_identity_service_config"></a>The `identity_service_config` block supports: 434 435 * `enabled` - (Optional) Whether to enable the Identity Service component. It is disabled by default. Set `enabled=true` to enable. 436 437 <a name="nested_istio_config"></a>The `istio_config` block supports: 438 439 * `disabled` - (Optional) The status of the Istio addon, which makes it easy to set up Istio for services in a 440 cluster. It is disabled by default. Set `disabled = false` to enable. 441 442 * `auth` - (Optional) The authentication type between services in Istio. Available options include `AUTH_MUTUAL_TLS`. 443 444 <a name="nested_cluster_autoscaling"></a>The `cluster_autoscaling` block supports: 445 446 * `enabled` - (Required) Whether node auto-provisioning is enabled. Resource 447 limits for `cpu` and `memory` must be defined to enable node auto-provisioning. 448 449 * `resource_limits` - (Optional) Global constraints for machine resources in the 450 cluster. Configuring the `cpu` and `memory` types is required if node 451 auto-provisioning is enabled. These limits will apply to node pool autoscaling 452 in addition to node auto-provisioning. Structure is [documented below](#nested_resource_limits). 453 454 * `auto_provisioning_defaults` - (Optional) Contains defaults for a node pool created by NAP. 455 Structure is [documented below](#nested_auto_provisioning_defaults). 456 457 * `autoscaling_profile` - (Optional, [Beta](https://terraform.io/docs/providers/google/provider_versions.html)) Configuration 458 options for the [Autoscaling profile](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler#autoscaling_profiles) 459 feature, which lets you choose whether the cluster autoscaler should optimize for resource utilization or resource availability 460 when deciding to remove nodes from a cluster. Can be `BALANCED` or `OPTIMIZE_UTILIZATION`. Defaults to `BALANCED`. 461 462 <a name="nested_resource_limits"></a>The `resource_limits` block supports: 463 464 * `resource_type` - (Required) The type of the resource. For example, `cpu` and 465 `memory`. See the [guide to using Node Auto-Provisioning](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning) 466 for a list of types. 467 468 * `minimum` - (Optional) Minimum amount of the resource in the cluster. 469 470 * `maximum` - (Optional) Maximum amount of the resource in the cluster. 471 472 <a name="nested_auto_provisioning_defaults"></a>The `auto_provisioning_defaults` block supports: 473 474 * `min_cpu_platform` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) 475 Minimum CPU platform to be used for NAP created node pools. The instance may be scheduled on the 476 specified or newer CPU platform. Applicable values are the friendly names of CPU platforms, such 477 as "Intel Haswell" or "Intel Sandy Bridge". 478 479 * `oauth_scopes` - (Optional) Scopes that are used by NAP when creating node pools. Use the "https://www.googleapis.com/auth/cloud-platform" scope to grant access to all APIs. It is recommended that you set `service_account` to a non-default service account and grant IAM roles to that service account for only the resources that it needs. 480 481 -> `monitoring.write` is always enabled regardless of user input. `monitoring` and `logging.write` may also be enabled depending on the values for `monitoring_service` and `logging_service`. 482 483 * `service_account` - (Optional) The Google Cloud Platform Service Account to be used by the node VMs. 484 485 * `image_type` - (Optional) The default image type used by NAP once a new node pool is being created. Please note that according to the [official documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-provisioning#default-image-type) the value must be one of the [COS_CONTAINERD, COS, UBUNTU_CONTAINERD, UBUNTU]. __NOTE__ : COS AND UBUNTU are deprecated as of `GKE 1.24` 486 487 <a name="nested_authenticator_groups_config"></a>The `authenticator_groups_config` block supports: 488 489 * `security_group` - (Required) The name of the RBAC security group for use with Google security groups in Kubernetes RBAC. Group name must be in format `gke-security-groups@yourdomain.com`. 490 491 <a name="nested_logging_config"></a>The `logging_config` block supports: 492 493 * `enable_components` - (Required) The GKE components exposing logs. Supported values include: 494 `SYSTEM_COMPONENTS` and `WORKLOADS`. 495 496 <a name="nested_monitoring_config"></a>The `monitoring_config` block supports: 497 498 * `enable_components` - (Required) The GKE components exposing logs. `SYSTEM_COMPONENTS` and in beta provider, both `SYSTEM_COMPONENTS` and `WORKLOADS` are supported. 499 500 <a name="nested_maintenance_policy"></a>The `maintenance_policy` block supports: 501 * `daily_maintenance_window` - (Optional) structure documented below. 502 * `recurring_window` - (Optional) structure documented below 503 * `maintenance_exclusion` - (Optional) structure documented below 504 505 In beta, one or the other of `recurring_window` and `daily_maintenance_window` is required if a `maintenance_policy` block is supplied. 506 507 * `daily_maintenance_window` - Time window specified for daily maintenance operations. 508 Specify `start_time` in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) format "HH:MM”, 509 where HH : \[00-23\] and MM : \[00-59\] GMT. For example: 510 511 Examples: 512 ```hcl 513 maintenance_policy { 514 daily_maintenance_window { 515 start_time = "03:00" 516 } 517 } 518 ``` 519 520 * `recurring_window` - Time window for recurring maintenance operations. 521 522 Specify `start_time` and `end_time` in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) "Zulu" date format. The start time's date is 523 the initial date that the window starts, and the end time is used for calculating duration. Specify `recurrence` in 524 [RFC5545](https://tools.ietf.org/html/rfc5545#section-3.8.5.3) RRULE format, to specify when this recurs. 525 Note that GKE may accept other formats, but will return values in UTC, causing a permanent diff. 526 527 Examples: 528 ``` 529 maintenance_policy { 530 recurring_window { 531 start_time = "2019-08-01T02:00:00Z" 532 end_time = "2019-08-01T06:00:00Z" 533 recurrence = "FREQ=DAILY" 534 } 535 } 536 ``` 537 538 ``` 539 maintenance_policy { 540 recurring_window { 541 start_time = "2019-01-01T09:00:00Z" 542 end_time = "2019-01-01T17:00:00Z" 543 recurrence = "FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR" 544 } 545 } 546 ``` 547 548 * `maintenance_exclusion` - Exceptions to maintenance window. Non-emergency maintenance should not occur in these windows. A cluster can have up to three maintenance exclusions at a time [Maintenance Window and Exclusions](https://cloud.google.com/kubernetes-engine/docs/concepts/maintenance-windows-and-exclusions) 549 550 <a name="nested_maintenance_exclusion"></a>The `maintenance_exclusion` block supports: 551 * `exclusion_options` - (Optional) MaintenanceExclusionOptions provides maintenance exclusion related options. 552 553 554 <a name="nested_exclusion_options"></a>The `exclusion_options` block supports: 555 * `scope` - (Required) The scope of automatic upgrades to restrict in the exclusion window. One of: **NO_UPGRADES | NO_MINOR_UPGRADES | NO_MINOR_OR_NODE_UPGRADES** 556 557 Specify `start_time` and `end_time` in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) "Zulu" date format. The start time's date is 558 the initial date that the window starts, and the end time is used for calculating duration.Specify `recurrence` in 559 [RFC5545](https://tools.ietf.org/html/rfc5545#section-3.8.5.3) RRULE format, to specify when this recurs. 560 Note that GKE may accept other formats, but will return values in UTC, causing a permanent diff. 561 562 Examples: 563 564 ``` 565 maintenance_policy { 566 recurring_window { 567 start_time = "2019-01-01T00:00:00Z" 568 end_time = "2019-01-02T00:00:00Z" 569 recurrence = "FREQ=DAILY" 570 } 571 maintenance_exclusion{ 572 exclusion_name = "batch job" 573 start_time = "2019-01-01T00:00:00Z" 574 end_time = "2019-01-02T00:00:00Z" 575 exclusion_options { 576 scope = "NO_UPGRADES" 577 } 578 } 579 maintenance_exclusion{ 580 exclusion_name = "holiday data load" 581 start_time = "2019-05-01T00:00:00Z" 582 end_time = "2019-05-02T00:00:00Z" 583 exclusion_options { 584 scope = "NO_MINOR_UPGRADES" 585 } 586 } 587 } 588 ``` 589 590 <a name="nested_ip_allocation_policy"></a>The `ip_allocation_policy` block supports: 591 592 * `cluster_secondary_range_name` - (Optional) The name of the existing secondary 593 range in the cluster's subnetwork to use for pod IP addresses. Alternatively, 594 `cluster_ipv4_cidr_block` can be used to automatically create a GKE-managed one. 595 596 * `services_secondary_range_name` - (Optional) The name of the existing 597 secondary range in the cluster's subnetwork to use for service `ClusterIP`s. 598 Alternatively, `services_ipv4_cidr_block` can be used to automatically create a 599 GKE-managed one. 600 601 * `cluster_ipv4_cidr_block` - (Optional) The IP address range for the cluster pod IPs. 602 Set to blank to have a range chosen with the default size. Set to /netmask (e.g. /14) 603 to have a range chosen with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) 604 from the RFC-1918 private networks (e.g. 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) to 605 pick a specific range to use. 606 607 * `services_ipv4_cidr_block` - (Optional) The IP address range of the services IPs in this cluster. 608 Set to blank to have a range chosen with the default size. Set to /netmask (e.g. /14) 609 to have a range chosen with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) 610 from the RFC-1918 private networks (e.g. 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) to 611 pick a specific range to use. 612 613 <a name="nested_master_auth"></a>The `master_auth` block supports: 614 615 * `client_certificate_config` - (Required) Whether client certificate authorization is enabled for this cluster. For example: 616 617 ```hcl 618 master_auth { 619 client_certificate_config { 620 issue_client_certificate = false 621 } 622 } 623 ``` 624 625 This block also contains several computed attributes, documented below. 626 627 <a name="nested_master_authorized_networks_config"></a>The `master_authorized_networks_config` block supports: 628 629 * `cidr_blocks` - (Optional) External networks that can access the 630 Kubernetes cluster master through HTTPS. 631 632 The `master_authorized_networks_config.cidr_blocks` block supports: 633 634 * `cidr_block` - (Optional) External network that can access Kubernetes master through HTTPS. 635 Must be specified in CIDR notation. 636 637 * `display_name` - (Optional) Field for users to identify CIDR blocks. 638 639 <a name="nested_network_policy"></a>The `network_policy` block supports: 640 641 * `provider` - (Optional) The selected network policy provider. Defaults to PROVIDER_UNSPECIFIED. 642 643 * `enabled` - (Required) Whether network policy is enabled on the cluster. 644 645 <a name="nested_node_config"></a>The `node_config` block supports: 646 647 * `disk_size_gb` - (Optional) Size of the disk attached to each node, specified 648 in GB. The smallest allowed disk size is 10GB. Defaults to 100GB. 649 650 * `disk_type` - (Optional) Type of the disk attached to each node 651 (e.g. 'pd-standard', 'pd-balanced' or 'pd-ssd'). If unspecified, the default disk type is 'pd-standard' 652 653 * `ephemeral_storage_config` - (Optional, [Beta]) Parameters for the ephemeral storage filesystem. If unspecified, ephemeral storage is backed by the boot disk. Structure is [documented below](#nested_ephemeral_storage_config). 654 655 ```hcl 656 ephemeral_storage_config { 657 local_ssd_count = 2 658 } 659 ``` 660 661 * `gcfs_config` - (Optional) Parameters for the Google Container Filesystem (GCFS). 662 If unspecified, GCFS will not be enabled on the node pool. When enabling this feature you must specify `image_type = "COS_CONTAINERD"` and `node_version` from GKE versions 1.19 or later to use it. 663 For GKE versions 1.19, 1.20, and 1.21, the recommended minimum `node_version` would be 1.19.15-gke.1300, 1.20.11-gke.1300, and 1.21.5-gke.1300 respectively. 664 A `machine_type` that has more than 16 GiB of memory is also recommended. 665 GCFS must be enabled in order to use [image streaming](https://cloud.google.com/kubernetes-engine/docs/how-to/image-streaming). 666 Structure is [documented below](#nested_gcfs_config). 667 668 ```hcl 669 gcfs_config { 670 enabled = true 671 } 672 ``` 673 674 675 * `gvnic` - (Optional) Google Virtual NIC (gVNIC) is a virtual network interface. 676 Installing the gVNIC driver allows for more efficient traffic transmission across the Google network infrastructure. 677 gVNIC is an alternative to the virtIO-based ethernet driver. GKE nodes must use a Container-Optimized OS node image. 678 GKE node version 1.15.11-gke.15 or later 679 Structure is [documented below](#nested_gvnic). 680 681 682 ```hcl 683 gvnic { 684 enabled = true 685 } 686 ``` 687 688 * `guest_accelerator` - (Optional) List of the type and count of accelerator cards attached to the instance. 689 Structure [documented below](#nested_guest_accelerator). 690 To support removal of guest_accelerators in Terraform 0.12 this field is an 691 [Attribute as Block](/docs/configuration/attr-as-blocks.html) 692 693 * `image_type` - (Optional) The image type to use for this node. Note that changing the image type 694 will delete and recreate all nodes in the node pool. 695 696 * `labels` - (Optional) The Kubernetes labels (key/value pairs) to be applied to each node. The kubernetes.io/ and k8s.io/ prefixes are 697 reserved by Kubernetes Core components and cannot be specified. 698 699 * `local_ssd_count` - (Optional) The amount of local SSD disks that will be 700 attached to each cluster node. Defaults to 0. 701 702 * `machine_type` - (Optional) The name of a Google Compute Engine machine type. 703 Defaults to `e2-medium`. To create a custom machine type, value should be set as specified 704 [here](https://cloud.google.com/compute/docs/reference/latest/instances#machineType). 705 706 * `metadata` - (Optional) The metadata key/value pairs assigned to instances in 707 the cluster. From GKE `1.12` onwards, `disable-legacy-endpoints` is set to 708 `true` by the API; if `metadata` is set but that default value is not 709 included, Terraform will attempt to unset the value. To avoid this, set the 710 value in your config. 711 712 * `min_cpu_platform` - (Optional) Minimum CPU platform to be used by this instance. 713 The instance may be scheduled on the specified or newer CPU platform. Applicable 714 values are the friendly names of CPU platforms, such as `Intel Haswell`. See the 715 [official documentation](https://cloud.google.com/compute/docs/instances/specify-min-cpu-platform) 716 for more information. 717 718 * `oauth_scopes` - (Optional) The set of Google API scopes to be made available 719 on all of the node VMs under the "default" service account. 720 Use the "https://www.googleapis.com/auth/cloud-platform" scope to grant access to all APIs. It is recommended that you set `service_account` to a non-default service account and grant IAM roles to that service account for only the resources that it needs. 721 722 See the [official documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/access-scopes) for information on migrating off of legacy access scopes. 723 724 * `preemptible` - (Optional) A boolean that represents whether or not the underlying node VMs 725 are preemptible. See the [official documentation](https://cloud.google.com/container-engine/docs/preemptible-vm) 726 for more information. Defaults to false. 727 728 * `spot` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) A boolean 729 that represents whether the underlying node VMs are spot. See the [official documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms) 730 for more information. Defaults to false. 731 732 * `sandbox_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) [GKE Sandbox](https://cloud.google.com/kubernetes-engine/docs/how-to/sandbox-pods) configuration. When enabling this feature you must specify `image_type = "COS_CONTAINERD"` and `node_version = "1.12.7-gke.17"` or later to use it. 733 Structure is [documented below](#nested_sandbox_config). 734 735 * `boot_disk_kms_key` - (Optional) The Customer Managed Encryption Key used to encrypt the boot disk attached to each node in the node pool. This should be of the form projects/[KEY_PROJECT_ID]/locations/[LOCATION]/keyRings/[RING_NAME]/cryptoKeys/[KEY_NAME]. For more information about protecting resources with Cloud KMS Keys please see: https://cloud.google.com/compute/docs/disks/customer-managed-encryption 736 737 * `service_account` - (Optional) The service account to be used by the Node VMs. 738 If not specified, the "default" service account is used. 739 740 * `shielded_instance_config` - (Optional) Shielded Instance options. Structure is [documented below](#nested_shielded_instance_config). 741 742 * `tags` - (Optional) The list of instance tags applied to all nodes. Tags are used to identify 743 valid sources or targets for network firewalls. 744 745 * `taint` - (Optional) A list of [Kubernetes taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) 746 to apply to nodes. GKE's API can only set this field on cluster creation. 747 However, GKE will add taints to your nodes if you enable certain features such 748 as GPUs. If this field is set, any diffs on this field will cause Terraform to 749 recreate the underlying resource. Taint values can be updated safely in 750 Kubernetes (eg. through `kubectl`), and it's recommended that you do not use 751 this field to manage taints. If you do, `lifecycle.ignore_changes` is 752 recommended. Structure is [documented below](#nested_taint). 753 754 * `workload_metadata_config` - (Optional) Metadata configuration to expose to workloads on the node pool. 755 Structure is [documented below](#nested_workload_metadata_config). 756 757 * `kubelet_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) 758 Kubelet configuration, currently supported attributes can be found [here](https://cloud.google.com/sdk/gcloud/reference/beta/container/node-pools/create#--system-config-from-file). 759 Structure is [documented below](#nested_kubelet_config). 760 761 ``` 762 kubelet_config { 763 cpu_manager_policy = "static" 764 cpu_cfs_quota = true 765 cpu_cfs_quota_period = "100us" 766 } 767 ``` 768 769 * `linux_node_config` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) 770 Linux node configuration, currently supported attributes can be found [here](https://cloud.google.com/sdk/gcloud/reference/beta/container/node-pools/create#--system-config-from-file). 771 Note that validations happen all server side. All attributes are optional. 772 Structure is [documented below](#nested_linux_node_config). 773 774 ```hcl 775 linux_node_config { 776 sysctls = { 777 "net.core.netdev_max_backlog" = "10000" 778 "net.core.rmem_max" = "10000" 779 } 780 } 781 ``` 782 783 * `node_group` - (Optional) Setting this field will assign instances of this pool to run on the specified node group. This is useful for running workloads on [sole tenant nodes](https://cloud.google.com/compute/docs/nodes/sole-tenant-nodes). 784 785 <a name="nested_network_config"></a>The `network_config` block supports: 786 787 * `create_pod_range` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) Whether to create a new range for pod IPs in this node pool. Defaults are provided for `pod_range` and `pod_ipv4_cidr_block` if they are not specified. 788 789 * `pod_ipv4_cidr_block` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) The IP address range for pod IPs in this node pool. Only applicable if createPodRange is true. Set to blank to have a range chosen with the default size. Set to /netmask (e.g. /14) to have a range chosen with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) to pick a specific range to use. 790 791 * `pod_range` - (Optional, [Beta](https://terraform.io/docs/providers/google/guides/provider_versions.html)) The ID of the secondary range for pod IPs. If `create_pod_range` is true, this ID is used for the new range. If `create_pod_range` is false, uses an existing secondary range with this ID. 792 793 <a name="nested_ephemeral_storage_config"></a>The `ephemeral_storage_config` block supports: 794 795 * `local_ssd_count` (Required) - Number of local SSDs to use to back ephemeral storage. Uses NVMe interfaces. Each local SSD is 375 GB in size. If zero, it means to disable using local SSDs as ephemeral storage. 796 797 <a name="nested_gcfs_config"></a>The `gcfs_config` block supports: 798 799 * `enabled` (Required) - Whether or not the Google Container Filesystem (GCFS) is enabled 800 801 <a name="nested_gvnic"></a>The `gvnic` block supports: 802 803 * `enabled` (Required) - Whether or not the Google Virtual NIC (gVNIC) is enabled 804 805 <a name="nested_guest_accelerator"></a>The `guest_accelerator` block supports: 806 807 * `type` (Required) - The accelerator type resource to expose to this instance. E.g. `nvidia-tesla-k80`. 808 809 * `count` (Required) - The number of the guest accelerator cards exposed to this instance. 810 811 * `gpu_partition_size` (Optional) - Size of partitions to create on the GPU. Valid values are described in the NVIDIA mig [user guide](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/#partitioning). 812 813 <a name="nested_workload_identity_config"></a> The `workload_identity_config` block supports: 814 815 * `workload_pool` (Optional) - The workload pool to attach all Kubernetes service accounts to. 816 817 ```hcl 818 workload_identity_config { 819 workload_pool = "${data.google_project.project.project_id}.svc.id.goog" 820 } 821 ``` 822 823 <a name="nested_notification_config"></a>The `notification_config` block supports: 824 825 * `pubsub` (Required) - The pubsub config for the cluster's upgrade notifications. 826 827 The `pubsub` block supports: 828 829 * `enabled` (Required) - Whether or not the notification config is enabled 830 831 * `topic` (Optional) - The pubsub topic to push upgrade notifications to. Must be in the same project as the cluster. Must be in the format: `projects/{project}/topics/{topic}`. 832 833 ```hcl 834 notification_config { 835 pubsub { 836 enabled = true 837 topic = google_pubsub_topic.notifications.id 838 } 839 } 840 ``` 841 842 <a name="nested_confidential_nodes"></a> The `confidential_nodes` block supports: 843 844 * `enabled` (Required) - Enable Confidential Nodes for this cluster. 845 846 <a name="nested_pod_security_policy_config"></a>The `pod_security_policy_config` block supports: 847 848 * `enabled` (Required) - Enable the PodSecurityPolicy controller for this cluster. 849 If enabled, pods must be valid under a PodSecurityPolicy to be created. 850 851 <a name="nested_private_cluster_config"></a>The `private_cluster_config` block supports: 852 853 * `enable_private_nodes` (Optional) - Enables the private cluster feature, 854 creating a private endpoint on the cluster. In a private cluster, nodes only 855 have RFC 1918 private addresses and communicate with the master's private 856 endpoint via private networking. 857 858 * `enable_private_endpoint` (Optional) - When `true`, the cluster's private 859 endpoint is used as the cluster endpoint and access through the public endpoint 860 is disabled. When `false`, either endpoint can be used. This field only applies 861 to private clusters, when `enable_private_nodes` is `true`. 862 863 * `master_ipv4_cidr_block` (Optional) - The IP range in CIDR notation to use for 864 the hosted master network. This range will be used for assigning private IP 865 addresses to the cluster master(s) and the ILB VIP. This range must not overlap 866 with any other ranges in use within the cluster's network, and it must be a /28 867 subnet. See [Private Cluster Limitations](https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#req_res_lim) 868 for more details. This field only applies to private clusters, when 869 `enable_private_nodes` is `true`. 870 871 * `master_global_access_config` (Optional) - Controls cluster master global 872 access settings. If unset, Terraform will no longer manage this field and will 873 not modify the previously-set value. Structure is [documented below](#nested_master_global_access_config). 874 875 In addition, the `private_cluster_config` allows access to the following read-only fields: 876 877 * `peering_name` - The name of the peering between this cluster and the Google owned VPC. 878 879 * `private_endpoint` - The internal IP address of this cluster's master endpoint. 880 881 * `public_endpoint` - The external IP address of this cluster's master endpoint. 882 883 !> The Google provider is unable to validate certain configurations of 884 `private_cluster_config` when `enable_private_nodes` is `false`. It's 885 recommended that you omit the block entirely if the field is not set to `true`. 886 887 <a name="nested_master_global_access_config"></a>The `private_cluster_config.master_global_access_config` block supports: 888 889 * `enabled` (Optional) - Whether the cluster master is accessible globally or 890 not. 891 892 <a name="nested_sandbox_config"></a>The `sandbox_config` block supports: 893 894 * `sandbox_type` (Required) Which sandbox to use for pods in the node pool. 895 Accepted values are: 896 897 * `"gvisor"`: Pods run within a gVisor sandbox. 898 899 <a name="nested_release_channel"></a>The `release_channel` block supports: 900 901 * `channel` - (Required) The selected release channel. 902 Accepted values are: 903 * UNSPECIFIED: Not set. 904 * RAPID: Weekly upgrade cadence; Early testers and developers who requires new features. 905 * REGULAR: Multiple per month upgrade cadence; Production users who need features not yet offered in the Stable channel. 906 * STABLE: Every few months upgrade cadence; Production users who need stability above all else, and for whom frequent upgrades are too risky. 907 908 <a name="nested_resource_usage_export_config"></a>The `resource_usage_export_config` block supports: 909 910 * `enable_network_egress_metering` (Optional) - Whether to enable network egress metering for this cluster. If enabled, a daemonset will be created 911 in the cluster to meter network egress traffic. 912 913 * `enable_resource_consumption_metering` (Optional) - Whether to enable resource 914 consumption metering on this cluster. When enabled, a table will be created in 915 the resource export BigQuery dataset to store resource consumption data. The 916 resulting table can be joined with the resource usage table or with BigQuery 917 billing export. Defaults to `true`. 918 919 * `bigquery_destination` (Required) - Parameters for using BigQuery as the destination of resource usage export. 920 921 * `bigquery_destination.dataset_id` (Required) - The ID of a BigQuery Dataset. For Example: 922 923 ```hcl 924 resource_usage_export_config { 925 enable_network_egress_metering = false 926 enable_resource_consumption_metering = true 927 928 bigquery_destination { 929 dataset_id = "cluster_resource_usage" 930 } 931 } 932 ``` 933 934 <a name="nested_shielded_instance_config"></a>The `shielded_instance_config` block supports: 935 936 * `enable_secure_boot` (Optional) - Defines if the instance has Secure Boot enabled. 937 938 Secure Boot helps ensure that the system only runs authentic software by verifying the digital signature of all boot components, and halting the boot process if signature verification fails. Defaults to `false`. 939 940 * `enable_integrity_monitoring` (Optional) - Defines if the instance has integrity monitoring enabled. 941 942 Enables monitoring and attestation of the boot integrity of the instance. The attestation is performed against the integrity policy baseline. This baseline is initially derived from the implicitly trusted boot image when the instance is created. Defaults to `true`. 943 944 <a name="nested_taint"></a>The `taint` block supports: 945 946 * `key` (Required) Key for taint. 947 948 * `value` (Required) Value for taint. 949 950 * `effect` (Required) Effect for taint. Accepted values are `NO_SCHEDULE`, `PREFER_NO_SCHEDULE`, and `NO_EXECUTE`. 951 952 <a name="nested_workload_metadata_config"></a>The `workload_metadata_config` block supports: 953 954 * `mode` (Required) How to expose the node metadata to the workload running on the node. 955 Accepted values are: 956 * UNSPECIFIED: Not Set 957 * GCE_METADATA: Expose all Compute Engine metadata to pods. 958 * GKE_METADATA: Run the GKE Metadata Server on this node. The GKE Metadata Server exposes a metadata API to workloads that is compatible with the V1 Compute Metadata APIs exposed by the Compute Engine and App Engine Metadata Servers. This feature can only be enabled if [workload identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) is enabled at the cluster level. 959 960 <a name="nested_kubelet_config"></a>The `kubelet_config` block supports: 961 962 * `cpu_manager_policy` - (Required) The CPU management policy on the node. See 963 [K8S CPU Management Policies](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/). 964 One of `"none"` or `"static"`. Defaults to `none` when `kubelet_config` is unset. 965 966 * `cpu_cfs_quota` - (Optional) If true, enables CPU CFS quota enforcement for 967 containers that specify CPU limits. 968 969 * `cpu_cfs_quota_period` - (Optional) The CPU CFS quota period value. Specified 970 as a sequence of decimal numbers, each with optional fraction and a unit suffix, 971 such as `"300ms"`. Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", 972 "h". The value must be a positive duration. 973 974 -> Note: At the time of writing (2020/08/18) the GKE API rejects the `none` 975 value and accepts an invalid `default` value instead. While this remains true, 976 not specifying the `kubelet_config` block should be the equivalent of specifying 977 `none`. 978 979 <a name="nested_linux_node_config"></a>The `linux_node_config` block supports: 980 981 * `sysctls` - (Required) The Linux kernel parameters to be applied to the nodes 982 and all pods running on the nodes. Specified as a map from the key, such as 983 `net.core.wmem_max`, to a string value. 984 985 <a name="nested_vertical_pod_autoscaling"></a>The `vertical_pod_autoscaling` block supports: 986 987 * `enabled` (Required) - Enables vertical pod autoscaling 988 989 <a name="nested_dns_config"></a>The `dns_config` block supports: 990 991 * `cluster_dns` - (Optional) Which in-cluster DNS provider should be used. `PROVIDER_UNSPECIFIED` (default) or `PLATFORM_DEFAULT` or `CLOUD_DNS`. 992 993 * `cluster_dns_scope` - (Optional) The scope of access to cluster DNS records. `DNS_SCOPE_UNSPECIFIED` (default) or `CLUSTER_SCOPE` or `VPC_SCOPE`. 994 995 * `cluster_dns_domain` - (Optional) The suffix used for all cluster service records. 996 997 ## Attributes Reference 998 999 In addition to the arguments listed above, the following computed attributes are 1000 exported: 1001 1002 * `id` - an identifier for the resource with format `projects/{{project}}/locations/{{zone}}/clusters/{{name}}` 1003 1004 * `self_link` - The server-defined URL for the resource. 1005 1006 * `endpoint` - The IP address of this cluster's Kubernetes master. 1007 1008 * `label_fingerprint` - The fingerprint of the set of labels for this cluster. 1009 1010 * `maintenance_policy.0.daily_maintenance_window.0.duration` - Duration of the time window, automatically chosen to be 1011 smallest possible in the given scenario. 1012 Duration will be in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) format "PTnHnMnS". 1013 1014 * `master_auth.0.client_certificate` - Base64 encoded public certificate 1015 used by clients to authenticate to the cluster endpoint. 1016 1017 * `master_auth.0.client_key` - Base64 encoded private key used by clients 1018 to authenticate to the cluster endpoint. 1019 1020 * `master_auth.0.cluster_ca_certificate` - Base64 encoded public certificate 1021 that is the root certificate of the cluster. 1022 1023 * `master_version` - The current version of the master in the cluster. This may 1024 be different than the `min_master_version` set in the config if the master 1025 has been updated by GKE. 1026 1027 * `tpu_ipv4_cidr_block` - The IP address range of the Cloud TPUs in this cluster, in 1028 [CIDR](http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) 1029 notation (e.g. `1.2.3.4/29`). 1030 1031 * `services_ipv4_cidr` - The IP address range of the Kubernetes services in this 1032 cluster, in [CIDR](http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) 1033 notation (e.g. `1.2.3.4/29`). Service addresses are typically put in the last 1034 `/16` from the container CIDR. 1035 1036 ## Timeouts 1037 1038 This resource provides the following 1039 [Timeouts](/docs/configuration/resources.html#timeouts) configuration options: 1040 1041 - `create` - Default is 40 minutes. 1042 - `read` - Default is 40 minutes. 1043 - `update` - Default is 60 minutes. 1044 - `delete` - Default is 40 minutes. 1045 1046 ## Import 1047 1048 GKE clusters can be imported using the `project` , `location`, and `name`. If the project is omitted, the default 1049 provider value will be used. Examples: 1050 1051 ``` 1052 $ terraform import google_container_cluster.mycluster projects/my-gcp-project/locations/us-east1-a/clusters/my-cluster 1053 1054 $ terraform import google_container_cluster.mycluster my-gcp-project/us-east1-a/my-cluster 1055 1056 $ terraform import google_container_cluster.mycluster us-east1-a/my-cluster 1057 ``` 1058 1059 ~> **Note:** This resource has several fields that control Terraform-specific behavior and aren't present in the API. If they are set in config and you import a cluster, Terraform may need to perform an update immediately after import. Most of these updates should be no-ops but some may modify your cluster if the imported state differs. 1060 1061 For example, the following fields will show diffs if set in config: 1062 1063 - `min_master_version` 1064 - `remove_default_node_pool` 1065 1066 ## User Project Overrides 1067 1068 This resource supports [User Project Overrides](https://www.terraform.io/docs/providers/google/guides/provider_reference.html#user_project_override).