github.com/crossplane/upjet@v1.3.0/docs/configuring-a-resource.md (about) 1 <!-- 2 SPDX-FileCopyrightText: 2023 The Crossplane Authors <https://crossplane.io> 3 4 SPDX-License-Identifier: CC-BY-4.0 5 --> 6 # Configuring a resource 7 8 [Upjet] generates as much as it could using the available information in the 9 Terraform resource schema. This includes an XRM-conformant schema of the 10 resource, controller logic, late initialization, sensitive data handling, etc. 11 However, there are still information that requires some input configuration 12 which can be found by checking the Terraform documentation of the resource: 13 14 - [External name] 15 - [Cross Resource Referencing] 16 - [Additional Sensitive Fields and Custom Connection Details] 17 - [Late Initialization Behavior] 18 - [Overriding Terraform Resource Schema] 19 - [Initializers] 20 21 ## External Name 22 23 Crossplane uses an annotation in managed resource CR to identify the external 24 resource which is managed by Crossplane. See [the external name documentation] 25 for more details. The format and source of the external name depends on the 26 cloud provider; sometimes it could simply be the name of resource (e.g. S3 27 Bucket), and sometimes it is an auto-generated id by cloud API (e.g. VPC id ). 28 This is something specific to resource, and we need some input configuration for 29 upjet to appropriately generate a resource. 30 31 Since Terraform already needs [a similar identifier] to import a resource, most 32 helpful part of resource documentation is the [import section]. 33 34 Upjet performs some back and forth conversions between Crossplane resource model 35 and Terraform configuration. We need a custom, per resource configuration to 36 adapt Crossplane `external name` from Terraform `id`. 37 38 Here are [the types for the External Name configuration]: 39 40 ```go 41 // SetIdentifierArgumentsFn sets the name of the resource in Terraform attributes map, 42 // i.e. Main HCL file. 43 type SetIdentifierArgumentsFn func(base map[string]any, externalName string) 44 // GetExternalNameFn returns the external name extracted from the TF State. 45 type GetExternalNameFn func(tfstate map[string]any) (string, error) 46 // GetIDFn returns the ID to be used in TF State file, i.e. "id" field in 47 // terraform.tfstate. 48 type GetIDFn func(ctx context.Context, externalName string, parameters map[string]any, providerConfig map[string]any) (string, error) 49 50 // ExternalName contains all information that is necessary for naming operations, 51 // such as removal of those fields from spec schema and calling Configure function 52 // to fill attributes with information given in external name. 53 type ExternalName struct { 54 // SetIdentifierArgumentFn sets the name of the resource in Terraform argument 55 // map. In many cases, there is a field called "name" in the HCL schema, however, 56 // there are cases like RDS DB Cluster where the name field in HCL is called 57 // "cluster_identifier". This function is the place that you can take external 58 // name and assign it to that specific key for that resource type. 59 SetIdentifierArgumentFn SetIdentifierArgumentsFn 60 61 // GetExternalNameFn returns the external name extracted from TF State. In most cases, 62 // "id" field contains all the information you need. You'll need to extract 63 // the format that is decided for external name annotation to use. 64 // For example the following is an Azure resource ID: 65 // /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/mygroup1 66 // The function should return "mygroup1" so that it can be used to set external 67 // name if it was not set already. 68 GetExternalNameFn GetExternalNameFn 69 70 // GetIDFn returns the string that will be used as "id" key in TF state. In 71 // many cases, external name format is the same as "id" but when it is not 72 // we may need information from other places to construct it. For example, 73 // the following is an Azure resource ID: 74 // /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/mygroup1 75 // The function here should use information from supplied arguments to 76 // construct this ID, i.e. "mygroup1" from external name, subscription ID 77 // from providerConfig, and others from parameters map if needed. 78 GetIDFn GetIDFn 79 80 // OmittedFields are the ones you'd like to be removed from the schema since 81 // they are specified via external name. For example, if you set 82 // "cluster_identifier" in SetIdentifierArgumentFn, then you need to omit 83 // that field. 84 // You can omit only the top level fields. 85 // No field is omitted by default. 86 OmittedFields []string 87 88 // DisableNameInitializer allows you to specify whether the name initializer 89 // that sets external name to metadata.name if none specified should be disabled. 90 // It needs to be disabled for resources whose external identifier is randomly 91 // assigned by the provider, like AWS VPC where it gets vpc-21kn123 identifier 92 // and not let you name it. 93 DisableNameInitializer bool 94 } 95 ``` 96 97 Comments explain the purpose of each field but let's clarify further with some 98 example cases. 99 100 ### Case 1: Name as External Name and Terraform ID 101 102 This is the simplest and most straightforward case with the following 103 conditions: 104 105 - Terraform resource uses the `name` argument to identify the resources 106 - Terraform resource can be imported with `name`, i.e. `id`=`name` 107 108 [aws_iam_user] is a good example here. In this case, we can just use the 109 [NameAsIdentifier] config of Upjet as follows: 110 111 ```go 112 import ( 113 "github.com/crossplane/upjet/pkg/config" 114 ... 115 ) 116 117 ... 118 p.AddResourceConfigurator("aws_iam_user", func(r *config.Resource) { 119 r.ExternalName = config.NameAsIdentifier 120 ... 121 } 122 ``` 123 124 There are some resources which fits into this case with an exception by 125 expecting an argument other than `name` to name/identify a resource, for 126 example, [bucket] for [aws_s3_bucket] and [cluster_identifier] for 127 [aws_rds_cluster]. 128 129 Let's check [aws_s3_bucket] further. Reading the [import section of s3 bucket] 130 we see that bucket is imported with its **name**, however, checking _arguments_ 131 section we see that this name is provided with the [bucket] argument. We also 132 notice, there is also another argument as `bucket_prefix` which conflicts with 133 `bucket` argument. We can just use the [NameAsIdentifier] config, however, we 134 also need to configure the `bucket` argument with `SetIdentifierArgumentFn` and 135 also omit `bucket` and `bucket_prefix` arguments from the spec with 136 `OmittedFields`: 137 138 ```go 139 import ( 140 "github.com/crossplane/upjet/pkg/config" 141 ... 142 ) 143 144 ... 145 p.AddResourceConfigurator("aws_s3_bucket", func(r *config.Resource) { 146 r.ExternalName = config.NameAsIdentifier 147 r.ExternalName.SetIdentifierArgumentFn = func(base map[string]any, externalName string) { 148 base["bucket"] = externalName 149 } 150 r.ExternalName.OmittedFields = []string{ 151 "bucket", 152 "bucket_prefix", 153 } 154 ... 155 } 156 ``` 157 158 ### Case 2: Identifier from Provider 159 160 In this case, the (cloud) provider generates an identifier for the resource 161 independent of what we provided as arguments. 162 163 Checking the [import section of aws_vpc], we see that this resource is being 164 imported with `vpc id`. When we check the [arguments list] and provided [example 165 usages], it is clear that this **id** is **not** something that user provides, 166 rather generated by AWS API. 167 168 Here, we can just use [IdentifierFromProvider] configuration: 169 170 ```go 171 import ( 172 "github.com/crossplane/upjet/pkg/config" 173 ... 174 ) 175 176 ... 177 p.AddResourceConfigurator("aws_vpc", func(r *config.Resource) { 178 r.ExternalName = config.IdentifierFromProvider 179 ... 180 } 181 ``` 182 183 ### Case 3: Terraform ID as a Formatted String 184 185 For some resources, Terraform uses a formatted string as `id` which include 186 resource identifier that Crossplane uses as external name but may also contain 187 some other parameters. 188 189 Most `azurerm` resources fall into this category. Checking the [import section 190 of azurerm_sql_server], we see that can be imported with an `id` in the 191 following format: 192 193 ```text 194 /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myresourcegroup/providers/Microsoft.Sql/servers/myserver 195 ``` 196 197 To properly set external name for such a resource, we need to configure how to 198 extract external name from this string (`GetExternalNameFn`) and how to build 199 this id back (`GetIDFn`). 200 201 ```go 202 import ( 203 "github.com/crossplane/upjet/pkg/config" 204 ... 205 ) 206 207 func getNameFromFullyQualifiedID(tfstate map[string]any) (string, error) { 208 id, ok := tfstate["id"] 209 if !ok { 210 return "", errors.Errorf(ErrFmtNoAttribute, "id") 211 } 212 idStr, ok := id.(string) 213 if !ok { 214 return "", errors.Errorf(ErrFmtUnexpectedType, "id") 215 } 216 words := strings.Split(idStr, "/") 217 return words[len(words)-1], nil 218 } 219 220 func getFullyQualifiedIDfunc(ctx context.Context, externalName string, parameters map[string]any, providerConfig map[string]any) (string, error) { 221 subID, ok := providerConfig["subscription_id"] 222 if !ok { 223 return "", errors.Errorf(ErrFmtNoAttribute, "subscription_id") 224 } 225 subIDStr, ok := subID.(string) 226 if !ok { 227 return "", errors.Errorf(ErrFmtUnexpectedType, "subscription_id") 228 } 229 rg, ok := parameters["resource_group_name"] 230 if !ok { 231 return "", errors.Errorf(ErrFmtNoAttribute, "resource_group_name") 232 } 233 rgStr, ok := rg.(string) 234 if !ok { 235 return "", errors.Errorf(ErrFmtUnexpectedType, "resource_group_name") 236 } 237 238 name, ok := parameters["name"] 239 if !ok { 240 return "", errors.Errorf(ErrFmtNoAttribute, "name") 241 } 242 nameStr, ok := rg.(string) 243 if !ok { 244 return "", errors.Errorf(ErrFmtUnexpectedType, "name") 245 } 246 247 return fmt.Sprintf("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Sql/servers/%s", subIDStr, rgStr, nameStr), nil 248 } 249 250 ... 251 p.AddResourceConfigurator("azurerm_sql_server", func(r *config.Resource) { 252 r.ExternalName = config.NameAsIdentifier 253 r.ExternalName.GetExternalNameFn = getNameFromFullyQualifiedID 254 r.ExternalName.GetIDFn = getFullyQualifiedIDfunc 255 ... 256 } 257 ``` 258 259 With this, we have covered most common scenarios for configuring external name. 260 You can always check resource configurations of existing jet Providers as 261 further examples under `config/<group>/config.go` in their repositories. 262 263 _Please see [this figure] to understand why we really need 3 different functions 264 to configure external names and, it visualizes which is used how:_ 265  _Note that, initially, GetIDFn 266 will use the external-name annotation to set the terraform.tfstate id and, after 267 that, it uses the terraform.tfstate id to update the external-name annotation. 268 For cases where both values are different, both GetIDFn and GetExternalNameFn 269 must be set in order to have the correct configuration._ 270 271 ### Cross Resource Referencing 272 273 Crossplane uses cross resource referencing to [handle dependencies] between 274 managed resources. For example, if you have an IAM User defined as a Crossplane 275 managed resource, and you want to create an Access Key for that user, you would 276 need to refer to the User CR from the Access Key resource. This is handled by 277 cross resource referencing. 278 279 See how the [user] referenced at `forProvider.userRef.name` field of the Access 280 Key in the following example: 281 282 ```yaml 283 apiVersion: iam.aws.tf.crossplane.io/v1alpha1 284 kind: User 285 metadata: 286 name: sample-user 287 spec: 288 forProvider: {} 289 --- 290 apiVersion: iam.aws.tf.crossplane.io/v1alpha1 291 kind: AccessKey 292 metadata: 293 name: sample-access-key 294 spec: 295 forProvider: 296 userRef: 297 name: sample-user 298 writeConnectionSecretToRef: 299 name: sample-access-key-secret 300 namespace: crossplane-system 301 ``` 302 303 Historically, reference resolution method were written by hand which requires 304 some effort, however, with the latest Crossplane code generation tooling, it is 305 now possible to [generate reference resolution methods] by just adding some 306 marker on the fields. Now, the only manual step for generating cross resource 307 references is to provide which field of a resource depends on which information 308 (e.g. `id`, `name`, `arn` etc.) from the other. 309 310 In Upjet, we have a [configuration] to provide this information for a field: 311 312 ```go 313 // Reference represents the Crossplane options used to generate 314 // reference resolvers for fields 315 type Reference struct { 316 // Type is the type name of the CRD if it is in the same package or 317 // <package-path>.<type-name> if it is in a different package. 318 Type string 319 // TerraformName is the name of the Terraform resource 320 // which will be referenced. The supplied resource name is 321 // converted to a type name of the corresponding CRD using 322 // the configured TerraformTypeMapper. 323 TerraformName string 324 // Extractor is the function to be used to extract value from the 325 // referenced type. Defaults to getting external name. 326 // Optional 327 Extractor string 328 // RefFieldName is the field name for the Reference field. Defaults to 329 // <field-name>Ref or <field-name>Refs. 330 // Optional 331 RefFieldName string 332 // SelectorFieldName is the field name for the Selector field. Defaults to 333 // <field-name>Selector. 334 // Optional 335 SelectorFieldName string 336 } 337 ``` 338 339 For a resource that we want to generate, we need to check its argument list in 340 Terraform documentation and figure out which field needs reference to which 341 resource. 342 343 Let's check [iam_access_key] as an example. In the argument list, we see the 344 [user] field which requires a reference to a IAM user. So, we need to the 345 following referencing configuration: 346 347 ```go 348 func Configure(p *config.Provider) { 349 p.AddResourceConfigurator("aws_iam_access_key", func (r *config.Resource) { 350 r.References["user"] = config.Reference{ 351 Type: "User", 352 } 353 }) 354 } 355 ``` 356 357 Please note the value of `Type` field needs to be a string representing the Go 358 type of the resource. Since, `AccessKey` and `User` resources are under the same 359 go package, we don't need to provide the package path. However, this is not 360 always the case and referenced resources might be in different package. In that 361 case, we would need to provide the full path. Referencing to a [kms key] from 362 `aws_ebs_volume` resource is a good example here: 363 364 ```go 365 func Configure(p *config.Provider) { 366 p.AddResourceConfigurator("aws_ebs_volume", func(r *config.Resource) { 367 r.References["kms_key_id"] = config.Reference{ 368 Type: "github.com/crossplane-contrib/provider-tf-aws/apis/kms/v1alpha1.Key", 369 } 370 }) 371 } 372 ``` 373 374 ### Auto Cross Resource Reference Generation 375 376 Cross Resource Referencing is one of the key concepts of the resource 377 configuration. As a very common case, cloud services depend on other cloud 378 services. For example, AWS Subnet resource needs an AWS VPC for creation. So, 379 for creating a Subnet successfully, before you have to create a VPC resource. 380 Please see the [Managed Resources] documentation for more details. 381 382 These documentations focus on the general concepts and manual configurations 383 of Cross Resource References. However, the main topic of this documentation is 384 automatic example&reference generation. 385 386 Upjet has a scraper tool for scraping provider metadata from the Terraform 387 Registry. The scraped metadata are: 388 389 - Resource Descriptions 390 - Examples of Resources (in HCL format) 391 - Field Documentations 392 - Import Statements 393 394 These are very critical information for our automation processes. We use this 395 scraped metadata in many contexts. For example, field documentation of 396 resources and descriptions are used as Golang comments for schema fields and 397 CRDs. 398 399 Another important scraped information is examples of resources. As a part 400 of testing efforts, finding the correct combination of field values is not easy 401 for every scenario. So, having a working example (combination) is very important 402 for easy testing. 403 404 At this point, this example that is in HCL format is converted to a Managed 405 Resource manifest, and we can use this manifest in our test efforts. 406 407 This is an example from Terraform Registry AWS Ebs Volume resource: 408 409 ```hcl 410 resource "aws_ebs_volume" "example" { 411 availability_zone = "us-west-2a" 412 size = 40 413 414 tags = { 415 Name = "HelloWorld" 416 } 417 } 418 419 resource "aws_ebs_snapshot" "example_snapshot" { 420 volume_id = aws_ebs_volume.example.id 421 422 tags = { 423 Name = "HelloWorld_snap" 424 } 425 } 426 ``` 427 428 The generated example: 429 430 ```yaml 431 apiVersion: ec2.aws.upbound.io/v1beta1 432 kind: EBSSnapshot 433 metadata: 434 annotations: 435 meta.upbound.io/example-id: ec2/v1beta1/ebssnapshot 436 labels: 437 testing.upbound.io/example-name: example_snapshot 438 name: example-snapshot 439 spec: 440 forProvider: 441 region: us-west-1 442 tags: 443 Name: HelloWorld_snap 444 volumeIdSelector: 445 matchLabels: 446 testing.upbound.io/example-name: example 447 448 --- 449 450 apiVersion: ec2.aws.upbound.io/v1beta1 451 kind: EBSVolume 452 metadata: 453 annotations: 454 meta.upbound.io/example-id: ec2/v1beta1/ebssnapshot 455 labels: 456 testing.upbound.io/example-name: example 457 name: example 458 spec: 459 forProvider: 460 availabilityZone: us-west-2a 461 region: us-west-1 462 size: 40 463 tags: 464 Name: HelloWorld 465 ``` 466 467 Here, there are three very important points that scraper makes easy our life: 468 469 - We do not have to find the correct value combinations for fields. So, we can 470 easily use the generated example manifest in our tests. 471 - The HCL example was scraped from registry documentation of the 472 `aws_ebs_snapshot` resource. In the example, you also see the `aws_ebs_volume` 473 resource manifest because, for the creation of an EBS Snapshot, you need an 474 EBS Volume resource. Thanks to the source Registry, (in many cases, there are 475 the dependent resources of target resources) we can also scrape the 476 dependencies of target resources. 477 - The last item is actually what is intended to be explained in this document. 478 For using the Cross Resource References, as I mentioned above, you need to add 479 some references to the resource configuration. But, in many cases, if in the 480 scraped example, the mentioned dependencies are already described you do not 481 have to write explicit references to resource configuration. The Cross 482 Resource Reference generator generates the mentioned references. 483 484 ### Validating the Cross Resource References 485 486 As I mentioned, many references are generated from scraped metadata by an auto 487 reference generator. However, there are two cases where we miss generating the 488 references. 489 490 The first one is related to some bugs or improvement points in the generator. 491 This means that the generator can handle many references in the scraped examples 492 and correctly generate them. But we cannot say that the ratio is 100%. For some 493 cases, the generator cannot generate references, even though they are in the 494 scraped example manifests. 495 496 The second one is related to the scraped example itself. As I mentioned above, 497 the source of the generator is the scraped example manifest. So, it checks the 498 manifest and tries to generate the found cross-resource references. In some 499 cases, although there are other reference fields, these do not exist in the 500 example manifest. They can only be mentioned in schema/field documentation. 501 502 For these types of situations, you must configure cross-resource references 503 explicitly. 504 505 ### Removing Auto-Generated Cross Resource References In Some Corner Cases 506 507 In some cases, the generated references can narrow the reference pool covered by 508 the field. For example, X resource has an A field and Y and Z resources can be 509 referenced via this field. However, since the reference to Y is mentioned in the 510 example manifest, this reference field will only be defined over Y. In this 511 case, since the reference pool of the relevant field will be narrowed, it would 512 be more appropriate to delete this reference. For example, 513 514 ```hcl 515 resource "aws_route53_record" "www" { 516 zone_id = aws_route53_zone.primary.zone_id 517 name = "example.com" 518 type = "A" 519 520 alias { 521 name = aws_elb.main.dns_name 522 zone_id = aws_elb.main.zone_id 523 evaluate_target_health = true 524 } 525 } 526 ``` 527 528 Route53 Record resource’s alias.name field has a reference. In the example, this 529 reference is shown by using the `aws_elb` resource. However, when we check the 530 field documentation, we see that this field can also be referenced by other 531 resources: 532 533 ```text 534 Alias 535 Alias records support the following: 536 537 name - (Required) DNS domain name for a CloudFront distribution, S3 bucket, ELB, 538 or another resource record set in this hosted zone. 539 ``` 540 541 ### Conclusion 542 543 As a result, mentioned scraper and example&reference generators are very useful 544 for making easy the test efforts. But using these functionalities, we must be 545 careful to avoid undesired states. 546 547 [Managed Resources]: https://docs.crossplane.io/latest/concepts/managed-resources/#referencing-other-resources 548 549 ## Additional Sensitive Fields and Custom Connection Details 550 551 Crossplane stores sensitive information of a managed resource in a Kubernetes 552 secret, together with some additional fields that would help consumption of the 553 resource, a.k.a. [connection details]. 554 555 In Upjet, we already handle sensitive fields that are marked as sensitive in 556 Terraform schema and no further action required for them. Upjet will properly 557 hide these fields from CRD spec and status by converting to a secret reference 558 or storing in connection details secret respectively. However, we still have 559 some custom configuration API that would allow including additional fields into 560 connection details secret no matter they are sensitive or not. 561 562 As an example, let's use `aws_iam_access_key`. Currently, Upjet stores all 563 sensitive fields in Terraform schema as prefixed with `attribute.`, so without 564 any `AdditionalConnectionDetailsFn`, connection resource will have 565 `attribute.id` and `attribute.secret` corresponding to [id] and [secret] fields 566 respectively. To see them with more common keys, i.e. `aws_access_key_id` and 567 `aws_secret_access_key`, we would need to make the following configuration: 568 569 ```go 570 func Configure(p *config.Provider) { 571 p.AddResourceConfigurator("aws_iam_access_key", func(r *config.Resource) { 572 r.Sensitive.AdditionalConnectionDetailsFn = func(attr map[string]any) (map[string][]byte, error) { 573 conn := map[string][]byte{} 574 if a, ok := attr["id"].(string); ok { 575 conn["aws_access_key_id"] = []byte(a) 576 } 577 if a, ok := attr["secret"].(string); ok { 578 conn["aws_secret_access_key"] = []byte(a) 579 } 580 return conn, nil 581 } 582 }) 583 } 584 ``` 585 586 This will produce a connection details secret as follows: 587 588 ```yaml 589 apiVersion: v1 590 data: 591 attribute.id: QUtJQVk0QUZUVFNFNDI2TlhKS0I= 592 attribute.secret: ABCxyzRedacted== 593 attribute.ses_smtp_password_v4: QQ00REDACTED== 594 aws_access_key_id: QUtJQVk0QUZUVFNFNDI2TlhKS0I= 595 aws_secret_access_key: ABCxyzRedacted== 596 kind: Secret 597 ``` 598 599 ### Late Initialization Configuration 600 601 Late initialization configuration is only required if there are conflicting 602 arguments in terraform resource configuration. Unfortunately, there is _no easy 603 way_ to figure that out without testing the resource, _so feel free to skip this 604 configuration_ at the first place and revisit _only if_ you have errors like 605 below while testing the resource. 606 607 ```text 608 observe failed: cannot run refresh: refresh failed: Invalid combination of arguments: 609 "address_prefix": only one of `address_prefix,address_prefixes` can be specified, but `address_prefix,address_prefixes` were specified.: File name: main.tf.json 610 ``` 611 612 If you would like to have the late-initialization library _not_ to process the 613 [`address_prefix`] parameter field, then the following configuration where we 614 specify the parameter field path is sufficient: 615 616 ```go 617 func Configure(p *config.Provider) { 618 p.AddResourceConfigurator("azurerm_subnet", func(r *config.Resource) { 619 r.LateInitializer = config.LateInitializer{ 620 IgnoredFields: []string{"address_prefix"}, 621 } 622 }) 623 } 624 ``` 625 626 _Please note that, there could be errors looking slightly different from above, 627 so please consider configuring late initialization behaviour whenever you got 628 some unexpected error starting with `observe failed:`, once you are sure that 629 you provided all necessary parameters to your resource._ 630 631 ### Further details on Late Initialization 632 633 Upjet runtime automatically performs late-initialization during an 634 [`external.Observe`] call with means of runtime reflection. State of the world 635 observed by Terraform CLI is used to initialize any `nil`-valued pointer 636 parameters in the managed resource's `spec`. In most of the cases no custom 637 configuration should be necessary for late-initialization to work. However, 638 there are certain cases where you will want/need to customize 639 late-initialization behaviour. Thus, Upjet provides an extensible 640 [late-initialization customization API] that controls late-initialization 641 behaviour. 642 643 The associated resource struct is defined 644 [here](https://github.com/crossplane/upjet/blob/c9e21387298d8ed59fcd71c7f753ec401a3383a5/pkg/config/resource.go#L91) 645 as follows: 646 647 ```go 648 // LateInitializer represents configurations that control 649 // late-initialization behaviour 650 type LateInitializer struct { 651 // IgnoredFields are the canonical field names to be skipped during 652 // late-initialization 653 IgnoredFields []string 654 } 655 ``` 656 657 Currently, it only involves a configuration option to specify certain `spec` 658 parameters to be ignored during late-initialization. Each element of the 659 `LateInitializer.IgnoredFields` slice represents the canonical path relative to 660 the parameters struct for the managed resource's `Spec` using `Go` type names as 661 path elements. As an example, with the following type definitions: 662 663 ```go 664 type Subnet struct { 665 metav1.TypeMeta `json:",inline"` 666 metav1.ObjectMeta `json:"metadata,omitempty"` 667 Spec SubnetSpec `json:"spec"` 668 Status SubnetStatus `json:"status,omitempty"` 669 } 670 671 type SubnetSpec struct { 672 ForProvider SubnetParameters `json:"forProvider"` 673 ... 674 } 675 676 type DelegationParameters struct { 677 // +kubebuilder:validation:Required 678 Name *string `json:"name" tf:"name,omitempty"` 679 ... 680 } 681 682 type SubnetParameters struct { 683 // +kubebuilder:validation:Optional 684 AddressPrefix *string `json:"addressPrefix,omitempty" tf:"address_prefix,omitempty"` 685 // +kubebuilder:validation:Optional 686 Delegation []DelegationParameters `json:"delegation,omitempty" tf:"delegation,omitempty"` 687 ... 688 } 689 ``` 690 691 In most cases, custom late-initialization configuration will not be necessary. 692 However, after generating a new managed resource and observing its behaviour (at 693 runtime), it may turn out that late-initialization behaviour needs 694 customization. For certain resources like the `provider-tf-azure`'s 695 `PostgresqlServer` resource, we have observed that Terraform state contains 696 values for mutually exclusive parameters, e.g., for `PostgresqlServer`, both 697 `StorageMb` and `StorageProfile[].StorageMb` get late-initialized. Upon next 698 reconciliation, we generate values for both parameters in the Terraform 699 configuration, and although they happen to have the same value, Terraform 700 configuration validation requires them to be mutually exclusive. Currently, we 701 observe this behaviour at runtime, and upon observing that the resource cannot 702 transition to the `Ready` state and acquires the Terraform validation error 703 message in its `status.conditions`, we do the `LateInitializer.IgnoreFields` 704 custom configuration detailed above to skip one of the mutually exclusive fields 705 during late-initialization. 706 707 ## Overriding Terraform Resource Schema 708 709 Upjet generates Crossplane resource schemas (CR spec/status) using the 710 [Terraform schema of the resource]. As of today, Upjet leverages the following 711 attributes in the schema: 712 713 - [Type] and [Elem] to identify the type of the field. 714 - [Sensitive] to see if we need to keep it in a Secret instead of CR. 715 - [Description] to add as a description to the field in CRD. 716 - [Optional] and [Computed] to identify whether the fields go under spec or 717 status: 718 - Not Optional & Not Computed => Spec (required) 719 - Optional & Not Computed => Spec (optional) 720 - Optional & Computed => Spec (optional, to be late-initialized) 721 - Not Optional & Computed => Status 722 723 Usually, we don't need to make any modifications in the resource schema and 724 resource schema just works as is. However, there could be some rare edge cases 725 like: 726 727 - Field contains sensitive information but not marked as `Sensitive` or vice 728 versa. 729 - An attribute does not make sense to have in CRD schema, like [tags_all for jet 730 AWS resources]. 731 - Moving parameters from Terraform provider config to resources schema to fit 732 Crossplane model, e.g. [AWS region] parameter is part of provider config in 733 Terraform but Crossplane expects it in CR spec. 734 735 Schema of a resource could be overridden as follows: 736 737 ```go 738 p.AddResourceConfigurator("aws_autoscaling_group", func(r *config.Resource) { 739 // Managed by Attachment resource. 740 if s, ok := r.TerraformResource.Schema["load_balancers"]; ok { 741 s.Optional = false 742 s.Computed = true 743 } 744 if s, ok := r.TerraformResource.Schema["target_group_arns"]; ok { 745 s.Optional = false 746 s.Computed = true 747 } 748 }) 749 ``` 750 751 ## Initializers 752 753 Initializers involve the operations that run before beginning of reconciliation. 754 This configuration option will provide that setting initializers for per 755 resource. 756 757 Many resources in aws have `tags` field in their schema. Also, in Crossplane 758 there is a [tagging convention]. To implement the tagging convention for jet-aws 759 provider, this initializer configuration support was provided. 760 761 There is a common struct (`Tagger`) in upjet to use the tagging convention: 762 763 ```go 764 // Tagger implements the Initialize function to set external tags 765 type Tagger struct { 766 kube client.Client 767 fieldName string 768 } 769 770 // NewTagger returns a Tagger object. 771 func NewTagger(kube client.Client, fieldName string) *Tagger { 772 return &Tagger{kube: kube, fieldName: fieldName} 773 } 774 775 // Initialize is a custom initializer for setting external tags 776 func (t *Tagger) Initialize(ctx context.Context, mg xpresource.Managed) error { 777 paved, err := fieldpath.PaveObject(mg) 778 if err != nil { 779 return err 780 } 781 pavedByte, err := setExternalTagsWithPaved(xpresource.GetExternalTags(mg), paved, t.fieldName) 782 if err != nil { 783 return err 784 } 785 if err := json.Unmarshal(pavedByte, mg); err != nil { 786 return err 787 } 788 if err := t.kube.Update(ctx, mg); err != nil { 789 return err 790 } 791 return nil 792 } 793 ``` 794 795 As seen above, the `Tagger` struct accepts a `fieldName`. This `fieldName` 796 specifies which value of field to set in the resource's spec. You can use the 797 common `Initializer` by specifying the field name that points to the external 798 tags in the configured resource. 799 800 There is also a default initializer for tagging convention, `TagInitializer`. It 801 sets the value of `fieldName` to `tags` as default: 802 803 ```go 804 // TagInitializer returns a tagger to use default tag initializer. 805 var TagInitializer NewInitializerFn = func(client client.Client) managed.Initializer { 806 return NewTagger(client, "tags") 807 } 808 ``` 809 810 In jet-aws provider, as a default process, if a resource has `tags` field in its 811 schema, then the default initializer (`TagInitializer`) is added to Initializer 812 list of resource: 813 814 ```go 815 // AddExternalTagsField adds ExternalTagsFieldName configuration for resources that have tags field. 816 func AddExternalTagsField() tjconfig.ResourceOption { 817 return func(r *tjconfig.Resource) { 818 if s, ok := r.TerraformResource.Schema["tags"]; ok && s.Type == schema.TypeMap { 819 r.InitializerFns = append(r.InitializerFns, tjconfig.TagInitializer) 820 } 821 } 822 } 823 ``` 824 825 However, if the field name that used for the external label is different from 826 the `tags`, the `NewTagger` function can be called and the specific `fieldName` 827 can be passed to this: 828 829 ```go 830 r.InitializerFns = append(r.InitializerFns, func(client client.Client) managed.Initializer { 831 return tjconfig.NewTagger(client, "example_tags_name") 832 }) 833 ``` 834 835 If the above tagging convention logic does not work for you, and you want to use 836 this configuration option for a reason other than tagging convention (for 837 another custom initializer operation), you need to write your own struct in 838 provider and have this struct implement the `Initializer` function with a custom 839 logic. 840 841 This configuration option is set by using the [InitializerFns] field that is a 842 list of [NewInitializerFn]: 843 844 ```go 845 // NewInitializerFn returns the Initializer with a client. 846 type NewInitializerFn func(client client.Client) managed.Initializer 847 ``` 848 849 Initializer is an interface in [crossplane-runtime]: 850 851 ```go 852 type Initializer interface { 853 Initialize(ctx context.Context, mg resource.Managed) error 854 } 855 ``` 856 857 So, an interface must be passed to the related configuration field for adding 858 initializers for a resource. 859 860 [Upjet]: https://github.com/crossplane/upjet 861 [External name]: #external-name 862 [Cross Resource Referencing]: #cross-resource-referencing 863 [Additional Sensitive Fields and Custom Connection Details]: #additional-sensitive-fields-and-custom-connection-details 864 [Late Initialization Behavior]: #late-initialization-configuration 865 [Overriding Terraform Resource Schema]: #overriding-terraform-resource-schema 866 [the external name documentation]: https://docs.crossplane.io/master/concepts/managed-resources/#naming-external-resources 867 [import section]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#import 868 [the types for the External Name configuration]: https://github.com/crossplane/upjet/blob/main/pkg/config/resource.go#L68 869 [aws_iam_user]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user 870 [NameAsIdentifier]: https://github.com/crossplane/upjet/blob/main/pkg/config/externalname.go#L28 871 [aws_s3_bucket]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket 872 [import section of s3 bucket]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket#import 873 [bucket]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket#bucket 874 [cluster_identifier]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster#cluster_identifier 875 [aws_rds_cluster]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster. 876 [import section of aws_vpc]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc#import 877 [arguments list]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc#argument-reference 878 [example usages]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc#example-usage 879 [IdentifierFromProvider]: https://github.com/crossplane/upjet/blob/main/pkg/config/externalname.go#L42 880 [a similar identifier]: https://www.terraform.io/docs/glossary#id 881 [import section of azurerm_sql_server]: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/sql_server#import 882 [handle dependencies]: https://docs.crossplane.io/master/concepts/managed-resources/#referencing-other-resources 883 [user]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#user 884 [generate reference resolution methods]: https://github.com/crossplane/crossplane-tools/pull/35 885 [configuration]: https://github.com/crossplane/upjet/blob/main/pkg/config/resource.go#L123 886 [iam_access_key]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#argument-reference 887 [kms key]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ebs_volume#kms_key_id 888 [connection details]: https://docs.crossplane.io/master/concepts/managed-resources/#writeconnectionsecrettoref 889 [id]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#id 890 [secret]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#secret 891 [`external.Observe`]: https://github.com/crossplane/upjet/blob/main/pkg/controller/external.go#L175 892 [late-initialization customization API]: https://github.com/crossplane/upjet/blob/main/pkg/resource/lateinit.go#L45 893 [`address_prefix`]: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/subnet#address_prefix 894 [Terraform schema of the resource]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L34 895 [Type]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L52 896 [Elem]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L151 897 [Sensitive]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L244 898 [Description]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L120 899 [Optional]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L80 900 [Computed]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L139 901 [tags_all for jet AWS resources]: https://github.com/upbound/provider-aws/blob/main/config/overrides.go#L62 902 [AWS region]: https://github.com/upbound/provider-aws/blob/main/config/overrides.go#L32 903 [this figure]: ../docs/images/upjet-externalname.png 904 [Initializers]: #initializers 905 [InitializerFns]: https://github.com/crossplane/upjet/blob/main/pkg/config/resource.go#L297 906 [NewInitializerFn]: https://github.com/crossplane/upjet/blob/main/pkg/config/resource.go#L210 907 [crossplane-runtime]: https://github.com/crossplane/crossplane-runtime/blob/428b7c3903756bb0dcf5330f40298e1fa0c34301/pkg/reconciler/managed/reconciler.go#L138 908 [tagging convention]: https://github.com/crossplane/crossplane/blob/60c7df9/design/one-pager-managed-resource-api-design.md#external-resource-labeling