github.com/pulumi/terraform@v1.4.0/website/docs/language/modules/develop/composition.mdx (about) 1 --- 2 page_title: Module Composition 3 description: |- 4 Module composition allows infrastructure to be described from modular 5 building blocks. 6 --- 7 8 # Module Composition 9 10 In a simple Terraform configuration with only one root module, we create a 11 flat set of resources and use Terraform's expression syntax to describe the 12 relationships between these resources: 13 14 ```hcl 15 resource "aws_vpc" "example" { 16 cidr_block = "10.1.0.0/16" 17 } 18 19 resource "aws_subnet" "example" { 20 vpc_id = aws_vpc.example.id 21 22 availability_zone = "us-west-2b" 23 cidr_block = cidrsubnet(aws_vpc.example.cidr_block, 4, 1) 24 } 25 ``` 26 27 When we introduce `module` blocks, our configuration becomes hierarchical 28 rather than flat: each module contains its own set of resources, and possibly 29 its own child modules, which can potentially create a deep, complex tree of 30 resource configurations. 31 32 However, in most cases we strongly recommend keeping the module tree flat, 33 with only one level of child modules, and use a technique similar to the 34 above of using expressions to describe the relationships between the modules: 35 36 ```hcl 37 module "network" { 38 source = "./modules/aws-network" 39 40 base_cidr_block = "10.0.0.0/8" 41 } 42 43 module "consul_cluster" { 44 source = "./modules/aws-consul-cluster" 45 46 vpc_id = module.network.vpc_id 47 subnet_ids = module.network.subnet_ids 48 } 49 ``` 50 51 We call this flat style of module usage _module composition_, because it 52 takes multiple [composable](https://en.wikipedia.org/wiki/Composability) 53 building-block modules and assembles them together to produce a larger system. 54 Instead of a module _embedding_ its dependencies, creating and managing its 55 own copy, the module _receives_ its dependencies from the root module, which 56 can therefore connect the same modules in different ways to produce different 57 results. 58 59 The rest of this page discusses some more specific composition patterns that 60 may be useful when describing larger systems with Terraform. 61 62 ## Dependency Inversion 63 64 In the example above, we saw a `consul_cluster` module that presumably describes 65 a cluster of [HashiCorp Consul](https://www.consul.io/) servers running in 66 an AWS VPC network, and thus it requires as arguments the identifiers of both 67 the VPC itself and of the subnets within that VPC. 68 69 An alternative design would be to have the `consul_cluster` module describe 70 its _own_ network resources, but if we did that then it would be hard for 71 the Consul cluster to coexist with other infrastructure in the same network, 72 and so where possible we prefer to keep modules relatively small and pass in 73 their dependencies. 74 75 This [dependency inversion](https://en.wikipedia.org/wiki/Dependency_inversion_principle) 76 approach also improves flexibility for future 77 refactoring, because the `consul_cluster` module doesn't know or care how 78 those identifiers are obtained by the calling module. A future refactor may 79 separate the network creation into its own configuration, and thus we may 80 pass those values into the module from data sources instead: 81 82 ```hcl 83 data "aws_vpc" "main" { 84 tags = { 85 Environment = "production" 86 } 87 } 88 89 data "aws_subnet_ids" "main" { 90 vpc_id = data.aws_vpc.main.id 91 } 92 93 module "consul_cluster" { 94 source = "./modules/aws-consul-cluster" 95 96 vpc_id = data.aws_vpc.main.id 97 subnet_ids = data.aws_subnet_ids.main.ids 98 } 99 ``` 100 101 ### Conditional Creation of Objects 102 103 In situations where the same module is used across multiple environments, 104 it's common to see that some necessary object already exists in some 105 environments but needs to be created in other environments. 106 107 For example, this can arise in development environment scenarios: for cost 108 reasons, certain infrastructure may be shared across multiple development 109 environments, while in production the infrastructure is unique and managed 110 directly by the production configuration. 111 112 Rather than trying to write a module that itself tries to detect whether something 113 exists and create it if not, we recommend applying the dependency inversion 114 approach: making the module accept the object it needs as an argument, via 115 an input variable. 116 117 For example, consider a situation where a Terraform module deploys compute 118 instances based on a disk image, and in some environments there is a 119 specialized disk image available while other environments share a common 120 base disk image. Rather than having the module itself handle both of these 121 scenarios, we can instead declare an input variable for an object representing 122 the disk image. Using AWS EC2 as an example, we might declare a common subtype 123 of the `aws_ami` resource type and data source schemas: 124 125 ```hcl 126 variable "ami" { 127 type = object({ 128 # Declare an object using only the subset of attributes the module 129 # needs. Terraform will allow any object that has at least these 130 # attributes. 131 id = string 132 architecture = string 133 }) 134 } 135 ``` 136 137 The caller of this module can now itself directly represent whether this is 138 an AMI to be created inline or an AMI to be retrieved from elsewhere: 139 140 ```hcl 141 # In situations where the AMI will be directly managed: 142 143 resource "aws_ami_copy" "example" { 144 name = "local-copy-of-ami" 145 source_ami_id = "ami-abc123" 146 source_ami_region = "eu-west-1" 147 } 148 149 module "example" { 150 source = "./modules/example" 151 152 ami = aws_ami_copy.example 153 } 154 ``` 155 156 ```hcl 157 # Or, in situations where the AMI already exists: 158 159 data "aws_ami" "example" { 160 owner = "9999933333" 161 162 tags = { 163 application = "example-app" 164 environment = "dev" 165 } 166 } 167 168 module "example" { 169 source = "./modules/example" 170 171 ami = data.aws_ami.example 172 } 173 ``` 174 175 This is consistent with Terraform's declarative style: rather than creating 176 modules with complex conditional branches, we directly describe what 177 should already exist and what we want Terraform to manage itself. 178 179 By following this pattern, we can be explicit about in which situations we 180 expect the AMI to already be present and which we don't. A future reader 181 of the configuration can then directly understand what it is intending to do 182 without first needing to inspect the state of the remote system. 183 184 In the above example, the object to be created or read is simple enough to 185 be given inline as a single resource, but we can also compose together multiple 186 modules as described elsewhere on this page in situations where the 187 dependencies themselves are complicated enough to benefit from abstractions. 188 189 ## Assumptions and Guarantees 190 191 Every module has implicit assumptions and guarantees that define what data it expects and what data it produces for consumers. 192 193 - **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. 194 - **Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. 195 196 We recommend using [custom conditions](/language/expressions/custom-conditions) help capture and test for assumptions and guarantees. This helps future maintainers understand the configuration design and intent. Custom conditions also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. 197 198 The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume. 199 200 ```hcl 201 output "api_base_url" { 202 value = "https://${aws_instance.example.private_dns}:8433/" 203 204 # The EC2 instance must have an encrypted root volume. 205 precondition { 206 condition = data.aws_ebs_volume.example.encrypted 207 error_message = "The server's root volume is not encrypted." 208 } 209 } 210 ``` 211 212 ## Multi-cloud Abstractions 213 214 Terraform itself intentionally does not attempt to abstract over similar 215 services offered by different vendors, because we want to expose the full 216 functionality in each offering and yet unifying multiple offerings behind a 217 single interface will tend to require a "lowest common denominator" approach. 218 219 However, through composition of Terraform modules it is possible to create 220 your own lightweight multi-cloud abstractions by making your own tradeoffs 221 about which platform features are important to you. 222 223 Opportunities for such abstractions arise in any situation where multiple 224 vendors implement the same concept, protocol, or open standard. For example, 225 the basic capabilities of the domain name system are common across all vendors, 226 and although some vendors differentiate themselves with unique features such 227 as geolocation and smart load balancing, you may conclude that in your use-case 228 you are willing to eschew those features in return for creating modules that 229 abstract the common DNS concepts across multiple vendors: 230 231 ```hcl 232 module "webserver" { 233 source = "./modules/webserver" 234 } 235 236 locals { 237 fixed_recordsets = [ 238 { 239 name = "www" 240 type = "CNAME" 241 ttl = 3600 242 records = [ 243 "webserver01", 244 "webserver02", 245 "webserver03", 246 ] 247 }, 248 ] 249 server_recordsets = [ 250 for i, addr in module.webserver.public_ip_addrs : { 251 name = format("webserver%02d", i) 252 type = "A" 253 records = [addr] 254 } 255 ] 256 } 257 258 module "dns_records" { 259 source = "./modules/route53-dns-records" 260 261 route53_zone_id = var.route53_zone_id 262 recordsets = concat(local.fixed_recordsets, local.server_recordsets) 263 } 264 ``` 265 266 In the above example, we've created a lightweight abstraction in the form of 267 a "recordset" object. This contains the attributes that describe the general 268 idea of a DNS recordset that should be mappable onto any DNS provider. 269 270 We then instantiate one specific _implementation_ of that abstraction as a 271 module, in this case deploying our recordsets to Amazon Route53. 272 273 If we later wanted to switch to a different DNS provider, we'd need only to 274 replace the `dns_records` module with a new implementation targeting that 275 provider, and all of the configuration that _produces_ the recordset 276 definitions can remain unchanged. 277 278 We can create lightweight abstractions like these by defining Terraform object 279 types representing the concepts involved and then using these object types 280 for module input variables. In this case, all of our "DNS records" 281 implementations would have the following variable declared: 282 283 ```hcl 284 variable "recordsets" { 285 type = list(object({ 286 name = string 287 type = string 288 ttl = number 289 records = list(string) 290 })) 291 } 292 ``` 293 294 While DNS serves as a simple example, there are many more opportunities to 295 exploit common elements across vendors. A more complex example is Kubernetes, 296 where there are now many different vendors offering hosted Kubernetes clusters 297 and even more ways to run Kubernetes yourself. 298 299 If the common functionality across all of these implementations is sufficient 300 for your needs, you may choose to implement a set of different modules that 301 describe a particular Kubernetes cluster implementation and all have the common 302 trait of exporting the hostname of the cluster as an output value: 303 304 ```hcl 305 output "hostname" { 306 value = azurerm_kubernetes_cluster.main.fqdn 307 } 308 ``` 309 310 You can then write _other_ modules that expect only a Kubernetes cluster 311 hostname as input and use them interchangeably with any of your Kubernetes 312 cluster modules: 313 314 ```hcl 315 module "k8s_cluster" { 316 source = "modules/azurerm-k8s-cluster" 317 318 # (Azure-specific configuration arguments) 319 } 320 321 module "monitoring_tools" { 322 source = "modules/monitoring_tools" 323 324 cluster_hostname = module.k8s_cluster.hostname 325 } 326 ``` 327 328 ## Data-only Modules 329 330 Most modules contain `resource` blocks and thus describe infrastructure to be 331 created and managed. It may sometimes be useful to write modules that do not 332 describe any new infrastructure at all, but merely retrieve information about 333 existing infrastructure that was created elsewhere using 334 [data sources](/language/data-sources). 335 336 As with conventional modules, we suggest using this technique only when the 337 module raises the level of abstraction in some way, in this case by 338 encapsulating exactly how the data is retrieved. 339 340 A common use of this technique is when a system has been decomposed into several 341 subsystem configurations but there is certain infrastructure that is shared 342 across all of the subsystems, such as a common IP network. In this situation, 343 we might write a shared module called `join-network-aws` which can be called 344 by any configuration that needs information about the shared network when 345 deployed in AWS: 346 347 ```hcl 348 module "network" { 349 source = "./modules/join-network-aws" 350 351 environment = "production" 352 } 353 354 module "k8s_cluster" { 355 source = "./modules/aws-k8s-cluster" 356 357 subnet_ids = module.network.aws_subnet_ids 358 } 359 ``` 360 361 The `network` module itself could retrieve this data in a number of different 362 ways: it could query the AWS API directly using 363 [`aws_vpc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) 364 and 365 [`aws_subnet_ids`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet_ids) 366 data sources, or it could read saved information from a Consul cluster using 367 [`consul_keys`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/data-sources/keys), 368 or it might read the outputs directly from the state of the configuration that 369 manages the network using 370 [`terraform_remote_state`](/language/state/remote-state-data). 371 372 The key benefit of this approach is that the source of this information can 373 change over time without updating every configuration that depends on it. 374 Furthermore, if you design your data-only module with a similar set of outputs 375 as a corresponding management module, you can swap between the two relatively 376 easily when refactoring.