github.com/pulumi/terraform@v1.4.0/website/docs/language/modules/develop/composition.mdx (about)

     1  ---
     2  page_title: Module Composition
     3  description: |-
     4    Module composition allows infrastructure to be described from modular
     5    building blocks.
     6  ---
     7  
     8  # Module Composition
     9  
    10  In a simple Terraform configuration with only one root module, we create a
    11  flat set of resources and use Terraform's expression syntax to describe the
    12  relationships between these resources:
    13  
    14  ```hcl
    15  resource "aws_vpc" "example" {
    16    cidr_block = "10.1.0.0/16"
    17  }
    18  
    19  resource "aws_subnet" "example" {
    20    vpc_id = aws_vpc.example.id
    21  
    22    availability_zone = "us-west-2b"
    23    cidr_block        = cidrsubnet(aws_vpc.example.cidr_block, 4, 1)
    24  }
    25  ```
    26  
    27  When we introduce `module` blocks, our configuration becomes hierarchical
    28  rather than flat: each module contains its own set of resources, and possibly
    29  its own child modules, which can potentially create a deep, complex tree of
    30  resource configurations.
    31  
    32  However, in most cases we strongly recommend keeping the module tree flat,
    33  with only one level of child modules, and use a technique similar to the
    34  above of using expressions to describe the relationships between the modules:
    35  
    36  ```hcl
    37  module "network" {
    38    source = "./modules/aws-network"
    39  
    40    base_cidr_block = "10.0.0.0/8"
    41  }
    42  
    43  module "consul_cluster" {
    44    source = "./modules/aws-consul-cluster"
    45  
    46    vpc_id     = module.network.vpc_id
    47    subnet_ids = module.network.subnet_ids
    48  }
    49  ```
    50  
    51  We call this flat style of module usage _module composition_, because it
    52  takes multiple [composable](https://en.wikipedia.org/wiki/Composability)
    53  building-block modules and assembles them together to produce a larger system.
    54  Instead of a module _embedding_ its dependencies, creating and managing its
    55  own copy, the module _receives_ its dependencies from the root module, which
    56  can therefore connect the same modules in different ways to produce different
    57  results.
    58  
    59  The rest of this page discusses some more specific composition patterns that
    60  may be useful when describing larger systems with Terraform.
    61  
    62  ## Dependency Inversion
    63  
    64  In the example above, we saw a `consul_cluster` module that presumably describes
    65  a cluster of [HashiCorp Consul](https://www.consul.io/) servers running in
    66  an AWS VPC network, and thus it requires as arguments the identifiers of both
    67  the VPC itself and of the subnets within that VPC.
    68  
    69  An alternative design would be to have the `consul_cluster` module describe
    70  its _own_ network resources, but if we did that then it would be hard for
    71  the Consul cluster to coexist with other infrastructure in the same network,
    72  and so where possible we prefer to keep modules relatively small and pass in
    73  their dependencies.
    74  
    75  This [dependency inversion](https://en.wikipedia.org/wiki/Dependency_inversion_principle)
    76  approach also improves flexibility for future
    77  refactoring, because the `consul_cluster` module doesn't know or care how
    78  those identifiers are obtained by the calling module. A future refactor may
    79  separate the network creation into its own configuration, and thus we may
    80  pass those values into the module from data sources instead:
    81  
    82  ```hcl
    83  data "aws_vpc" "main" {
    84    tags = {
    85      Environment = "production"
    86    }
    87  }
    88  
    89  data "aws_subnet_ids" "main" {
    90    vpc_id = data.aws_vpc.main.id
    91  }
    92  
    93  module "consul_cluster" {
    94    source = "./modules/aws-consul-cluster"
    95  
    96    vpc_id     = data.aws_vpc.main.id
    97    subnet_ids = data.aws_subnet_ids.main.ids
    98  }
    99  ```
   100  
   101  ### Conditional Creation of Objects
   102  
   103  In situations where the same module is used across multiple environments,
   104  it's common to see that some necessary object already exists in some
   105  environments but needs to be created in other environments.
   106  
   107  For example, this can arise in development environment scenarios: for cost
   108  reasons, certain infrastructure may be shared across multiple development
   109  environments, while in production the infrastructure is unique and managed
   110  directly by the production configuration.
   111  
   112  Rather than trying to write a module that itself tries to detect whether something
   113  exists and create it if not, we recommend applying the dependency inversion
   114  approach: making the module accept the object it needs as an argument, via
   115  an input variable.
   116  
   117  For example, consider a situation where a Terraform module deploys compute
   118  instances based on a disk image, and in some environments there is a
   119  specialized disk image available while other environments share a common
   120  base disk image. Rather than having the module itself handle both of these
   121  scenarios, we can instead declare an input variable for an object representing
   122  the disk image. Using AWS EC2 as an example, we might declare a common subtype
   123  of the `aws_ami` resource type and data source schemas:
   124  
   125  ```hcl
   126  variable "ami" {
   127    type = object({
   128      # Declare an object using only the subset of attributes the module
   129      # needs. Terraform will allow any object that has at least these
   130      # attributes.
   131      id           = string
   132      architecture = string
   133    })
   134  }
   135  ```
   136  
   137  The caller of this module can now itself directly represent whether this is
   138  an AMI to be created inline or an AMI to be retrieved from elsewhere:
   139  
   140  ```hcl
   141  # In situations where the AMI will be directly managed:
   142  
   143  resource "aws_ami_copy" "example" {
   144    name              = "local-copy-of-ami"
   145    source_ami_id     = "ami-abc123"
   146    source_ami_region = "eu-west-1"
   147  }
   148  
   149  module "example" {
   150    source = "./modules/example"
   151  
   152    ami = aws_ami_copy.example
   153  }
   154  ```
   155  
   156  ```hcl
   157  # Or, in situations where the AMI already exists:
   158  
   159  data "aws_ami" "example" {
   160    owner = "9999933333"
   161  
   162    tags = {
   163      application = "example-app"
   164      environment = "dev"
   165    }
   166  }
   167  
   168  module "example" {
   169    source = "./modules/example"
   170  
   171    ami = data.aws_ami.example
   172  }
   173  ```
   174  
   175  This is consistent with Terraform's declarative style: rather than creating
   176  modules with complex conditional branches, we directly describe what
   177  should already exist and what we want Terraform to manage itself.
   178  
   179  By following this pattern, we can be explicit about in which situations we
   180  expect the AMI to already be present and which we don't. A future reader
   181  of the configuration can then directly understand what it is intending to do
   182  without first needing to inspect the state of the remote system.
   183  
   184  In the above example, the object to be created or read is simple enough to
   185  be given inline as a single resource, but we can also compose together multiple
   186  modules as described elsewhere on this page in situations where the
   187  dependencies themselves are complicated enough to benefit from abstractions.
   188  
   189  ## Assumptions and Guarantees
   190  
   191  Every module has implicit assumptions and guarantees that define what data it expects and what data it produces for consumers.
   192  
   193  - **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture.
   194  - **Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record.
   195  
   196  We recommend using [custom conditions](/language/expressions/custom-conditions) help capture and test for assumptions and guarantees. This helps future maintainers understand the configuration design and intent. Custom conditions also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations.
   197  
   198  The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume.
   199  
   200  ```hcl
   201  output "api_base_url" {
   202    value = "https://${aws_instance.example.private_dns}:8433/"
   203  
   204    # The EC2 instance must have an encrypted root volume.
   205    precondition {
   206      condition     = data.aws_ebs_volume.example.encrypted
   207      error_message = "The server's root volume is not encrypted."
   208    }
   209  }
   210  ```
   211  
   212  ## Multi-cloud Abstractions
   213  
   214  Terraform itself intentionally does not attempt to abstract over similar
   215  services offered by different vendors, because we want to expose the full
   216  functionality in each offering and yet unifying multiple offerings behind a
   217  single interface will tend to require a "lowest common denominator" approach.
   218  
   219  However, through composition of Terraform modules it is possible to create
   220  your own lightweight multi-cloud abstractions by making your own tradeoffs
   221  about which platform features are important to you.
   222  
   223  Opportunities for such abstractions arise in any situation where multiple
   224  vendors implement the same concept, protocol, or open standard. For example,
   225  the basic capabilities of the domain name system are common across all vendors,
   226  and although some vendors differentiate themselves with unique features such
   227  as geolocation and smart load balancing, you may conclude that in your use-case
   228  you are willing to eschew those features in return for creating modules that
   229  abstract the common DNS concepts across multiple vendors:
   230  
   231  ```hcl
   232  module "webserver" {
   233    source = "./modules/webserver"
   234  }
   235  
   236  locals {
   237    fixed_recordsets = [
   238      {
   239        name = "www"
   240        type = "CNAME"
   241        ttl  = 3600
   242        records = [
   243          "webserver01",
   244          "webserver02",
   245          "webserver03",
   246        ]
   247      },
   248    ]
   249    server_recordsets = [
   250      for i, addr in module.webserver.public_ip_addrs : {
   251        name    = format("webserver%02d", i)
   252        type    = "A"
   253        records = [addr]
   254      }
   255    ]
   256  }
   257  
   258  module "dns_records" {
   259    source = "./modules/route53-dns-records"
   260  
   261    route53_zone_id = var.route53_zone_id
   262    recordsets      = concat(local.fixed_recordsets, local.server_recordsets)
   263  }
   264  ```
   265  
   266  In the above example, we've created a lightweight abstraction in the form of
   267  a "recordset" object. This contains the attributes that describe the general
   268  idea of a DNS recordset that should be mappable onto any DNS provider.
   269  
   270  We then instantiate one specific _implementation_ of that abstraction as a
   271  module, in this case deploying our recordsets to Amazon Route53.
   272  
   273  If we later wanted to switch to a different DNS provider, we'd need only to
   274  replace the `dns_records` module with a new implementation targeting that
   275  provider, and all of the configuration that _produces_ the recordset
   276  definitions can remain unchanged.
   277  
   278  We can create lightweight abstractions like these by defining Terraform object
   279  types representing the concepts involved and then using these object types
   280  for module input variables. In this case, all of our "DNS records"
   281  implementations would have the following variable declared:
   282  
   283  ```hcl
   284  variable "recordsets" {
   285    type = list(object({
   286      name    = string
   287      type    = string
   288      ttl     = number
   289      records = list(string)
   290    }))
   291  }
   292  ```
   293  
   294  While DNS serves as a simple example, there are many more opportunities to
   295  exploit common elements across vendors. A more complex example is Kubernetes,
   296  where there are now many different vendors offering hosted Kubernetes clusters
   297  and even more ways to run Kubernetes yourself.
   298  
   299  If the common functionality across all of these implementations is sufficient
   300  for your needs, you may choose to implement a set of different modules that
   301  describe a particular Kubernetes cluster implementation and all have the common
   302  trait of exporting the hostname of the cluster as an output value:
   303  
   304  ```hcl
   305  output "hostname" {
   306    value = azurerm_kubernetes_cluster.main.fqdn
   307  }
   308  ```
   309  
   310  You can then write _other_ modules that expect only a Kubernetes cluster
   311  hostname as input and use them interchangeably with any of your Kubernetes
   312  cluster modules:
   313  
   314  ```hcl
   315  module "k8s_cluster" {
   316    source = "modules/azurerm-k8s-cluster"
   317  
   318    # (Azure-specific configuration arguments)
   319  }
   320  
   321  module "monitoring_tools" {
   322    source = "modules/monitoring_tools"
   323  
   324    cluster_hostname = module.k8s_cluster.hostname
   325  }
   326  ```
   327  
   328  ## Data-only Modules
   329  
   330  Most modules contain `resource` blocks and thus describe infrastructure to be
   331  created and managed. It may sometimes be useful to write modules that do not
   332  describe any new infrastructure at all, but merely retrieve information about
   333  existing infrastructure that was created elsewhere using
   334  [data sources](/language/data-sources).
   335  
   336  As with conventional modules, we suggest using this technique only when the
   337  module raises the level of abstraction in some way, in this case by
   338  encapsulating exactly how the data is retrieved.
   339  
   340  A common use of this technique is when a system has been decomposed into several
   341  subsystem configurations but there is certain infrastructure that is shared
   342  across all of the subsystems, such as a common IP network. In this situation,
   343  we might write a shared module called `join-network-aws` which can be called
   344  by any configuration that needs information about the shared network when
   345  deployed in AWS:
   346  
   347  ```hcl
   348  module "network" {
   349    source = "./modules/join-network-aws"
   350  
   351    environment = "production"
   352  }
   353  
   354  module "k8s_cluster" {
   355    source = "./modules/aws-k8s-cluster"
   356  
   357    subnet_ids = module.network.aws_subnet_ids
   358  }
   359  ```
   360  
   361  The `network` module itself could retrieve this data in a number of different
   362  ways: it could query the AWS API directly using
   363  [`aws_vpc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc)
   364  and
   365  [`aws_subnet_ids`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet_ids)
   366  data sources, or it could read saved information from a Consul cluster using
   367  [`consul_keys`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/data-sources/keys),
   368  or it might read the outputs directly from the state of the configuration that
   369  manages the network using
   370  [`terraform_remote_state`](/language/state/remote-state-data).
   371  
   372  The key benefit of this approach is that the source of this information can
   373  change over time without updating every configuration that depends on it.
   374  Furthermore, if you design your data-only module with a similar set of outputs
   375  as a corresponding management module, you can swap between the two relatively
   376  easily when refactoring.