github.com/terraform-linters/tflint@v0.51.2-0.20240520175844-3750771571b6/docs/developer-guide/architecture.md (about)

     1  # Architecture
     2  
     3  TFLint is a pluggable linter and does not contain any rule implementations. Rules are provided as plugins, these are launched by TFLint as subprocesses and communicate over gRPC.
     4  
     5  An important part of understanding TFLint's architecture is that TFLint (host) and a plugin act as both gRPC server/client.
     6  
     7  For example, the plugin (client) to the host (server) requests to:
     8  
     9  - Retrieve Terraform configs (e.g. `aws_instance.main`)
    10  - Evaluate expressions (e.g. `var.foo`)
    11  - Save reported issues to the host server
    12  
    13  The host (client) to the plugin (server) requests to:
    14  
    15  - Apply plugin configs
    16  - Request to run inspections
    17  
    18  The plugin system is implemented by [TFLint plugin SDK](https://github.com/terraform-linters/tflint-plugin-sdk). If you want to know more about `*.proto` and detailed gRPC server/client implementation, check out the SDK.
    19  
    20  ## Inspection Flow Diagram
    21  
    22  The following diagram explains how an inspection is performed when a user executes command:
    23  
    24  ```mermaid
    25  flowchart TB
    26    CLI["CLI<br/>(cmd package)"]-->configLoad["Load TFLint config<br/>(tflint.LoadConfig)"]
    27    configLoad-->tfLoad["Load Terraform config<br/>(terraform.LoadConfig)"]
    28    tfLoad-->discoverPlugin["Discover plugins<br/>(plugin.Discovery)"]
    29    discoverPlugin-->launchPlugin["Launch plugin processes<br/>(go-plugin)"]
    30    launchPlugin-->startRulesetServer["Start RuleSet gRPC server<br/>(go-plugin)"]
    31    launchPlugin-->applyPluginConfig["Apply configs to plugins<br/>(ruleset.ApplyConfig)"]
    32    applyPluginConfig-->requestCheck["Request inspection to plugins<br/>(ruleset.Check)"]
    33    requestCheck-->startRunnerServer["Start Runner gRPC server<br/>(tflint-plugin-sdk)"]
    34    requestCheck-->printIssues["Print issues<br/>(formatter.Print)"]
    35    subgraph goroutie
    36      startRunnerServer-->respondTerraformConfig["Respond to requests for Terraform config<br/>(plugin.GRPCServer)"]
    37      respondTerraformConfig-->saveIssues["Save issues emitted by plugins<br/>(plugin.GRPCServer)"]
    38    end
    39    subgraph plugin
    40      startRulesetServer-->appliedPluginConfig["Apply plugin config sent from TFLint<br/>(tflint-plugin-sdk)"]
    41      appliedPluginConfig-->respondCheck["Respond to inspection request<br/>(tflint-plugin-sdk)"]
    42      respondCheck-->runInspection["Run inspection on each rule<br/>(tflint-plugin-sdk)"]
    43      runInspection-->requestTerraformConfig["Request to get Terraform config<br/>(tflint-plugin-sdk)"]
    44      requestTerraformConfig-->requestEmitIssue["Emit issues to TFLint<br/>(tflint-plugin-sdk)"]
    45    end
    46    requestEmitIssue-->printIssues
    47    saveIssues-->printIssues
    48  ```
    49  
    50  ### CLI (`cmd` package)
    51  
    52  [The `cmd` package](https://github.com/terraform-linters/tflint/tree/master/cmd) is the entrypoint of the CLI. `cmd.CLI` has streams to stdout/stderr and prints a result to the screen.
    53  
    54  Depending on the user's instructions, it does the following:
    55  
    56  - Run inspection
    57  - Initialize TFLint (install plugins)
    58  - Start language server
    59  - Print version info
    60  - Run bundled plugin (internal use)
    61  
    62  This package is responsible for parsing CLI flags and arguments. The parsed `cmd.Option` is converted to `tflint.Config` and merged with a config file.
    63  
    64  ### Load TFLint config (`tflint.LoadConfig`)
    65  
    66  [The `tflint` package](https://github.com/terraform-linters/tflint/tree/master/tflint) provides many features related to TFLint, such as loading a config file (`.tflint.hcl`) and parsing annotations (`# tflint-ignore` comments).
    67  
    68  The `tflint.LoadConfig` loads a config file and returns `tflint.Config`. This config will be used in later steps.
    69  
    70  ### Load Terraform config (`terraform.LoadConfig`)
    71  
    72  [The `terraform` package](https://github.com/terraform-linters/tflint/tree/master/terraform) is a fork of [github.com/hashicorp/terraform/internal](https://pkg.go.dev/github.com/hashicorp/terraform/internal). This package is responsible for processing the Terraform semantics, such as parsing `*.tf` files, evaluating expressions, and loading modules.
    73  
    74  The `terraform.LoadConfig` reads `*.tf` files as a `terraform.Config` in the given directory. These structures are designed to be as similar to Terraform core. See "The Design of `terraform` Package" section below for details.
    75  
    76  ### Discover plugins (`plugin.Discovery`)
    77  
    78  [The `plugin` package](https://github.com/terraform-linters/tflint/tree/master/plugin) is responsible for the plugin system. This package contains gRPC server implementation, installation, discovery, etc.
    79  
    80  The `plugin.Discovery` discovers installed plugins and launches plugin binaries as subprocesses. This detailed implementation is hidden by [github.com/hashicorp/go-plugin](https://github.com/hashicorp/go-plugin).
    81  
    82  ### Launch plugin processes (go-plugin)
    83  
    84  The go-plugin launches a plugin binary as a subprocess. The launched plugin acts as a gRPC server and communicates with the host process.
    85  
    86  The plugin server is called "RuleSet" server. Its behavior is implemented by a plugin developer.
    87  
    88  ### Apply configs to plugins (`ruleset.ApplyConfig`)
    89  
    90  The `plugin.Discovery` returns a client for the RuleSet server it started. Use the `ApplyConfig` method to send the plugin config described in the config file to the server.
    91  
    92  ### Request inspection to plugins (`ruleset.Check`)
    93  
    94  Similarly, send inspection requests to the server. The server responds to requests and runs an inspection, but the plugin needs access to `terraform.Config` (imagine `runner.GetResourceContent`).
    95  
    96  For this, the host process launches a gRPC server to respond such requests. Ths host server is called "Runner" server. Its behavior is implemented by TFLint. The plugin is passed a client that corresponds to the Runner server.
    97  
    98  ### Respond to requests for Terraform config (`plugin.GRPCServer`)
    99  
   100  The Runner server responds to requests from plugins to retrieve Terraform configs, evaluate expressions, etc. This implementation is contained in the `plugin` package.
   101  
   102  ### Save issues emitted by plugins (`plugin.GRPCServer`)
   103  
   104  The Runner server saves issues emitted by plugins (imagine `runner.EmitIssue`). The saved issues will be printed to the screen in the next step.
   105  
   106  ### Print issues (`formatter.Print`)
   107  
   108  [The `formatter` package](https://github.com/terraform-linters/tflint/tree/master/formatter)  processes and outputs issues in formats such as default, JSON, and SARIF.
   109  
   110  ## Inspection Sequence Diagram
   111  
   112  The following diagram explains how the host process, RuleSet server, and Runner server each behave in inspection:
   113  
   114  ```mermaid
   115  sequenceDiagram
   116    participant TFLint as TFLint (host)
   117    participant RuleSet as RuleSet (plugin)
   118    participant Runner as Runner (host as goroutie)
   119    TFLint->>RuleSet: Start server
   120    TFLint->>RuleSet: Apply plugin configs
   121    TFLint->>Runner: Start server
   122    TFLint->>RuleSet: Request to run inspection
   123    RuleSet->>Runner: Request to get Terraform configs
   124    Runner-->>RuleSet: Return resources/modules
   125    RuleSet->>Runner: Request to evaluate expressions
   126    Runner-->>RuleSet: Return evaluated values
   127    RuleSet->>Runner: Emit issues
   128    RuleSet-->>TFLint: End of inspection
   129    Runner-->>TFLint: Return issues
   130  ```
   131  
   132  ## The Design of `terraform` Package
   133  
   134  The `terraform` package is a fork of Terraform internal packages. It is based on the same Terraform core, but with some implementation changes specifically for static analysis.
   135  
   136  The following diagram explains the dependencies of each component:
   137  
   138  ```mermaid
   139  flowchart TB
   140    files["Terraform config files<br/>(*.tf)"]-->parser["Language parser<br/>(terraform.Parser)"]
   141    subgraph child module
   142      modFiles["Terraform config files<br/>(*.tf)"]-->modParser["Language parser<br/>(terraform.Parser)"]
   143    end
   144    modParser-->modWalker["Module walker<br/>(terraform.ModuleWalker)"]  
   145    parser-- terraform.Module -->loader["Config loader<br/>(terraform.LoadConfig)"]
   146    modWalker-- terraform.Module -->loader
   147    loader-- terraform.Config -->gatherVars["Gather variables<br/>(terraform.VariableValues)"]
   148    valuesFile["Variable definitions files<br/>(*.tfvars)"]-- terraform.InputValues -->gatherVars
   149    cliFlag["CLI flags<br/>(--var)"]-- terraform.InputValues -->gatherVars
   150    gatherVars-- cty.Value -->evaluator["Evaluator<br/>(terraform.Evaluator)"]
   151    loader-- terraform.Config -->evaluator
   152  ```
   153  
   154  ### HCL and cty
   155  
   156  The underlying technologies of Terraform language are [HCL](https://github.com/hashicorp/hcl) and [cty](https://github.com/zclconf/go-cty). The Terraform language implements its own semantics on top of these.
   157  
   158  TFLint also leverages these technologies, making it highly compatible with Terraform.
   159  
   160  ### Differences from Terraform
   161  
   162  The basic architecture is the same as Terraform. The biggest difference is that the `terraform.Evaluator` is state (`terraform.State`) independent. See also [Terraform Core Architecture Summary
   163  ](https://github.com/hashicorp/terraform/blob/main/docs/architecture.md).
   164  
   165  State is not always available for static analysis as it can only be obtained by running terraform plan/apply. TFLint solves this problem in the same way as Terraform. Terraform has the ability to handle unknown values during first-time planning, and TFLint take advantage of this to always treat dynamic values as unknowns.
   166  
   167  It has the same basic design as Terraform, so it has the advantage of being able to easily support future language extensions of Terraform.
   168  
   169  ### Lazy Schema Evaluation
   170  
   171  Terraform has a predefined schema and uses it to decode HCL body. This is necessary to strictly define the syntax, but TFLint does not necessarily require a strict schema in order to support many versions of the Terraform language.
   172  
   173  Against this background, `terraform.Module` in TFLint decodes minimal structure (e.g. `terraform.Variable`, `terraform.Resource`). The `terraform.Module` has native `hcl.File` and returns bodies based on the schema requested during inspection. In other words, it does lazy schema evaluation to achieve syntactic robustness.
   174  
   175  For this reason, TFLint take a different approach to implementing `for_each` and `count` meta-arguments, and dynamic blocks than Terraform. Instead of being pre-expanded, it is expanded as an extension of `hcl.Body` when extracted by the schema.