github.com/yanndegat/hiera@v0.6.8/README.md (about) 1 # Hiera lookup framework 2 3 [](https://goreportcard.com/report/github.com/yanndegat/hiera) 4 [](https://godoc.org/github.com/yanndegat/hiera) 5 [](https://github.com/yanndegat/hiera/actions) 6 7 ## Introduction 8 9 Hiera is a flexible, powerful tool for resolving values for variable lookups, which was first popularised by its use in [Puppet](https://puppet.com/docs/puppet/5.5/hiera.html). 10 11 This module is a "clean-room" Go implementation of the Hiera framework, suitable for use as a library from other tools. 12 13 ## Details 14 15 Hiera uses the concept of "managing by exception": you design a *hierarchy* of data sources, with the most specific source at the top and least-specific defaults at the bottom. Hiera searches for keys starting at the top, allowing more-specific sources to override defaults. Sources are usually YAML files stored on the filesystem, and layers usually use variable interpolation to find the right file, allowing the context of the lookup to pick the right file. 16 17 ## How to run it 18 19 ### Standalone execution 20 21 #### Install the module 22 23 To install the module under $GOPATH/src: 24 25 go get github.com/yanndegat/hiera 26 27 #### Install the lookup binary 28 29 Install the lookup binary under $GOPATH/bin: 30 31 go install github.com/yanndegat/hiera/lookup 32 33 #### Run the binary 34 35 lookup --help 36 37 38 ## Pass values for interpolation 39 40 If your hierarchy config contains variable interpolation, you can provide context for the lookup using the `var` query parameter. Repeated `var` parameters will create an array of available parameters. The values should be colon-separated variable-value pairs: 41 42 curl 'http://localhost:8080/lookup/aws.tags?var=environment:production&var=hostname:specialhost' 43 44 TODO: Nested variable lookups such like `os.family` are not yet working. 45 46 ## Hiera configuration and directory structure 47 48 Much of hiera's power lies in its ability to interpolate variables in the hierarchy's configuration. A lookup provides values, and hiera maps the interpolated values onto the filesystem (or other back-end data structure). A common example uses two levels of override: one for specific hosts, a higher layer for environment-wide settings, and finally a fall-through default. A functional `hiera.yaml` which implements this policy looks like: 49 50 --- 51 version: 5 52 defaults: 53 datadir: hiera 54 data_hash: yaml_data 55 56 hierarchy: 57 - name: "Host-specific overrides" 58 path: "hosts/%{hostname}.yaml" 59 - name: "Environmental overrides" 60 path: "environments/%{environment}.yaml" 61 - name: "Fall through defaults" 62 path: "defaults.yaml" 63 64 This maps to a directory structure based in the `hiera` subdirectory (due to the `datadir` top level key) containing yaml files like: 65 66 hiera 67 ├── defaults.yaml 68 ├── environments 69 │ └── production.yaml 70 └── hosts 71 └── specialhost.yaml 72 73 ## Extending Hiera 74 75 When Hiera performs a lookup it uses a lookup function. Unless the function embedded in the hiera binary, it will 76 make an attempt to load a RESTful Plugin that provides the needed function. Such plugins are enabled by using the API 77 provided by the [hierasdk library](https://github.com/lyraproj/hierasdk). 78 79 #### How Hiera finds its plugins 80 The resolution of a plugin can be controlled using a "plugindir" key in the Hiera configuration file. As with "datadir", 81 the "plugindir" can be specified both in the defaults hierarchy or explicitly in a specific hierarchy. Unless specified, 82 the "plugindir" is assumed to be the directory "plugin" present in the same directory as the configuration file. 83 84 In addition to "plugindir", a hierarchy may also specify a "pluginfile". Unless specified, the "pluginfile" is assumed 85 to be equal to the name of the lookup function (with the extension ".exe" in case of Windows). 86 87 ## Environment Variables 88 89 The following environment variables can be set as an alternative to CLI options. 90 91 * `HIERA_CONFIGFILE` - `--config` 92 93 Values passed as CLI options will take precendence over the environment variables. 94 95 The following environment variables can be set as an alternative to setting values in the `defaults` hash. 96 97 * `HIERA_DATADIR` - `datadir` 98 * `HIERA_PLUGINDIR` - `plugindir` 99 100 Values set in `hiera.yaml` will take precedence over the environment variables. 101 102 ### Containerized extension 103 104 In order to include an extension in a Hiera Docker image you need to: 105 106 1. Copy the source (or clone the git repository) of the desired extensions into the hiera plugin directory (don't worry, 107 this directory will be ignored by git when doing commits to hiera). 108 2. For each extension, add a line like the following line to the Hiera Dockerfile below the comment 109 `# Add plugin builds here`: 110 ``` 111 RUN (cd plugin/hiera_terraform && go build -o ../terraform_backend) 112 ``` 113 3. Run the docker build. 114 115 ### Useful extensions 116 117 * [Azure Key Vault lookup_key](https://github.com/lyraproj/hiera_azure). Allows you to lookup single values stored as 118 secrets from the Azure Key Vault. 119 * [Terraform Backend data_hash](https://github.com/lyraproj/hiera_terrform). Allows hiera to query data from a Terraform 120 backend. 121 122 ## Implementation status 123 124 * [x] lookup CLI 125 * [x] lookup function 126 * [x] lookup context 127 * [x] dotted keys (dig functionality) 128 * [x] interpolation using scope, lookup/hiera, alias, or literal function 129 * [x] Hiera version 5 configuration in hiera.yaml 130 * [x] merge strategies (first, unique, hash, deep) 131 * [x] YAML data 132 * [x] JSON data 133 * [x] lookup options stored adjacent to data 134 * [x] convert_to type coercions 135 * [x] Sensitive data 136 * [ ] configurable deep merge 137 * [x] pluggable back ends 138 * [x] `explain` functionality to show traversal 139 * [x] containerized REST-based microservice 140 * [x] JSON and YAML schema for the hiera.yaml config file (see schema/hiera_v5.yaml)