github.com/jaylevin/jenkins-library@v1.230.4/documentation/docs/infrastructure/vault.md (about)

     1  # Vault for Pipeline Secrets
     2  
     3  Project "Piper" supports fetching your pipeline secrets directly from [Vault](https://www.hashicorp.com/products/vault).
     4  Currently, Vault's key value engine is supported in version 1 and 2, although we recommend version 2 since it supports
     5  the versioning of secrets
     6  
     7  Parameters that support being fetched from Vault are marked with the Vault Label in the Step Documentation.
     8  
     9  ![Vault Label](../images/parameter-with-vault-support.png)
    10  
    11  ## Authenticating Piper to Vault
    12  
    13  Piper currently supports Vault's `AppRole` and `Token` authentication. However, `AppRole` authentication is recommended
    14  since Piper is able to regularly rotate the SecretID, which is not possible with a Token.
    15  
    16  ### AppRole Authentication
    17  
    18  To authenticate against Vault, using [AppRole](https://www.vaultproject.io/docs/auth/approle) authentication you need to
    19  do the following things
    20  
    21  - Enable AppRole authentication in your Vault instance.
    22  - After that you have
    23    to [create an AppRole Role](https://www.vaultproject.io/api-docs/auth/approle#create-update-approle) for Piper
    24  - Assign the necessary policies to your newly created AppRole.
    25  - Take the **AppRole ID** and create a Jenkins `Secret Text` credential.
    26  - Take the **AppRole Secret ID** and create a Jenkins `Secret Text` credential.
    27  
    28  ![Create two jenkins secret text credentials](../images/jenkins-vault-credential.png)
    29  
    30  ### Token Authentication
    31  
    32  First step to use Token authentication is
    33  to [Create a Vault Token](https://www.vaultproject.io/api/auth/token#create-token)
    34  In order to use a Vault token for authentication you need to store the Vault token inside your Jenkins instance as shown
    35  below.
    36  
    37  ![Create a Jenkins secret text credential](../images/jenkins-vault-token-credential.png)
    38  
    39  ## Setup a Secret Store in Vault
    40  
    41  The first step to store your pipeline secrets in Vault, is to enable a the
    42  [Key-Value Engine](https://www.vaultproject.io/docs/secrets/kv/kv-v2). Then create a policy which grants read access to
    43  the key value engine.
    44  
    45  ![Enable a new secret engine in vault](../images/vault-secret-engine-enable.png)
    46  
    47  ## Pipeline Configuration
    48  
    49  For pipelines to actually use the secrets stored in Vault you need to adjust your `config.yml`:
    50  
    51  ```yml
    52  general:
    53    ...
    54    vaultServerUrl: '<YOUR_VAULT_SERVER_URL>'
    55    vaultNamespace: '<YOUR_NAMESPACE_NAME>' # if you are not using vault's namespace feature you can remove this line
    56    vaultPath: 'kv/my-pipeline' # the path under which your jenkins secrets are stored
    57    ...
    58  ```
    59  
    60  To authenticate you need to provide `PIPER_vaultAppRoleID` and `PIPER_vaultAppRoleSecretID` if you use app role authentication or `PIPER_vaultToken` if you use token authentication.
    61  
    62  !!! note "Jenkins"
    63      When running a step via the Jenkins library you can use Jenkins credentials for pass this values. Use `vaultAppRoleTokenCredentialsId` and `vaultAppRoleSecretTokenCredentialsId` or `vaultTokenCredentialsId` in your `config.yml`.
    64  
    65  ## Configuring the Secret Lookup
    66  
    67  When Piper is configured to lookup secrets in Vault, there are some aspects that need to be considered.
    68  
    69  ### Overwriting of Parameters
    70  
    71  Whenever a parameter is provided via `config.yml` or passed to the CLI it gets overwritten when a secret is found in
    72  Vault. To disable overriding parameters put a `vaultDisableOverwrite: false` on `Step` `Stage` or `General` Section in
    73  your config.
    74  
    75  ```yaml
    76  general:
    77    ...
    78    vaultDisableOverwrite: true
    79    ...
    80  steps:
    81    executeBuild:
    82      vaultDisableOverwrite: false
    83      ...
    84  ```
    85  
    86  ### Skipping Vault Secret Lookup
    87  
    88  It is also possible to skip Vault for `Steps`, `Stages` or in `General` by using the `skipVault` config parameter as
    89  shown below.
    90  
    91  ```yaml
    92  ...
    93  steps:
    94    executeBuild:
    95      skipVault: true   # Skip Vault Secret Lookup for this step
    96  ```
    97  
    98  ## Using Vault for general purpose and test credentials
    99  
   100  Vault can be used with piper to fetch any credentials, e.g. when they need to be appended to custom piper extensions or when they need to be appended to test command. The configuration for Vault general purpose credentials can be added to **any** piper golang-based step. The configuration has to be done as follows:
   101  
   102  ```yaml
   103  general:
   104    < your Vault configuration > # see above
   105  ...
   106  steps:
   107    < piper go step >:
   108      vaultCredentialPath: 'myStepCredentials'
   109      vaultCredentialKeys: ['myAppId', 'myAppSecret']
   110  ```
   111  
   112  The `vaultCredentialPath` parameter is the endpoint of your credential path in Vault. Depending on your _general_ config, the lookup for the credential IDs will be done in the following order respectively locations. The first path with found general purpose credentials will be used.
   113  
   114  1. `<vaultPath>/<vaultCredentialPath>`
   115  2. `<vaultBasePath>/<vaultPipelineName>/<vaultCredentialPath>`
   116  3. `<vaultBasePath>/GROUP-SECRETS/<vaultCredentialPath>`
   117  
   118  The `vaultCredentialKeys`parameter is a list of credential IDs. The secret value of the credential will be exposed as an environment variable prefixed by "PIPER_VAULTCREDENTIAL_" and transformed to a valid variable name. For a credential ID named `myAppId` the forwarded environment variable to the step will be `PIPER_VAULTCREDENTIAL_MYAPPID` containing the secret. The Base64 encoded secret value will be exposed as environment variable to the step as  `PIPER_VAULTCREDENTIAL_MYAPPID_BASE64`. Hyphens will be replaced by underscores and other non-alphanumeric characters will be removed.
   119  
   120  !!! hint "Using a custom prefix for test credentials"
   121      By default the prefix for test credentials is `PIPER_VAULTCREDENTIAL_`.
   122  
   123      It is possible to use a custom prefix by setting for example `vaultCredentialEnvPrefix: MY_CUSTOM_PREFIX` in your configuration.
   124      With this above credential ID named `myAppId` will be populated into an environment variable with the name `MY_CUSTOM_PREFIX_MYAPPID`.
   125  
   126  Extended logging for Vault secret fetching (e.g. found credentials and environment variable names) can be activated via `verbose: true` configuration.
   127  
   128  ## Using Vault for test credentials (Deprecated : use general purpose and test credentials as above)
   129  
   130  Vault can be used with piper to fetch any credentials, e.g. when they need to be appended to test command. The configuration for Vault test credentials can be added to **any** piper golang-based step. The configuration has to be done as follows:
   131  
   132  ```yaml
   133  general:
   134    < your Vault configuration > # see above
   135  ...
   136  steps:
   137    < piper go step >:
   138      vaultTestCredentialPath: 'myTestStepCrecetials'
   139      vaultTestCredentialKeys: ['myAppId', 'myAppSecret']
   140  ```
   141  
   142  The `vaultTestCredentialPath` parameter is the endpoint of your credential path in vault. Depending on your _general_ config, the lookup for the credential IDs will be done in the following order respectively locations. The first path with found test credentials will be used.
   143  
   144  1. `<vaultPath>/<vaultTestCredentialPath>`
   145  2. `<vaultBasePath>/<vaultPipelineName>/<vaultTestCredentialPath>`
   146  3. `<vaultBasePath>/GROUP-SECRETS/<vaultTestCredentialPath>`
   147  
   148  The `vaultTestCredentialKeys`parameter is a list of credential IDs. The secret value of the credential will be exposed as an environment variable prefixed by "PIPER_TESTCREDENTIAL_" and transformed to a valid variable name. For a credential ID named `myAppId` the forwarded environment variable to the step will be `PIPER_TESTCREDENTIAL_MYAPPID` containing the secret. Hyphens will be replaced by underscores and other non-alphanumeric characters will be removed.
   149  
   150  !!! hint "Using a custom prefix for test credentials"
   151      By default the prefix for test credentials is `PIPER_TESTCREDENTIAL_`.
   152  
   153      It is possible to use a custom prefix by setting for example `vaultTestCredentialEnvPrefix: MY_CUSTOM_PREFIX_` in your configuration.
   154      With this above credential ID named `myAppId` will be populated into an environment variable with the name `MY_CUSTOM_PREFIX_MYAPPID`.
   155  
   156  Extended logging for Vault secret fetching (e.g. found credentials and environment variable names) can be activated via `verbose: true` configuration.