github.com/StackExchange/dnscontrol/v4@v4.11.0/documentation/ci-cd-gitlab.md (about)

     1  # GitLab CI/CD example
     2  
     3  Before discussing the GitLab CI/CD setup, let's assume you already have a working DNSControl setup. Aren't you there yet? Then first check out the '[Getting Started](getting-started.md)' section.
     4  
     5  ## DNSControl - Demo setup
     6  
     7  For this tutorial, there is a [GitLab repository](https://gitlab.com/cafferata/dnscontrol/) ready with an example DNSControl setup/domain.
     8  
     9  This is based on:
    10  - The domain `cafferata.dev`.
    11  - The DNS provider [TransIP](https://www.transip.nl/cp/account/api/).
    12  - The TransIP account `cafferatax`.
    13  
    14  For convenience, both configuration files are shown below.
    15  
    16  {% code title="dnsconfig.js" %}
    17  ```javascript
    18  var PROVIDER_NONE = NewRegistrar("none");
    19  var PROVIDER_TRANSIP = NewDnsProvider("transip");
    20  
    21  D("cafferata.dev",
    22      PROVIDER_NONE,
    23      DnsProvider(PROVIDER_TRANSIP),
    24      DefaultTTL("1d"),
    25      TXT("spf", [
    26          "v=spf1",
    27          "-all"
    28      ].join(" ")),
    29  END);
    30  ```
    31  {% endcode %}
    32  
    33  {% code title="creds.json" %}
    34  ```json
    35  {
    36    "transip": {
    37      "TYPE": "TRANSIP",
    38      "AccountName": "cafferatax",
    39      "PrivateKey": "$TRANSIP_PRIVATE_KEY"
    40    }
    41  }
    42  ```
    43  {% endcode %}
    44  
    45  ## GitLab CI - Preparation
    46  
    47  You may have noticed that the `creds.json` file contains a variable `$TRANSIP_PRIVATE_KEY`. This variable is populated from the GitLab CI variables and contain the TransIP API key.
    48  
    49  ```text
    50  -----BEGIN PRIVATE KEY-----
    51  MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDgJjUQcUrijxul
    52  NbbN+KD3tDISPDczuJW8HYZRafOc9BgHWuYiBcE+1+9mzHi7FOujuHuTBD84vvRN
    53  x9buaOLoJNjCvtHDtotYQMhUWF8bnNYyoIIdJGXXcRG0G0+NNreVQiZ+5IQIvhRU
    54  PXyjvUdQoIaRc9S5NxAMbGLJUHtZ4LEdJKa7HmCXAA5dBYaMMSZjpdBJImSJAtxj
    55  msseUkc0EiznGgsBVMOkn3VMba2gjo5RDGbfXXLUX3DTOKdwY+hdWktG/gO/3D9l
    56  hY/gnT/MmXXko3YAcI4eQL8=
    57  -----END PRIVATE KEY-----
    58  ```
    59  
    60  _Example of variable `$TRANSIP_PRIVATE_KEY` contents._
    61  
    62  ![GitLab CI/CD variables](assets/ci-cd-gitlab/settings-ci-cd-variables.png)
    63  
    64  ![Insert GitLab CI/CD variable TRANSIP_PRIVATE_KEY](assets/ci-cd-gitlab/settings-ci-cd-variables-insert.png)
    65  
    66  ## GitLab CI - DNSControl preview
    67  
    68  Now it's time to apply the power of DNSControl within GitLab CI merge requests. We'll start by adding the basic GitLab CI setup. You can view the git diff online in the [GitLab merge request #1](https://gitlab.com/cafferata/dnscontrol/-/merge_requests/1/diffs). The GitLab CI setup has also been added for convenience.
    69  
    70  `.gitlab-ci.yml`
    71  
    72  ```yaml
    73  dnscontrol-preview:
    74    stage: 'test'
    75    image:
    76      name: 'stackexchange/dnscontrol'
    77      entrypoint: ['']
    78    script:
    79      - '/usr/local/bin/dnscontrol version'
    80      - '/usr/local/bin/dnscontrol check'
    81      - '/usr/local/bin/dnscontrol preview'
    82    rules:
    83      - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    84        changes:
    85          - 'dnsconfig.js'
    86  ```
    87  
    88  What does this YAML configuration mean?
    89  
    90  - The `dnscontrol preview` is run within the GitLab CI [predefined stage](https://docs.gitlab.com/ee/ci/yaml/#stages) `test` using the Docker image [stackexchange/dnscontrol](https://hub.docker.com/r/stackexchange/dnscontrol).
    91    - A conscious decision has been made to always use the latest version so that no maintenance is required. Of course you can choose to include a Docker image version. You do this by choosing from the [available versions](https://hub.docker.com/r/stackexchange/dnscontrol/tags), and including it in `image:` for example: `name: 'stackexchange/dnscontrol:v3.20.0'`
    92  - Because the choice was made not to adopt a version, it's nice to know from the GitLab CI jobs which version DNSControl is running.
    93    We check and validate the DNSControl set-up `dnsconfig.js`.
    94  - Then we ask TransIP which DNS diff there is.
    95  - **(!)** This only happens in the context of a GitLab merge request and (_**very important**_) only when there is a change in the DNSControl configuration (`dnsconfig.js`).
    96  
    97  Because the above GitLab CI configuration expects a diff, we apply this by (_for example_) adding the Google Workspace SPF include.
    98  
    99  `dnsconfig.js`
   100  
   101  ```diff
   102  var PROVIDER_NONE = NewRegistrar("none");
   103  var PROVIDER_TRANSIP = NewDnsProvider("transip");
   104  
   105  D("cafferata.dev",
   106      PROVIDER_NONE,
   107      DnsProvider(PROVIDER_TRANSIP),
   108      DefaultTTL("1d"),
   109      TXT("spf", [
   110          "v=spf1",
   111  +       "include:_spf.google.com",
   112          "-all"
   113      ].join(" "))
   114  );
   115  ```
   116  
   117  From that moment everything comes together! Within the [GitLab merge request #1](https://gitlab.com/cafferata/dnscontrol/-/merge_requests/1), a [GitLab pipeline](https://gitlab.com/cafferata/dnscontrol/-/pipelines/656368053) with a [GitLab job](https://gitlab.com/cafferata/dnscontrol/-/jobs/3115895010) starts running containing the command `dnscontrol preview`. The outcome of this job? The desired change that will be made within TransIP. Wow this is cool!
   118  
   119  ```shell
   120  /usr/local/bin/dnscontrol version
   121  ```
   122  ```shell
   123  dnscontrol "3.20.0" ("8bb63be8f5ed996a7ae0a21091954fcab996621b") built 26 Aug 22 14:59 UTC
   124  ```
   125  ```shell
   126  /usr/local/bin/dnscontrol preview
   127  ```
   128  ```shell
   129  ******************** Domain: cafferata.dev
   130  1 correction
   131  #1: MODIFY TXT spf.cafferata.dev: ("v=spf1 -all" ttl=86400) -> ("v=spf1 include:_spf.google.com -all" ttl=86400)
   132  Done. 1 corrections.
   133  ```
   134  
   135  ![CI/CD job output for DNSControl preview](assets/ci-cd-gitlab/ci-cd-job-output-dnscontrol-preview.png)
   136  
   137  ## GitLab CI - DNSControl push
   138  
   139  We just saw that we can view the DNSControl diff from the [GitLab job](https://gitlab.com/cafferata/dnscontrol/-/jobs/3115895010). Now it's time to make GitLab CI responsible for the command `dnscontrol push`.
   140  
   141  From here several choices can be made. You can choose to have the `dnscontrol push` run as soon as a merge request is pushed to default branch (e.g. `main`), or from a GitLab pipeline trigger within the [GitLab web interface](https://gitlab.com/cafferata/dnscontrol/-/pipelines/new). We have opted for the [GitLab pipeline web interface](https://gitlab.com/cafferata/dnscontrol/-/pipelines/new) so that it cannot happen that DNS changes are made from previous merge requests in default branch (e.g. `main`).
   142  
   143  It will probably not surprise you that the basis of this GitLab YAML configuration corresponds for 90% with the DNSControl preview. See the [GitLab merge request #2](https://gitlab.com/cafferata/dnscontrol/-/merge_requests/2) here.
   144  
   145  `.gitlab-ci.yml`
   146  
   147  ```yaml
   148  dnscontrol-push:
   149    stage: 'deploy'
   150    image:
   151      name: 'stackexchange/dnscontrol'
   152      entrypoint: ['']
   153    script:
   154      - '/usr/local/bin/dnscontrol version'
   155      - '/usr/local/bin/dnscontrol push'
   156    rules:
   157      - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "web"'
   158  ```
   159  
   160  What does this (new) YAML configuration mean?
   161  
   162  - The `dnscontrol push` is run within the GitLab CI [predefined stage](https://docs.gitlab.com/ee/ci/yaml/#stages) `deploy`.
   163  - This only happens when you start a GitLab pipeline from the [GitLab web interface](https://gitlab.com/cafferata/dnscontrol/-/pipelines/new) for the default branch (e.g. `main`).
   164  
   165  ![Start new CI/CD pipeline from the GitLab web interface](assets/ci-cd-gitlab/ci-cd-pipelines-new.png)
   166  
   167  When we start the new [GitLab pipeline](https://gitlab.com/cafferata/dnscontrol/-/pipelines/656368384) from the [GitLab web interface](https://gitlab.com/cafferata/dnscontrol/-/pipelines/new), we see the GitLab job [dnscontrol-push](https://gitlab.com/cafferata/dnscontrol/-/jobs/3115896199) which makes the changes within the DNS provider TransIP.
   168  
   169  ```shell
   170  /usr/local/bin/dnscontrol version
   171  ```
   172  ```shell
   173  dnscontrol "3.20.0" ("8bb63be8f5ed996a7ae0a21091954fcab996621b") built 26 Aug 22 14:59 UTC
   174  ```
   175  ```shell
   176  /usr/local/bin/dnscontrol push
   177  ```
   178  ```shell
   179  ******************** Domain: cafferata.dev
   180  1 correction
   181  #1: MODIFY TXT spf.cafferata.dev: ("v=spf1 -all" ttl=86400) -> ("v=spf1 include:_spf.google.com -all" ttl=86400)
   182  SUCCESS!
   183  Done. 1 corrections.
   184  ```
   185  
   186  ![CI/CD job output for DNSControl push](assets/ci-cd-gitlab/ci-cd-job-output-dnscontrol-push.png)
   187  
   188  ## GitLab CI - Duplicate YAML configuration
   189  
   190  We have a working setup at this point that includes a `dnscontrol preview` and a `dnscontrol push` command. Well done! You might consider cleaning up the duplicate GitLab YAML configuration. We can move the DNSControl `image` `name` and `entrypoint` to a GitLab YAML `extends`. Then we can also move the duplicate `dnscontrol version` command to a GitLab `before_script`. See the third (_and also last_) [GitLab merge request #3](https://gitlab.com/cafferata/dnscontrol/-/merge_requests/3).
   191  
   192  This eventually brings us to the following GitLab CI setup.
   193  
   194  `.gitlab-ci.yml`
   195  
   196  ```yaml
   197  .dnscontrol:
   198    image:
   199      name: 'stackexchange/dnscontrol'
   200      entrypoint: ['']
   201    before_script:
   202      - '/usr/local/bin/dnscontrol version'
   203  
   204  dnscontrol-preview:
   205    extends: '.dnscontrol'
   206    stage: 'test'
   207    script:
   208      - '/usr/local/bin/dnscontrol check'
   209      - '/usr/local/bin/dnscontrol preview'
   210    rules:
   211      - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
   212        changes:
   213          - 'dnsconfig.js'
   214  
   215  dnscontrol-push:
   216    extends: '.dnscontrol'
   217    stage: 'deploy'
   218    script:
   219      - '/usr/local/bin/dnscontrol push'
   220    rules:
   221      - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "web"'
   222  ```
   223  
   224  If you are unexpectedly unable to set up this setup, feel free to [ask questions](https://github.com/StackExchange/dnscontrol/issues/new) about it.