github.com/StackExchange/dnscontrol/v4@v4.11.0/documentation/byo-secrets.md (about)

     1  # Bring-Your-Own-Secrets for automated testing
     2  
     3  Goal: Enable automated integration testing without accidentally
     4  leaking credentials (API keys and other secrets); at the same time permit everyone
     5  to automate their own tests without having to share their credentials.
     6  
     7  The instructions in this document will enable automated tests to run in these situations:
     8  
     9  * PR from a project member:
    10    * All officially supported providers plus many others too.
    11  * PR from an external people:
    12    * Automated tests run for providers that don't require secrets, which is currently only `BIND`.
    13  * PR on a fork of DNSControl:
    14    * The forker can set up secrets in their fork and only those providers with secrets will be tested. They can "set it and forget it" and all their future PRs will receive all the benefits of automated testing.
    15  
    16  # Background: How GitHub Actions protects secrets
    17  
    18  GitHub Actions has a secure
    19  [secrets storage system](https://docs.github.com/en/free-pro-team@latest/actions/reference/encrypted-secrets).
    20  Those secrets are available to GitHub Actions and are required for the
    21  integration tests to communicate with the various DNS providers that
    22  DNSControl supports.
    23  
    24  For security reasons, those secrets are unavailable if the PR comes
    25  from outside the project (a forked repo).  This is a good thing.  If
    26  it didn't work that way, a third-party could write a PR that leaks the
    27  secrets without the owners of the project knowing.
    28  
    29  The docs (and many blog posts) describe this as forked repos don't
    30  have access to secrets, and instead receive null strings. That's not
    31  actually what's happening.
    32  
    33  Actually what happens is the secrets come from the forked repo.  Or,
    34  more precisely, the secrets offered to a PR come from the repo that the
    35  PR came from.  A PR from DNSControl's owners gets secrets from
    36  [github.com/StackExchange/dnscontrol's secret store](https://github.com/StackExchange/dnscontrol/settings/secrets/actions)
    37  but a PR from a fork, such as
    38  [https://github.com/TomOnTime/dnscontrol](https://github.com/TomOnTime/dnscontrol)
    39  gets its secrets from TomOnTime's secrets.
    40  
    41  Our automated integration tests leverages this info to have tests
    42  only run if they have access to the secrets they will need.
    43  
    44  # Which providers are selected for testing?
    45  
    46  Tests are executed if the env variable`*_DOMAIN` exists where `*` is the name of the provider.  If the value is empty or
    47  unset, the test is skipped.
    48  For example, if a provider is called `FANCYDNS`, there must
    49  be a variable called `FANCYDNS_DOMAIN`.
    50  
    51  # Bring your own secrets
    52  
    53  This section describes how to add a provider to the "Actions" part of GitHub.
    54  
    55  Step 1: Create a branch
    56  
    57  Create a branch as you normally would to submit a PR to the project.
    58  
    59  Step 2: Update `pr_test.yml`
    60  
    61  Edit `.github/workflows/pr_test.yml`
    62  
    63  1. Add the provider to the `PROVIDERS` list.
    64  
    65  * Add the name of the provider to the PROVIDERS list.
    66  * Please keep this list sorted alphabetically.
    67  
    68  The line looks something like:
    69  
    70  {% code title=".github/workflows/pr_test.yml" %}
    71  ```
    72          PROVIDERS: "['AZURE_DNS','BIND','CLOUDFLAREAPI','CLOUDNS','DIGITALOCEAN','GANDI_V5','GCLOUD','HEDNS','HEXONET','INWX','NAMEDOTCOM','NS1','POWERDNS','ROUTE53','TRANSIP']"
    73  ```
    74  {% endcode %}
    75  
    76  2. Add your providers `_DOMAIN` env variable:
    77  
    78  * Add it to the `env` section of `integration-tests`.
    79  * Please keep this list sorted alphabetically.
    80  
    81  To find this section, search for `PROVIDER SECRET LIST`.
    82  
    83  For example, the entry for BIND looks like:
    84  
    85  {% code title=".github/workflows/pr_test.yml" %}
    86  ```
    87          BIND_DOMAIN: ${{ vars.BIND_DOMAIN }}
    88  ```
    89  {% endcode %}
    90  
    91  3. Add your providers other ENV variables:
    92  
    93  Every provider requires different variables set to perform the integration tests.  The list of such variables is in `integrationTest/providers.json`.
    94  
    95  You've already added `*_DOMAIN` to `pr_test.yml`. Now we're going to add the remaining ones.
    96  
    97  To find this section, search for `PROVIDER SECRET LIST`.
    98  
    99  For example, the entry for CLOUDFLAREAPI looks like this:
   100  
   101  {% code title=".github/workflows/pr_test.yml" %}
   102  ```
   103          CLOUDFLAREAPI_ACCOUNTID: ${{ secrets.CLOUDFLAREAPI_ACCOUNTID }}
   104          CLOUDFLAREAPI_TOKEN: ${{ secrets.CLOUDFLAREAPI_TOKEN }}
   105  ```
   106  {% endcode %}
   107  
   108  Step 3. Add the secrets to the repo.
   109  
   110  The `*_DOMAIN` variable is stored as a "variable" while the others are stored as "secrets".
   111  
   112  1. Go to Settings -> Secrets and variables -> Actions.
   113  
   114  2. On the "Variables" tab, add `*_DOMAIN` with the name of a test domain. This domain must already exist in the account. The DNS records of the domain will be deleted, so please use a test domain or other disposable domain.
   115  
   116  {% hint style="info" %}
   117  For the main project, **variables** are added here: [https://github.com/StackExchange/dnscontrol/settings/variables/actions](https://github.com/StackExchange/dnscontrol/settings/variables/actions)
   118  {% endhint %}
   119  
   120  3. On the "Secrets" tab, add the other env variables.
   121  
   122  {% hint style="info" %}
   123  For the main project, **secrets** are added here: [https://github.com/StackExchange/dnscontrol/settings/secrets/actions](https://github.com/StackExchange/dnscontrol/settings/secrets/actions)
   124  {% endhint %}
   125  
   126  If you have forked the project, add these to the settings of that fork.
   127  
   128  Step 4. Submit this PR like any other.
   129  
   130  GitHub Actions should kick and and run the tests.
   131  
   132  The tests will fail if a secret is wrong or missing.  It may take a few iterations to get everything working because... computers.
   133  
   134  # Donate secrets to the project
   135  
   136  The DNSControl project would like to have all providers automatically tested.
   137  However, we can't fund purchasing domains or maintaining credentials at every
   138  provider. Instead we depend on volunteers to maintain (and pay for) such
   139  accounts.
   140  
   141  We recommend the domain be named `dnscontroltest-PROVIDER.com` (or similar)
   142  where PROVIDER is replaced by the name of your provider or an abbreviation. For
   143  example `dnscontroltest-r53.com` and `dnscontroltest-gcloud.com`.
   144  
   145  When possible, use an OTE or free domain. Don't spend money if you don't have
   146  to. This isn't just to be thrifty! It avoids renewals and other hassles too.
   147  You'd be surprised at how many providers (such as Google and Azure) permit DNS
   148  zones to be created in your account without registering them.
   149  
   150  For actual DNS domains, please select the "private registration" option if it
   151  is available. Otherwise you will get spam phones calls and emails. The phone
   152  calls will make you wish you didn't own a phone.
   153  
   154  {% hint style="danger" %}
   155  Some rules:
   156  
   157  * The account/credentials should only access the test domain. Don't send your company's actual credentials and trust us to only touch the test domain. (this hasn't happened yet, thankfully!)
   158  * Renew the domain in a timely manner. This may be monitoring an email inbox you don't normally monitor.
   159  * Don't do anything that will get you in trouble with your employer, like charging it to your employer without permission. (this hasn't happend yet either, thankfully!)
   160  {% endhint %}
   161  
   162  Now that we've covered all that...
   163  
   164  Create a new Github issue with a subject "Add PROVIDER to automated tests" where "PROVIDER" is the name of the provider. DO NOT SEND THE CREDENTIALS IN THE GITHUB ISSUE.  Write that you understand the above rules and would like to volunteer to maintain the credentials and account.
   165  
   166  To securely send the credentials to the project, use this link: [https://transfer.secretoverflow.com/u/tlimoncelli](https://transfer.secretoverflow.com/u/tlimoncelli)
   167  
   168  You'll hear back within a week.
   169  
   170  Thank you for contributing credentials. The more providers we can test automatically with each PR, the better. It "shifts left" finding bugs and API changes and makes less work for everyone.