github.com/StackExchange/dnscontrol/v4@v4.11.0/documentation/byo-secrets.md (about) 1 # Bring-Your-Own-Secrets for automated testing 2 3 Goal: Enable automated integration testing without accidentally 4 leaking credentials (API keys and other secrets); at the same time permit everyone 5 to automate their own tests without having to share their credentials. 6 7 The instructions in this document will enable automated tests to run in these situations: 8 9 * PR from a project member: 10 * All officially supported providers plus many others too. 11 * PR from an external people: 12 * Automated tests run for providers that don't require secrets, which is currently only `BIND`. 13 * PR on a fork of DNSControl: 14 * The forker can set up secrets in their fork and only those providers with secrets will be tested. They can "set it and forget it" and all their future PRs will receive all the benefits of automated testing. 15 16 # Background: How GitHub Actions protects secrets 17 18 GitHub Actions has a secure 19 [secrets storage system](https://docs.github.com/en/free-pro-team@latest/actions/reference/encrypted-secrets). 20 Those secrets are available to GitHub Actions and are required for the 21 integration tests to communicate with the various DNS providers that 22 DNSControl supports. 23 24 For security reasons, those secrets are unavailable if the PR comes 25 from outside the project (a forked repo). This is a good thing. If 26 it didn't work that way, a third-party could write a PR that leaks the 27 secrets without the owners of the project knowing. 28 29 The docs (and many blog posts) describe this as forked repos don't 30 have access to secrets, and instead receive null strings. That's not 31 actually what's happening. 32 33 Actually what happens is the secrets come from the forked repo. Or, 34 more precisely, the secrets offered to a PR come from the repo that the 35 PR came from. A PR from DNSControl's owners gets secrets from 36 [github.com/StackExchange/dnscontrol's secret store](https://github.com/StackExchange/dnscontrol/settings/secrets/actions) 37 but a PR from a fork, such as 38 [https://github.com/TomOnTime/dnscontrol](https://github.com/TomOnTime/dnscontrol) 39 gets its secrets from TomOnTime's secrets. 40 41 Our automated integration tests leverages this info to have tests 42 only run if they have access to the secrets they will need. 43 44 # Which providers are selected for testing? 45 46 Tests are executed if the env variable`*_DOMAIN` exists where `*` is the name of the provider. If the value is empty or 47 unset, the test is skipped. 48 For example, if a provider is called `FANCYDNS`, there must 49 be a variable called `FANCYDNS_DOMAIN`. 50 51 # Bring your own secrets 52 53 This section describes how to add a provider to the "Actions" part of GitHub. 54 55 Step 1: Create a branch 56 57 Create a branch as you normally would to submit a PR to the project. 58 59 Step 2: Update `pr_test.yml` 60 61 Edit `.github/workflows/pr_test.yml` 62 63 1. Add the provider to the `PROVIDERS` list. 64 65 * Add the name of the provider to the PROVIDERS list. 66 * Please keep this list sorted alphabetically. 67 68 The line looks something like: 69 70 {% code title=".github/workflows/pr_test.yml" %} 71 ``` 72 PROVIDERS: "['AZURE_DNS','BIND','CLOUDFLAREAPI','CLOUDNS','DIGITALOCEAN','GANDI_V5','GCLOUD','HEDNS','HEXONET','INWX','NAMEDOTCOM','NS1','POWERDNS','ROUTE53','TRANSIP']" 73 ``` 74 {% endcode %} 75 76 2. Add your providers `_DOMAIN` env variable: 77 78 * Add it to the `env` section of `integration-tests`. 79 * Please keep this list sorted alphabetically. 80 81 To find this section, search for `PROVIDER SECRET LIST`. 82 83 For example, the entry for BIND looks like: 84 85 {% code title=".github/workflows/pr_test.yml" %} 86 ``` 87 BIND_DOMAIN: ${{ vars.BIND_DOMAIN }} 88 ``` 89 {% endcode %} 90 91 3. Add your providers other ENV variables: 92 93 Every provider requires different variables set to perform the integration tests. The list of such variables is in `integrationTest/providers.json`. 94 95 You've already added `*_DOMAIN` to `pr_test.yml`. Now we're going to add the remaining ones. 96 97 To find this section, search for `PROVIDER SECRET LIST`. 98 99 For example, the entry for CLOUDFLAREAPI looks like this: 100 101 {% code title=".github/workflows/pr_test.yml" %} 102 ``` 103 CLOUDFLAREAPI_ACCOUNTID: ${{ secrets.CLOUDFLAREAPI_ACCOUNTID }} 104 CLOUDFLAREAPI_TOKEN: ${{ secrets.CLOUDFLAREAPI_TOKEN }} 105 ``` 106 {% endcode %} 107 108 Step 3. Add the secrets to the repo. 109 110 The `*_DOMAIN` variable is stored as a "variable" while the others are stored as "secrets". 111 112 1. Go to Settings -> Secrets and variables -> Actions. 113 114 2. On the "Variables" tab, add `*_DOMAIN` with the name of a test domain. This domain must already exist in the account. The DNS records of the domain will be deleted, so please use a test domain or other disposable domain. 115 116 {% hint style="info" %} 117 For the main project, **variables** are added here: [https://github.com/StackExchange/dnscontrol/settings/variables/actions](https://github.com/StackExchange/dnscontrol/settings/variables/actions) 118 {% endhint %} 119 120 3. On the "Secrets" tab, add the other env variables. 121 122 {% hint style="info" %} 123 For the main project, **secrets** are added here: [https://github.com/StackExchange/dnscontrol/settings/secrets/actions](https://github.com/StackExchange/dnscontrol/settings/secrets/actions) 124 {% endhint %} 125 126 If you have forked the project, add these to the settings of that fork. 127 128 Step 4. Submit this PR like any other. 129 130 GitHub Actions should kick and and run the tests. 131 132 The tests will fail if a secret is wrong or missing. It may take a few iterations to get everything working because... computers. 133 134 # Donate secrets to the project 135 136 The DNSControl project would like to have all providers automatically tested. 137 However, we can't fund purchasing domains or maintaining credentials at every 138 provider. Instead we depend on volunteers to maintain (and pay for) such 139 accounts. 140 141 We recommend the domain be named `dnscontroltest-PROVIDER.com` (or similar) 142 where PROVIDER is replaced by the name of your provider or an abbreviation. For 143 example `dnscontroltest-r53.com` and `dnscontroltest-gcloud.com`. 144 145 When possible, use an OTE or free domain. Don't spend money if you don't have 146 to. This isn't just to be thrifty! It avoids renewals and other hassles too. 147 You'd be surprised at how many providers (such as Google and Azure) permit DNS 148 zones to be created in your account without registering them. 149 150 For actual DNS domains, please select the "private registration" option if it 151 is available. Otherwise you will get spam phones calls and emails. The phone 152 calls will make you wish you didn't own a phone. 153 154 {% hint style="danger" %} 155 Some rules: 156 157 * The account/credentials should only access the test domain. Don't send your company's actual credentials and trust us to only touch the test domain. (this hasn't happened yet, thankfully!) 158 * Renew the domain in a timely manner. This may be monitoring an email inbox you don't normally monitor. 159 * Don't do anything that will get you in trouble with your employer, like charging it to your employer without permission. (this hasn't happend yet either, thankfully!) 160 {% endhint %} 161 162 Now that we've covered all that... 163 164 Create a new Github issue with a subject "Add PROVIDER to automated tests" where "PROVIDER" is the name of the provider. DO NOT SEND THE CREDENTIALS IN THE GITHUB ISSUE. Write that you understand the above rules and would like to volunteer to maintain the credentials and account. 165 166 To securely send the credentials to the project, use this link: [https://transfer.secretoverflow.com/u/tlimoncelli](https://transfer.secretoverflow.com/u/tlimoncelli) 167 168 You'll hear back within a week. 169 170 Thank you for contributing credentials. The more providers we can test automatically with each PR, the better. It "shifts left" finding bugs and API changes and makes less work for everyone.