github.com/anth0d/nomad@v0.0.0-20221214183521-ae3a0a2cad06/e2e/README.md (about) 1 # End to End Tests 2 3 This package contains integration tests. Unlike tests alongside Nomad code, 4 these tests expect there to already be a functional Nomad cluster accessible 5 (either on localhost or via the `NOMAD_ADDR` env var). 6 7 See [`framework/doc.go`](framework/doc.go) for how to write tests. 8 9 The `NOMAD_E2E=1` environment variable must be set for these tests to run. 10 11 ## Provisioning Test Infrastructure on AWS 12 13 The `terraform/` folder has provisioning code to spin up a Nomad cluster on 14 AWS. You'll need both Terraform and AWS credentials to setup AWS instances on 15 which e2e tests will run. See the 16 [README](https://github.com/hashicorp/nomad/blob/main/e2e/terraform/README.md) 17 for details. The number of servers and clients is configurable, as is the 18 specific build of Nomad to deploy and the configuration file for each client 19 and server. 20 21 ## Provisioning Local Clusters 22 23 To run tests against a local cluster, you'll need to make sure the following 24 environment variables are set: 25 26 * `NOMAD_ADDR` should point to one of the Nomad servers 27 * `CONSUL_HTTP_ADDR` should point to one of the Consul servers 28 * `NOMAD_E2E=1` 29 30 _TODO: the scripts in `./bin` currently work only with Terraform, it would be 31 nice for us to have a way to deploy Nomad to Vagrant or local clusters._ 32 33 ## Running 34 35 After completing the provisioning step above, you can set the client 36 environment for `NOMAD_ADDR` and run the tests as shown below: 37 38 ```sh 39 # from the ./e2e/terraform directory, set your client environment 40 # if you haven't already 41 $(terraform output environment) 42 43 cd .. 44 go test -v ./... 45 ``` 46 47 If you want to run a specific suite, you can specify the `-suite` flag as 48 shown below. Only the suite with a matching `Framework.TestSuite.Component` 49 will be run, and all others will be skipped. 50 51 ```sh 52 go test -v -suite=Consul . 53 ``` 54 55 If you want to run a specific test, you'll need to regex-escape some of the 56 test's name so that the test runner doesn't skip over framework struct method 57 names in the full name of the tests: 58 59 ```sh 60 go test -v . -run 'TestE2E/Consul/\*consul\.ScriptChecksE2ETest/TestGroup' 61 ^ ^ ^ ^ 62 | | | | 63 Component | | Test func 64 | | 65 Go Package Struct 66 ``` 67 68 We're also in the process of migrating to "stdlib-style" tests that 69 use the standard go `testing` package without a notion of "suite". You 70 can run these with `-run` regexes the same way you would any other go 71 test: 72 73 ```sh 74 go test -v . -run TestExample/TestExample_Simple 75 ``` 76 77 78 ## I Want To... 79 80 ### ...SSH Into One Of The Test Machines 81 82 You can use the Terraform output to find the IP address. The keys will 83 in the `./terraform/keys/` directory. 84 85 ```sh 86 ssh -i keys/nomad-e2e-*.pem ubuntu@${EC2_IP_ADDR} 87 ``` 88 89 Run `terraform output` for IP addresses and details. 90 91 ### ...Deploy a Cluster of Mixed Nomad Versions 92 93 The `variables.tf` file describes the `nomad_version`, and 94 `nomad_local_binary` variables that can be used for most circumstances. But if 95 you want to deploy mixed Nomad versions, you can provide a list of versions in 96 your `terraform.tfvars` file. 97 98 For example, if you want to provision 3 servers all using Nomad 0.12.1, and 2 99 Linux clients using 0.12.1 and 0.12.2, you can use the following variables: 100 101 ```hcl 102 # will be used for servers 103 nomad_version = "0.12.1" 104 105 # will override the nomad_version for Linux clients 106 nomad_version_client_linux = [ 107 "0.12.1", 108 "0.12.2" 109 ] 110 ``` 111 112 ### ...Deploy Custom Configuration Files 113 114 Set the `profile` field to `"custom"` and put the configuration files in 115 `./terraform/config/custom/` as described in the 116 [README](https://github.com/hashicorp/nomad/blob/main/e2e/terraform/README.md#Profiles). 117 118 ### ...Deploy More Than 4 Linux Clients 119 120 Use the `"custom"` profile as described above. 121 122 ### ...Change the Nomad Version After Provisioning 123 124 You can update the `nomad_version` variable, or simply rebuild the binary you 125 have at the `nomad_local_binary` path so that Terraform picks up the 126 changes. Then run `terraform plan`/`terraform apply` again. This will update 127 Nomad in place, making the minimum amount of changes necessary.