github.com/Tyktechnologies/tyk@v2.9.5+incompatible/TESTING.md (about) 1 Table of Contents 2 ================= 3 4 * [Tyk testing guide](#tyk-testing-guide) 5 * [Initializing test server](#initializing-test-server) 6 * [Loading and configuring APIs](#loading-and-configuring-apis) 7 * [Running the tests](#running-the-tests) 8 * [Changing config variables](#changing-config-variables) 9 * [Upstream test server](#upstream-test-server) 10 * [Coprocess plugin testing](#coprocess-plugin-testing) 11 * [Creating user sessions](#creating-user-sessions) 12 * [Mocking dashboard](#mocking-dashboard) 13 * [Mocking RPC (Hybrid)](#mocking-rpc-hybrid) 14 * [Mocking DNS](#mocking-dns) 15 * [Test Framework](#test-framework) 16 17 18 ## Tyk testing guide 19 20 When it comes to the tests, one of the main questions is how to keep balance between expressivity, extendability, repeatability and performance. There are countless discussions if you should write integration or unit tests, should your mock or not, should you write tests first or after and etc. Since you will never find the right answer, on a growing code base, multiple people start introducing own methodology and distinct test helpers. Even looking at our quite small code base, you can find like 3-4 ways to write the same test. 21 22 This document describes Tyk test framework and unified guidelines on writing tests. 23 24 Main points of the test framework are: 25 - All tests run HTTP requests though the full HTTP stack, same as user will do 26 - Test definition logic separated from test runner. 27 - Official mocks for the Dashboard, RPC, and Bundler 28 29 Framework located inside "github.com/TykTechnologies/tyk/test" package. 30 See its API docs https://godoc.org/github.com/TykTechnologies/tyk/test 31 32 Let’s learn by example: 33 34 ```go 35 func genAuthHeader(username, password string) string { 36 toEncode := strings.Join([]string{username, password}, ":") 37 encodedPass := base64.StdEncoding.EncodeToString([]byte(toEncode)) 38 return fmt.Sprintf("Basic %s", encodedPass) 39 } 40 41 func TestBasicAuth(t *testing.T) { 42 // Start the test server 43 ts := newTykTestServer() 44 defer ts.Close() 45 46 // Configure and load API definition 47 buildAndLoadAPI(func(spec *APISpec) { 48 spec.UseBasicAuth = true 49 spec.UseKeylessAccess = false 50 spec.Proxy.ListenPath = "/" 51 spec.OrgID = "default" 52 }) 53 54 // Prepare data which will be used in tests 55 56 session := createStandardSession() 57 session.BasicAuthData.Password = "password" 58 session.AccessRights = map[string]user.AccessDefinition{"test": {APIID: "test", Versions: []string{"v1"}}} 59 60 validPassword := map[string]string{"Authorization": genAuthHeader("user", "password")} 61 wrongPassword := map[string]string{"Authorization": genAuthHeader("user", "wrong")} 62 wrongFormat := map[string]string{"Authorization": genAuthHeader("user", "password:more")} 63 malformed := map[string]string{"Authorization": "not base64"} 64 65 // Running tests one by one, based on our definition 66 ts.Run(t, []test.TestCase{ 67 // Create base auth based key 68 {Method: "POST", Path: "/tyk/keys/defaultuser", Data: session, AdminAuth: true, Code: 200}, 69 {Method: "GET", Path: "/", Code: 401, BodyMatch: `Authorization field missing`}, 70 {Method: "GET", Path: "/", Headers: validPassword, Code: 200}, 71 {Method: "GET", Path: "/", Headers: wrongPassword, Code: 401}, 72 {Method: "GET", Path: "/", Headers: wrongFormat, Code: 400, BodyMatch: `Attempted access with malformed header, values not in basic auth format`}, 73 {Method: "GET", Path: "/", Headers: malformed, Code: 400, BodyMatch: `Attempted access with malformed header, auth data not encoded correctly`}, 74 }...) 75 } 76 ``` 77 78 [Direct Github link](https://github.com/matiasinsaurralde/tyk/blob/4b6e0290ee36f6721b8d5343051bf343900b5943/mw_basic_auth_test.go) 79 80 And now compare it with previous Go style approach: 81 82 ```go 83 func TestBasicAuthWrongPassword(t *testing.T) { 84 spec := createSpecTest(t, basicAuthDef) 85 session := createBasicAuthSession() 86 username := "4321" 87 88 // Basic auth sessions are stored as {org-id}{username}, so we need to append it here when we create the session. 89 spec.SessionManager.UpdateSession("default4321", session, 60) 90 91 to_encode := strings.Join([]string{username, "WRONGPASSTEST"}, ":") 92 encodedPass := base64.StdEncoding.EncodeToString([]byte(to_encode)) 93 94 recorder := httptest.NewRecorder() 95 req := testReq(t, "GET", "/", nil) 96 req.Header.Set("Authorization", fmt.Sprintf("Basic %s", encodedPass)) 97 98 chain := getBasicAuthChain(spec) 99 chain.ServeHTTP(recorder, req) 100 101 if recorder.Code == 200 { 102 t.Error("Request should have failed and returned non-200 code!: \n", recorder.Code) 103 } 104 105 if recorder.Code != 401 { 106 t.Error("Request should have returned 401 code!: \n", recorder.Code) 107 } 108 109 if recorder.Header().Get("WWW-Authenticate") == "" { 110 t.Error("Request should have returned WWW-Authenticate header!: \n") 111 } 112 } 113 ``` 114 115 [Direct Github link](https://github.com/matiasinsaurralde/tyk/blob/2a9d7d5b6c289ac75dfa9b4e9c4527f1041d7daf/mw_basic_auth_test.go#L246:1) 116 117 Note that in the last “classic” way we defined only 1 test case, while in with our new framework we defined 6, all of them repeatable, and share the same assertion and test runner logic provided by framework. 118 119 Now lets review tests written with a new framework piece by piece. 120 ### Initializing test server 121 One of the core ideas, is that tests should be as close as possible to real users. In order to implement it, framework provides you a way to programmatically start and stop full Gateway HTTP stack using `tykTestServer` object, like this: 122 123 ```go 124 ts := newTykTestServer() 125 defer ts.Close() 126 ``` 127 128 When you create a new server, it initialize gateway itself, starts listener on random port, setup required global variables and etc. It is very similar to what happens when you start gateway process, but in this case you can start and stop it on demand. 129 130 You can configure server behavior using few variable, like setting control API on a separate port, by providing `tykTestServerConfig` object, to `newTykTestServer` as argument. Here is the list of all possible arguments: 131 132 ```go 133 ts := newTykTestServer(tykTestServerConfig{ 134 // Run control API on a separate port 135 sepatateControlAPI: true, 136 // Add delay after each test case, if you code depend on timing 137 // Bad practice, but sometimes needed 138 delay: 10 * time.Millisecond, 139 // Emulate that Gateway restarted using SIGUSR2 140 hotReload: true, 141 // Emulate that listener will 142 overrideDefaults, true, 143 }) 144 ``` 145 146 To close the server simply call `tykTestServer#Close` method, which will ensure that all the listeners will be properly closed. 147 148 ### Loading and configuring APIs 149 150 ```go 151 buildAndLoadAPI(func(spec *APISpec) { 152 spec.UseBasicAuth = true 153 spec.UseKeylessAccess = false 154 spec.Proxy.ListenPath = "/" 155 spec.OrgID = "default" 156 }) 157 ``` 158 159 Basic idea that you have default bare minimum API definition, which you can configure using generator function, to set state required for the test. API then will be loaded into the Gateway, and will be ready to be used inside tests. 160 161 If you need to load multiple APIs at the same time, `buildAndLoadAPI` support variadic number of arguments: `buildAndLoadAPI(<fn1>, <fn2>, ...)` 162 163 You can also call it without arguments at all, in this case it will load default API definition: `buildAndLoadAPI()` 164 165 In fact, this function is mashup of 2 lower level functions: `buildAPI` and `loadAPI`, both returning `[]*APISpec` array. In some cases you may need to build API template, and with some smaller modifications load it on demand in different tests. So it can look like: 166 167 ```go 168 spec := buildAPI(<fn>) 169 ... 170 spec.SomeField = "Case1" 171 loadAPI(spec) 172 ... 173 spec.SomeField = "Case2" 174 loadAPI(spec) 175 ``` 176 177 Updating variables inside API version can be tricky, because API version object is inside `Versions` map, and direct manipulations with map value is prohibited. To simplify this process, there is special helper `updateAPIVersion`, which can be used like this: 178 179 ```go 180 updateAPIVersion(spec, "v1", func(v *apidef.VersionInfo) { 181 v.Paths.BlackList = []string{"/blacklist/literal", "/blacklist/{id}/test"} 182 v.UseExtendedPaths = false 183 }) 184 ``` 185 186 In some cases updating API definition via Go structures can be a bit complex, and you may want to update API definition directly via JSON unmarshaling: 187 188 ```go 189 updateAPIVersion(spec, "v1", func(v *apidef.VersionInfo) { 190 json.Unmarshal([]byte(`[ 191 { 192 "path": "/ignored/literal", 193 "method_actions": {"GET": {"action": "no_action"}} 194 }, 195 { 196 "path": "/ignored/{id}/test", 197 "method_actions": {"GET": {"action": "no_action"}} 198 } 199 ]`), &v.ExtendedPaths.Ignored) 200 }) 201 ``` 202 203 ### Running the tests 204 205 ```go 206 ts.Run(t, []test.TestCase{ 207 // Create base auth based key 208 {Method: "POST", Path: "/tyk/keys/defaultuser", Data: session, AdminAuth: true, Code: 200}, 209 {Method: "GET", Path: "/", Code: 401, BodyMatch: `Authorization field missing`}, 210 {Method: "GET", Path: "/", Headers: validPassword, Code: 200}, 211 {Method: "GET", Path: "/", Headers: wrongPassword, Code: 401}, 212 {Method: "GET", Path: "/", Headers: wrongFormat, Code: 400, BodyMatch: `Attempted access with malformed header, values not in basic auth format`}, 213 {Method: "GET", Path: "/", Headers: malformed, Code: 400, BodyMatch: `Attempted access with malformed header, auth data not encoded correctly`}, 214 }...) 215 } 216 ``` 217 218 Tests are defined using new `test` package `TestCase` structure, which allows you to define both http request details and response assertions. For example `{Method: "GET", Path: "/", Headers: validPassword, Code: 200}` tells to make a `GET` request to `/` path, with specified headers. After request is made, it will assert response status code with given value. 219 220 ```go 221 type TestCase struct { 222 Method, Path string `json:",omitempty"` 223 Domain string `json:",omitempty"` 224 Proto string `json:",omitempty"` 225 Code int `json:",omitempty"` 226 Data interface{} `json:",omitempty"` 227 Headers map[string]string `json:",omitempty"` 228 PathParams map[string]string `json:",omitempty"` 229 Cookies []*http.Cookie `json:",omitempty"` 230 Delay time.Duration `json:",omitempty"` 231 BodyMatch string `json:",omitempty"` 232 BodyMatchFunc func([]byte) bool `json:",omitempty"` 233 BodyNotMatch string `json:",omitempty"` 234 HeadersMatch map[string]string `json:",omitempty"` 235 HeadersNotMatch map[string]string `json:",omitempty"` 236 JSONMatch map[string]string `json:",omitempty"` 237 ErrorMatch string `json:",omitempty"` 238 BeforeFn func() `json:"-"` 239 Client *http.Client `json:"-"` 240 241 AdminAuth bool `json:",omitempty"` 242 ControlRequest bool `json:",omitempty"` 243 } 244 ``` 245 246 `tykTestServer` provides a test runner, which generate HTTP requests based on specification and does assertions. Most of the time you going to use `tykTestServer#Run(t *testing.T, test.TestCase...) (*http.Response, error)`function. Note that it use variadic number of arguments, so if you need to pass multiple test cases, pass it like in example above: `[]test.TestCase{<tc1>,<tc2>}...`, with 3 dots in the end. 247 248 Additionally there is `RunEx` function, with exactly same definition, but internally it runs test cases multiple times (4 right now) with different combinations of `overrideDefaults` and `hotReload` options. This can be handy if you need to test functionality that tightly depends hot reload functionality, like reloading APIs, loading plugin bundles or listener itself. 249 250 Both `Run` and `RunEx` also return response and error of the last test case, in case if you need it. 251 ### Changing config variables 252 In lot of cases tests depend on various config variables. You can can update them directly on `config.Global` object, and restore default config using `resetTestConfig` function. 253 254 ```go 255 config.Global.HttpServerOptions.OverrideDefaults = true 256 config.Global.HttpServerOptions.SkipURLCleaning = true 257 defer resetTestConfig() 258 ``` 259 260 ### Upstream test server 261 You may notice that default API already targets some upstream mock, created for testing purpose. Url of the upstream hold in `testHttpAny` variable, but in most cases you do not need it, because APIs created by default already embed it. By default this upstream mock will successfully respond to any url, and response will contain details of the request in the following format: 262 263 ```go 264 type testHttpResponse struct { 265 Method string 266 Url string 267 Headers map[string]string 268 Form map[string]string 269 } 270 ``` 271 272 Note that it include final request details, so, for example if you need to test URL rewriting functionality, URL of original request will differ from URL in response of upstream mock, and you can assert it with: BodyMatch: "Url":"<assert-url>". Also notice how we used simple BodyMatch string assertion to validation JSON response. 273 274 There is also few special URLs with specific behavior: 275 - `/get` accepts only `GET` requests 276 - `/post` accepts only `POST` requests 277 - `/jwk.json` used for cases when JWK token downloaded from upsteram 278 - `/ws` used for testing WebSockets 279 - `/bundles` built in plugin bundle web server, more details below 280 281 ### Coprocess plugin testing 282 If you want use Python, Lua or GRPC plugins, you need bundle manifest file and scripts to ZIP file, upload them somewhere on external file webserver, and point Gateway to bundle location. 283 284 Our test framework include built-in bundle file server, and for simplicity, you provide only content of the of the bundle files, and it will automatically server it as ZIP file. 285 1. Create `map[string]string` object with file contents, where key is file name 286 2. Call `registerBundle("<unique-plugin-id>", <map-with-files>)` which will return unique bundle ID. 287 3. When creating API set `spec.CustomMiddlewareBundle` to bundle ID returned by `registerBundle` 288 289 Example of loading `python` auth plugin: 290 291 ```go 292 var pythonBundleWithAuthCheck = map[string]string{ 293 "manifest.json": ` 294 { 295 "file_list": [ 296 "middleware.py" 297 ], 298 "custom_middleware": { 299 "driver": "python", 300 "auth_check": { 301 "name": "MyAuthHook" 302 } 303 } 304 } 305 `, 306 "middleware.py": ` 307 from tyk.decorators import * 308 from gateway import TykGateway as tyk 309 @Hook 310 def MyAuthHook(request, session, metadata, spec): 311 print("MyAuthHook is called") 312 auth_header = request.get_header('Authorization') 313 if auth_header == 'valid_token': 314 session.rate = 1000.0 315 session.per = 1.0 316 metadata["token"] = "valid_token" 317 return request, session, metadata 318 `, 319 } 320 321 func TestPython(t *testing.T) { 322 ts := newTykTestServer() 323 defer ts.Close() 324 325 bundleID := registerBundle("python_with_auth_check", pythonBundleWithAuthCheck) 326 327 buildAndLoadAPI(func(spec *APISpec) { 328 spec.UseKeylessAccess = false 329 spec.EnableCoProcessAuth = true 330 spec.CustomMiddlewareBundle = bundleID 331 }) 332 // test code goes here 333 } 334 ``` 335 336 ### Creating user sessions 337 You can create a user session, similar to API, by calling `createSession` function: 338 ```go 339 key := createSession(func(s *user.SessionState) { 340 s.QuotaMax = 2 341 }) 342 ``` 343 You can call it without arguments as well, if you are ok with default settings `createSession()` 344 345 If you need to create session object without adding it to database, for example if you need to create key explicitly via API, you can use `createStandardSession()` function, which returns `*user.SessionState` object. 346 347 ### Custom upstream mock 348 If you need to create custom upstream test server, for example if you need custom TLS settings for Mutual TLS testing, the easiest way is to use standard Go `net/http/httptest` package and override `spec.Proxy.TargetURL` API URL to test server. 349 350 ```go 351 upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { 352 // custom logic 353 })) 354 355 buildAndLoadAPI(func(spec *APISpec) { 356 spec.Proxy.TargetURL = upstream.URL 357 }) 358 ``` 359 360 ### Mocking dashboard 361 There is no any specific object to mock the dashboard (yet), but since Dashboard is a standard HTTP server, you can use approach similar to described in **Custom upstream mock** section: 362 363 ```go 364 dashboard := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { 365 if r.URL.Path == "/system/apis" { 366 w.Write([]byte(`{"Status": "OK", "Nonce": "1", "Message": [{"api_definition": {}}]}`)) 367 } else { 368 t.Fatal("Unknown dashboard API request", r) 369 } 370 })) 371 372 config.Global.UseDBAppConfigs = true 373 config.Global.AllowInsecureConfigs = true 374 config.Global.DBAppConfOptions.ConnectionString = dashboard.URL 375 ``` 376 377 ### Mocking RPC (Hybrid) 378 When Gateway works in Hybrid mode, it talks with MDCB instance via RPC channel using `gorpc` library. You can use `startRPCMock` and `stopRPCMock` functions to mock RPC server. `startRPCMock` internally sets required config variables to enable RPC mode. 379 380 ```go 381 func TestSyncAPISpecsRPCSuccess(t *testing.T) { 382 // Mock RPC 383 dispatcher := gorpc.NewDispatcher() 384 dispatcher.AddFunc("GetApiDefinitions", func(clientAddr string, dr *DefRequest) (string, error) { 385 return "[{}]", nil 386 }) 387 dispatcher.AddFunc("Login", func(clientAddr, userKey string) bool { 388 return true 389 }) 390 rpc := startRPCMock(dispatcher) 391 defer stopRPCMock(rpc) 392 count := syncAPISpecs() 393 if count != 1 { 394 t.Error("Should return array with one spec", apiSpecs) 395 } 396 } 397 ``` 398 399 ### DNS mocks 400 Inside tests we override default network resolver to use custom DNS server mock, creating using awesome `github.com/miekg/dns` library. Domain -\> IP mapping set via map inside `helpers_test.go` file. By default you have access to domains: `localhost`, `host1.local`, `host2.local` and `host3.local`. Access to all unknown domains will cause panic. 401 402 Using DNS mock means that you are able to create tests with APIs on multiple domains, without modifying machine `/etc/hosts` file. 403 404 ## Test Framework 405 406 Usage of framework described above is not limited by Tyk Gateway, and it is used across variety of Tyk projects. 407 The main building block is the test runner. 408 ```go 409 type HTTPTestRunner struct { 410 Do func(*http.Request, *TestCase) (*http.Response, error) 411 Assert func(*http.Response, *TestCase) error 412 RequestBuilder func(*TestCase) (*http.Request, error) 413 } 414 func (r HTTPTestRunner) Run(t testing.TB, testCases ...TestCase) { 415 ... 416 } 417 ``` 418 By overriding its variables, you can tune runner behavior. 419 For example http runner can be look like: 420 ``` 421 import "github.com/TykTechnologies/tyk/test" 422 423 ... 424 baseURL := "http://example.com" 425 runner := test.HTTPTestRunner{ 426 Do: func(r *http.Request, tc *TestCase) (*http.Response, error) { 427 return tc.Client.Do(r) 428 } 429 RequestBuilder: func(tc *TestCase) (*http.Request, error) { 430 tc.BaseURL = baseURL 431 return NewRequest(tc) 432 }, 433 } 434 runner.Run(t, testCases...) 435 ... 436 ``` 437 And Unit testing of http handlers can be: 438 ``` 439 import "github.com/TykTechnologies/tyk/test" 440 441 ... 442 handler := func(wr http.RequestWriter, r *http.Request){...} 443 runner := test.HTTPTestRunner{ 444 Do: func(r *http.Request, _ *TestCase) (*http.Response, error) { 445 rec := httptest.NewRecorder() 446 handler(rec, r) 447 return rec.Result(), nil 448 }, 449 } 450 runner.Run(t, testCases...) 451 ... 452 ``` 453 454 This package already exports functions for cases mentioned above: 455 - `func TestHttpServer(t testing.TB, baseURL string, testCases ...TestCase)` 456 - `func TestHttpHandler(t testing.TB, handle http.HandlerFunc, testCases ...TestCase)`