github.com/google/fleetspeak@v0.1.15-0.20240426164851-4f31f62c1aea/fleetspeak/src/e2etesting/README.md (about) 1 # End-to-End testing framework design 2 3 ## FRR 4 5 The 6 [FRR service](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto) 7 is a dummy/mock fleetspeak service meant for load/integration testing. It 8 consists of a protocol specification and a GO implementation of a FRR client and 9 server. Multiple FRR clients and servers can run in-process with a fleetspeak 10 client and server. The GO FRR implementation is optimized for load testing - to 11 put as much load on the database as possible. However, one focus of this project 12 is correct operation of clients/servers running in separate processes and 13 testing of the python connector libraries. 14 15 In this project: - FRR protocol is reused. - A new 16 [python implementation](https://github.com/google/fleetspeak/tree/master/frr_python) 17 of a subset of the FRR server and client is implemented. - The implementation is 18 using the connector libraries and running in separate processes. 19 20 The following is implemented: 1. 21 [frr_client.py](https://github.com/google/fleetspeak/blob/master/frr_python/frr_client.py) - 22 Runs in a separate process. - Uses the 23 [client connector](https://github.com/google/fleetspeak/tree/master/fleetspeak_python/fleetspeak/client_connector) 24 as an interface to fleetspeak. - Receives fleetspeak messages. - If the message 25 type equals "TrafficRequest", deserializes the payload as an 26 [TrafficRequestData](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto#L15). - 27 Replies with an appropriate 28 [TrafficResponseData](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto#L36). 29 30 1. [frr_server.py](https://github.com/google/fleetspeak/blob/master/frr_python/frr_server.py) 31 2. Runs in a separate process. 32 3. Has a GRPC connection to the FRR master server. 33 4. Uses the 34 [server connector](https://github.com/google/fleetspeak/tree/master/fleetspeak_python/fleetspeak/server_connector) 35 as an interface to fleetspeak. 36 5. Receives a fleetspeak message. 37 6. If the message type equals "TrafficResponse", deserializes the payload as 38 TrafficResponseData. 39 7. Forwards the received TrafficResponseData to the FRR master server. 40 41 8. [frr_master_server_main.go](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/e2etesting/frr_master_server_main/frr_master_server_main.go) 42 43 9. [frr.go](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/frr.go) 44 contains the logic to run a FRR master server, however the logic can't be 45 run stand-alone, it doesn't have its own main(). frr_master_server_main.go 46 is created to run the FRR server as a standalone binary. 47 48 Extensions to the FRR master server: - The Master service in 49 [frr.proto](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto) 50 is extended. - A new rpc CompletedRequests() is added. This rpc exposes the 51 respective 52 [implementation](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/frr.go#L454) 53 as an RPC method. - A new rpc CreateHunt() is added. It exposes the 54 [respective function](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/frr.go#L542) 55 and sends either broadcast request or unicast messages to all the clients. 56 57 ## Test cases 58 59 The system supports multiple test cases. However, the main focus of the project 60 is a single test case (the end-to-end test service "basically working") executed 61 in multiple configurations. Tests can be run with a local installation (all the 62 components are started in separate processes on one machine) and with a 63 [distributed installation](https://github.com/google/fleetspeak/tree/master/terraform). 64 65 ## Test framework 66 67 The job of the test framework is to set up a working fleetspeak + FRR testing 68 environment. In particular: - It runs the fleetspeak-config binary to generate 69 configuration files for servers and clients. - It generates service 70 configuration files for the FRR service. - It starts the fleetspeak clients, 71 servers, the FRR python binaries, the Load balancer and the Master server as 72 separate processes. - It provides functionality to stop all the processes it 73 started. 74 75 ## Load balancer 76 77 A simple 78 [load balancer](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/e2etesting/balancer/balancer.go) 79 was implemented for the local testing. It picks a random active server and 80 forwards client's messages to it. The load balancer implements 81 [PROXY protocol Version 1](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt). 82 83 ## Local testing 84 85 Testing framework is used for the local testing. Local testing is added to CI. 86 To run the tests on your machine you have to create fleetspeak database and 87 associated user: 88 89 Run mysql console: `$ mysql --user root` Create database and user: `mysql> 90 CREATE USER fs_user IDENTIFIED BY "fs_password"; mysql> CREATE DATABASE 91 fs_test_db; mysql> GRANT ALL PRIVILEGES ON fs_test_db.* TO fs_user;` Exit mysql 92 and set MySQL parameters in environment variables: `export 93 MYSQL_TEST_USER="fs_user" export MYSQL_TEST_PASS="fs_password" export 94 MYSQL_TEST_ADDR="127.0.0.1:3306" export MYSQL_TEST_E2E_DB="fs_test_db"` `cd` to 95 [fleetspeak/src/e2etesting/localtesting](https://github.com/google/fleetspeak/tree/master/fleetspeak/src/e2etesting/localtesting) 96 and run `go test` 97 98 or 99 100 Run all the tests by running `fleetspeak/test.sh`. This script will also run the 101 local end-to-end tests. 102 103 ## Cloud testing 104 105 The testing framework is also used to run the tests with distributed fleetspeak 106 installation in a cloud. The code and a guide how to test a distributed 107 installation using Terraform can be found in 108 [terraform directory](https://github.com/google/fleetspeak/tree/master/terraform).