github.com/google/fleetspeak@v0.1.15-0.20240426164851-4f31f62c1aea/fleetspeak/src/e2etesting/README.md (about)

     1  # End-to-End testing framework design
     2  
     3  ## FRR
     4  
     5  The
     6  [FRR service](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto)
     7  is a dummy/mock fleetspeak service meant for load/integration testing. It
     8  consists of a protocol specification and a GO implementation of a FRR client and
     9  server. Multiple FRR clients and servers can run in-process with a fleetspeak
    10  client and server. The GO FRR implementation is optimized for load testing - to
    11  put as much load on the database as possible. However, one focus of this project
    12  is correct operation of clients/servers running in separate processes and
    13  testing of the python connector libraries.
    14  
    15  In this project: - FRR protocol is reused. - A new
    16  [python implementation](https://github.com/google/fleetspeak/tree/master/frr_python)
    17  of a subset of the FRR server and client is implemented. - The implementation is
    18  using the connector libraries and running in separate processes.
    19  
    20  The following is implemented: 1.
    21  [frr_client.py](https://github.com/google/fleetspeak/blob/master/frr_python/frr_client.py) -
    22  Runs in a separate process. - Uses the
    23  [client connector](https://github.com/google/fleetspeak/tree/master/fleetspeak_python/fleetspeak/client_connector)
    24  as an interface to fleetspeak. - Receives fleetspeak messages. - If the message
    25  type equals "TrafficRequest", deserializes the payload as an
    26  [TrafficRequestData](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto#L15). -
    27  Replies with an appropriate
    28  [TrafficResponseData](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto#L36).
    29  
    30  1.  [frr_server.py](https://github.com/google/fleetspeak/blob/master/frr_python/frr_server.py)
    31  2.  Runs in a separate process.
    32  3.  Has a GRPC connection to the FRR master server.
    33  4.  Uses the
    34      [server connector](https://github.com/google/fleetspeak/tree/master/fleetspeak_python/fleetspeak/server_connector)
    35      as an interface to fleetspeak.
    36  5.  Receives a fleetspeak message.
    37  6.  If the message type equals "TrafficResponse", deserializes the payload as
    38      TrafficResponseData.
    39  7.  Forwards the received TrafficResponseData to the FRR master server.
    40  
    41  8.  [frr_master_server_main.go](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/e2etesting/frr_master_server_main/frr_master_server_main.go)
    42  
    43  9.  [frr.go](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/frr.go)
    44      contains the logic to run a FRR master server, however the logic can't be
    45      run stand-alone, it doesn't have its own main(). frr_master_server_main.go
    46      is created to run the FRR server as a standalone binary.
    47  
    48  Extensions to the FRR master server: - The Master service in
    49  [frr.proto](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/inttesting/frr/proto/fleetspeak_frr/frr.proto)
    50  is extended. - A new rpc CompletedRequests() is added. This rpc exposes the
    51  respective
    52  [implementation](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/frr.go#L454)
    53  as an RPC method. - A new rpc CreateHunt() is added. It exposes the
    54  [respective function](https://github.com/google/fleetspeak/blob/690991be00993813230a8f6c3aad703b21dfb0c5/fleetspeak/src/inttesting/frr/frr.go#L542)
    55  and sends either broadcast request or unicast messages to all the clients.
    56  
    57  ## Test cases
    58  
    59  The system supports multiple test cases. However, the main focus of the project
    60  is a single test case (the end-to-end test service "basically working") executed
    61  in multiple configurations. Tests can be run with a local installation (all the
    62  components are started in separate processes on one machine) and with a
    63  [distributed installation](https://github.com/google/fleetspeak/tree/master/terraform).
    64  
    65  ## Test framework
    66  
    67  The job of the test framework is to set up a working fleetspeak + FRR testing
    68  environment. In particular: - It runs the fleetspeak-config binary to generate
    69  configuration files for servers and clients. - It generates service
    70  configuration files for the FRR service. - It starts the fleetspeak clients,
    71  servers, the FRR python binaries, the Load balancer and the Master server as
    72  separate processes. - It provides functionality to stop all the processes it
    73  started.
    74  
    75  ## Load balancer
    76  
    77  A simple
    78  [load balancer](https://github.com/google/fleetspeak/blob/master/fleetspeak/src/e2etesting/balancer/balancer.go)
    79  was implemented for the local testing. It picks a random active server and
    80  forwards client's messages to it. The load balancer implements
    81  [PROXY protocol Version 1](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt).
    82  
    83  ## Local testing
    84  
    85  Testing framework is used for the local testing. Local testing is added to CI.
    86  To run the tests on your machine you have to create fleetspeak database and
    87  associated user:
    88  
    89  Run mysql console: `$ mysql --user root` Create database and user: `mysql>
    90  CREATE USER fs_user IDENTIFIED BY "fs_password"; mysql> CREATE DATABASE
    91  fs_test_db; mysql> GRANT ALL PRIVILEGES ON fs_test_db.* TO fs_user;` Exit mysql
    92  and set MySQL parameters in environment variables: `export
    93  MYSQL_TEST_USER="fs_user" export MYSQL_TEST_PASS="fs_password" export
    94  MYSQL_TEST_ADDR="127.0.0.1:3306" export MYSQL_TEST_E2E_DB="fs_test_db"` `cd` to
    95  [fleetspeak/src/e2etesting/localtesting](https://github.com/google/fleetspeak/tree/master/fleetspeak/src/e2etesting/localtesting)
    96  and run `go test`
    97  
    98  or
    99  
   100  Run all the tests by running `fleetspeak/test.sh`. This script will also run the
   101  local end-to-end tests.
   102  
   103  ## Cloud testing
   104  
   105  The testing framework is also used to run the tests with distributed fleetspeak
   106  installation in a cloud. The code and a guide how to test a distributed
   107  installation using Terraform can be found in
   108  [terraform directory](https://github.com/google/fleetspeak/tree/master/terraform).