github.com/eth-easl/loader@v0.0.0-20230908084258-8a37e1d94279/README.md (about) 1 # Loader 2 3 A load generator for benchmarking serverless systems. 4 5 ## Prerequisites 6 7 The experiments require a server-grade node running Linux (tested on Ubuntu 20, Intel Xeon). On CloudLab, one 8 can choose the APT cluster `d430` node. 9 10 ## Create a cluster 11 12 First, configure `script/setup/setup.cfg`. You can specify there which vHive branch to use, loader branch, operation mode 13 (sandbox type), maximum number of pods per node, and the Github token. All these configurations are mandatory. 14 We currently support the following modes: containerd (`container`), Firecracker (`firecracker`), and Firecracker with 15 snapshots (`firecracker_snapshots`). 16 The token needs `repo` and `admin:public_key` permissions and will be used for adding SSH key to the user's account for 17 purpose of cloning Github private repositories. 18 Loader will be cloned on every node specified as argument of the cluster create script. The same holds for Kubernetes 19 API server certificate. 20 21 * To create a multi-node cluster, specify the node addresses as the arguments and run the following command: 22 23 ```bash 24 $ bash ./scripts/setup/create_multinode.sh <master_node@IP> <loader_node@IP> <worker_node@IP> ... 25 ``` 26 27 This command will create the following setup: control plane is placed on master node, loader node is used for running 28 loader and monitoring pods (mostly, Prometheus, if enabled in setup config), workers are used purely for working pods. In 29 this setup, neither control plane nor workers are affected by loader and monitoring, creating more reliable measurements 30 of performance. 31 32 * Single-node cluster (experimental) 33 34 ```bash 35 $ bash ./scripts/setup/create_singlenode_container.sh <node@IP> 36 ``` 37 38 This mode is only for debugging purposes, and there is no guarantees of isolation between the loader and the master-node 39 components. 40 41 ### Check cluster health (on the master node) 42 43 Once the setup scripts are finished, we need to check if they have completed their jobs fully, because, e.g., there 44 might be race conditions where a few nodes are unavailable. 45 46 * First, log out the status of the control plane components on the master node and monitoring deployments by running the 47 following command: 48 49 ```bash 50 $ bash ./scripts/util/log_kn_status.sh 51 ``` 52 53 * If you see everything is `Running`, check if the cluster capacity is stretched to the desired capacity by running the 54 following script: 55 56 ```bash 57 $ bash ./scripts/util/check_node_capacity.sh 58 ``` 59 60 If you want to check 61 the [pod CIDR range](https://www.ibm.com/docs/en/cloud-private/3.1.2?topic=networking-kubernetes-network-model), run the 62 following 63 64 ```bash 65 $ bash ./scripts/util/get_pod_cidr.sh 66 ``` 67 68 * Next, try deploying a function (`myfunc`) with the desired `scale` (i.e., the number of instances that must be active 69 at all times). 70 71 ```bash 72 $ bash ./scripts/util/set_function_scale.sh <scale> 73 ``` 74 75 * One should verify that the system was able to start the requested number of function instances, by using the following 76 command. 77 78 ```bash 79 $ kubectl -n default get podautoscalers 80 ``` 81 82 ## Tune the timing for the benchmark function 83 84 Before start any experiments, the timing of the benchmark function should be tuned so that it consumes the required 85 service time more precisely. 86 87 First, run the following command to deploy the timing benchmark that yields the number of execution units* the function 88 needs to run given a required service time. 89 90 * The execution unit is approximately 100 `SQRTSD` x86 instructions. 91 92 ```bash 93 $ kubectl apply -f server/benchmark/timing.yaml 94 ``` 95 96 Then, monitor and collect the `ITERATIONS_MULTIPLIER` from the job logs as follows: 97 98 ```bash 99 $ watch kubectl logs timing 100 ``` 101 102 Finally, set the `ITERATIONS_MULTIPLIER` in the function template `workloads/$SANDBOX_TYPE/trace_func_go.yaml` to the 103 value previously collected. 104 105 To account for difference in CPU performance set `ITERATIONS_MULTIPLIER=102` if using 106 Cloudlab `xl170` or `d430` machines. (Date of measurement: 18-Oct-2022) 107 108 ## Single execution 109 110 To run load generator use the following command: 111 112 ```bash 113 $ go run cmd/loader.go --config cmd/config.json 114 ``` 115 116 Additionally, one can specify log verbosity argument as `--verbosity [info, debug, trace]`. The default value is `info`. 117 118 For to configure the workload for load generator, please refer to `docs/configuration.md`. 119 120 There are a couple of constants that should not be exposed to the users. They can be examined and changed 121 in `pkg/common/constants.go`. 122 123 ## Build the image for a synthetic function 124 125 The reason for existence of Firecracker and container version is because of different ports for gRPC server. Firecracker 126 uses port 50051, whereas containers are accessible via port 80. 127 128 Synthetic function runs for a given time and allocates the required amount of memory: 129 130 * `trace-firecracker` - for Firecracker 131 * `trace-container` - for regular containers 132 133 For testing cold start performance: 134 135 * `empty-firecracker` - for Firecracker - returns immediately 136 * `empty-container` - for regular containers - returns immediately 137 138 ```bash 139 $ make build <trace-firecracker|trace-container|empty-firecracker|empty-container> 140 ``` 141 142 ## Clean up between runs 143 144 ```bash 145 $ make clean 146 ``` 147 148 ## Running the Experiment Driver 149 150 Within the tools/driver folder, Configure the driverConfig.json file based on your username on Cloudlab, 151 the address of the loader node that you are running the experiment on and any experiment parameters you'd like to set. 152 Note that you need to have the relevant trace files on the machine running the experiment driver, they will then get 153 transferred to the loader node. 154 Then run the following command to launch an experiment: 155 ```bash 156 $ go run experiment_driver.go -c driverConfig.json 157 ``` 158 For more details take a look at the README in the tools/driver folder. 159 160 --- 161 162 For more options, please see the `Makefile`. 163 164 ## Using OpenWhisk 165 166 For instructions on how to use the loader with OpenWhisk go to `openwhisk_setup/README.md`. 167