go.dedis.ch/onet/v3@v3.2.11-0.20210930124529-e36530bca7ef/simul/platform/MININET.md (about) 1 Navigation: [DEDIS](https://github.com/dedis/doc/tree/master/README.md) :: 2 [Onet](../../README.md) :: 3 [Simulation](../README.md) :: 4 Mininet Simulation 5 6 # Mininet Simulation 7 8 Mininet allows to have a better control over the delay and bandwidth restrictions 9 during the simulation. While [Deterlab](DETERLAB.md) only has these restrictions 10 between servers, but not necessarily between all nodes, Mininet will set up the 11 same restriction between any two nodes that communcate with each other. 12 13 The Mininet simulation uses http://mininet.org/ to simulate a network where each 14 host represents a Cothority node. It has been extensively tested on the 15 hardware of https://iccluster.epfl.ch, but should theoretically also run on 16 other kinds of hardware. As it needs to set up the mininet environment, it is 17 more fragile than the Deterlab simulations. But server availability in EPFL 18 is often better than on Deterlab. 19 20 Each T3-server on the iccluster is a 24-core server with 256GB of RAM and can 21 run around 300 cothority nodes simultaneously. So if you want to run a 22 simulation with 2000 nodes, you need at least 7 physical servers. 23 24 ## Setting up ICCluster 25 26 Supposing you want to run your simulation using the iccluster-network, you 27 first need to reserve the machines. You have to do so one day in advance, as 28 it is not possible to reserve the machines on the same day. Once the machines 29 are ready, you need to install them. 30 31 1. In the ICCluster-admin interface, you need to go to `My Servers` -> `Setup`. 32 2. Chose the servers you reserved - take care, as there might be servers from 33 other people in the same lab! 34 3. `Choose a boot option` - please take _Ubuntu focal 2004_ for best results. 35 As of Dec 2020, Ubuntu 14.04 still appears to work, but it is no longer supported by 36 Ubuntu and should not be used for new work. 37 4. `Customization` - chose a password (it can be very simple) for the setup. 38 5. `Run Setup` and confirm the setup. THIS WILL DELETE ALL DATA ON THE SERVERS! 39 40 ### Verifying everything is correctly set up 41 42 The power-cycle and OS installation process takes several minutes, and the IC Cluster 43 UI does not have any feedback to know the state of the servers. As a result, 44 you need to use `ping` and `ssh` to monitor them. 45 46 ```bash 47 ping iccluster033.iccluster.epfl.ch 48 ``` 49 50 If you are outside of EPFL, you need to use the VPN, as iccluster is only 51 reachable from inside EPFL. 52 53 You can also connect to the server with 54 55 ```bash 56 ssh root@iccluster033.iccluster.epfl.ch 57 ``` 58 59 and enter your password that you gave in the `Customization` step above. 60 61 ## Running a simulation 62 63 Now you are finally ready to run your simulation. We suppose that you have 64 your simulation running successfully under [Localhost](LOCALHOST.md). Then 65 all you need to do is: 66 67 ```bash 68 go build && ./simul -platform mininet simul.toml 69 ``` 70 71 If it is the first time the simulation is run, it will ask you whether you want 72 to use iccluster or not. If you reply `Y` (or simply press `Enter`), you can 73 give the names of the servers you reserved and installed. An example is: 74 `31 32 33`. You don't need to enter leading `0`s, this will be automatically added, 75 as well as the `icculster.epfl.ch`. 76 77 During the first run, the simulation will make sure that mininet is correctly 78 installed on the servers and will try to install it if not. If not all of the 79 servers are up (pingable and have SSH running) then you will receive 80 feedback in the terminal as to which is not up yet. 81 82 You can remove the file `server_list` in order to trigger the first time 83 server-config and setup behaviour again. 84 85 ### Debugging an installation 86 87 If something goes wrong, you can always try to run the 88 `onet/simul/platform/mininet/setup_iccluster.sh` 89 bash file to have mininet correctly installed. It is idempotent, you can 90 run it as many times as you need, in order to catch all the servers once they are 91 up. 92 93 ```bash 94 ./setup_iccluster.sh 31 32 33 95 ``` 96 97 Another simple test is to ssh to the remote machine and to run: 98 99 ```bash 100 mn -c 101 ``` 102 103 which should remove all running mininet sessions and return to the command line. 104 105 ### Monitor port occupied 106 107 Sometimes the ssh-forwarding is misbehaving, and you cannot run your simulation. 108 The simplest solution is to restart the servers (not reinstall, just power cycle it). 109 A faster option is to ssh to the first server and check if you can find the rogue 110 `sshd` process. 111 112 Take care, there is a `sshd`-process that listens for incoming 113 connections - if you kill that one, you will have to restart the server, as you 114 won't have access to it. 115 116 ### Other problems 117 118 You can always run the simulation with `-debug 3` to get more information and 119 see what is going wrong: 120 121 ```bash 122 go build && ./simul -platform mininet -debug 3 simul.toml 123 ``` 124 125 Then you will see more details of what is happening and you may see what 126 you need to change. 127 128 ### Development 129 130 You can run an Ubuntu server in a local virtual machine on your laptop. 131 You must configure it for bridged networking, sharing the same network as your 132 current network connection. When it boots, log in on it's console to find the 133 IP address. (There is some support for connecting via NAT port forwarding, 134 but full simulations can only be run with a real IP address, because the monitor 135 port needs to know the real IP address of the gateway machine.) 136 137 In order to match the configuration of the IC Cluster's Ubuntu 138 installs, you need to: 139 140 1. Use `passwd root` to set a root password. 141 1. Use `passwd -u root` to unlock it. 142 1. Edit /etc/ssh/sshd_config to set `PermitRootLogin yes`. Use `service ssh restart` afterwards. 143