go.dedis.ch/onet/v3@v3.2.11-0.20210930124529-e36530bca7ef/simul/platform/MININET.md (about)

     1  Navigation: [DEDIS](https://github.com/dedis/doc/tree/master/README.md) ::
     2  [Onet](../../README.md) ::
     3  [Simulation](../README.md) ::
     4  Mininet Simulation
     5  
     6  # Mininet Simulation
     7  
     8  Mininet allows to have a better control over the delay and bandwidth restrictions
     9  during the simulation. While [Deterlab](DETERLAB.md) only has these restrictions
    10  between servers, but not necessarily between all nodes, Mininet will set up the
    11  same restriction between any two nodes that communcate with each other.
    12  
    13  The Mininet simulation uses http://mininet.org/ to simulate a network where each
    14  host represents a Cothority node. It has been extensively tested on the
    15  hardware of https://iccluster.epfl.ch, but should theoretically also run on
    16  other kinds of hardware. As it needs to set up the mininet environment, it is
    17  more fragile than the Deterlab simulations. But server availability in EPFL
    18  is often better than on Deterlab.
    19  
    20  Each T3-server on the iccluster is a 24-core server with 256GB of RAM and can
    21  run around 300 cothority nodes simultaneously. So if you want to run a
    22  simulation with 2000 nodes, you need at least 7 physical servers.
    23  
    24  ## Setting up ICCluster
    25  
    26  Supposing you want to run your simulation using the iccluster-network, you
    27  first need to reserve the machines. You have to do so one day in advance, as
    28  it is not possible to reserve the machines on the same day. Once the machines
    29  are ready, you need to install them.
    30  
    31  1. In the ICCluster-admin interface, you need to go to `My Servers` -> `Setup`.
    32  2. Chose the servers you reserved - take care, as there might be servers from
    33  other people in the same lab!
    34  3. `Choose a boot option` - please take _Ubuntu focal 2004_ for best results.
    35  As of Dec 2020, Ubuntu 14.04 still appears to work, but it is no longer supported by
    36  Ubuntu and should not be used for new work.
    37  4. `Customization` - chose a password (it can be very simple) for the setup.
    38  5. `Run Setup` and confirm the setup. THIS WILL DELETE ALL DATA ON THE SERVERS!
    39  
    40  ### Verifying everything is correctly set up
    41  
    42  The power-cycle and OS installation process takes several minutes, and the IC Cluster
    43  UI does not have any feedback to know the state of the servers. As a result,
    44  you need to use `ping` and `ssh` to monitor them.
    45  
    46  ```bash
    47  ping iccluster033.iccluster.epfl.ch
    48  ```
    49  
    50  If you are outside of EPFL, you need to use the VPN, as iccluster is only
    51  reachable from inside EPFL.
    52  
    53  You can also connect to the server with
    54  
    55  ```bash
    56  ssh root@iccluster033.iccluster.epfl.ch
    57  ```
    58  
    59  and enter your password that you gave in the `Customization` step above.
    60  
    61  ## Running a simulation
    62  
    63  Now you are finally ready to run your simulation. We suppose that you have
    64  your simulation running successfully under [Localhost](LOCALHOST.md). Then
    65  all you need to do is:
    66  
    67  ```bash
    68  go build && ./simul -platform mininet simul.toml
    69  ```
    70  
    71  If it is the first time the simulation is run, it will ask you whether you want
    72  to use iccluster or not. If you reply `Y` (or simply press `Enter`), you can
    73  give the names of the servers you reserved and installed. An example is:
    74  `31 32 33`. You don't need to enter leading `0`s, this will be automatically added,
    75  as well as the `icculster.epfl.ch`.
    76  
    77  During the first run, the simulation will make sure that mininet is correctly
    78  installed on the servers and will try to install it if not. If not all of the
    79  servers are up (pingable and have SSH running) then you will receive
    80  feedback in the terminal as to which is not up yet.
    81  
    82  You can remove the file `server_list` in order to trigger the first time
    83  server-config and setup behaviour again.
    84  
    85  ### Debugging an installation
    86  
    87  If something goes wrong, you can always try to run the
    88  `onet/simul/platform/mininet/setup_iccluster.sh`
    89  bash file to have mininet correctly installed. It is idempotent, you can
    90  run it as many times as you need, in order to catch all the servers once they are
    91  up.
    92  
    93  ```bash
    94  ./setup_iccluster.sh 31 32 33
    95  ```
    96  
    97  Another simple test is to ssh to the remote machine and to run:
    98  
    99  ```bash
   100  mn -c
   101  ```
   102  
   103  which should remove all running mininet sessions and return to the command line.
   104  
   105  ### Monitor port occupied
   106  
   107  Sometimes the ssh-forwarding is misbehaving, and you cannot run your simulation.
   108  The simplest solution is to restart the servers (not reinstall, just power cycle it).
   109  A faster option is to ssh to the first server and check if you can find the rogue
   110  `sshd` process.
   111  
   112  Take care, there is a `sshd`-process that listens for incoming
   113  connections - if you kill that one, you will have to restart the server, as you
   114  won't have access to it.
   115  
   116  ### Other problems
   117  
   118  You can always run the simulation with `-debug 3` to get more information and
   119  see what is going wrong:
   120  
   121  ```bash
   122  go build && ./simul -platform mininet -debug 3 simul.toml
   123  ```
   124  
   125  Then you will see more details of what is happening and you may see what
   126  you need to change.
   127  
   128  ### Development
   129  
   130  You can run an Ubuntu server in a local virtual machine on your laptop.
   131  You must configure it for bridged networking, sharing the same network as your
   132  current network connection. When it boots, log in on it's console to find the
   133  IP address. (There is some support for connecting via NAT port forwarding,
   134  but full simulations can only be run with a real IP address, because the monitor
   135  port needs to know the real IP address of the gateway machine.)
   136  
   137  In order to match the configuration of the IC Cluster's Ubuntu
   138  installs, you need to:
   139  
   140  1. Use `passwd root` to set a root password.
   141  1. Use `passwd -u root` to unlock it.
   142  1. Edit /etc/ssh/sshd_config to set `PermitRootLogin yes`. Use `service ssh restart` afterwards.
   143