github.com/number571/tendermint@v0.34.11-gost/docs/networks/terraform-and-ansible.md (about)

     1  ---
     2  order: 3
     3  ---
     4  
     5  # Terraform & Ansible
     6  
     7  > Note: These commands/files are not being maintained by the tendermint team currently. Please use them carefully.
     8  
     9  Automated deployments are done using
    10  [Terraform](https://www.terraform.io/) to create servers on Digital
    11  Ocean then [Ansible](http://www.ansible.com/) to create and manage
    12  testnets on those servers.
    13  
    14  ## Install
    15  
    16  NOTE: see the [integration bash
    17  script](https://github.com/number571/tendermint/blob/master/networks/remote/integration.sh)
    18  that can be run on a fresh DO droplet and will automatically spin up a 4
    19  node testnet. The script more or less does everything described below.
    20  
    21  - Install [Terraform](https://www.terraform.io/downloads.html) and
    22    [Ansible](http://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
    23    on a Linux machine.
    24  - Create a [DigitalOcean API
    25    token](https://cloud.digitalocean.com/settings/api/tokens) with read
    26    and write capability.
    27  - Install the python dopy package (`pip install dopy`)
    28  - Create SSH keys (`ssh-keygen`)
    29  - Set environment variables:
    30  
    31  ```sh
    32  export DO_API_TOKEN="abcdef01234567890abcdef01234567890"
    33  export SSH_KEY_FILE="$HOME/.ssh/id_rsa.pub"
    34  ```
    35  
    36  These will be used by both `terraform` and `ansible`.
    37  
    38  ## Terraform
    39  
    40  This step will create four Digital Ocean droplets. First, go to the
    41  correct directory:
    42  
    43  ```sh
    44  cd $GOPATH/src/github.com/number571/tendermint/networks/remote/terraform
    45  ```
    46  
    47  then:
    48  
    49  ```sh
    50  terraform init
    51  terraform apply -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
    52  ```
    53  
    54  and you will get a list of IP addresses that belong to your droplets.
    55  
    56  With the droplets created and running, let's setup Ansible.
    57  
    58  ## Ansible
    59  
    60  The playbooks in [the ansible
    61  directory](https://github.com/number571/tendermint/tree/master/networks/remote/ansible)
    62  run ansible roles to configure the sentry node architecture. You must
    63  switch to this directory to run ansible
    64  (`cd $GOPATH/src/github.com/number571/tendermint/networks/remote/ansible`).
    65  
    66  There are several roles that are self-explanatory:
    67  
    68  First, we configure our droplets by specifying the paths for tendermint
    69  (`BINARY`) and the node files (`CONFIGDIR`). The latter expects any
    70  number of directories named `node0, node1, ...` and so on (equal to the
    71  number of droplets created).
    72  
    73  To create the node files run:
    74  
    75  ```sh
    76  tendermint testnet
    77  ```
    78  
    79  Then, to configure our droplets run:
    80  
    81  ```sh
    82  ansible-playbook -i inventory/digital_ocean.py -l sentrynet config.yml -e BINARY=$GOPATH/src/github.com/number571/tendermint/build/tendermint -e CONFIGDIR=$GOPATH/src/github.com/number571/tendermint/networks/remote/ansible/mytestnet
    83  ```
    84  
    85  Voila! All your droplets now have the `tendermint` binary and required
    86  configuration files to run a testnet.
    87  
    88  Next, we run the install role:
    89  
    90  ```sh
    91  ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
    92  ```
    93  
    94  which as you'll see below, executes
    95  `tendermint node --proxy-app=kvstore` on all droplets. Although we'll
    96  soon be modifying this role and running it again, this first execution
    97  allows us to get each `node_info.id` that corresponds to each
    98  `node_info.listen_addr`. (This part will be automated in the future). In
    99  your browser (or using `curl`), for every droplet, go to IP:26657/status
   100  and note the two just mentioned `node_info` fields. Notice that blocks
   101  aren't being created (`latest_block_height` should be zero and not
   102  increasing).
   103  
   104  Next, open `roles/install/templates/systemd.service.j2` and look for the
   105  line `ExecStart` which should look something like:
   106  
   107  ```sh
   108  ExecStart=/usr/bin/tendermint node --proxy-app=kvstore
   109  ```
   110  
   111  and add the `--p2p.persistent-peers` flag with the relevant information
   112  for each node. The resulting file should look something like:
   113  
   114  ```sh
   115  [Unit]
   116  Description={{service}}
   117  Requires=network-online.target
   118  After=network-online.target
   119  
   120  [Service]
   121  Restart=on-failure
   122  User={{service}}
   123  Group={{service}}
   124  PermissionsStartOnly=true
   125  ExecStart=/usr/bin/tendermint node --proxy-app=kvstore --p2p.persistent-peers=167b80242c300bf0ccfb3ced3dec60dc2a81776e@165.227.41.206:26656,3c7a5920811550c04bf7a0b2f1e02ab52317b5e6@165.227.43.146:26656,303a1a4312c30525c99ba66522dd81cca56a361a@159.89.115.32:26656,b686c2a7f4b1b46dca96af3a0f31a6a7beae0be4@159.89.119.125:26656
   126  ExecReload=/bin/kill -HUP $MAINPID
   127  KillSignal=SIGTERM
   128  
   129  [Install]
   130  WantedBy=multi-user.target
   131  ```
   132  
   133  Then, stop the nodes:
   134  
   135  ```sh
   136  ansible-playbook -i inventory/digital_ocean.py -l sentrynet stop.yml
   137  ```
   138  
   139  Finally, we run the install role again:
   140  
   141  ```sh
   142  ansible-playbook -i inventory/digital_ocean.py -l sentrynet install.yml
   143  ```
   144  
   145  to re-run `tendermint node` with the new flag, on all droplets. The
   146  `latest_block_hash` should now be changing and `latest_block_height`
   147  increasing. Your testnet is now up and running :)
   148  
   149  Peek at the logs with the status role:
   150  
   151  ```sh
   152  ansible-playbook -i inventory/digital_ocean.py -l sentrynet status.yml
   153  ```
   154  
   155  ## Logging
   156  
   157  The crudest way is the status role described above. You can also ship
   158  logs to Logz.io, an Elastic stack (Elastic search, Logstash and Kibana)
   159  service provider. You can set up your nodes to log there automatically.
   160  Create an account and get your API key from the notes on [this
   161  page](https://app.logz.io/#/dashboard/data-sources/Filebeat), then:
   162  
   163  ```sh
   164  yum install systemd-devel || echo "This will only work on RHEL-based systems."
   165  apt-get install libsystemd-dev || echo "This will only work on Debian-based systems."
   166  
   167  go get github.com/mheese/journalbeat
   168  ansible-playbook -i inventory/digital_ocean.py -l sentrynet logzio.yml -e LOGZIO_TOKEN=ABCDEFGHIJKLMNOPQRSTUVWXYZ012345
   169  ```
   170  
   171  ## Cleanup
   172  
   173  To remove your droplets, run:
   174  
   175  ```sh
   176  terraform destroy -var DO_API_TOKEN="$DO_API_TOKEN" -var SSH_KEY_FILE="$SSH_KEY_FILE"
   177  ```