github.com/choria-io/go-choria@v0.28.1-0.20240416190746-b3bf9c7d5a45/aagent/watchers/machineswatcher/README.md (about)

     1  ## Machines Watchers for Choria Autonomous Agents
     2  
     3  This contains an [Autonomous Agent](https://choria.io/docs/autoagents/) Watcher plugin capable of
     4  managing the typical `/etc/choria/machines` directory via Choria Key-Value Store and the `archive`
     5  watcher.
     6  
     7  In effect this allows you to Configuration Manage sets of Autonomous Agents on a fleet where you do not have
     8  other Configuration Management tools or where you just want to manage these out of band.
     9  
    10  ## Goals
    11  
    12  Create an opinionated manager for other machines that will safely and securely set up a server for hosting autonomous
    13  agents using the properties of the `archive` watcher.
    14  
    15  If this watcher is used it is one that would be compiled into the Choria binary and configured using KV.
    16  
    17  ## Autonomous Agent Archives
    18  
    19  These archives are prepared as per the instructions in the [archive watcher](../archivewatcher/README.md) with the following hard constraints:
    20  
    21  * The checksums file must be `SHA256SUMS` and must be present
    22  * The tar file must create a directory matching the name exactly, `yourmachine-1.2.3.tar.gz` must create `yourmachine`
    23  * Checksums of the `SHA256SUMS` file and the archive must be specified
    24  
    25  ### Configuring
    26  
    27  An Autonomous agent must be created that polls the Key-Value store and then configures the `machines` type watcher:
    28  
    29  ```yaml
    30  watchers:
    31    - name: desired_state
    32      type: kv
    33      interval: 1m
    34      state_match: [MANAGE]
    35      properties:
    36         bucket: MACHINES
    37         key: machines
    38         mode: poll
    39         bucket_prefix: false
    40      
    41    - name: manage_machines
    42      state_match: [MANAGE]
    43      type: machines
    44      interval: 1m
    45      state_match:
    46        - MANAGE
    47      properties:
    48        data_item: machines
    49        purge_unknown: true
    50        machine_manage_interval: 1m
    51        public_key: 64031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da
    52  ```
    53  
    54  Here we set an optional `public_key`, when this is set to a ed25519 public key it will verify and only accept data from the data store that has a valid signature signed using the corresponding private key.
    55  
    56  A keypair can be created using the signer command:
    57  
    58  ```go
    59  go run cmd/mms.go keys
    60   Public Key: 64031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da
    61  Private Key: d8bd4d6392af154e996a18a4ccd5f51931d8e861d42966a677d85fbb598b66d364031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da
    62  ```
    63  
    64  The data can now be created:
    65  
    66  ```nohighlight
    67  $ cat machines.json
    68  [
    69   {
    70     "name": "facts",
    71       "source": "https://my.example.net/metadata/metadata-machine-1.0.0.tgz",
    72       "verify": "SHA256SUMS",
    73       "verify_checksum": "1e85719c6959eb0f2c8f2166e30ae952ccaef2c286f31868ea1d311d3738a339",
    74       "checksum": "f11ea2005de97bf309bafac46e77c01925307a26675f44f388d4502d2b9d00bf",
    75       "match": "has_command('facter')"
    76   }
    77  ]
    78  $ go run cmd/mms.go pack machines.json d8bd4d6392af154e996a18a4ccd5f51931d8e861d42966a677d85fbb598b66d364031219d4922eed63a5f567303e98607c632139c01bc9fa4ca2514c2d9d30da > spec.json
    79  $ cat spec.json | choria kv put MACHINES machines -
    80  {"machines":"WwogewogICAibmFtZSI6ICJmYWN0cyIsCiAgICAgInNvdXJjZSI6ICJodHRwczovL215LmV4YW1wbGUubmV0L21ldGFkYXRhL21ldGFkYXRhLW1hY2hpbmUtMS4wLjAudGd6IiwKICAgICAidmVyaWZ5IjogIlNIQTI1NlNVTVMiLAogICAgICJ2ZXJpZnlfY2hlY2tzdW0iOiAiMWU4NTcxOWM2OTU5ZWIwZjJjOGYyMTY2ZTMwYWU5NTJjY2FlZjJjMjg2ZjMxODY4ZWExZDMxMWQzNzM4YTMzOSIsCiAgICAgImNoZWNrc3VtIjogImYxMWVhMjAwNWRlOTdiZjMwOWJhZmFjNDZlNzdjMDE5MjUzMDdhMjY2NzVmNDRmMzg4ZDQ1MDJkMmI5ZDAwYmYiLAogICAgICJtYXRjaCI6ICJoYXNfY29tbWFuZCgnZmFjdGVyJykiCiB9Cl0K","signature":"f06d4a1cfe9ac79d26b5e6646fdfa9d845a5506c9a2fe0a71fb8416f6f7edd253a1eb46363c12ca5f6148b19ab1ed9a5f25c89b09b3360a09b7d054bf4b55204"}
    81  ```
    82  
    83  After this the machines will be downloaded and maintained. In the `pack` command above the key is optional so the same command can be used to encode the specification without signing. They key can be read from the environment variable `KEY`.
    84  
    85  Note the `has_command('facter')` for the `matcher` key, this is a small [expr](https://github.com/expr-lang/expr) expression
    86  that is run on the node to determine if a specific machine should go on a node. The Key-Value is for the entire connected
    87  DC so in order to allow heterogeneous environments machines that should not go on the entire fleet can be limited using matchers.
    88  
    89  |Function|Description|
    90  |--------|-----------|
    91  |identity|Regular expression match over the machine identity|
    92  |has_file|Determines if a regular file is present on the machine|
    93  |has_directory|Determines if a directory is present on the machine|
    94  |has_command|Searches `PATH` for a command, note the `PATH` choria runs with is quite limited|
    95  
    96  The expression format is the typical used by Choria for example a match might be `identity('^web') && has_command('facter')`
    97  would do pretty much the right thing.
    98  
    99  ## Compiling Autonomous Agents into Choria
   100  
   101  Current `main` of Choria also supports compiling Autonomous Agents into Choria, if you're really paranoid or strict
   102  you would compile the above autonomous agent into Choria and use it to bootstrap others in a trusted manner from a trusted
   103  source allowing just the properties you want to be adjusted via Key-Value Store.
   104  
   105  In this mode you can even forgo the entire Key-Value integration and compile urls and all checksums right into the binary
   106  for the truly paranoid.
   107  
   108  ```go
   109  package metamgr
   110  
   111  import (
   112  	"github.com/choria-io/go-choria/aagent/machine"
   113  	"github.com/choria-io/go-choria/aagent/plugin"
   114  )
   115  
   116  func ChoriaPlugin() *plugin.MachinePlugin {
   117  	return plugin.NewMachinePlugin("metamgr", &machine.Machine{
   118  		MachineName: "metamgr",
   119  		InitialState: "MANAGE",
   120  		// rest of the autonomous agent
   121      })
   122  }
   123  ```
   124  
   125  You can now include this file in the `user_plugins.yaml` and it will be compiled in, see below example.  This way you have
   126  an unmodifiable way to bootstrap a trusted set of Autonomous Agents onto new servers without needing Configuration Management
   127  
   128  ## Compiling into Choria
   129  
   130  Compiling this into Choria is reasonably simple, assuming you already figured out how to compile choria :-)
   131  
   132  ```yaml
   133  # packager/user_plugins.yaml
   134  archive_watcher: github.com/choria-io/go-choria/aagent/watchers/archivewatcher
   135  machines_watcher: github.com/choria-io/go-choria/aagent/watchers/machineswatcher
   136  machines_manager: github.com/choria-io/go-choria/aagent/watchers/machineswatcher/manager
   137  ```
   138  
   139  Do `go generate` and recompile, this will include the watcher.