github.com/influxdata/telegraf@v1.30.3/docs/developers/STATE_PERSISTENCE.md (about)

     1  # State-persistence for plugins
     2  
     3  ## Purpose
     4  
     5  Plugin state-persistence allows a plugin to save its state across restarts of
     6  Telegraf. This might be necessary if data-input (or output) is stateful and
     7  depends on the result of a previous operation.
     8  
     9  If you for example query data from a service providing a `next` token, your
    10  plugin would need to know the last token received in order to make the next
    11  query. However, this token is lost after a restart of Telegraf if not persisted
    12  and thus your only chance is to restart the query chain potentially resulting
    13  in handling redundant data producing unnecessary traffic.
    14  
    15  This is where state-persistence comes into play. The state-persistence framework
    16  allows your plugin to store a _state_ on shutdown and load that _state_ again
    17  on startup of Telegraf.
    18  
    19  ## State format
    20  
    21  The _state_ of a plugin can be any structure or datatype that is serializable
    22  using Golang's JSON serializer. It can be a key-value map or a more complex
    23  structure. E.g.
    24  
    25  ```go
    26  type MyState struct {
    27      CurrentToken string
    28      LastToken    string
    29      NextToken    string
    30      FilterIDs    []int64
    31  }
    32  ```
    33  
    34  would represent a valid state.
    35  
    36  ## Implementation
    37  
    38  To enable state-persistence in your plugin you need to implement the
    39  `StatefulPlugin` interface defined in `plugin.go`. The interface looks as
    40  follows:
    41  
    42  ```go
    43  type StatefulPlugin interface {
    44      GetState() interface{}
    45      SetState(state interface{}) error
    46  }
    47  ```
    48  
    49  The `GetState()` function should return the current state of the plugin
    50  (see [state format](#state-format)). Please note that this function should
    51  _always_ succeed and should always be callable directly after `Init()`. So make
    52  sure your relevant data-structures are initialized in `Init` to prevent panics.
    53  
    54  Telegraf will call the `GetState()` function on shutdown and will then compile
    55  an overall Telegraf state from the information of all stateful plugins. This
    56  state is then persisted to disk if (and only if) the `statefile` option in the
    57  `agent` section is set. You do _not_ need take care of any serialization or
    58  writing, Telegraf will handle this for you.
    59  
    60  When starting Telegraf, the overall persisted Telegraf state will be restored,
    61  if `statefile` is set. To do so, the `SetState()` function is called with the
    62  deserialized state of the plugin. Please note that this function is called
    63  directly _after_ the `Init()` function of your plugin. You need to make sure
    64  that the given state is what you expect using a type-assertion! Make sure this
    65  won't panic but rather return a meaningful error.
    66  
    67  To assign the state to the correct plugin, Telegraf relies on a plugin ID.
    68  See the ["State assignment" section](#state-assignment) for more details on
    69  the procedure and ["Plugin Identifier" section](#plugin-identifier) for more
    70  details on ID generation.
    71  
    72  ## State assignment
    73  
    74  When restoring the state on loading, Telegraf needs to ensure that each plugin
    75  _instance_ gets the correct state. To do so, a plugin ID is used. By default
    76  this ID is generated automatically for each plugin instance but can be
    77  overwritten if necessary (see [Plugin Identifier](#plugin-identifier)).
    78  
    79  State assignment needs to be able to handle multiple instances of the same
    80  plugin type correctly, e.g. if the user has configured multiple instances of
    81  your plugin with different `server` settings. Here, the state saved for
    82  `foo.example.com` needs to be restored to the plugin instance handling
    83  `foo.example.com` on next startup of Telegraf and should _not_ end up at server
    84  `bar.example.com`. So the plugin identifier used for the assignment should be
    85  consistent over restarts of Telegraf.
    86  
    87  In case plugin instances are added to the configuration between restarts, no
    88  state is restored _for those instances_. Furthermore, all states referencing
    89  plugin identifier that are no-longer valid are dropped and will be ignored. This
    90  can happen in case plugin instances are removed or changed in ID.
    91  
    92  ## Plugin Identifier
    93  
    94  As outlined above, the plugin identifier (plugin ID) is crucial when assigning
    95  states to plugin instances. By default, Telegraf will automatically generate an
    96  identifier for each plugin configured when starting up. The ID is consistent
    97  over restarts of Telegraf and is based on the _entire configuration_ of the
    98  plugin. This means for each plugin instance, all settings in the configuration
    99  will be concatenated and hashed to derive the ID. The resulting ID will then be
   100  used in both save and restore operations making sure the state ends up in a
   101  plugin with _exactly_ the same configuration that created the state.
   102  
   103  However, this also means that the plugin identifier _is changing_ whenever _any_
   104  of the configuration setting is changed! For example if your plugin is defined
   105  as
   106  
   107  ```go
   108  type MyPlugin struct {
   109      Server  string          `toml:"server"`
   110      Token   string          `toml:"token"`
   111      Timeout config.Duration `toml:"timeout"`
   112  
   113      offset int
   114  }
   115  ```
   116  
   117  with `offset` being your state, the plugin ID will change if a user changes the
   118  `timeout` setting in the configuration file. As a consequence the state cannot
   119  be restored. This might be undesirable for your plugin, therefore you can
   120  overwrite the ID generation by implementing the `PluginWithID` interface (see
   121  `plugin.go`). This interface defines a `ID() string` function returning the
   122  identifier o the current plugin _instance_. When implementing this function you
   123  should take the following criteria into account:
   124  
   125  1. The identifier has to be _unique_ for your plugin _instance_ (not only for
   126     the plugin type) to make sure the state is assigned to the correct instance.
   127  1. The identifier has to be _consistent_ across startups/restarts of Telegraf
   128     as otherwise the state cannot be restored. Make sure the order of
   129     configuration settings doesn't matter.
   130  1. Make sure to _include all settings relevant for state assignment_. In
   131     the example above, the plugin's `token` setting might or might not be
   132     relevant to identify the plugin instance.
   133  1. Make sure to _leave out all settings irrelevant for state assignment_. In
   134     the example above, the plugin's `timeout` setting likely is not relevant
   135     for the state and can be left out.
   136  
   137  Which settings are relevant for the state are plugin specific. For example, if
   138  the `offset` is a property of the _server_ the `token` setting is irrelevant.
   139  However, if the `offset` is specific for a certain user suddenly the `token`
   140  setting is relevant.
   141  
   142  Alternatively to generating an identifier automatically, the plugin can allow
   143  the user to specify that ID directly in a configuration setting. However, please
   144  not that this might lead to colliding IDs in larger setups and should thus be
   145  avoided.