github.com/influxdata/telegraf@v1.30.3/docs/developers/STATE_PERSISTENCE.md (about) 1 # State-persistence for plugins 2 3 ## Purpose 4 5 Plugin state-persistence allows a plugin to save its state across restarts of 6 Telegraf. This might be necessary if data-input (or output) is stateful and 7 depends on the result of a previous operation. 8 9 If you for example query data from a service providing a `next` token, your 10 plugin would need to know the last token received in order to make the next 11 query. However, this token is lost after a restart of Telegraf if not persisted 12 and thus your only chance is to restart the query chain potentially resulting 13 in handling redundant data producing unnecessary traffic. 14 15 This is where state-persistence comes into play. The state-persistence framework 16 allows your plugin to store a _state_ on shutdown and load that _state_ again 17 on startup of Telegraf. 18 19 ## State format 20 21 The _state_ of a plugin can be any structure or datatype that is serializable 22 using Golang's JSON serializer. It can be a key-value map or a more complex 23 structure. E.g. 24 25 ```go 26 type MyState struct { 27 CurrentToken string 28 LastToken string 29 NextToken string 30 FilterIDs []int64 31 } 32 ``` 33 34 would represent a valid state. 35 36 ## Implementation 37 38 To enable state-persistence in your plugin you need to implement the 39 `StatefulPlugin` interface defined in `plugin.go`. The interface looks as 40 follows: 41 42 ```go 43 type StatefulPlugin interface { 44 GetState() interface{} 45 SetState(state interface{}) error 46 } 47 ``` 48 49 The `GetState()` function should return the current state of the plugin 50 (see [state format](#state-format)). Please note that this function should 51 _always_ succeed and should always be callable directly after `Init()`. So make 52 sure your relevant data-structures are initialized in `Init` to prevent panics. 53 54 Telegraf will call the `GetState()` function on shutdown and will then compile 55 an overall Telegraf state from the information of all stateful plugins. This 56 state is then persisted to disk if (and only if) the `statefile` option in the 57 `agent` section is set. You do _not_ need take care of any serialization or 58 writing, Telegraf will handle this for you. 59 60 When starting Telegraf, the overall persisted Telegraf state will be restored, 61 if `statefile` is set. To do so, the `SetState()` function is called with the 62 deserialized state of the plugin. Please note that this function is called 63 directly _after_ the `Init()` function of your plugin. You need to make sure 64 that the given state is what you expect using a type-assertion! Make sure this 65 won't panic but rather return a meaningful error. 66 67 To assign the state to the correct plugin, Telegraf relies on a plugin ID. 68 See the ["State assignment" section](#state-assignment) for more details on 69 the procedure and ["Plugin Identifier" section](#plugin-identifier) for more 70 details on ID generation. 71 72 ## State assignment 73 74 When restoring the state on loading, Telegraf needs to ensure that each plugin 75 _instance_ gets the correct state. To do so, a plugin ID is used. By default 76 this ID is generated automatically for each plugin instance but can be 77 overwritten if necessary (see [Plugin Identifier](#plugin-identifier)). 78 79 State assignment needs to be able to handle multiple instances of the same 80 plugin type correctly, e.g. if the user has configured multiple instances of 81 your plugin with different `server` settings. Here, the state saved for 82 `foo.example.com` needs to be restored to the plugin instance handling 83 `foo.example.com` on next startup of Telegraf and should _not_ end up at server 84 `bar.example.com`. So the plugin identifier used for the assignment should be 85 consistent over restarts of Telegraf. 86 87 In case plugin instances are added to the configuration between restarts, no 88 state is restored _for those instances_. Furthermore, all states referencing 89 plugin identifier that are no-longer valid are dropped and will be ignored. This 90 can happen in case plugin instances are removed or changed in ID. 91 92 ## Plugin Identifier 93 94 As outlined above, the plugin identifier (plugin ID) is crucial when assigning 95 states to plugin instances. By default, Telegraf will automatically generate an 96 identifier for each plugin configured when starting up. The ID is consistent 97 over restarts of Telegraf and is based on the _entire configuration_ of the 98 plugin. This means for each plugin instance, all settings in the configuration 99 will be concatenated and hashed to derive the ID. The resulting ID will then be 100 used in both save and restore operations making sure the state ends up in a 101 plugin with _exactly_ the same configuration that created the state. 102 103 However, this also means that the plugin identifier _is changing_ whenever _any_ 104 of the configuration setting is changed! For example if your plugin is defined 105 as 106 107 ```go 108 type MyPlugin struct { 109 Server string `toml:"server"` 110 Token string `toml:"token"` 111 Timeout config.Duration `toml:"timeout"` 112 113 offset int 114 } 115 ``` 116 117 with `offset` being your state, the plugin ID will change if a user changes the 118 `timeout` setting in the configuration file. As a consequence the state cannot 119 be restored. This might be undesirable for your plugin, therefore you can 120 overwrite the ID generation by implementing the `PluginWithID` interface (see 121 `plugin.go`). This interface defines a `ID() string` function returning the 122 identifier o the current plugin _instance_. When implementing this function you 123 should take the following criteria into account: 124 125 1. The identifier has to be _unique_ for your plugin _instance_ (not only for 126 the plugin type) to make sure the state is assigned to the correct instance. 127 1. The identifier has to be _consistent_ across startups/restarts of Telegraf 128 as otherwise the state cannot be restored. Make sure the order of 129 configuration settings doesn't matter. 130 1. Make sure to _include all settings relevant for state assignment_. In 131 the example above, the plugin's `token` setting might or might not be 132 relevant to identify the plugin instance. 133 1. Make sure to _leave out all settings irrelevant for state assignment_. In 134 the example above, the plugin's `timeout` setting likely is not relevant 135 for the state and can be left out. 136 137 Which settings are relevant for the state are plugin specific. For example, if 138 the `offset` is a property of the _server_ the `token` setting is irrelevant. 139 However, if the `offset` is specific for a certain user suddenly the `token` 140 setting is relevant. 141 142 Alternatively to generating an identifier automatically, the plugin can allow 143 the user to specify that ID directly in a configuration setting. However, please 144 not that this might lead to colliding IDs in larger setups and should thus be 145 avoided.