github.com/eliastor/durgaform@v0.0.0-20220816172711-d0ab2d17673e/docs/plugin-protocol/object-wire-format.md (about) 1 # Wire Format for Durgaform Objects and Associated Values 2 3 The provider wire protocol (as of major version 5) includes a protobuf message 4 type `DynamicValue` which Durgaform uses to represent values from the Terraform 5 Language type system, which result from evaluating the content of `resource`, 6 `data`, and `provider` blocks, based on a schema defined by the corresponding 7 provider. 8 9 Because the structure of these values is determined at runtime, `DynamicValue` 10 uses one of two possible dynamic serialization formats for the values 11 themselves: MessagePack or JSON. Durgaform most commonly uses MessagePack, 12 because it offers a compact binary representation of a value. However, a server 13 implementation of the provider protocol should fall back to JSON if the 14 MessagePack field is not populated, in order to support both formats. 15 16 The remainder of this document describes how Durgaform translates from its own 17 type system into the type system of the two supported serialization formats. 18 A server implementation of the Durgaform provider protocol can use this 19 information to decode `DynamicValue` values from incoming messages into 20 whatever representation is convenient for the provider implementation. 21 22 A server implementation must also be able to _produce_ `DynamicValue` messages 23 as part of various response messages. When doing so, servers should always 24 use MessagePack encoding, because Durgaform does not consistently support 25 JSON responses across all request types and all Durgaform versions. 26 27 Both the MessagePack and JSON serializations are driven by information the 28 provider previously returned in a `Schema` message. Durgaform will encode each 29 value depending on the type constraint given for it in the corresponding schema, 30 using the closest possible MessagePack or JSON type to the Durgaform language 31 type. Therefore a server implementation can decode a serialized value using a 32 standard MessagePack or JSON library and assume it will conform to the 33 serialization rules described below. 34 35 ## MessagePack Serialization Rules 36 37 The MessagePack types referenced in this section are those defined in 38 [The MessagePack type system specification](https://github.com/msgpack/msgpack/blob/master/spec.md#type-system). 39 40 Note that MessagePack defines several possible serialization formats for each 41 type, and Durgaform may choose any of the formats of a specified type. 42 The exact serialization chosen for a given value may vary between Durgaform 43 versions, but the types given here are contractual. 44 45 Conversely, server implementations that are _producing_ MessagePack-encoded 46 values are free to use any of the valid serialization formats for a particular 47 type. However, we recommend choosing the most compact format that can represent 48 the value without a loss of range. 49 50 ### `Schema.Block` Mapping Rules for MessagePack 51 52 To represent the content of a block as MessagePack, Durgaform constructs a 53 MessagePack map that contains one key-value pair per attribute and one 54 key-value pair per distinct nested block described in the `Schema.Block` message. 55 56 The key-value pairs representing attributes have values based on 57 [the `Schema.Attribute` mapping rules](#Schema.Attribute-mapping-rules-for-messagepack). 58 The key-value pairs representing nested block types have values based on 59 [the `Schema.NestedBlock` mapping rules](#Schema.NestedBlock-mapping-rules-for-messagepack). 60 61 ### `Schema.Attribute` Mapping Rules for MessagePack 62 63 The MessagePack serialization of an attribute value depends on the value of the 64 `type` field of the corresponding `Schema.Attribute` message. The `type` field is 65 a compact JSON serialization of a 66 [Durgaform type constraint](https://www.durgaform.io/docs/configuration/types.html), 67 which consists either of a single 68 string value (for primitive types) or a two-element array giving a type kind 69 and a type argument. 70 71 The following table describes the type-specific mapping rules. Along with those 72 type-specific rules there are two special rules that override the mappings 73 in the table below, regardless of type: 74 75 * A null value is represented as a MessagePack nil value. 76 * An unknown value (that is, a placeholder for a value that will be decided 77 only during the apply operation) is represented as a 78 [MessagePack extension](https://github.com/msgpack/msgpack/blob/master/spec.md#extension-types) 79 value whose type identifier is zero and whose value is unspecified and 80 meaningless. 81 82 | `type` Pattern | MessagePack Representation | 83 |---|---| 84 | `"string"` | A MessagePack string containing the Unicode characters from the string value serialized as normalized UTF-8. | 85 | `"number"` | Either MessagePack integer, MessagePack float, or MessagePack string representing the number. If a number is represented as a string then the string contains a decimal representation of the number which may have a larger mantissa than can be represented by a 64-bit float. | 86 | `"bool"` | A MessagePack boolean value corresponding to the value. | 87 | `["list",T]` | A MessagePack array with the same number of elements as the list value, each of which is represented by the result of applying these same mapping rules to the nested type `T`. | 88 | `["set",T]` | Identical in representation to `["list",T]`, but the order of elements is undefined because Durgaform sets are unordered. | 89 | `["map",T]` | A MessagePack map with one key-value pair per element of the map value, where the element key is serialized as the map key (always a MessagePack string) and the element value is represented by a value constructed by applying these same mapping rules to the nested type `T`. | 90 | `["object",ATTRS]` | A MessagePack map with one key-value pair per attribute defined in the `ATTRS` object. The attribute name is serialized as the map key (always a MessagePack string) and the attribute value is represented by a value constructed by applying these same mapping rules to each attribute's own type. | 91 | `["tuple",TYPES]` | A MessagePack array with one element per element described by the `TYPES` array. The element values are constructed by applying these same mapping rules to the corresponding element of `TYPES`. | 92 | `"dynamic"` | A MessagePack array with exactly two elements. The first element is a MessagePack binary value containing a JSON-serialized type constraint in the same format described in this table. The second element is the result of applying these same mapping rules to the value with the type given in the first element. This special type constraint represents values whose types will be decided only at runtime. | 93 94 ### `Schema.NestedBlock` Mapping Rules for MessagePack 95 96 The MessagePack serialization of a collection of blocks of a particular type 97 depends on the `nesting` field of the corresponding `Schema.NestedBlock` message. 98 The `nesting` field is a value from the `Schema.NestingBlock.NestingMode` 99 enumeration. 100 101 All `nesting` values cause the individual blocks of a type to be represented 102 by applying 103 [the `Schema.Block` mapping rules](#Schema.Block-mapping-rules-for-messagepack) 104 to the block's contents based on the `block` field, producing what we'll call 105 a _block value_ in the table below. 106 107 The `nesting` value then in turn defines how Durgaform will collect all of the 108 individual block values together to produce a single property value representing 109 the nested block type. For all `nesting` values other than `MAP`, blocks may 110 not have any labels. For the `nesting` value `MAP`, blocks must have exactly 111 one label, which is a string we'll call a _block label_ in the table below. 112 113 | `nesting` Value | MessagePack Representation | 114 |---|---| 115 | `SINGLE` | The block value of the single block of this type, or nil if there is no block of that type. | 116 | `LIST` | A MessagePack array of all of the block values, preserving the order of definition of the blocks in the configuration. | 117 | `SET` | A MessagePack array of all of the block values in no particular order. | 118 | `MAP` | A MessagePack map with one key-value pair per block value, where the key is the block label and the value is the block value. | 119 | `GROUP` | The same as with `SINGLE`, except that if there is no block of that type Durgaform will synthesize a block value by pretending that all of the declared attributes are null and that there are zero blocks of each declared block type. | 120 121 For the `LIST` and `SET` nesting modes, Durgaform guarantees that the 122 MessagePack array will have a number of elements between the `min_items` and 123 `max_items` values given in the schema, _unless_ any of the block values contain 124 nested unknown values. When unknown values are present, Durgaform considers 125 the value to be potentially incomplete and so Durgaform defers validation of 126 the number of blocks. For example, if the configuration includes a `dynamic` 127 block whose `for_each` argument is unknown then the final number of blocks is 128 not predictable until the apply phase. 129 130 ## JSON Serialization Rules 131 132 The JSON serialization is a secondary representation for `DynamicValue`, with 133 MessagePack preferred due to its ability to represent unknown values via an 134 extension. 135 136 The JSON encoding described in this section is also used for the `json` field 137 of the `RawValue` message that forms part of an `UpgradeResourceState` request. 138 However, in that case the data is serialized per the schema of the provider 139 version that created it, which won't necessarily match the schema of the 140 _current_ version of that provider. 141 142 ### `Schema.Block` Mapping Rules for JSON 143 144 To represent the content of a block as JSON, Durgaform constructs a 145 JSON object that contains one property per attribute and one property per 146 distinct nested block described in the `Schema.Block` message. 147 148 The properties representing attributes have property values based on 149 [the `Schema.Attribute` mapping rules](#Schema.Attribute-mapping-rules-for-json). 150 The properties representing nested block types have property values based on 151 [the `Schema.NestedBlock` mapping rules](#Schema.NestedBlock-mapping-rules-for-json). 152 153 ### `Schema.Attribute` Mapping Rules for JSON 154 155 The JSON serialization of an attribute value depends on the value of the `type` 156 field of the corresponding `Schema.Attribute` message. The `type` field is 157 a compact JSON serialization of a 158 [Durgaform type constraint](https://www.durgaform.io/docs/configuration/types.html), 159 which consists either of a single 160 string value (for primitive types) or a two-element array giving a type kind 161 and a type argument. 162 163 The following table describes the type-specific mapping rules. Along with those 164 type-specific rules there is one special rule that overrides the rules in the 165 table regardless of type: 166 167 * A null value is always represented as JSON `null`. 168 169 | `type` Pattern | JSON Representation | 170 |---|---| 171 | `"string"` | A JSON string containing the Unicode characters from the string value. | 172 | `"number"` | A JSON number representing the number value. Durgaform numbers are arbitrary-precision floating point, so the value may have a larger mantissa than can be represented by a 64-bit float. | 173 | `"bool"` | Either JSON `true` or JSON `false`, depending on the boolean value. | 174 | `["list",T]` | A JSON array with the same number of elements as the list value, each of which is represented by the result of applying these same mapping rules to the nested type `T`. | 175 | `["set",T]` | Identical in representation to `["list",T]`, but the order of elements is undefined because Durgaform sets are unordered. | 176 | `["map",T]` | A JSON object with one property per element of the map value, where the element key is serialized as the property name string and the element value is represented by a property value constructed by applying these same mapping rules to the nested type `T`. | 177 | `["object",ATTRS]` | A JSON object with one property per attribute defined in the `ATTRS` object. The attribute name is serialized as the property name string and the attribute value is represented by a property value constructed by applying these same mapping rules to each attribute's own type. | 178 | `["tuple",TYPES]` | A JSON array with one element per element described by the `TYPES` array. The element values are constructed by applying these same mapping rules to the corresponding element of `TYPES`. | 179 | `"dynamic"` | A JSON object with two properties: `"type"` specifying one of the `type` patterns described in this table in-band, giving the exact runtime type of the value, and `"value"` specifying the result of applying these same mapping rules to the table for the specified runtime type. This special type constraint represents values whose types will be decided only at runtime. | 180 181 ### `Schema.NestedBlock` Mapping Rules for JSON 182 183 The JSON serialization of a collection of blocks of a particular type depends 184 on the `nesting` field of the corresponding `Schema.NestedBlock` message. 185 The `nesting` field is a value from the `Schema.NestingBlock.NestingMode` 186 enumeration. 187 188 All `nesting` values cause the individual blocks of a type to be represented 189 by applying 190 [the `Schema.Block` mapping rules](#Schema.Block-mapping-rules-for-json) 191 to the block's contents based on the `block` field, producing what we'll call 192 a _block value_ in the table below. 193 194 The `nesting` value then in turn defines how Durgaform will collect all of the 195 individual block values together to produce a single property value representing 196 the nested block type. For all `nesting` values other than `MAP`, blocks may 197 not have any labels. For the `nesting` value `MAP`, blocks must have exactly 198 one label, which is a string we'll call a _block label_ in the table below. 199 200 | `nesting` Value | JSON Representation | 201 |---|---| 202 | `SINGLE` | The block value of the single block of this type, or `null` if there is no block of that type. | 203 | `LIST` | A JSON array of all of the block values, preserving the order of definition of the blocks in the configuration. | 204 | `SET` | A JSON array of all of the block values in no particular order. | 205 | `MAP` | A JSON object with one property per block value, where the property name is the block label and the value is the block value. | 206 | `GROUP` | The same as with `SINGLE`, except that if there is no block of that type Durgaform will synthesize a block value by pretending that all of the declared attributes are null and that there are zero blocks of each declared block type. | 207 208 For the `LIST` and `SET` nesting modes, Durgaform guarantees that the JSON 209 array will have a number of elements between the `min_items` and `max_items` 210 values given in the schema.