github.com/observiq/carbon@v0.9.11-0.20200820160507-1b872e368a5e/docs/operators/restructure.md (about)

     1  ## `restructure` operator
     2  
     3  The `restructure` operator facilitates changing the structure of a record by adding, removing, moving, and flattening fields.
     4  
     5  The operator is configured with a list of ops, which are small operations that are applied to a record in the order
     6  they are defined.
     7  
     8  ### Configuration Fields
     9  
    10  | Field      | Default          | Description                                                                                     |
    11  | ---        | ---              | ---                                                                                             |
    12  | `id`       | `restructure`    | A unique identifier for the operator                                                            |
    13  | `output`   | Next in pipeline | The connected operator(s) that will receive all outbound entries                                |
    14  | `ops`      | required         | A list of ops. The available op types are defined below                                         |
    15  | `on_error` | `send`           | The behavior of the operator if it encounters an error. See [on_error](/docs/types/on_error.md) |
    16  
    17  ### Op types
    18  
    19  #### Add
    20  
    21  The `add` op adds a field to a record. It must have a `field` key and exactly one of `value` or `value_expr`.
    22  
    23  `field` is a [field](/docs/types/field.md) that will be set to `value` or the result of `value_expr`
    24  
    25  `value` is a static string that will be added to each entry at the field defined by `field`
    26  
    27  `value_expr` is an [expression](/docs/types/expression.md) with access to the `record` object
    28  
    29  Example usage:
    30  ```yaml
    31  - type: restructure
    32    ops:
    33      - add:
    34          field: "key1"
    35          value: "val1"
    36      - add:
    37          field: "key2"
    38          value_expr: 'record["key1"] + "-suffix"'
    39  ```
    40  
    41  <table>
    42  <tr><td> Input record </td> <td> Output record </td></tr>
    43  <tr>
    44  <td>
    45  
    46  ```json
    47  {}
    48  ```
    49  
    50  </td>
    51  <td>
    52  
    53  ```json
    54  {
    55    "key1": "val1",
    56    "key2": "val1-suffix"
    57  }
    58  ```
    59  
    60  </td>
    61  </tr>
    62  </table>
    63  
    64  #### Remove
    65  
    66  The `remove` op removes a field from a record.
    67  
    68  Example usage:
    69  ```yaml
    70  - type: restructure
    71    ops:
    72      - remove: "key1"
    73  ```
    74  
    75  <table>
    76  <tr><td> Input record </td> <td> Output record </td></tr>
    77  <tr>
    78  <td>
    79  
    80  ```json
    81  {
    82    "key1": "val1",
    83    "key2": "val2"
    84  }
    85  ```
    86  
    87  </td>
    88  <td>
    89  
    90  ```json
    91  {
    92    "key2": "val2"
    93  }
    94  ```
    95  
    96  </td>
    97  </tr>
    98  </table>
    99  
   100  #### Retain
   101  
   102  The `retain` op keeps the specified list of fields, and removes the rest.
   103  
   104  Example usage:
   105  ```yaml
   106  - type: restructure
   107    ops:
   108      - retain:
   109        - "key1"
   110        - "key2"
   111  ```
   112  
   113  <table>
   114  <tr><td> Input record </td> <td> Output record </td></tr>
   115  <tr>
   116  <td>
   117  
   118  ```json
   119  {
   120    "key1": "val1",
   121    "key2": "val2",
   122    "key3": "val3",
   123    "key4": "val4"
   124  }
   125  ```
   126  
   127  </td>
   128  <td>
   129  
   130  ```json
   131  {
   132    "key1": "val1",
   133    "key2": "val2"
   134  }
   135  ```
   136  
   137  </td>
   138  </tr>
   139  </table>
   140  
   141  #### Move
   142  
   143  The `move` op moves (or renames) a field from one location to another. Both the `from` and `to` fields are required.
   144  
   145  Example usage:
   146  ```yaml
   147  - type: restructure
   148    ops:
   149      - move:
   150          from: "key1"
   151          to: "key3"
   152  ```
   153  
   154  <table>
   155  <tr><td> Input record </td> <td> Output record </td></tr>
   156  <tr>
   157  <td>
   158  
   159  ```json
   160  {
   161    "key1": "val1",
   162    "key2": "val2"
   163  }
   164  ```
   165  
   166  </td>
   167  <td>
   168  
   169  ```json
   170  {
   171    "key3": "val1",
   172    "key2": "val2"
   173  }
   174  ```
   175  
   176  </td>
   177  </tr>
   178  </table>
   179  
   180  #### Flatten
   181  
   182  The `flatten` op flattens a field by moving its children up to the same level as the field.
   183  
   184  Example usage:
   185  ```yaml
   186  - type: restructure
   187    ops:
   188      - flatten: "key1"
   189  ```
   190  
   191  <table>
   192  <tr><td> Input record </td> <td> Output record </td></tr>
   193  <tr>
   194  <td>
   195  
   196  ```json
   197  {
   198    "key1": {
   199      "nested1": "nestedval1",
   200      "nested2": "nestedval2"
   201    },
   202    "key2": "val2"
   203  }
   204  ```
   205  
   206  </td>
   207  <td>
   208  
   209  ```json
   210  {
   211    "nested1": "nestedval1",
   212    "nested2": "nestedval2",
   213    "key2": "val2"
   214  }
   215  ```
   216  
   217  </td>
   218  </tr>
   219  </table>