github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/unarchive.md (about)

     1  ---
     2  title: unarchive
     3  type: processor
     4  status: stable
     5  categories: ["Parsing","Utility"]
     6  ---
     7  
     8  <!--
     9       THIS FILE IS AUTOGENERATED!
    10  
    11       To make changes please edit the contents of:
    12       lib/processor/unarchive.go
    13  -->
    14  
    15  import Tabs from '@theme/Tabs';
    16  import TabItem from '@theme/TabItem';
    17  
    18  
    19  Unarchives messages according to the selected archive [format](#formats) into
    20  multiple messages within a [batch](/docs/configuration/batching).
    21  
    22  
    23  <Tabs defaultValue="common" values={[
    24    { label: 'Common', value: 'common', },
    25    { label: 'Advanced', value: 'advanced', },
    26  ]}>
    27  
    28  <TabItem value="common">
    29  
    30  ```yaml
    31  # Common config fields, showing default values
    32  label: ""
    33  unarchive:
    34    format: binary
    35  ```
    36  
    37  </TabItem>
    38  <TabItem value="advanced">
    39  
    40  ```yaml
    41  # All config fields, showing default values
    42  label: ""
    43  unarchive:
    44    format: binary
    45    parts: []
    46  ```
    47  
    48  </TabItem>
    49  </Tabs>
    50  
    51  When a message is unarchived the new messages replace the original message in
    52  the batch. Messages that are selected but fail to unarchive (invalid format)
    53  will remain unchanged in the message batch but will be flagged as having failed,
    54  allowing you to [error handle them](/docs/configuration/error_handling).
    55  
    56  For the unarchive formats that contain file information (tar, zip), a metadata
    57  field is added to each message called `archive_filename` with the
    58  extracted filename.
    59  
    60  ## Fields
    61  
    62  ### `format`
    63  
    64  The unarchive [format](#formats) to use.
    65  
    66  
    67  Type: `string`  
    68  Default: `"binary"`  
    69  Options: `tar`, `zip`, `binary`, `lines`, `json_documents`, `json_array`, `json_map`, `csv`.
    70  
    71  ### `parts`
    72  
    73  An optional array of message indexes of a batch that the processor should apply to.
    74  If left empty all messages are processed. This field is only applicable when
    75  batching messages [at the input level](/docs/configuration/batching).
    76  
    77  Indexes can be negative, and if so the part will be selected from the end
    78  counting backwards starting from -1.
    79  
    80  
    81  Type: `array`  
    82  Default: `[]`  
    83  
    84  ## Formats
    85  
    86  ### `tar`
    87  
    88  Extract messages from a unix standard tape archive.
    89  
    90  ### `zip`
    91  
    92  Extract messages from a zip file.
    93  
    94  ### `binary`
    95  
    96  Extract messages from a binary blob format consisting of:
    97  
    98  - Four bytes containing number of messages in the batch (in big endian)
    99  - For each message part:
   100    + Four bytes containing the length of the message (in big endian)
   101    + The content of message
   102  
   103  ### `lines`
   104  
   105  Extract the lines of a message each into their own message.
   106  
   107  ### `json_documents`
   108  
   109  Attempt to parse a message as a stream of concatenated JSON documents. Each
   110  parsed document is expanded into a new message.
   111  
   112  ### `json_array`
   113  
   114  Attempt to parse a message as a JSON array, and extract each element into its
   115  own message.
   116  
   117  ### `json_map`
   118  
   119  Attempt to parse the message as a JSON map and for each element of the map
   120  expands its contents into a new message. A metadata field is added to each
   121  message called `archive_key` with the relevant key from the top-level
   122  map.
   123  
   124  ### `csv`
   125  
   126  Attempt to parse the message as a csv file (header required) and for each row in 
   127  the file expands its contents into a json object in a new message.
   128