github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/processors/parse_log.md (about) 1 --- 2 title: parse_log 3 type: processor 4 status: stable 5 categories: ["Parsing"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/processor/parse_log.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 19 Parses common log [formats](#formats) into [structured data](#codecs). This is 20 easier and often much faster than [`grok`](/docs/components/processors/grok). 21 22 23 <Tabs defaultValue="common" values={[ 24 { label: 'Common', value: 'common', }, 25 { label: 'Advanced', value: 'advanced', }, 26 ]}> 27 28 <TabItem value="common"> 29 30 ```yaml 31 # Common config fields, showing default values 32 label: "" 33 parse_log: 34 format: syslog_rfc5424 35 codec: json 36 ``` 37 38 </TabItem> 39 <TabItem value="advanced"> 40 41 ```yaml 42 # All config fields, showing default values 43 label: "" 44 parse_log: 45 format: syslog_rfc5424 46 codec: json 47 best_effort: true 48 allow_rfc3339: true 49 default_year: current 50 default_timezone: UTC 51 parts: [] 52 ``` 53 54 </TabItem> 55 </Tabs> 56 57 ## Fields 58 59 ### `format` 60 61 A common log [format](#formats) to parse. 62 63 64 Type: `string` 65 Default: `"syslog_rfc5424"` 66 Options: `syslog_rfc5424`, `syslog_rfc3164`. 67 68 ### `codec` 69 70 Specifies the structured format to parse a log into. 71 72 73 Type: `string` 74 Default: `"json"` 75 Options: `json`. 76 77 ### `best_effort` 78 79 Still returns partially parsed messages even if an error occurs. 80 81 82 Type: `bool` 83 Default: `true` 84 85 ### `allow_rfc3339` 86 87 Also accept timestamps in rfc3339 format while parsing. Applicable to format `syslog_rfc3164`. 88 89 90 Type: `bool` 91 Default: `true` 92 93 ### `default_year` 94 95 Sets the strategy used to set the year for rfc3164 timestamps. Applicable to format `syslog_rfc3164`. When set to `current` the current year will be set, when set to an integer that value will be used. Leave this field empty to not set a default year at all. 96 97 98 Type: `string` 99 Default: `"current"` 100 101 ### `default_timezone` 102 103 Sets the strategy to decide the timezone for rfc3164 timestamps. Applicable to format `syslog_rfc3164`. This value should follow the [time.LoadLocation](https://golang.org/pkg/time/#LoadLocation) format. 104 105 106 Type: `string` 107 Default: `"UTC"` 108 109 ### `parts` 110 111 An optional array of message indexes of a batch that the processor should apply to. 112 If left empty all messages are processed. This field is only applicable when 113 batching messages [at the input level](/docs/configuration/batching). 114 115 Indexes can be negative, and if so the part will be selected from the end 116 counting backwards starting from -1. 117 118 119 Type: `array` 120 Default: `[]` 121 122 ## Codecs 123 124 Currently the only supported structured data codec is `json`. 125 126 ## Formats 127 128 ### `syslog_rfc5424` 129 130 Attempts to parse a log following the [Syslog rfc5424](https://tools.ietf.org/html/rfc5424) 131 spec. The resulting structured document may contain any of the following fields: 132 133 - `message` (string) 134 - `timestamp` (string, RFC3339) 135 - `facility` (int) 136 - `severity` (int) 137 - `priority` (int) 138 - `version` (int) 139 - `hostname` (string) 140 - `procid` (string) 141 - `appname` (string) 142 - `msgid` (string) 143 - `structureddata` (object) 144 145 ### `syslog_rfc3164` 146 147 Attempts to parse a log following the [Syslog rfc3164](https://tools.ietf.org/html/rfc3164) 148 spec. The resulting structured document may contain any of the following fields: 149 150 - `message` (string) 151 - `timestamp` (string, RFC3339) 152 - `facility` (int) 153 - `severity` (int) 154 - `priority` (int) 155 - `hostname` (string) 156 - `procid` (string) 157 - `appname` (string) 158 - `msgid` (string) 159 160