github.com/observiq/carbon@v0.9.11-0.20200820160507-1b872e368a5e/docs/types/severity.md (about)

     1  ## Severity Parsing
     2  
     3  `carbon` uses a flexible severity parsing system based on the integers 0 to 100. Standard severities are provided at multiples of 10.
     4  
     5  This severity system allows each output operator to interpret the values 0 to 100 as appropriate for the corresponding backend.
     6  
     7  The following named severity levels are supported.
     8  
     9  | Severity    | Numeric Value | Alias         |
    10  | ---         | ---           | ---           |
    11  | Default     |        0      | `default`     |
    12  | Trace       |       10      | `trace`       |
    13  | Debug       |       20      | `debug`       |
    14  | Info        |       30      | `info`        |
    15  | Notice      |       40      | `notice`      |
    16  | Warning     |       50      | `warning`     |
    17  | Error       |       60      | `error`       |
    18  | Critical    |       70      | `critical`    |
    19  | Alert       |       80      | `alert`       |
    20  | Emergency   |       90      | `emergency`   |
    21  | Catastrophe |      100      | `catastrophe` |
    22  
    23  
    24  ### `severity` parsing parameters
    25  
    26  Parser operators can parse a severity and attach the resulting value to a log entry.
    27  
    28  | Field          | Default   | Description                                                                        |
    29  | ---            | ---       | ---                                                                                |
    30  | `parse_from`   | required  | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON      |
    31  | `preserve`     | false     | Preserve the unparsed value on the record                                          |
    32  | `preset`       | `default` | A predefined set of values that should be interpretted at specific severity levels |
    33  | `mapping`      |           | A custom set of values that should be interpretted at designated severity levels   |
    34  
    35  
    36  ### How severity `mapping` works
    37  
    38  Severity parsing behavior is defined in a config file using a severity `mapping`. The general structure of the `mapping` is as follows:
    39  
    40  ```yaml
    41  ...
    42    mapping:
    43      severity_as_int_or_alias: value | list of values | range | special
    44      severity_as_int_or_alias: value | list of values | range | special
    45  ```
    46  
    47  The following example illustrates many of the ways in which mapping can configured:
    48  ```yaml
    49  ...
    50    mapping:
    51  
    52      # single value to be parsed as "error"
    53      error: oops
    54  
    55      # list of values to be parsed as "warning"
    56      warning:
    57        - hey!
    58        - YSK
    59  
    60      # range of values to be parsed as "info"
    61      info:
    62        - min: 300
    63          max: 399
    64  
    65      # special value representing the range 200-299, to be parsed as "debug"
    66      debug: 2xx
    67  
    68      # single value to be parsed as a custom level of 36
    69      36: medium
    70  
    71      # mix and match the above concepts
    72      95:
    73        - really serious
    74        - min: 9001
    75          max: 9050
    76        - 5xx
    77  ```
    78  
    79  ### How to simplify configuration with a `preset`
    80  
    81  A `preset` can reduce the amount of configuration needed in the `mapping` structure by initializing the severity mapping with common values. Values specified in the more verbose `mapping` structure will then be added to the severity map.
    82  
    83  By default, a common `preset` is used. Alternately, `preset: none` can be specified to start with an empty mapping.
    84  
    85  The following configurations are equivalent:
    86  
    87  ```yaml
    88  ...
    89    mapping:
    90      error: 404
    91  ```
    92  
    93  ```yaml
    94  ...
    95    preset: default
    96    mapping:
    97      error: 404
    98  ```
    99  
   100  ```yaml
   101  ...
   102    preset: none
   103    mapping:
   104      trace: trace
   105      debug: debug
   106      info: info
   107      notice: notice
   108      warning:
   109        - warning
   110        - warn
   111      error:
   112        - error
   113        - err
   114        - 404
   115      critical:
   116        - critical
   117        - crit
   118      alert: alert
   119      emergency: emergency
   120      catastrophe: catastrophe
   121  ```
   122  
   123  <sub>Additional built-in presets coming soon</sub>
   124  
   125  
   126  ### How to use severity parsing
   127  
   128  All parser operators, such as [`regex_parser`](/docs/operators/regex_parser.md) support these fields inside of a `severity` block.
   129  
   130  If a severity block is specified, the parser operator will perform the severity parsing _after_ performing its other parsing actions, but _before_ passing the entry to the specified output operator.
   131  
   132  ```yaml
   133  - type: regex_parser
   134    regexp: '^StatusCode=(?P<severity_field>\d{3}), Host=(?P<host>[^,]+)'
   135    severity:
   136      parse_from: severity_field
   137      mapping:
   138        critical: 5xx
   139        error: 4xx
   140        info: 3xx
   141        debug: 2xx
   142  ```
   143  
   144  ---
   145  
   146  As a special case, the [`severity_parser`](/docs/operators/severity_parser.md) operator supports these fields inline. This is because severity parsing is the primary purpose of the operator.
   147  ```yaml
   148  - type: severity_parser
   149    parse_from: severity_field
   150    mapping:
   151      critical: 5xx
   152      error: 4xx
   153      info: 3xx
   154      debug: 2xx
   155  ```
   156  
   157  ### Example Configurations
   158  
   159  #### Parse a severity from a standard value
   160  
   161  Configuration:
   162  ```yaml
   163  - type: severity_parser
   164    parse_from: severity_field
   165  ```
   166  
   167  Note that the default `preset` is in place, and no additional values have been specified.
   168  
   169  <table>
   170  <tr><td> Input entry </td> <td> Output entry </td></tr>
   171  <tr>
   172  <td>
   173  
   174  ```json
   175  {
   176    "severity": 0,
   177    "record": {
   178      "severity_field": "ERROR"
   179    }
   180  }
   181  ```
   182  
   183  </td>
   184  <td>
   185  
   186  ```json
   187  {
   188    "severity": 60,
   189    "record": {}
   190  }
   191  ```
   192  
   193  </td>
   194  </tr>
   195  </table>
   196  
   197  #### Parse a severity from a non-standard value
   198  
   199  Configuration:
   200  ```yaml
   201  - type: severity_parser
   202    parse_from: severity_field
   203    mapping:
   204      error: nooo!
   205  ```
   206  
   207  Note that the default `preset` is in place, and one additional values has been specified.
   208  
   209  <table>
   210  <tr><td> Input entry </td> <td> Output entry </td></tr>
   211  <tr>
   212  <td>
   213  
   214  ```json
   215  {
   216    "severity": 0,
   217    "record": {
   218      "severity_field": "nooo!"
   219    }
   220  }
   221  ```
   222  
   223  </td>
   224  <td>
   225  
   226  ```json
   227  {
   228    "severity": 60,
   229    "record": {}
   230  }
   231  ```
   232  
   233  </td>
   234  </tr>
   235  <tr>
   236  <td>
   237  
   238  ```json
   239  {
   240    "severity": 0,
   241    "record": {
   242      "severity_field": "ERROR"
   243    }
   244  }
   245  ```
   246  
   247  </td>
   248  <td>
   249  
   250  ```json
   251  {
   252    "severity": 60,
   253    "record": {}
   254  }
   255  ```
   256  
   257  </td>
   258  </tr>
   259  </table>
   260  
   261  #### Parse a severity from any of several non-standard values
   262  
   263  Configuration:
   264  ```yaml
   265  - type: severity_parser
   266    parse_from: severity_field
   267    mapping:
   268      error:
   269        - nooo!
   270        - nooooooo
   271      info: HEY
   272      debug: 1234
   273  ```
   274  
   275  <table>
   276  <tr><td> Input entry </td> <td> Output entry </td></tr>
   277  <tr>
   278  <td>
   279  
   280  ```json
   281  {
   282    "severity": 0,
   283    "record": {
   284      "severity_field": "nooo!"
   285    }
   286  }
   287  ```
   288  
   289  </td>
   290  <td>
   291  
   292  ```json
   293  {
   294    "severity": 60,
   295    "record": {}
   296  }
   297  ```
   298  
   299  </td>
   300  </tr>
   301  <tr>
   302  <td>
   303  
   304  ```json
   305  {
   306    "severity": 0,
   307    "record": {
   308      "severity_field": "nooooooo"
   309    }
   310  }
   311  ```
   312  
   313  </td>
   314  <td>
   315  
   316  ```json
   317  {
   318    "severity": 60,
   319    "record": {}
   320  }
   321  ```
   322  
   323  </td>
   324  </tr>
   325  <tr>
   326  <td>
   327  
   328  ```json
   329  {
   330    "severity": 0,
   331    "record": {
   332      "severity_field": "hey"
   333    }
   334  }
   335  ```
   336  
   337  </td>
   338  <td>
   339  
   340  ```json
   341  {
   342    "severity": 30,
   343    "record": {}
   344  }
   345  ```
   346  
   347  </td>
   348  </tr>
   349  <tr>
   350  <td>
   351  
   352  ```json
   353  {
   354    "severity": 0,
   355    "record": {
   356      "severity_field": 1234
   357    }
   358  }
   359  ```
   360  
   361  </td>
   362  <td>
   363  
   364  ```json
   365  {
   366    "severity": 20,
   367    "record": {}
   368  }
   369  ```
   370  
   371  </td>
   372  </tr>
   373  <tr>
   374  <td>
   375  
   376  ```json
   377  {
   378    "severity": 0,
   379    "record": {
   380      "severity_field": "unknown"
   381    }
   382  }
   383  ```
   384  
   385  </td>
   386  <td>
   387  
   388  ```json
   389  {
   390    "severity": 0,
   391    "record": {}
   392  }
   393  ```
   394  
   395  </td>
   396  </tr>
   397  </table>
   398  
   399  #### Parse a severity from a range of values
   400  
   401  Configuration:
   402  ```yaml
   403  - type: severity_parser
   404    parse_from: severity_field
   405    mapping:
   406      error:
   407        - min: 1
   408          max: 5
   409      alert:
   410        - min: 6
   411          max: 10
   412  ```
   413  
   414  <table>
   415  <tr><td> Input entry </td> <td> Output entry </td></tr>
   416  <tr>
   417  <td>
   418  
   419  ```json
   420  {
   421    "severity": 0,
   422    "record": {
   423      "severity_field": 3
   424    }
   425  }
   426  ```
   427  
   428  </td>
   429  <td>
   430  
   431  ```json
   432  {
   433    "severity": 60,
   434    "record": {}
   435  }
   436  ```
   437  
   438  </td>
   439  </tr>
   440  <tr>
   441  <td>
   442  
   443  ```json
   444  {
   445    "severity": 0,
   446    "record": {
   447      "severity_field": 9
   448    }
   449  }
   450  ```
   451  
   452  </td>
   453  <td>
   454  
   455  ```json
   456  {
   457    "severity": 80,
   458    "record": {}
   459  }
   460  ```
   461  
   462  </td>
   463  </tr>
   464  <tr>
   465  <td>
   466  
   467  ```json
   468  {
   469    "severity": 0,
   470    "record": {
   471      "severity_field": 12
   472    }
   473  }
   474  ```
   475  
   476  </td>
   477  <td>
   478  
   479  ```json
   480  {
   481    "severity": 0,
   482    "record": {}
   483  }
   484  ```
   485  
   486  </td>
   487  </tr>
   488  </table>
   489  
   490  #### Parse a severity from a HTTP Status Codes value
   491  
   492  Special values are provided to represent http status code ranges.
   493  
   494  | Value | Meaning   |
   495  | ---   | ---       |
   496  | 2xx   | 200 - 299 |
   497  | 3xx   | 300 - 399 |
   498  | 4xx   | 400 - 499 |
   499  | 5xx   | 500 - 599 |
   500  
   501  Configuration:
   502  ```yaml
   503  - type: severity_parser
   504    parse_from: severity_field
   505    mapping:
   506      critical: 5xx
   507      error: 4xx
   508      info: 3xx
   509      debug: 2xx
   510  ```
   511  
   512  Equivalent Configuration:
   513  ```yaml
   514  - id: my_severity_parser
   515    type: severity_parser
   516    parse_from: severity_field
   517    mapping:
   518      critical:
   519        - min: 500
   520          max: 599
   521      error:
   522        - min: 400
   523          max: 499
   524      info:
   525        - min: 300
   526          max: 399
   527      debug:
   528        - min: 200
   529          max: 299
   530    output: my_next_operator
   531  ```
   532  
   533  <table>
   534  <tr><td> Input entry </td> <td> Output entry </td></tr>
   535  <tr>
   536  <td>
   537  
   538  ```json
   539  {
   540    "severity": 0,
   541    "record": {
   542      "severity_field": 302
   543    }
   544  }
   545  ```
   546  
   547  </td>
   548  <td>
   549  
   550  ```json
   551  {
   552    "severity": 30,
   553    "record": {}
   554  }
   555  ```
   556  
   557  </td>
   558  </tr>
   559  <tr>
   560  <td>
   561  
   562  ```json
   563  {
   564    "severity": 0,
   565    "record": {
   566      "severity_field": 404
   567    }
   568  }
   569  ```
   570  
   571  </td>
   572  <td>
   573  
   574  ```json
   575  {
   576    "severity": 60,
   577    "record": {}
   578  }
   579  ```
   580  
   581  </td>
   582  </tr>
   583  <tr>
   584  <td>
   585  
   586  ```json
   587  {
   588    "severity": 0,
   589    "record": {
   590      "severity_field": 200
   591    }
   592  }
   593  ```
   594  
   595  </td>
   596  <td>
   597  
   598  ```json
   599  {
   600    "severity": 20,
   601    "record": {}
   602  }
   603  ```
   604  
   605  </td>
   606  </tr>
   607  </table>
   608  
   609  #### Parse a severity from a value without using the default preset
   610  
   611  Configuration:
   612  ```yaml
   613  - type: severity_parser
   614    parse_from: severity_field
   615    preset: none
   616    mapping:
   617      error: nooo!
   618  ```
   619  
   620  <table>
   621  <tr><td> Input entry </td> <td> Output entry </td></tr>
   622  <tr>
   623  <td>
   624  
   625  ```json
   626  {
   627    "severity": 0,
   628    "record": {
   629      "severity_field": "nooo!"
   630    }
   631  }
   632  ```
   633  
   634  </td>
   635  <td>
   636  
   637  ```json
   638  {
   639    "severity": 60,
   640    "record": {}
   641  }
   642  ```
   643  
   644  </td>
   645  </tr>
   646  <tr>
   647  <td>
   648  
   649  ```json
   650  {
   651    "severity": 0,
   652    "record": {
   653      "severity_field": "ERROR"
   654    }
   655  }
   656  ```
   657  
   658  </td>
   659  <td>
   660  
   661  ```json
   662  {
   663    "severity": 0,
   664    "record": {}
   665  }
   666  ```
   667  
   668  </td>
   669  </tr>
   670  </table>