github.com/observiq/carbon@v0.9.11-0.20200820160507-1b872e368a5e/docs/types/severity.md (about) 1 ## Severity Parsing 2 3 `carbon` uses a flexible severity parsing system based on the integers 0 to 100. Standard severities are provided at multiples of 10. 4 5 This severity system allows each output operator to interpret the values 0 to 100 as appropriate for the corresponding backend. 6 7 The following named severity levels are supported. 8 9 | Severity | Numeric Value | Alias | 10 | --- | --- | --- | 11 | Default | 0 | `default` | 12 | Trace | 10 | `trace` | 13 | Debug | 20 | `debug` | 14 | Info | 30 | `info` | 15 | Notice | 40 | `notice` | 16 | Warning | 50 | `warning` | 17 | Error | 60 | `error` | 18 | Critical | 70 | `critical` | 19 | Alert | 80 | `alert` | 20 | Emergency | 90 | `emergency` | 21 | Catastrophe | 100 | `catastrophe` | 22 23 24 ### `severity` parsing parameters 25 26 Parser operators can parse a severity and attach the resulting value to a log entry. 27 28 | Field | Default | Description | 29 | --- | --- | --- | 30 | `parse_from` | required | A [field](/docs/types/field.md) that indicates the field to be parsed as JSON | 31 | `preserve` | false | Preserve the unparsed value on the record | 32 | `preset` | `default` | A predefined set of values that should be interpretted at specific severity levels | 33 | `mapping` | | A custom set of values that should be interpretted at designated severity levels | 34 35 36 ### How severity `mapping` works 37 38 Severity parsing behavior is defined in a config file using a severity `mapping`. The general structure of the `mapping` is as follows: 39 40 ```yaml 41 ... 42 mapping: 43 severity_as_int_or_alias: value | list of values | range | special 44 severity_as_int_or_alias: value | list of values | range | special 45 ``` 46 47 The following example illustrates many of the ways in which mapping can configured: 48 ```yaml 49 ... 50 mapping: 51 52 # single value to be parsed as "error" 53 error: oops 54 55 # list of values to be parsed as "warning" 56 warning: 57 - hey! 58 - YSK 59 60 # range of values to be parsed as "info" 61 info: 62 - min: 300 63 max: 399 64 65 # special value representing the range 200-299, to be parsed as "debug" 66 debug: 2xx 67 68 # single value to be parsed as a custom level of 36 69 36: medium 70 71 # mix and match the above concepts 72 95: 73 - really serious 74 - min: 9001 75 max: 9050 76 - 5xx 77 ``` 78 79 ### How to simplify configuration with a `preset` 80 81 A `preset` can reduce the amount of configuration needed in the `mapping` structure by initializing the severity mapping with common values. Values specified in the more verbose `mapping` structure will then be added to the severity map. 82 83 By default, a common `preset` is used. Alternately, `preset: none` can be specified to start with an empty mapping. 84 85 The following configurations are equivalent: 86 87 ```yaml 88 ... 89 mapping: 90 error: 404 91 ``` 92 93 ```yaml 94 ... 95 preset: default 96 mapping: 97 error: 404 98 ``` 99 100 ```yaml 101 ... 102 preset: none 103 mapping: 104 trace: trace 105 debug: debug 106 info: info 107 notice: notice 108 warning: 109 - warning 110 - warn 111 error: 112 - error 113 - err 114 - 404 115 critical: 116 - critical 117 - crit 118 alert: alert 119 emergency: emergency 120 catastrophe: catastrophe 121 ``` 122 123 <sub>Additional built-in presets coming soon</sub> 124 125 126 ### How to use severity parsing 127 128 All parser operators, such as [`regex_parser`](/docs/operators/regex_parser.md) support these fields inside of a `severity` block. 129 130 If a severity block is specified, the parser operator will perform the severity parsing _after_ performing its other parsing actions, but _before_ passing the entry to the specified output operator. 131 132 ```yaml 133 - type: regex_parser 134 regexp: '^StatusCode=(?P<severity_field>\d{3}), Host=(?P<host>[^,]+)' 135 severity: 136 parse_from: severity_field 137 mapping: 138 critical: 5xx 139 error: 4xx 140 info: 3xx 141 debug: 2xx 142 ``` 143 144 --- 145 146 As a special case, the [`severity_parser`](/docs/operators/severity_parser.md) operator supports these fields inline. This is because severity parsing is the primary purpose of the operator. 147 ```yaml 148 - type: severity_parser 149 parse_from: severity_field 150 mapping: 151 critical: 5xx 152 error: 4xx 153 info: 3xx 154 debug: 2xx 155 ``` 156 157 ### Example Configurations 158 159 #### Parse a severity from a standard value 160 161 Configuration: 162 ```yaml 163 - type: severity_parser 164 parse_from: severity_field 165 ``` 166 167 Note that the default `preset` is in place, and no additional values have been specified. 168 169 <table> 170 <tr><td> Input entry </td> <td> Output entry </td></tr> 171 <tr> 172 <td> 173 174 ```json 175 { 176 "severity": 0, 177 "record": { 178 "severity_field": "ERROR" 179 } 180 } 181 ``` 182 183 </td> 184 <td> 185 186 ```json 187 { 188 "severity": 60, 189 "record": {} 190 } 191 ``` 192 193 </td> 194 </tr> 195 </table> 196 197 #### Parse a severity from a non-standard value 198 199 Configuration: 200 ```yaml 201 - type: severity_parser 202 parse_from: severity_field 203 mapping: 204 error: nooo! 205 ``` 206 207 Note that the default `preset` is in place, and one additional values has been specified. 208 209 <table> 210 <tr><td> Input entry </td> <td> Output entry </td></tr> 211 <tr> 212 <td> 213 214 ```json 215 { 216 "severity": 0, 217 "record": { 218 "severity_field": "nooo!" 219 } 220 } 221 ``` 222 223 </td> 224 <td> 225 226 ```json 227 { 228 "severity": 60, 229 "record": {} 230 } 231 ``` 232 233 </td> 234 </tr> 235 <tr> 236 <td> 237 238 ```json 239 { 240 "severity": 0, 241 "record": { 242 "severity_field": "ERROR" 243 } 244 } 245 ``` 246 247 </td> 248 <td> 249 250 ```json 251 { 252 "severity": 60, 253 "record": {} 254 } 255 ``` 256 257 </td> 258 </tr> 259 </table> 260 261 #### Parse a severity from any of several non-standard values 262 263 Configuration: 264 ```yaml 265 - type: severity_parser 266 parse_from: severity_field 267 mapping: 268 error: 269 - nooo! 270 - nooooooo 271 info: HEY 272 debug: 1234 273 ``` 274 275 <table> 276 <tr><td> Input entry </td> <td> Output entry </td></tr> 277 <tr> 278 <td> 279 280 ```json 281 { 282 "severity": 0, 283 "record": { 284 "severity_field": "nooo!" 285 } 286 } 287 ``` 288 289 </td> 290 <td> 291 292 ```json 293 { 294 "severity": 60, 295 "record": {} 296 } 297 ``` 298 299 </td> 300 </tr> 301 <tr> 302 <td> 303 304 ```json 305 { 306 "severity": 0, 307 "record": { 308 "severity_field": "nooooooo" 309 } 310 } 311 ``` 312 313 </td> 314 <td> 315 316 ```json 317 { 318 "severity": 60, 319 "record": {} 320 } 321 ``` 322 323 </td> 324 </tr> 325 <tr> 326 <td> 327 328 ```json 329 { 330 "severity": 0, 331 "record": { 332 "severity_field": "hey" 333 } 334 } 335 ``` 336 337 </td> 338 <td> 339 340 ```json 341 { 342 "severity": 30, 343 "record": {} 344 } 345 ``` 346 347 </td> 348 </tr> 349 <tr> 350 <td> 351 352 ```json 353 { 354 "severity": 0, 355 "record": { 356 "severity_field": 1234 357 } 358 } 359 ``` 360 361 </td> 362 <td> 363 364 ```json 365 { 366 "severity": 20, 367 "record": {} 368 } 369 ``` 370 371 </td> 372 </tr> 373 <tr> 374 <td> 375 376 ```json 377 { 378 "severity": 0, 379 "record": { 380 "severity_field": "unknown" 381 } 382 } 383 ``` 384 385 </td> 386 <td> 387 388 ```json 389 { 390 "severity": 0, 391 "record": {} 392 } 393 ``` 394 395 </td> 396 </tr> 397 </table> 398 399 #### Parse a severity from a range of values 400 401 Configuration: 402 ```yaml 403 - type: severity_parser 404 parse_from: severity_field 405 mapping: 406 error: 407 - min: 1 408 max: 5 409 alert: 410 - min: 6 411 max: 10 412 ``` 413 414 <table> 415 <tr><td> Input entry </td> <td> Output entry </td></tr> 416 <tr> 417 <td> 418 419 ```json 420 { 421 "severity": 0, 422 "record": { 423 "severity_field": 3 424 } 425 } 426 ``` 427 428 </td> 429 <td> 430 431 ```json 432 { 433 "severity": 60, 434 "record": {} 435 } 436 ``` 437 438 </td> 439 </tr> 440 <tr> 441 <td> 442 443 ```json 444 { 445 "severity": 0, 446 "record": { 447 "severity_field": 9 448 } 449 } 450 ``` 451 452 </td> 453 <td> 454 455 ```json 456 { 457 "severity": 80, 458 "record": {} 459 } 460 ``` 461 462 </td> 463 </tr> 464 <tr> 465 <td> 466 467 ```json 468 { 469 "severity": 0, 470 "record": { 471 "severity_field": 12 472 } 473 } 474 ``` 475 476 </td> 477 <td> 478 479 ```json 480 { 481 "severity": 0, 482 "record": {} 483 } 484 ``` 485 486 </td> 487 </tr> 488 </table> 489 490 #### Parse a severity from a HTTP Status Codes value 491 492 Special values are provided to represent http status code ranges. 493 494 | Value | Meaning | 495 | --- | --- | 496 | 2xx | 200 - 299 | 497 | 3xx | 300 - 399 | 498 | 4xx | 400 - 499 | 499 | 5xx | 500 - 599 | 500 501 Configuration: 502 ```yaml 503 - type: severity_parser 504 parse_from: severity_field 505 mapping: 506 critical: 5xx 507 error: 4xx 508 info: 3xx 509 debug: 2xx 510 ``` 511 512 Equivalent Configuration: 513 ```yaml 514 - id: my_severity_parser 515 type: severity_parser 516 parse_from: severity_field 517 mapping: 518 critical: 519 - min: 500 520 max: 599 521 error: 522 - min: 400 523 max: 499 524 info: 525 - min: 300 526 max: 399 527 debug: 528 - min: 200 529 max: 299 530 output: my_next_operator 531 ``` 532 533 <table> 534 <tr><td> Input entry </td> <td> Output entry </td></tr> 535 <tr> 536 <td> 537 538 ```json 539 { 540 "severity": 0, 541 "record": { 542 "severity_field": 302 543 } 544 } 545 ``` 546 547 </td> 548 <td> 549 550 ```json 551 { 552 "severity": 30, 553 "record": {} 554 } 555 ``` 556 557 </td> 558 </tr> 559 <tr> 560 <td> 561 562 ```json 563 { 564 "severity": 0, 565 "record": { 566 "severity_field": 404 567 } 568 } 569 ``` 570 571 </td> 572 <td> 573 574 ```json 575 { 576 "severity": 60, 577 "record": {} 578 } 579 ``` 580 581 </td> 582 </tr> 583 <tr> 584 <td> 585 586 ```json 587 { 588 "severity": 0, 589 "record": { 590 "severity_field": 200 591 } 592 } 593 ``` 594 595 </td> 596 <td> 597 598 ```json 599 { 600 "severity": 20, 601 "record": {} 602 } 603 ``` 604 605 </td> 606 </tr> 607 </table> 608 609 #### Parse a severity from a value without using the default preset 610 611 Configuration: 612 ```yaml 613 - type: severity_parser 614 parse_from: severity_field 615 preset: none 616 mapping: 617 error: nooo! 618 ``` 619 620 <table> 621 <tr><td> Input entry </td> <td> Output entry </td></tr> 622 <tr> 623 <td> 624 625 ```json 626 { 627 "severity": 0, 628 "record": { 629 "severity_field": "nooo!" 630 } 631 } 632 ``` 633 634 </td> 635 <td> 636 637 ```json 638 { 639 "severity": 60, 640 "record": {} 641 } 642 ``` 643 644 </td> 645 </tr> 646 <tr> 647 <td> 648 649 ```json 650 { 651 "severity": 0, 652 "record": { 653 "severity_field": "ERROR" 654 } 655 } 656 ``` 657 658 </td> 659 <td> 660 661 ```json 662 { 663 "severity": 0, 664 "record": {} 665 } 666 ``` 667 668 </td> 669 </tr> 670 </table>