github.com/Jeffail/benthos/v3@v3.65.0/website/docs/guides/bloblang/methods.md

---
title: Bloblang Methods
sidebar_label: Methods
description: A list of Bloblang methods
---

<!--
     THIS FILE IS AUTOGENERATED!

     To make changes please edit the contents of:
     internal/bloblang/query/methods.go
     internal/bloblang/query/methods_strings.go
     internal/docs/bloblang.go
-->

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

Methods provide most of the power in Bloblang as they allow you to augment values and can be added to any expression (including other methods):

```coffee
root.doc.id = this.thing.id.string().catch(uuid_v4())
root.doc.reduced_nums = this.thing.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})
root.has_good_taste = ["pikachu","mewtwo","magmar"].contains(this.user.fav_pokemon)
```

Methods support both named and nameless style arguments:

```coffee
root.foo_one = this.(bar | baz).trim().replace(old: "dog", new: "cat")
root.foo_two = this.(bar | baz).trim().replace("dog", "cat")
```

## General

### `apply`

Apply a declared mapping to a target value.

#### Parameters

**`mapping`** &lt;string&gt; The mapping to apply.  

#### Examples


```coffee
map thing {
  root.inner = this.first
}

root.foo = this.doc.apply("thing")

# In:  {"doc":{"first":"hello world"}}
# Out: {"foo":{"inner":"hello world"}}
```

```coffee
map create_foo {
  root.name = "a foo"
  root.purpose = "to be a foo"
}

root = this
root.foo = null.apply("create_foo")

# In:  {"id":"1234"}
# Out: {"foo":{"name":"a foo","purpose":"to be a foo"},"id":"1234"}
```

### `catch`

If the result of a target query fails (due to incorrect types, failed parsing, etc) the argument is returned instead.

#### Parameters

**`fallback`** &lt;query expression&gt; A value to yield, or query to execute, if the target query fails.  

#### Examples


```coffee
root.doc.id = this.thing.id.string().catch(uuid_v4())
```

When the input document is not structured attempting to reference structured fields with `this` will result in an error. Therefore, a convenient way to delete non-structured data is with a catch.

```coffee
root = this.catch(deleted())

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":{"foo":"bar"}}

# In:  not structured data
# Out: <Message deleted>
```

### `exists`

Checks that a field, identified via a [dot path][field_paths], exists in an object.

#### Parameters

**`path`** &lt;string&gt; A [dot path][field_paths] to a field.  

#### Examples


```coffee
root.result = this.foo.exists("bar.baz")

# In:  {"foo":{"bar":{"baz":"yep, I exist"}}}
# Out: {"result":true}

# In:  {"foo":{"bar":{}}}
# Out: {"result":false}

# In:  {"foo":{}}
# Out: {"result":false}
```

### `from`

Modifies a target query such that certain functions are executed from the perspective of another message in the batch. This allows you to mutate events based on the contents of other messages. Functions that support this behaviour are `content`, `json` and `meta`.

#### Parameters

**`index`** &lt;integer&gt; The message index to use as a perspective.  

#### Examples


For example, the following map extracts the contents of the JSON field `foo` specifically from message index `1` of a batch, effectively overriding the field `foo` for all messages of a batch to that of message 1:

```coffee
root = this
root.foo = json("foo").from(1)
```

### `from_all`

Modifies a target query such that certain functions are executed from the perspective of each message in the batch, and returns the set of results as an array. Functions that support this behaviour are `content`, `json` and `meta`.

#### Examples


```coffee
root = this
root.foo_summed = json("foo").from_all().sum()
```

### `or`

If the result of the target query fails or resolves to `null`, returns the argument instead. This is an explicit method alternative to the coalesce pipe operator `|`.

#### Parameters

**`fallback`** &lt;query expression&gt; A value to yield, or query to execute, if the target query fails or resolves to `null`.  

#### Examples


```coffee
root.doc.id = this.thing.id.or(uuid_v4())
```

## String Manipulation

### `capitalize`

Takes a string value and returns a copy with all Unicode letters that begin words mapped to their Unicode title case.

#### Examples


```coffee
root.title = this.title.capitalize()

# In:  {"title":"the foo bar"}
# Out: {"title":"The Foo Bar"}
```

### `contains`

Checks whether a string contains a substring and returns a boolean result.

#### Parameters

**`value`** &lt;unknown&gt; A value to test against elements of the target.  

#### Examples


```coffee
root.has_foo = this.thing.contains("foo")

# In:  {"thing":"this foo that"}
# Out: {"has_foo":true}

# In:  {"thing":"this bar that"}
# Out: {"has_foo":false}
```

### `escape_html`

Escapes a string so that special characters like `<` to become `&lt;`. It escapes only five such characters: `<`, `>`, `&`, `'` and `"` so that it can be safely placed within an HTML entity.

#### Examples


```coffee
root.escaped = this.value.escape_html()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo &amp; bar"}
```

### `escape_url_query`

Escapes a string so that it can be safely placed within a URL query.

#### Examples


```coffee
root.escaped = this.value.escape_url_query()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo+%26+bar"}
```

### `filepath_join`

Joins an array of path elements into a single file path. The separator depends on the operating system of the machine.

#### Examples


```coffee
root.path = this.path_elements.filepath_join()

# In:  {"path_elements":["/foo/","bar.txt"]}
# Out: {"path":"/foo/bar.txt"}
```

### `filepath_split`

Splits a file path immediately following the final Separator, separating it into a directory and file name component returned as a two element array of strings. If there is no Separator in the path, the first element will be empty and the second will contain the path. The separator depends on the operating system of the machine.

#### Examples


```coffee
root.path_sep = this.path.filepath_split()

# In:  {"path":"/foo/bar.txt"}
# Out: {"path_sep":["/foo/","bar.txt"]}

# In:  {"path":"baz.txt"}
# Out: {"path_sep":["","baz.txt"]}
```

### `format`

Use a value string as a format specifier in order to produce a new string, using any number of provided arguments.

#### Examples


```coffee
root.foo = "%s(%v): %v".format(this.name, this.age, this.fingers)

# In:  {"name":"lance","age":37,"fingers":13}
# Out: {"foo":"lance(37): 13"}
```

### `has_prefix`

Checks whether a string has a prefix argument and returns a bool.

#### Parameters

**`value`** &lt;string&gt; The string to test.  

#### Examples


```coffee
root.t1 = this.v1.has_prefix("foo")
root.t2 = this.v2.has_prefix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":true,"t2":false}
```

### `has_suffix`

Checks whether a string has a suffix argument and returns a bool.

#### Parameters

**`value`** &lt;string&gt; The string to test.  

#### Examples


```coffee
root.t1 = this.v1.has_suffix("foo")
root.t2 = this.v2.has_suffix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":false,"t2":true}
```

### `index_of`

Returns the starting index of the argument substring in a string target, or `-1` if the target doesn't contain the argument.

#### Parameters

**`value`** &lt;string&gt; A string to search for.  

#### Examples


```coffee
root.index = this.thing.index_of("bar")

# In:  {"thing":"foobar"}
# Out: {"index":3}
```

```coffee
root.index = content().index_of("meow")

# In:  the cat meowed, the dog woofed
# Out: {"index":8}
```

### `length`

Returns the length of a string.

#### Examples


```coffee
root.foo_len = this.foo.length()

# In:  {"foo":"hello world"}
# Out: {"foo_len":11}
```

### `lowercase`

Convert a string value into lowercase.

#### Examples


```coffee
root.foo = this.foo.lowercase()

# In:  {"foo":"HELLO WORLD"}
# Out: {"foo":"hello world"}
```

### `quote`

Quotes a target string using escape sequences (`\t`, `\n`, `\xFF`, `\u0100`) for control characters and non-printable characters.

#### Examples


```coffee
root.quoted = this.thing.quote()

# In:  {"thing":"foo\nbar"}
# Out: {"quoted":"\"foo\\nbar\""}
```

### `replace`

Replaces all occurrences of the first argument in a target string with the second argument.

#### Parameters

**`old`** &lt;string&gt; A string to match against.  
**`new`** &lt;string&gt; A string to replace with.  

#### Examples


```coffee
root.new_value = this.value.replace("foo","dog")

# In:  {"value":"The foo ate my homework"}
# Out: {"new_value":"The dog ate my homework"}
```

### `replace_many`

For each pair of strings in an argument array, replaces all occurrences of the first item of the pair with the second. This is a more compact way of chaining a series of `replace` methods.

#### Parameters

**`values`** &lt;array&gt; An array of values, each even value will be replaced with the following odd value.  

#### Examples


```coffee
root.new_value = this.value.replace_many([
  "<b>", "&lt;b&gt;",
  "</b>", "&lt;/b&gt;",
  "<i>", "&lt;i&gt;",
  "</i>", "&lt;/i&gt;",
])

# In:  {"value":"<i>Hello</i> <b>World</b>"}
# Out: {"new_value":"&lt;i&gt;Hello&lt;/i&gt; &lt;b&gt;World&lt;/b&gt;"}
```

### `reverse`

Returns the target string in reverse order.

#### Examples


```coffee
root.reversed = this.thing.reverse()

# In:  {"thing":"backwards"}
# Out: {"reversed":"sdrawkcab"}
```

```coffee
root = content().reverse()

# In:  {"thing":"backwards"}
# Out: }"sdrawkcab":"gniht"{
```

### `slice`

Extract a slice from a string by specifying two indices, a low and high bound, which selects a half-open range that includes the first character, but excludes the last one. If the second index is omitted then it defaults to the length of the input sequence.

#### Parameters

**`low`** &lt;integer&gt; The low bound, which is the first element of the selection, or if negative selects from the end.  
**`high`** &lt;(optional) integer&gt; An optional high bound.  

#### Examples


```coffee
root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":"foo bar"}
# Out: {"beginning":"fo","end":"bar"}
```

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned.

```coffee
root.last_chunk = this.value.slice(-4)
root.the_rest = this.value.slice(0, -4)

# In:  {"value":"foo bar"}
# Out: {"last_chunk":" bar","the_rest":"foo"}
```

### `split`

Split a string value into an array of strings by splitting it on a string separator.

#### Parameters

**`delimiter`** &lt;string&gt; The delimiter to split with.  

#### Examples


```coffee
root.new_value = this.value.split(",")

# In:  {"value":"foo,bar,baz"}
# Out: {"new_value":["foo","bar","baz"]}
```

### `strip_html`

Attempts to remove all HTML tags from a target string.

#### Parameters

**`preserve`** &lt;(optional) array&gt; An optional array of element types to preserve in the output.  

#### Examples


```coffee
root.stripped = this.value.strip_html()

# In:  {"value":"<p>the plain <strong>old text</strong></p>"}
# Out: {"stripped":"the plain old text"}
```

It's also possible to provide an explicit list of element types to preserve in the output.

```coffee
root.stripped = this.value.strip_html(["article"])

# In:  {"value":"<article><p>the plain <strong>old text</strong></p></article>"}
# Out: {"stripped":"<article>the plain old text</article>"}
```

### `trim`

Remove all leading and trailing characters from a string that are contained within an argument cutset. If no arguments are provided then whitespace is removed.

#### Parameters

**`cutset`** &lt;(optional) string&gt; An optional string of characters to trim from the target value.  

#### Examples


```coffee
root.title = this.title.trim("!?")
root.description = this.description.trim()

# In:  {"description":"  something happened and its amazing! ","title":"!!!watch out!?"}
# Out: {"description":"something happened and its amazing!","title":"watch out"}
```

### `unescape_html`

Unescapes a string so that entities like `&lt;` become `<`. It unescapes a larger range of entities than `escape_html` escapes. For example, `&aacute;` unescapes to `á`, as does `&#225;` and `&xE1;`.

#### Examples


```coffee
root.unescaped = this.value.unescape_html()

# In:  {"value":"foo &amp; bar"}
# Out: {"unescaped":"foo & bar"}
```

### `unescape_url_query`

Expands escape sequences from a URL query string.

#### Examples


```coffee
root.unescaped = this.value.unescape_url_query()

# In:  {"value":"foo+%26+bar"}
# Out: {"unescaped":"foo & bar"}
```

### `unquote`

Unquotes a target string, expanding any escape sequences (`\t`, `\n`, `\xFF`, `\u0100`) for control characters and non-printable characters.

#### Examples


```coffee
root.unquoted = this.thing.unquote()

# In:  {"thing":"\"foo\\nbar\""}
# Out: {"unquoted":"foo\nbar"}
```

### `uppercase`

Convert a string value into uppercase.

#### Examples


```coffee
root.foo = this.foo.uppercase()

# In:  {"foo":"hello world"}
# Out: {"foo":"HELLO WORLD"}
```

## Regular Expressions

### `re_find_all`

Returns an array containing all successive matches of a regular expression in a string.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  

#### Examples


```coffee
root.matches = this.value.re_find_all("a.")

# In:  {"value":"paranormal"}
# Out: {"matches":["ar","an","al"]}
```

### `re_find_all_object`

Returns an array of objects containing all matches of the regular expression and the matches of its subexpressions. The key of each match value is the name of the group when specified, otherwise it is the index of the matching group, starting with the expression as a whole at 0.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  

#### Examples


```coffee
root.matches = this.value.re_find_all_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[{"0":"axxb","foo":"xx"},{"0":"ab","foo":""}]}
```

```coffee
root.matches = this.value.re_find_all_object("(?m)(?P<key>\\w+):\\s+(?P<value>\\w+)$")

# In:  {"value":"option1: value1\noption2: value2\noption3: value3"}
# Out: {"matches":[{"0":"option1: value1","key":"option1","value":"value1"},{"0":"option2: value2","key":"option2","value":"value2"},{"0":"option3: value3","key":"option3","value":"value3"}]}
```

### `re_find_all_submatch`

Returns an array of arrays containing all successive matches of the regular expression in a string and the matches, if any, of its subexpressions.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  

#### Examples


```coffee
root.matches = this.value.re_find_all_submatch("a(x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[["axxb","xx"],["ab",""]]}
```

### `re_find_object`

Returns an object containing the first match of the regular expression and the matches of its subexpressions. The key of each match value is the name of the group when specified, otherwise it is the index of the matching group, starting with the expression as a whole at 0.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  

#### Examples


```coffee
root.matches = this.value.re_find_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":{"0":"axxb","foo":"xx"}}
```

```coffee
root.matches = this.value.re_find_object("(?P<key>\\w+):\\s+(?P<value>\\w+)")

# In:  {"value":"option1: value1"}
# Out: {"matches":{"0":"option1: value1","key":"option1","value":"value1"}}
```

### `re_match`

Checks whether a regular expression matches against any part of a string and returns a boolean.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  

#### Examples


```coffee
root.matches = this.value.re_match("[0-9]")

# In:  {"value":"there are 10 puppies"}
# Out: {"matches":true}

# In:  {"value":"there are ten puppies"}
# Out: {"matches":false}
```

### `re_replace`

Replaces all occurrences of the argument regular expression in a string with a value. Inside the value $ signs are interpreted as submatch expansions, e.g. `$1` represents the text of the first submatch.

#### Parameters

**`pattern`** &lt;string&gt; The pattern to match against.  
**`value`** &lt;string&gt; The value to replace with.  

#### Examples


```coffee
root.new_value = this.value.re_replace("ADD ([0-9]+)","+($1)")

# In:  {"value":"foo ADD 70"}
# Out: {"new_value":"foo +(70)"}
```

## Number Manipulation

### `abs`

Returns the absolute value of a number.

#### Examples


```coffee
root.new_value = this.value.abs()

# In:  {"value":5.3}
# Out: {"new_value":5.3}

# In:  {"value":-5.9}
# Out: {"new_value":5.9}
```

### `ceil`

Returns the least integer value greater than or equal to a number.

#### Examples


```coffee
root.new_value = this.value.ceil()

# In:  {"value":5.3}
# Out: {"new_value":6}

# In:  {"value":-5.9}
# Out: {"new_value":-5}
```

### `floor`

Returns the greatest integer value less than or equal to the target number.

#### Examples


```coffee
root.new_value = this.value.floor()

# In:  {"value":5.7}
# Out: {"new_value":5}
```

### `log`

Returns the natural logarithm of a number.

#### Examples


```coffee
root.new_value = this.value.log().round()

# In:  {"value":1}
# Out: {"new_value":0}

# In:  {"value":2.7183}
# Out: {"new_value":1}
```

### `log10`

Returns the decimal logarithm of a number.

#### Examples


```coffee
root.new_value = this.value.log10()

# In:  {"value":100}
# Out: {"new_value":2}

# In:  {"value":1000}
# Out: {"new_value":3}
```

### `max`

Returns the largest numerical value found within an array. All values must be numerical and the array must not be empty, otherwise an error is returned.

#### Examples


```coffee
root.biggest = this.values.max()

# In:  {"values":[0,3,2.5,7,5]}
# Out: {"biggest":7}
```

```coffee
root.new_value = [0,this.value].max()

# In:  {"value":-1}
# Out: {"new_value":0}

# In:  {"value":7}
# Out: {"new_value":7}
```

### `min`

Returns the smallest numerical value found within an array. All values must be numerical and the array must not be empty, otherwise an error is returned.

#### Examples


```coffee
root.smallest = this.values.min()

# In:  {"values":[0,3,-2.5,7,5]}
# Out: {"smallest":-2.5}
```

```coffee
root.new_value = [10,this.value].min()

# In:  {"value":2}
# Out: {"new_value":2}

# In:  {"value":23}
# Out: {"new_value":10}
```

### `round`

Rounds numbers to the nearest integer, rounding half away from zero.

#### Examples


```coffee
root.new_value = this.value.round()

# In:  {"value":5.3}
# Out: {"new_value":5}

# In:  {"value":5.9}
# Out: {"new_value":6}
```

## Timestamp Manipulation

### `format_timestamp`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to format a timestamp value as a string according to a specified format, or ISO 8601 by default. Timestamp values can either be a numerical unix time in seconds (with up to nanosecond precision via decimals), or a string in ISO 8601 format.

#### Parameters

**`format`** &lt;string, default `"2006-01-02T15:04:05.999999999Z07:00"`&gt; The output format to use.  
**`tz`** &lt;(optional) string&gt; An optional timezone to use, otherwise the timezone of the input string is used, or in the case of unix timestamps the local timezone is used.  

#### Examples


```coffee
root.something_at = (this.created_at + 300).format_timestamp()
```

An optional string argument can be used in order to specify the output format of the timestamp. The format is defined by showing how the reference time, defined to be Mon Jan 2 15:04:05 -0700 MST 2006, would be displayed if it were the value.

```coffee
root.something_at = (this.created_at + 300).format_timestamp("2006-Jan-02 15:04:05")
```

A second optional string argument can also be used in order to specify a timezone, otherwise the timezone of the input string is used, or in the case of unix timestamps the local timezone is used.

```coffee
root.something_at = this.created_at.format_timestamp(format: "2006-Jan-02 15:04:05", tz: "UTC")

# In:  {"created_at":1597405526}
# Out: {"something_at":"2020-Aug-14 11:45:26"}

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}
```

And `format_timestamp` supports up to nanosecond precision with floating point timestamp values.

```coffee
root.something_at = this.created_at.format_timestamp("2006-Jan-02 15:04:05.999999", "UTC")

# In:  {"created_at":1597405526.123456}
# Out: {"something_at":"2020-Aug-14 11:45:26.123456"}

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26.371"}
```

### `format_timestamp_strftime`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to format a timestamp value as a string according to a specified strftime-compatible format. Timestamp values can either be a numerical unix time in seconds (with up to nanosecond precision via decimals), or a string in ISO 8601 format.

#### Parameters

**`format`** &lt;string&gt; The output format to use.  
**`tz`** &lt;(optional) string&gt; An optional timezone to use, otherwise the timezone of the input string is used.  

#### Examples


The format consists of zero or more conversion specifiers and ordinary characters (except `%`). All ordinary characters are copied to the output string without modification. Each conversion specification begins with `%` character followed by the character that determines the behaviour of the specifier. Please refer to [man 3 strftime](https://linux.die.net/man/3/strftime) for the list of format specifiers.

```coffee
root.something_at = (this.created_at + 300).format_timestamp_strftime("%Y-%b-%d %H:%M:%S")
```

A second optional string argument can also be used in order to specify a timezone, otherwise the timezone of the input string is used, or in the case of unix timestamps the local timezone is used.

```coffee
root.something_at = this.created_at.format_timestamp_strftime("%Y-%b-%d %H:%M:%S", "UTC")

# In:  {"created_at":1597405526}
# Out: {"something_at":"2020-Aug-14 11:45:26"}

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}
```

### `format_timestamp_unix`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to format a timestamp value as a unix timestamp. Timestamp values can either be a numerical unix time in seconds (with up to nanosecond precision via decimals), or a string in ISO 8601 format. The [`parse_timestamp`](#parse_timestamp) method can be used in order to parse different timestamp formats.

#### Examples


```coffee
root.created_at_unix = this.created_at.format_timestamp_unix()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000}
```

### `format_timestamp_unix_nano`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to format a timestamp value as a unix timestamp with nanosecond precision. Timestamp values can either be a numerical unix time in seconds (with up to nanosecond precision via decimals), or a string in ISO 8601 format. The [`parse_timestamp`](#parse_timestamp) method can be used in order to parse different timestamp formats.

#### Examples


```coffee
root.created_at_unix = this.created_at.format_timestamp_unix_nano()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000000000}
```

### `parse_duration`

Attempts to parse a string as a duration and returns an integer of nanoseconds. A duration string is a possibly signed sequence of decimal numbers, each with an optional fraction and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".

#### Examples


```coffee
root.delay_for_ns = this.delay_for.parse_duration()

# In:  {"delay_for":"50us"}
# Out: {"delay_for_ns":50000}
```

```coffee
root.delay_for_s = this.delay_for.parse_duration() / 1000000000

# In:  {"delay_for":"2h"}
# Out: {"delay_for_s":7200}
```

### `parse_duration_iso8601`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to parse a string using ISO-8601 rules as a duration and returns an integer of nanoseconds. A duration string is represented by the format "P[n]Y[n]M[n]DT[n]H[n]M[n]S" or "P[n]W". In these representations, the "[n]" is replaced by the value for each of the date and time elements that follow the "[n]". For example, "P3Y6M4DT12H30M5S" represents a duration of "three years, six months, four days, twelve hours, thirty minutes, and five seconds". The last field of the format allows fractions with one decimal place, so "P3.5S" will return 3500000000ns. Any additional decimals will be truncated.

#### Examples


Arbitrary ISO-8601 duration string to nanoseconds:

```coffee
root.delay_for_ns = this.delay_for.parse_duration_iso8601()

# In:  {"delay_for":"P3Y6M4DT12H30M5S"}
# Out: {"delay_for_ns":110839937000000000}
```

Two hours ISO-8601 duration string to seconds:

```coffee
root.delay_for_s = this.delay_for.parse_duration_iso8601() / 1000000000

# In:  {"delay_for":"PT2H"}
# Out: {"delay_for_s":7200}
```

Two and a half seconds ISO-8601 duration string to seconds:

```coffee
root.delay_for_s = this.delay_for.parse_duration_iso8601() / 1000000000

# In:  {"delay_for":"PT2.5S"}
# Out: {"delay_for_s":2.5}
```

### `parse_timestamp`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to parse a string as a timestamp following a specified format and outputs a string following ISO 8601, which can then be fed into `format_timestamp`. The input format is defined by showing how the reference time, defined to be Mon Jan 2 15:04:05 -0700 MST 2006, would be displayed if it were the value.

#### Parameters

**`format`** &lt;string&gt; The format of the target string.  

#### Examples


```coffee
root.doc.timestamp = this.doc.timestamp.parse_timestamp("2006-Jan-02")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}
```

### `parse_timestamp_strptime`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to parse a string as a timestamp following a specified strptime-compatible format and outputs a string following ISO 8601, which can then be fed into `format_timestamp`.

#### Parameters

**`format`** &lt;string&gt; The format of the target string.  

#### Examples


The format consists of zero or more conversion specifiers and ordinary characters (except `%`). All ordinary characters are copied to the output string without modification. Each conversion specification begins with a `%` character followed by the character that determines the behaviour of the specifier. Please refer to [man 3 strptime](https://linux.die.net/man/3/strptime) for the list of format specifiers.

```coffee
root.doc.timestamp = this.doc.timestamp.parse_timestamp_strptime("%Y-%b-%d")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}
```

## Type Coercion

### `bool`

Attempt to parse a value into a boolean. An optional argument can be provided, in which case if the value cannot be parsed the argument will be returned instead. If the value is a number then any non-zero value will resolve to `true`, if the value is a string then any of the following values are considered valid: `1, t, T, TRUE, true, True, 0, f, F, FALSE`.

#### Parameters

**`default`** &lt;(optional) bool&gt; An optional value to yield if the target cannot be parsed as a boolean.  

#### Examples


```coffee
root.foo = this.thing.bool()
root.bar = this.thing.bool(true)
```

### `bytes`

Marshal a value into a byte array. If the value is already a byte array it is unchanged.

#### Examples


```coffee
root.first_byte = this.name.bytes().index(0)

# In:  {"name":"foobar bazson"}
# Out: {"first_byte":102}
```

### `not_empty`

Ensures that the given string, array or object value is not empty, and if so returns it, otherwise an error is returned.

#### Examples


```coffee
root.a = this.a.not_empty()

# In:  {"a":"foo"}
# Out: {"a":"foo"}

# In:  {"a":""}
# Out: Error("failed assignment (line 1): field `this.a`: string value is empty")

# In:  {"a":["foo","bar"]}
# Out: {"a":["foo","bar"]}

# In:  {"a":[]}
# Out: Error("failed assignment (line 1): field `this.a`: array value is empty")

# In:  {"a":{"b":"foo","c":"bar"}}
# Out: {"a":{"b":"foo","c":"bar"}}

# In:  {"a":{}}
# Out: Error("failed assignment (line 1): field `this.a`: object value is empty")
```

### `not_null`

Ensures that the given value is not `null`, and if so returns it, otherwise an error is returned.

#### Examples


```coffee
root.a = this.a.not_null()

# In:  {"a":"foobar","b":"barbaz"}
# Out: {"a":"foobar"}

# In:  {"b":"barbaz"}
# Out: Error("failed assignment (line 1): field `this.a`: value is null")
```

### `number`

Attempt to parse a value into a number. An optional argument can be provided, in which case if the value cannot be parsed into a number the argument will be returned instead.

#### Parameters

**`default`** &lt;(optional) float&gt; An optional value to yield if the target cannot be parsed as a number.  

#### Examples


```coffee
root.foo = this.thing.number() + 10
root.bar = this.thing.number(5) * 10
```

### `string`

Marshal a value into a string. If the value is already a string it is unchanged.

#### Examples


```coffee
root.nested_json = this.string()

# In:  {"foo":"bar"}
# Out: {"nested_json":"{\"foo\":\"bar\"}"}
```

```coffee
root.id = this.id.string()

# In:  {"id":228930314431312345}
# Out: {"id":"228930314431312345"}
```

### `type`

Returns the type of a value as a string, providing one of the following values: `string`, `bytes`, `number`, `bool`, `array`, `object` or `null`.

#### Examples


```coffee
root.bar_type = this.bar.type()
root.foo_type = this.foo.type()

# In:  {"bar":10,"foo":"is a string"}
# Out: {"bar_type":"number","foo_type":"string"}
```

## Object & Array Manipulation

### `all`

Checks each element of an array against a query and returns true if all elements passed. An error occurs if the target is not an array, or if any element results in the provided query returning a non-boolean result. Returns false if the target array is empty.

#### Parameters

**`test`** &lt;query expression&gt; A test query to apply to each element.  

#### Examples


```coffee
root.all_over_21 = this.patrons.all(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"all_over_21":false}

# In:  {"patrons":[{"id":"1","age":45},{"id":"2","age":23}]}
# Out: {"all_over_21":true}
```

### `any`

Checks the elements of an array against a query and returns true if any element passes. An error occurs if the target is not an array, or if an element results in the provided query returning a non-boolean result. Returns false if the target array is empty.

#### Parameters

**`test`** &lt;query expression&gt; A test query to apply to each element.  

#### Examples


```coffee
root.any_over_21 = this.patrons.any(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"any_over_21":true}

# In:  {"patrons":[{"id":"1","age":10},{"id":"2","age":12}]}
# Out: {"any_over_21":false}
```

### `append`

Returns an array with new elements appended to the end.

#### Examples


```coffee
root.foo = this.foo.append("and", "this")

# In:  {"foo":["bar","baz"]}
# Out: {"foo":["bar","baz","and","this"]}
```

### `assign`

Merge a source object into an existing destination object. When a collision is found within the merged structures (both a source and destination object contain the same non-object keys) the value in the destination object will be overwritten by that of source object. In order to preserve both values on collision use the [`merge`](#merge) method.

#### Parameters

**`with`** &lt;unknown&gt; A value to merge the target value with.  

#### Examples


```coffee
root = this.foo.assign(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":"foos","second_name":"barer"}
```

### `collapse`

Collapse an array or object into an object of key/value pairs for each field, where the key is the full path of the structured field in dot path notation. Empty arrays an objects are ignored by default.

#### Parameters

**`include_empty`** &lt;bool, default `false`&gt; Whether to include empty objects and arrays in the resulting object.  

#### Examples


```coffee
root.result = this.collapse()

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.2.bar":"2"}}
```

An optional boolean parameter can be set to true in order to include empty objects and arrays.

```coffee
root.result = this.collapse(include_empty: true)

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.1.bar":{},"foo.2.bar":"2","foo.3.bar":[]}}
```

### `contains`

Checks whether an array contains an element matching the argument, or an object contains a value matching the argument, and returns a boolean result. Numerical comparisons are made irrespective of the representation type (float versus integer).

#### Parameters

**`value`** &lt;unknown&gt; A value to test against elements of the target.  

#### Examples


```coffee
root.has_foo = this.thing.contains("foo")

# In:  {"thing":["this","foo","that"]}
# Out: {"has_foo":true}

# In:  {"thing":["this","bar","that"]}
# Out: {"has_foo":false}
```

```coffee
root.has_bar = this.thing.contains(20)

# In:  {"thing":[10.3,20.0,"huh",3]}
# Out: {"has_bar":true}

# In:  {"thing":[2,3,40,67]}
# Out: {"has_bar":false}
```

### `enumerated`

Converts an array into a new array of objects, where each object has a field index containing the `index` of the element and a field `value` containing the original value of the element.

#### Examples


```coffee
root.foo = this.foo.enumerated()

# In:  {"foo":["bar","baz"]}
# Out: {"foo":[{"index":0,"value":"bar"},{"index":1,"value":"baz"}]}
```

### `explode`

Explodes an array or object at a [field path][field_paths].

#### Parameters

**`path`** &lt;string&gt; A [dot path][field_paths] to a field to explode.  

#### Examples


##### On arrays

Exploding arrays results in an array containing elements matching the original document, where the target field of each element is an element of the exploded array:

```coffee
root = this.explode("value")

# In:  {"id":1,"value":["foo","bar","baz"]}
# Out: [{"id":1,"value":"foo"},{"id":1,"value":"bar"},{"id":1,"value":"baz"}]
```

##### On objects

Exploding objects results in an object where the keys match the target object, and the values match the original document but with the target field replaced by the exploded value:

```coffee
root = this.explode("value")

# In:  {"id":1,"value":{"foo":2,"bar":[3,4],"baz":{"bev":5}}}
# Out: {"bar":{"id":1,"value":[3,4]},"baz":{"id":1,"value":{"bev":5}},"foo":{"id":1,"value":2}}
```

### `filter`

Executes a mapping query argument for each element of an array or key/value pair of an object. If the query returns `false` the item is removed from the resulting array or object. The item will also be removed if the query returns any non-boolean value.

#### Parameters

**`test`** &lt;query expression&gt; A query to apply to each element, if this query resolves to any value other than a boolean `true` the element will be removed from the result.  

#### Examples


```coffee
root.new_nums = this.nums.filter(num -> num > 10)

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[11,17]}
```

##### On objects

When filtering objects the mapping query argument is provided a context with a field `key` containing the value key, and a field `value` containing the value.

```coffee
root.new_dict = this.dict.filter(item -> item.value.contains("foo"))

# In:  {"dict":{"first":"hello foo","second":"world","third":"this foo is great"}}
# Out: {"new_dict":{"first":"hello foo","third":"this foo is great"}}
```

### `flatten`

Iterates an array and any element that is itself an array is removed and has its elements inserted directly in the resulting array.

#### Examples


```coffee
root.result = this.flatten()

# In:  ["foo",["bar","baz"],"buz"]
# Out: {"result":["foo","bar","baz","buz"]}
```

### `fold`

Takes two arguments: an initial value, and a mapping query. For each element of an array the mapping context is an object with two fields `tally` and `value`, where `tally` contains the current accumulated value and `value` is the value of the current element. The mapping must return the result of adding the value to the tally.

The first argument is the value that `tally` will have on the first call.

#### Parameters

**`initial`** &lt;unknown&gt; The initial value to start the fold with. For example, an empty object `{}`, a zero count `0`, or an empty string `""`.  
**`query`** &lt;query expression&gt; A query to apply for each element. The query is provided an object with two fields; `tally` containing the current tally, and `value` containing the value of the current element. The query should result in a new tally to be passed to the next element query.  

#### Examples


```coffee
root.sum = this.foo.fold(0, item -> item.tally + item.value)

# In:  {"foo":[3,8,11]}
# Out: {"sum":22}
```

```coffee
root.result = this.foo.fold("", item -> "%v%v".format(item.tally, item.value))

# In:  {"foo":["hello ", "world"]}
# Out: {"result":"hello world"}
```

You can use fold to merge an array of objects together:

```coffee
root.smoothie = this.fruits.fold({}, item -> item.tally.merge(item.value))

# In:  {"fruits":[{"apple":5},{"banana":3},{"orange":8}]}
# Out: {"smoothie":{"apple":5,"banana":3,"orange":8}}
```

### `get`

Extract a field value, identified via a [dot path][field_paths], from an object.

#### Parameters

**`path`** &lt;string&gt; A [dot path][field_paths] identifying a field to obtain.  

#### Examples


```coffee
root.result = this.foo.get(this.target)

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"bar"}
# Out: {"result":"from bar"}

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"baz"}
# Out: {"result":"from baz"}
```

### `index`

Extract an element from an array by an index. The index can be negative, and if so the element will be selected from the end counting backwards starting from -1. E.g. an index of -1 returns the last element, an index of -2 returns the element before the last, and so on.

#### Parameters

**`index`** &lt;integer&gt; The index to obtain from an array.  

#### Examples


```coffee
root.last_name = this.names.index(-1)

# In:  {"names":["rachel","stevens"]}
# Out: {"last_name":"stevens"}
```

It is also possible to use this method on byte arrays, in which case the selected element will be returned as an integer.

```coffee
root.last_byte = this.name.bytes().index(-1)

# In:  {"name":"foobar bazson"}
# Out: {"last_byte":110}
```

### `join`

Join an array of strings with an optional delimiter into a single string.

#### Parameters

**`delimiter`** &lt;(optional) string&gt; An optional delimiter to add between each string.  

#### Examples


```coffee
root.joined_words = this.words.join()
root.joined_numbers = this.numbers.map_each(this.string()).join(",")

# In:  {"words":["hello","world"],"numbers":[3,8,11]}
# Out: {"joined_numbers":"3,8,11","joined_words":"helloworld"}
```

### `json_schema`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Checks a [JSON schema](https://json-schema.org/) against a value and returns the value if it matches or throws and error if it does not.

#### Parameters

**`schema`** &lt;string&gt; The schema to check values against.  

#### Examples


```coffee
root = this.json_schema("""{
  "type":"object",
  "properties":{
    "foo":{
      "type":"string"
    }
  }
}""")

# In:  {"foo":"bar"}
# Out: {"foo":"bar"}

# In:  {"foo":5}
# Out: Error("failed assignment (line 1): field `this`: foo invalid type. expected: string, given: integer")
```

In order to load a schema from a file use the `file` function.

```coffee
root = this.json_schema(file(var("BENTHOS_TEST_BLOBLANG_SCHEMA_FILE")))
```

### `key_values`

Returns the key/value pairs of an object as an array, where each element is an object with a `key` field and a `value` field. The order of the resulting array will be random.

#### Examples


```coffee
root.foo_key_values = this.foo.key_values().sort_by(pair -> pair.key)

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_key_values":[{"key":"bar","value":1},{"key":"baz","value":2}]}
```

### `keys`

Returns the keys of an object as an array.

#### Examples


```coffee
root.foo_keys = this.foo.keys()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_keys":["bar","baz"]}
```

### `length`

Returns the length of an array or object (number of keys).

#### Examples


```coffee
root.foo_len = this.foo.length()

# In:  {"foo":["first","second"]}
# Out: {"foo_len":2}

# In:  {"foo":{"first":"bar","second":"baz"}}
# Out: {"foo_len":2}
```

### `map_each`



#### Parameters

**`query`** &lt;query expression&gt; A query that will be used to map each element.  

#### Examples


##### On arrays

Apply a mapping to each element of an array and replace the element with the result. Within the argument mapping the context is the value of the element being mapped.

```coffee
root.new_nums = this.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[1,7]}
```

##### On objects

Apply a mapping to each value of an object and replace the value with the result. Within the argument mapping the context is an object with a field `key` containing the value key, and a field `value`.

```coffee
root.new_dict = this.dict.map_each(item -> item.value.uppercase())

# In:  {"dict":{"foo":"hello","bar":"world"}}
# Out: {"new_dict":{"bar":"WORLD","foo":"HELLO"}}
```

### `map_each_key`

Apply a mapping to each key of an object, and replace the key with the result, which must be a string.

#### Parameters

**`query`** &lt;query expression&gt; A query that will be used to map each key.  

#### Examples


```coffee
root.new_dict = this.dict.map_each_key(key -> key.uppercase())

# In:  {"dict":{"keya":"hello","keyb":"world"}}
# Out: {"new_dict":{"KEYA":"hello","KEYB":"world"}}
```

```coffee
root = this.map_each_key(key -> if key.contains("kafka") { "_" + key })

# In:  {"amqp_key":"foo","kafka_key":"bar","kafka_topic":"baz"}
# Out: {"_kafka_key":"bar","_kafka_topic":"baz","amqp_key":"foo"}
```

### `merge`

Merge a source object into an existing destination object. When a collision is found within the merged structures (both a source and destination object contain the same non-object keys) the result will be an array containing both values, where values that are already arrays will be expanded into the resulting array. In order to simply override destination fields on collision use the [`assign`](#assign) method.

#### Parameters

**`with`** &lt;unknown&gt; A value to merge the target value with.  

#### Examples


```coffee
root = this.foo.merge(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":["bars","foos"],"second_name":"barer"}
```

### `slice`

Extract a slice from an array by specifying two indices, a low and high bound, which selects a half-open range that includes the first element, but excludes the last one. If the second index is omitted then it defaults to the length of the input sequence.

#### Parameters

**`low`** &lt;integer&gt; The low bound, which is the first element of the selection, or if negative selects from the end.  
**`high`** &lt;(optional) integer&gt; An optional high bound.  

#### Examples


```coffee
root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"beginning":["foo","bar"],"end":["bev"]}
```

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned.

```coffee
root.last_chunk = this.value.slice(-2)
root.the_rest = this.value.slice(0, -2)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"last_chunk":["buz","bev"],"the_rest":["foo","bar","baz"]}
```

### `sort`

Attempts to sort the values of an array in increasing order. The type of all values must match in order for the ordering to succeed. Supports string and number values.

#### Parameters

**`compare`** &lt;(optional) query expression&gt; An optional query that should explicitly compare elements `left` and `right` and provide a boolean result.  

#### Examples


```coffee
root.sorted = this.foo.sort()

# In:  {"foo":["bbb","ccc","aaa"]}
# Out: {"sorted":["aaa","bbb","ccc"]}
```

It's also possible to specify a mapping argument, which is provided an object context with fields `left` and `right`, the mapping must return a boolean indicating whether the `left` value is less than `right`. This allows you to sort arrays containing non-string or non-number values.

```coffee
root.sorted = this.foo.sort(item -> item.left.v < item.right.v)

# In:  {"foo":[{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"},{"id":"baz","v":"aaa"}]}
# Out: {"sorted":[{"id":"baz","v":"aaa"},{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"}]}
```

### `sort_by`

Attempts to sort the elements of an array, in increasing order, by a value emitted by an argument query applied to each element. The type of all values must match in order for the ordering to succeed. Supports string and number values.

#### Parameters

**`query`** &lt;query expression&gt; A query to apply to each element that yields a value used for sorting.  

#### Examples


```coffee
root.sorted = this.foo.sort_by(ele -> ele.id)

# In:  {"foo":[{"id":"bbb","message":"bar"},{"id":"aaa","message":"foo"},{"id":"ccc","message":"baz"}]}
# Out: {"sorted":[{"id":"aaa","message":"foo"},{"id":"bbb","message":"bar"},{"id":"ccc","message":"baz"}]}
```

### `sum`

Sum the numerical values of an array.

#### Examples


```coffee
root.sum = this.foo.sum()

# In:  {"foo":[3,8,4]}
# Out: {"sum":15}
```

### `unique`

Attempts to remove duplicate values from an array. The array may contain a combination of different value types, but numbers and strings are checked separately (`"5"` is a different element to `5`).

#### Parameters

**`emit`** &lt;(optional) query expression&gt; An optional query that can be used in order to yield a value for each element to determine uniqueness.  

#### Examples


```coffee
root.uniques = this.foo.unique()

# In:  {"foo":["a","b","a","c"]}
# Out: {"uniques":["a","b","c"]}
```

### `values`

Returns the values of an object as an array. The order of the resulting array will be random.

#### Examples


```coffee
root.foo_vals = this.foo.values().sort()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_vals":[1,2]}
```

### `without`

Returns an object where one or more [field path][field_paths] arguments are removed. Each path specifies a specific field to be deleted from the input object, allowing for nested fields.

If a key within a nested path does not exist or is not an object then it is not removed.

#### Examples


```coffee
root = this.without("inner.a","inner.c","d")

# In:  {"inner":{"a":"first","b":"second","c":"third"},"d":"fourth","e":"fifth"}
# Out: {"e":"fifth","inner":{"b":"second"}}
```

## Parsing

### `bloblang`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Executes an argument Bloblang mapping on the target. This method can be used in order to execute dynamic mappings. Imports and functions that interact with the environment, such as `file` and `env`, or that access message information directly, such as `content` or `json`, are not enabled for dynamic Bloblang mappings.

#### Parameters

**`mapping`** &lt;string&gt; The mapping to execute.  

#### Examples


```coffee
root.body = this.body.bloblang(this.mapping)

# In:  {"body":{"foo":"hello world"},"mapping":"root.foo = this.foo.uppercase()"}
# Out: {"body":{"foo":"HELLO WORLD"}}

# In:  {"body":{"foo":"hello world 2"},"mapping":"root.foo = this.foo.capitalize()"}
# Out: {"body":{"foo":"Hello World 2"}}
```

### `format_json`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Serializes a target value into a pretty-printed JSON byte array (with 4 space indentation by default).

#### Parameters

**`indent`** &lt;(optional) string, default `"    "`&gt; Indentation string. Each element in a JSON object or array will begin on a new, indented line followed by one or more copies of indent according to the indentation nesting.  

#### Examples


```coffee
root = this.doc.format_json()

# In:  {"doc":{"foo":"bar"}}
# Out: {
#          "foo": "bar"
#      }
```

Provide an argument string in order to customise the indentation used.

```coffee
root = this.format_json("  ")

# In:  {"doc":{"foo":"bar"}}
# Out: {
#        "doc": {
#          "foo": "bar"
#        }
#      }
```

Use the `.string()` method in order to coerce the result into a string.

```coffee
root.doc = this.doc.format_json().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"{\n    \"foo\": \"bar\"\n}"}
```

### `format_msgpack`

Formats data as a [MessagePack](https://msgpack.org/) message in bytes format.

#### Examples


```coffee
root = this.format_msgpack().encode("hex")

# In:  {"foo":"bar"}
# Out: 81a3666f6fa3626172
```

```coffee
root.encoded = this.format_msgpack().encode("base64")

# In:  {"foo":"bar"}
# Out: {"encoded":"gaNmb2+jYmFy"}
```

### `format_yaml`

Serializes a target value into a YAML byte array.

#### Examples


```coffee
root = this.doc.format_yaml()

# In:  {"doc":{"foo":"bar"}}
# Out: foo: bar
```

Use the `.string()` method in order to coerce the result into a string.

```coffee
root.doc = this.doc.format_yaml().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"foo: bar\n"}
```

### `parse_csv`

Attempts to parse a string into an array of objects by following the CSV format described in RFC 4180. The first line is assumed to be a header row, which determines the keys of values in each object.

#### Examples


```coffee
root.orders = this.orders.parse_csv()

# In:  {"orders":"foo,bar\nfoo 1,bar 1\nfoo 2,bar 2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar 2","foo":"foo 2"}]}
```

### `parse_json`

Attempts to parse a string as a JSON document and returns the result.

#### Examples


```coffee
root.doc = this.doc.parse_json()

# In:  {"doc":"{\"foo\":\"bar\"}"}
# Out: {"doc":{"foo":"bar"}}
```

### `parse_msgpack`

Parses a [MessagePack](https://msgpack.org/) message into a structured document.

#### Examples


```coffee
root = content().decode("hex").parse_msgpack()

# In:  81a3666f6fa3626172
# Out: {"foo":"bar"}
```

```coffee
root = this.encoded.decode("base64").parse_msgpack()

# In:  {"encoded":"gaNmb2+jYmFy"}
# Out: {"foo":"bar"}
```

### `parse_xml`

BETA: This method is mostly stable but breaking changes could still be made outside of major version releases if a fundamental problem with it is found.

Attempts to parse a string as an XML document and returns a structured result, where elements appear as keys of an object according to the following rules:

- If an element contains attributes they are parsed by prefixing a hyphen, `-`, to the attribute label.
- If the element is a simple element and has attributes, the element value is given the key `#text`.
- XML comments, directives, and process instructions are ignored.
- When elements are repeated the resulting JSON value is an array.
- If cast is true, try to cast values to numbers and booleans instead of returning strings.

#### Parameters

**`cast`** &lt;(optional) bool&gt; whether to try to cast values that are numbers and booleans to the right type. default: false  

#### Examples


```coffee
root.doc = this.doc.parse_xml()

# In:  {"doc":"<root><title>This is a title</title><content>This is some content</content></root>"}
# Out: {"doc":{"root":{"content":"This is some content","title":"This is a title"}}}
```

```coffee
root.doc = this.doc.parse_xml(cast: false)

# In:  {"doc":"<root><title>This is a title</title><number id=99>123</number><bool>True</bool></root>"}
# Out: {"doc":{"root":{"bool":"True","number":{"#text":"123","-id":"99"},"title":"This is a title"}}}
```

```coffee
root.doc = this.doc.parse_xml(cast: true)

# In:  {"doc":"<root><title>This is a title</title><number id=99>123</number><bool>True</bool></root>"}
# Out: {"doc":{"root":{"bool":true,"number":{"#text":123,"-id":99},"title":"This is a title"}}}
```

### `parse_yaml`

Attempts to parse a string as a single YAML document and returns the result.

#### Examples


```coffee
root.doc = this.doc.parse_yaml()

# In:  {"doc":"foo: bar"}
# Out: {"doc":{"foo":"bar"}}
```

## Encoding and Encryption

### `decode`

Decodes an encoded string target according to a chosen scheme and returns the result as a byte array. When mapping the result to a JSON field the value should be cast to a string using the method [`string`][methods.string], or encoded using the method [`encode`][methods.encode], otherwise it will be base64 encoded by default.

Available schemes are: `base64`, `base64url`, `hex`, `ascii85`.

#### Parameters

**`scheme`** &lt;string&gt; The decoding scheme to use.  

#### Examples


```coffee
root.decoded = this.value.decode("hex").string()

# In:  {"value":"68656c6c6f20776f726c64"}
# Out: {"decoded":"hello world"}
```

```coffee
root = this.encoded.decode("ascii85")

# In:  {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}
# Out: this is totally unstructured data
```

### `decrypt_aes`

Decrypts an encrypted string or byte array target according to a chosen AES encryption method and returns the result as a byte array. The algorithms require a key and an initialization vector / nonce. Available schemes are: `ctr`, `ofb`, `cbc`.

#### Parameters

**`scheme`** &lt;string&gt; The scheme to use for decryption, one of `ctr`, `ofb`, `cbc`.  
**`key`** &lt;string&gt; A key to decrypt with.  
**`iv`** &lt;string&gt; An initialization vector / nonce.  

#### Examples


```coffee
let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.decrypted = this.value.decode("hex").decrypt_aes("ctr", $key, $vector).string()

# In:  {"value":"84e9b31ff7400bdf80be7254"}
# Out: {"decrypted":"hello world!"}
```

### `encode`

Encodes a string or byte array target according to a chosen scheme and returns a string result. Available schemes are: `base64`, `base64url`, `hex`, `ascii85`.

#### Parameters

**`scheme`** &lt;string&gt; The encoding scheme to use.  

#### Examples


```coffee
root.encoded = this.value.encode("hex")

# In:  {"value":"hello world"}
# Out: {"encoded":"68656c6c6f20776f726c64"}
```

```coffee
root.encoded = content().encode("ascii85")

# In:  this is totally unstructured data
# Out: {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}
```

### `encrypt_aes`

Encrypts a string or byte array target according to a chosen AES encryption method and returns a string result. The algorithms require a key and an initialization vector / nonce. Available schemes are: `ctr`, `ofb`, `cbc`.

#### Parameters

**`scheme`** &lt;string&gt; The scheme to use for encryption, one of `ctr`, `ofb`, `cbc`.  
**`key`** &lt;string&gt; A key to encrypt with.  
**`iv`** &lt;string&gt; An initialization vector / nonce.  

#### Examples


```coffee
let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.encrypted = this.value.encrypt_aes("ctr", $key, $vector).encode("hex")

# In:  {"value":"hello world!"}
# Out: {"encrypted":"84e9b31ff7400bdf80be7254"}
```

### `hash`

Hashes a string or byte array according to a chosen algorithm and returns the result as a byte array. When mapping the result to a JSON field the value should be cast to a string using the method [`string`][methods.string], or encoded using the method [`encode`][methods.encode], otherwise it will be base64 encoded by default.

Available algorithms are: `hmac_sha1`, `hmac_sha256`, `hmac_sha512`, `md5`, `sha1`, `sha256`, `sha512`, `xxhash64`.

The following algorithms require a key, which is specified as a second argument: `hmac_sha1`, `hmac_sha256`, `hmac_sha512`.

#### Parameters

**`algorithm`** &lt;string&gt; The hasing algorithm to use.  
**`key`** &lt;(optional) string&gt; An optional key to use.  

#### Examples


```coffee
root.h1 = this.value.hash("sha1").encode("hex")
root.h2 = this.value.hash("hmac_sha1","static-key").encode("hex")

# In:  {"value":"hello world"}
# Out: {"h1":"2aae6c35c94fcfb415dbe95f408b9ce91ee846ed","h2":"d87e5f068fa08fe90bb95bc7c8344cb809179d76"}
```

## GeoIP

### `geoip_anonymous_ip`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the anonymous IP associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_asn`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the ASN associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_city`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the city associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_connection_type`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the connection type associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_country`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the country associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_domain`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the domain associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_enterprise`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the enterprise associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

### `geoip_isp`

EXPERIMENTAL: Looks up an IP address against a [MaxMind database file](https://www.maxmind.com/en/home) and, if found, returns an object describing the ISP associated with it.

#### Parameters

**`path`** &lt;string&gt; A path to an mmdb (maxmind) file.  

## Deprecated

### `parse_timestamp_unix`

Attempts to parse a string as a timestamp, following ISO 8601 format by default, and returns the unix epoch.

#### Parameters

**`format`** &lt;string, default `"2006-01-02T15:04:05.999999999Z07:00"`&gt; An optional format to use.  

#### Examples


```coffee
root.doc.timestamp = this.doc.timestamp.parse_timestamp_unix()

# In:  {"doc":{"timestamp":"2020-08-14T11:45:26.371Z"}}
# Out: {"doc":{"timestamp":1597405526}}
```

An optional string argument can be used in order to specify the expected format of the timestamp. The format is defined by showing how the reference time, defined to be Mon Jan 2 15:04:05 -0700 MST 2006, would be displayed if it were the value.

```coffee
root.doc.timestamp = this.doc.timestamp.parse_timestamp_unix("2006-Jan-02")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":1597363200}}
```

[field_paths]: /docs/configuration/field_paths
[methods.encode]: #encode
[methods.string]: #string