github.com/lmorg/murex@v0.0.0-20240217211045-e081c89cd4ef/lang/stdio/interface_doc.yaml

- DocumentID: ReadArray
  Title: >-
    `ReadArray()` (type)
  CategoryID: apis
  Summary: >-
    Read from a data type one array element at a time
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by builtins to allow them to read data structures one array element
    at a time.
    
    The purpose of this function is to allow builtins to support sequential reads
    (where possible) and also create a standard interface for builtins, thus
    allowing them to be data-type agnostic.
  Usage: |-
    Registering your `ReadArray()`
    
    ```go
    // To avoid confusion, this should only happen inside func init()
    stdio.RegisterReadArray(/* your type name */, /* your readArray func */)
    ```
  Examples: |-
    Example `ReadArray()` function:

    ```go
    {{ include "builtins/types/string/array_read.go" }}
    ```
  Detail: |-
    If your data type is not a stream-able array, it is then recommended that
    you pass your array to  `lang.ArrayTemplate()` which is a handler to convert Go
    structures into Murex arrays. This also makes writing `ReadArray()` handlers
    easier since you can just pass `lang.ArrayTemplate()` your marshaller.
    For example:

    ```go
    {{ include "builtins/types/json/array_read.go" }}
    ```

    The downside of this is that you're then unmarshalling the entire file, which
    could be slow on large files and also breaks the streaming nature of UNIX
    pipelines.
  Parameters:
  - "`stdio.Io`: stream to read from (eg STDIN)"
  - "`func([]byte)`: callback function. Each callback will be a []byte slice containing an array element"
  Related:
  - WriteArray
  - ReadMap
  - lang.ArrayTemplate
  - lang.ArrayWithTypeTemplate
  - ReadIndex
  - ReadNotIndex


- DocumentID: lang.ArrayTemplate
  Title: >-
    `lang.ArrayTemplate()` (template API)
  CategoryID: apis
  Summary: >-
    Unmarshals a data type into a Go struct and returns the results as an array
  Description: |-
    This is a template API you can use for your custom data types to wrap around an
    existing Go marshaller and return a Murex array which is consistent with
    other structures such as nested JSON or YAML documents.

    It should only be called from `ReadArray()` functions.

    Because `lang.ArrayTemplate()` relies on a marshaller, it means any types that
    rely on this API are not going to be stream-able.
  Usage:
  Examples: |-
    Example calling `lang.ArrayTemplate()` function:

    ```go
    {{ include "builtins/types/json/array_read.go" }}
    ```
  Detail: |-
    ### API Source:

    ```go
    {{ include "lang/define_array.go" }}
    ```
  Parameters:
  - "`func(interface{}) ([]byte, error)`: data type's marshaller"
  - "`func([]byte, interface{}) error`: data type's unmarshaller"
  - "`stdio.Io`: stream to read from (eg STDIN)"
  - "`func([]byte)`: callback function to write each array element"
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - ReadMap
  - ReadIndex
  - ReadNotIndex
  - lang.IndexTemplateObject
  - lang.IndexTemplateTable


- DocumentID: ReadArrayWithType
  Title: >-
    `ReadArrayWithType()` (type)
  CategoryID: apis
  Summary: >-
    Read from a data type one array element at a time and return the elements
    contents and data type
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by builtins to allow them to read data structures one array element
    at a time.
    
    The purpose of this function is to allow builtins to support sequential reads
    (where possible) and also create a standard interface for builtins, thus
    allowing them to be data-type agnostic.
    
    This differs from ReadArray() because it also returns the data type.

    There is a good chance ReadArray() might get deprecated in the medium to long
    term.
  Usage: |-
    Registering your `ReadArrayWithType()`
    
    ```go
    // To avoid confusion, this should only happen inside func init()
    stdio.RegisterReadArrayWithType(/* your type name */, /* your readArray func */)
    ```
  Examples: |-
    Example `ReadArrayWithType()` function:

    ```go
    {{ include "builtins/types/string/array_read_type.go" }}
    ```
  Detail: |-
    If your data type is not a stream-able array, it is then recommended that
    you pass your array to  `lang.ArrayWithTypeTemplate()` which is a handler to
    convert Go structures into Murex arrays. This also makes writing `ReadArray()`
    handlers easier since you can just pass `lang.ArrayTemplate()` your marshaller.
    For example:

    ```go
    {{ include "builtins/types/json/array_read_type.go" }}
    ```

    The downside of this is that you're then unmarshalling the entire file, which
    could be slow on large files and also breaks the streaming nature of UNIX
    pipelines.
  Parameters:
  - "`stdio.Io`: stream to read from (eg STDIN)"
  - "`func(interface{}, string)`: callback function. Each callback will be the value in its native Go data type (eg string, int, float64, bool) for an array element"
  Related:
  - WriteArray
  - ReadMap
  - lang.ArrayTemplate
  - lang.ArrayWithTypeTemplate
  - ReadIndex
  - ReadNotIndex


- DocumentID: lang.ArrayWithTypeTemplate
  Title: >-
    `lang.ArrayWithTypeTemplate()` (template API)
  CategoryID: apis
  Summary: >-
    Unmarshals a data type into a Go struct and returns the results as an array
    with data type included
  Description: |-
    This is a template API you can use for your custom data types to wrap around an
    existing Go marshaller and return a Murex array which is consistent with
    other structures such as nested JSON or YAML documents.

    It should only be called from `ReadArrayWithType()` functions.

    Because `lang.ArrayTemplateWithType()` relies on a marshaller, it means any types that
    rely on this API are not going to be stream-able.
  Usage:
  Examples: |-
    Example calling `lang.ArrayTemplate()` function:

    ```go
    {{ include "builtins/types/json/array_read.go" }}
    ```
  Detail: |-
    ### API Source:

    ```go
    {{ include "lang/define_array_type.go" }}
    ```
  Parameters:
  - "`func(interface{}) ([]byte, error)`: data type's marshaller"
  - "`func([]byte, interface{}) error`: data type's unmarshaller"
  - "`stdio.Io`: stream to read from (eg STDIN)"
  - "`func(interface{}, string)`: callback function to write each array element, with data type"
  Related:
  - ReadArrayWithType
  - ReadArray
  - WriteArray
  - ReadMap
  - ReadIndex
  - ReadNotIndex
  - lang.IndexTemplateObject
  - lang.IndexTemplateTable



- DocumentID: WriteArray
  Title: >-
    `WriteArray()` (type)
  CategoryID: apis
  Summary: >-
    Write a data type, one array element at a time
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by builtins to allow them to write data structures one array
    element at a time.

    The purpose of this function is to allow builtins to support sequential writes
    (where possible) and also create a standard interface for builtins, thus
    allowing them to be data-type agnostic.

    ### A Collection of Functions

    `WriteArray()` should return a `struct` that satisfies the following
    `interface{}`:
    
    ```go
    {{ include "lang/stdio/interface_aw.go" }}
    ```
  Usage: |-
    Registering your `WriteArray()`
    
    ```go
    // To avoid confusion, this should only happen inside func init()
    stdio.RegisterWriteArray(/* your type name */, /* your writeArray func */)
    ```
  Examples: |-
    Example `WriteArray()` function:

    ```go
    {{ include "builtins/types/string/array_write.go" }}
    ```
  Detail: |-
    Since not all data types will be stream-able (for example `json`), some types
    may need to cache the array and then to write it once the array writer has been
    closed.

    ```go
    {{ include "builtins/types/json/array_write.go" }}
    ```
  Related:
  - ReadArray
  - ReadArrayWithType
  - ReadMap
  - ReadIndex
  - ReadNotIndex

- DocumentID: ReadMap
  Title: >-
    `ReadMap()` (type)
  CategoryID: apis
  Summary: >-
    Treat data type as a key/value structure and read its contents
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by builtins to allow them to read data structures one key/value
    pair at a time.

    The purpose of this function is to allow builtins to support sequential reads
    (where possible) and also create a standard interface for builtins, thus
    allowing them to be data-type agnostic.
  Usage: |-
    Registering your `ReadMap()`
    
    ```go
    // To avoid confusion, this should only happen inside func init()
    stdio.RegisterReadMap(/* your type name */, /* your readMap func */)
    ```
  Examples: |-
    Example `ReadMap()` function:

    ```go
    {{ include "builtins/types/json/map.go" }}
    ```
  Detail: |-
    There isn't (yet) a template read function for types to call. However that
    might follow in a future release of Murex.
  Parameters:
  - "`stdio.Io`: stream to read from (eg STDIN)"
  - "`*config.Config`: scoped config (eg your data type might have configurable parsing rules)"
  - "`func(key, value string, last bool)`: callback function: key and value of map plus boolean which is true if last element in row (eg reading from tables rather than key/values)"
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - ReadIndex
  - ReadNotIndex


- DocumentID: ReadIndex
  Title: >-
    `ReadIndex()` (type)
  CategoryID: apis
  Summary: >-
    Data type handler for the index, `[`, builtin
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by the index, `[`, builtin.

    The purpose of this function is to allow builtins to support sequential reads
    (where possible) and also create a standard interface for `[` (index), thus
    allowing it to be data-type agnostic.
  Usage: |-
    Registering your `ReadIndex()`
    
    ```go
    // To avoid data races, this should only happen inside func init()
    lang.ReadIndexes[ /* your type name */ ] = /* your readIndex func */
    ```
  Examples: |-
    Example `ReadIndex()` function:

    ```go
    {{ include "builtins/types/json/index.go" }}
    ```
  Detail: |-
    While there is support for a dedicated `ReadNotIndex()` for instances of `![`,
    the template APIs `lang.IndexTemplateObject` and `lang.IndexTemplateTable` are
    both agnostic to the bang prefix.
  Parameters:
  - "`*lang.Process`: Process's runtime state. Typically expressed as the variable `p` "
  - "`[]string`: slice of parameters used in `[` "
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - lang.IndexTemplateObject
  - lang.IndexTemplateTable
  - ReadNotIndex
  - index
  - element
  - bang-prefix



- DocumentID: ReadNotIndex
  Title: >-
    `ReadNotIndex()` (type)
  CategoryID: apis
  Summary: >-
    Data type handler for the bang-prefixed index, `![`, builtin
  Description: |-
    This is a function you would write when programming a Murex data-type.
    
    It's called by the index, `![`, builtin.

    The purpose of this function is to allow builtins to support sequential reads
    (where possible) and also create a standard interface for `![` (index), thus
    allowing it to be data-type agnostic.
  Usage: |-
    Registering your `ReadNotIndex()`
    
    ```go
    // To avoid data races, this should only happen inside func init()
    lang.ReadNotIndexes[ /* your type name */ ] = /* your readIndex func */
    ```
  Examples: |-
    Example `ReadIndex()` function (the code structure is the same for `ReadIndex`
    and `ReadNotIndex`):

    ```go
    {{ include "builtins/types/json/index.go" }}
    ```
  Detail: |-
    While there is support for a dedicated `ReadNotIndex()` for instances of `![`,
    the template APIs `lang.IndexTemplateObject` and `lang.IndexTemplateTable` are
    both agnostic to the bang prefix.
  Parameters:
  - "`*lang.Process`: Process's runtime state. Typically expressed as the variable `p` "
  - "`[]string`: slice of parameters used in `![` "
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - lang.IndexTemplateObject
  - lang.IndexTemplateTable
  - ReadIndex
  - index
  - element
  - bang-prefix


- DocumentID: lang.IndexTemplateObject
  Title: >-
    `lang.IndexTemplateObject()` (template API)
  CategoryID: apis
  Summary: >-
    Returns element(s) from a data structure
  Description: |-
    This is a template API you can use for your custom data types.

    It should only be called from `ReadIndex()` and `ReadNotIndex()` functions.

    This function ensures consistency with the index, `[`, builtin when used with
    different Murex data types. Thus making indexing a data type agnostic
    capability.
  Usage:
  Examples: |-
    Example calling `lang.IndexTemplateObject()` function:

    ```go
    {{ include "builtins/types/json/index.go" }}
    ```
  Detail: |-
    ### API Source:

    ```go
    {{ include "lang/define_index_objects.go" }}
    ```
  Parameters:
  - "`*lang.Process`: Process's runtime state. Typically expressed as the variable `p` "
  - "`[]string`: slice of parameters used in `[` / `![` "
  - "`*interface{}`: a pointer to the data structure being indexed"
  - "`func(interface{}) ([]byte, error)`: data type marshaller function"
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - ReadMap
  - ReadIndex
  - ReadNotIndex
  - lang.IndexTemplateTable
  - index


- DocumentID: lang.IndexTemplateTable
  Title: >-
    `lang.IndexTemplateTable()` (template API)
  CategoryID: apis
  Summary: >-
    Returns element(s) from a table
  Description: |-
    This is a template API you can use for your custom data types.

    It should only be called from `ReadIndex()` and `ReadNotIndex()` functions.

    This function ensures consistency with the index, `[`, builtin when used with
    different Murex data types. Thus making indexing a data type agnostic
    capability.
  Usage:
  Examples: |-
    Example calling `lang.IndexTemplateTable()` function:

    ```go
    {{ include "builtins/types/generic/index.go" }}
    ```
  Detail: |-
    ### API Source:

    ```go
    {{ include "lang/define_index_tables.go" }}
    ```
  Parameters:
  - "`*lang.Process`: Process's runtime state. Typically expressed as the variable `p` "
  - "`[]string`: slice of parameters used in `[` / `![` "
  - "`chan []string`: a channel for rows (each element in the slice is a column within the row). This allows tables to be stream-able"
  - "`func(interface{}) ([]byte, error)`: data type marshaller function"
  Related:
  - ReadArray
  - ReadArrayWithType
  - WriteArray
  - ReadMap
  - ReadIndex
  - ReadNotIndex
  - lang.IndexTemplateObject
  - index