github.com/xiaq/elvish@v0.12.0/website/src/ref/language.md

<!-- toc number-sections -->

# Introduction

This document describes the Elvish programming language. It tries to be both a
specification and an advanced tutorial; if it turns out to be impossible to
do these two things at the same time, this document will evolve to a formal
specification, and more readable tutorials will be created.

Examples for one construct might use constructs that have not yet been
introduced, so some familiarity with the language is assumed. If you are new
to Elvish, start with the [learning materials](/learn).

**Note to the reader**. Like Elvish itself, this document is a work in
progress. Some materials are missing, and some are documented sparingly. If
you have found something that should be improved -- even if there is already a
"TODO" for it -- please feel free to ask on any of the chat channels
advertised on the [homepage](/). Some developer will explain to you, and then
update the document. Question-driven documentation :)


# Syntax Convention

Elvish source code must be UTF-8-encoded. In this document, **character** is a
synonym of [Unicode codepoint](https://en.wikipedia.org/wiki/Code_point) or
its UTF-8 encoding.

Also like most shells, Elvish uses whitespaces -- instead of commas, periods
or semicolons -- to separate constructs. In this document, an **inline
whitespace** is any of:

*   A space (ASCII 0x20) or tab (ASCII 0x9, `"\t"`);

*   A comment: starting with `#` and ending before the next newline or end of
    file;

*   Line continuation: a backslash followed by a newline.

A **whitespace** is either an **inline whitespace** or a newline (`"\n"`).

Like most shells, Elvish has a syntax structure that can be divided into two
levels: a **statement** level and an **expression** level. For instance, on
the expression level, `"echo"` is a quoted string that evaluates to `echo`;
but on the statement level, it is a command that outputs an empty line. This
distinction is often clear from the context and sometime the verb used. A
statements **executes** to produce side effects; an expression **evaluates**
to some values. (The traditional terms for the two levels are "commands" and
"words", but those terms are quite ambiguous.)


# String

The most common data structure in shells is the string. String literals can be
quoted or unquoted (barewords).

## Quoted

There are two types of quoted strings in Elvish, single-quoted strings and
double-quoted strings.

In single-quoted strings, all characters represent themselves, except single
quotes, which need to be doubled. For instance, ``'*\'`` evaluates to ``*\``,
and ``'it''s'`` evaluates to ``it's``.

In double-quoted strings, the backslash ``\`` introduces a **escape
sequence**. For instance, ``"\n"`` evaluates to a newline; ``"\\"`` evaluates
to a backslash; invalid escape sequences like ``"\*"`` result in a syntax
error.

**TODO**: Document the full list of supported escape sequences.

Unlike most other shells, double-quoted strings do not support interpolation.
For instance, `"$USER"` simply evaluates to the string `$USER`. To get a
similar effect, simply concatenate strings: instead of `"my name is $name"`,
write `"my name is "$name`. Under the hood this is a [compound
expression](#compound-expression-and-braced-lists).

## Barewords

If a string only consists of bareword characters, it can be written without
any quote; this is called a **bareword**. Examples are `a.txt`,
`long-bareword`, and `/usr/local/bin`. The set of bareword characters include:

*   ASCII letters (a-z and A-Z) and numbers (0-9);

*   The symbols `-_:%+,./@!`;

*   Non-ASCII codepoints that are printable, as defined by
    [unicode.IsPrint](https://godoc.org/unicode#IsPrint) in Go's standard
    library.

The following are bareword characters depending on their position:

*   The tilde `~`, unless it appears at the beginning of a compound
    expression, in which case it is subject to [tilde
    expansion](#tilde-expansion);

*   The equal sign `=`, unless it is used for terminating [map keys](#map) or
    [option keys](#arguments-and-options), or denoting
    [assignments](#assignment) or [temporary
    assignments](#temporary-assignment).

Unlike traditional shells, an unquoted backslash ``\`` does not escape
metacharacters; use quoted strings instead. For instance, to echo a star, write
`echo "*"` or `echo '*'`, not ``echo \*``. Unquote backslashes are now only
used in line continuations; their use elsewhere is reserved will cause a
syntax error.

## Notes

The three syntaxes above all evaluate to strings, and they are
interchangeable. For instance, `xyz`, `'xyz'` and `"xyz"` are different
syntaxes for the same string, and they are always equivalent.

Elvish does **not** have a separate number type. For instance, in the command
`+ 1 2`, both `1` and `2` are strings, and it is the command `+` that knows to
treat its arguments as numbers. This design is driven by syntax -- because
barewords are always treated as strings, and digits are barewords, we cannot
treat words like `1` as number literals. At some point the language may get a
dedicated number type (or several number types), but they will likely need to
be constructed explicitly, e.g. `(number 1)`.


# List and Map

Lists and maps are the basic container types in Elvish.

## List

Lists are surround by square brackets `[ ]`, with elements separated by
whitespaces. Examples:

```elvish-transcript
~> put [lorem ipsum]
▶ [lorem ipsum]
~> put [lorem
        ipsum
        foo
        bar]
▶ [lorem ipsum foo bar]
```

Note that commas have no special meanings and are valid bareword characters,
so don't use them to separate elements:

```elvish-transcript
~> li = [a, b]
~> put $li
▶ [a, b]
~> put $li[0]
▶ a,
```

## Map

Maps are also surrounded by square brackets; a key/value pair is written
`&key=value` (reminiscent to HTTP query parameters), and pairs are separated
by whitespaces. Whitespaces are allowed after `=`, but not before `=`.
Examples:

```elvish-transcript
~> put [&foo=bar &lorem=ipsum]
▶ [&foo=bar &lorem=ipsum]
~> put [&a=   10
        &b=   23
        &sum= (+ 10 23)]
▶ [&a=10 &b=23 &sum=33]
```

An empty map is written as `[&]`.


# Variable

Variables are named holders of values. The following characters can be used in
variable names (a subset of bareword characters):

*   ASCII letters (a-z and A-Z) and numbers (0-9);

*   The symbols `-_:~`. The colon `:` is special; it is normally used for
    separating namespaces or denoting namespace variables;

*   Non-ASCII codepoints that are printable, as defined by
    [unicode.IsPrint](https://godoc.org/unicode#IsPrint) in Go's standard
    library.

In most other shells, variables can map directly to environmental variables:
`$PATH` is the same as the `PATH` environment variable. This is not the case
in Elvish. Instead, environment variables are put in a dedicated `E:`
namespace; the environment variable `PATH` is known as `$E:PATH`. The `$PATH`
variable, on the other hand, does not exist initially, and if you have defined
it, only lives in a certain lexical scope within the Elvish interpreter.

You will notice that variables sometimes have a leading dollar `$`, and
sometimes not. The tradition is that they do when they are used for their
values, and do not otherwise (e.g. in assignment). This is consistent with
most other shells.

## Assignment

A variable can be assigned by writing its name, `=`, and the value to assign.
There must be inline whitespaces both before and after `=`. Example:

```elvish-transcript
~> foo = bar
```

You can assign multiple values to multiple variables simultaneously, simply by writing several variable names (separated by inline whitespaces) on the left-hand side, and several values on the right-hand side:

```elvish-transcript
~> x y = 3 4
```

## Referencing

Use a variable by adding `$` before the name:

```elvish-transcript
~> foo = bar
~> x y = 3 4
~> put $foo
▶ bar
~> put $x
▶ 3
```

Variables must be assigned before use. Attempting to use an unassigned
variable causes a compilation error:

```elvish-transcript
~> echo $x
Compilation error: variable $x not found
[tty], line 1: echo $x
~> { echo $x }
Compilation error: variable $x not found
[tty], line 1: { echo $x }
```

## Explosion and Rest Variable

If a variable contains a list value, you can add `@` before the variable name
to get all its element values. This is called **exploding** the variable:

```elvish-transcript
~> li = [lorem ipsum foo bar]
~> put $li
▶ [lorem ipsum foo bar]
~> put $@li
▶ lorem
▶ ipsum
▶ foo
▶ bar
```

(This notation is restricted to exploding variables. To explode arbitrary values, use the builtin [explode](/ref/builtin.html#explode) command.)

When assigning variables, if you prefix the name of the last variable with
`@`, it gets assigned a list containing all remaining values. That variable is
called a **rest variable**. Example:

```elvish-transcript
~> a b @rest = 1 2 3 4 5 6 7
~> put $a $b $rest
▶ 1
▶ 2
▶ [3 4 5 6 7]
```

Schematically this is a reverse operation to variable explosion, which is why
they share the `@` sign.


## Temporary Assignment

You can prepend a command with **temporary assignments**, which gives
variables temporarily values during the execution of that command.

In the following
example, `$x` and `$y` are temporarily assigned 100 and 200:

```elvish-transcript
~> x y = 1 2
~> x=100 y=200 + $x $y
▶ 300
~> echo $x $y
1 2
```

In contrary to normal assignments, there should be no whitespaces around the
equal sign `=`. To have multiple variables in the left-hand side, use braces:

```elvish-transcript
~> x y = 1 2
~> fn f { put 100 200 }
~> {x,y}=(f) + $x $y
▶ 300
```

If you use a previously undefined variable in a temporary assignment, its
value will become the empty string after the command finishes. This behavior
will likely change; don't rely on it.

Since ordinary assignments are also a kind of command, they can also be
prepended with temporary assignments:

```elvish-transcript
~> x=1
~> x=100 y = (+ 133 $x)
~> put $x $y
▶ 1
▶ 233
```

Temporary assignments must all appear before the command. As soon as something
that is not a temporary assignments is parsed, Elvish no longer parses
temporary assignments. For instance, in `x=1 echo x=1`, the second `x=1` is not
a temporary assignment, but a bareword.

**Note**: Elvish's behavior differs from bash (or zsh) in one important place.
In bash, temporary assignments to variables do not affect their direct
appearance in the command:

```sh-transcript
bash-4.4$ x=1
bash-4.4$ x=100 echo $x
1
```


## Scoping rule

Elvish has lexical scoping. Scopes are introduced by [lambdas](#lambda) or
[user-defined modules](#user-defined-modules).

When you use a variable, Elvish looks for it in the current lexical scope,
then its parent lexical scope and so forth, until the outermost scope:

```elvish-transcript
~> x = 12
~> { echo $x } # $x is in the global scope
12
~> { y = bar; { echo $y } } # $y is in the outer scope
bar
```

If a variable is not in any of the lexical scopes, Elvish tries to resolve it
in the `builtin:` namespace, and if that also fails, cause an error:

```elvish-transcript
~> echo $pid # builtin
36613
~> echo $nonexistent
Compilation error: variable $nonexistent not found
  [interactive], line 1:
    echo $nonexistent
```

Note that Elvish resolves all variables in a code chunk before starting to
execute any of it; that is why the error message above says *compilation
error*. This can be more clearly observed in the following example:

```elvish-transcript
~> echo pre-error; echo $nonexistent
Compilation error: variable $nonexistent not found
[tty], line 1: echo pre-error; echo $nonexistent
```

When you assign a variable, Elvish does a similar searching. If the variable
cannot be found, it will be created in the current scope:

```elvish-transcript
~> x = 12
~> { x = 13 } # assigns to x in the global scope
~> echo $x
13
~> { z = foo } # creates z in the inner scope
~> echo $z
Compilation error: variable $z not found
[tty], line 1: echo $z
```

One implication of this behavior is that Elvish will not shadow your variable
in outer scopes.

There is a `local:` namespace that always refers to the current scope, and by
using it it is possible to force Elvish to shadow variables:

```elvish-transcript
~> x = 12
~> { local:x = 13; echo $x } # force shadowing
13
~> echo $x
12
```

After force shadowing, you can still access the variable in the outer scope
using the `up:` namespace, which always **skips** the innermost scope:

```elvish-transcript
~> x = 12
~> { local:x = 14; echo $x $up:x }
14 12
```

The `local:` and `up:` namespaces can also be used on unshadowed variables,
although they are not useful in those cases:

```elvish-transcript
~> foo = a
~> { echo $up:foo } # $up:foo is the same as $foo
a
~> { bar = b; echo $local:bar } # $local:bar is the same as $bar
b
```

It is not possible to refer to a specific outer scope.

You cannot create new variables in the `builtin:` namespace, although existing
variables in it can be assigned new values.


# Lambda

A function literal, or lambda, is a [code chunk](#code-chunk) surrounded by
curly braces:

```elvish-transcript
~> f = { echo "Inside a lambda" }
~> put $f
▶ <closure 0x18a1a340>
```

One or more whitespace characters after `{` is required: Elvish relies on the
presence of whitespace to disambiguate lambda literals and [braced
lists](#braced-lists). It is good style to put some whitespace before the
closing `}` as well, but this is not required by the syntax.

Functions are first-class values in Elvish. They can be kept in variables, used
as arguments, output on the value channel, and embedded in other data
structures. They can also be used as commands:

```elvish-transcript
~> $f
Inside a lambda
~> { echo "Inside a literal lambda" }
Inside a literal lambda
```

The last command resembles a code block in C-like languages in syntax. But
under the hood, it defines a function on the fly and calls it immediately.

Functions defined using the basic syntax above do not accept any arguments or
options. To do so, you need to write a signature.

## Signature

A **signature** specifies the arguments a function can accept:

```elvish-transcript
~> f = [a b]{ put $b $a }
~> $f lorem ipsum
▶ ipsum
▶ lorem
```

There should be no space between `]` and `{`; otherwise Elvish will parse the
signature as a list, followed by a lambda without signature:

```elvish-transcript
~> put [a]{ nop }
▶ <closure 0xc420153d80>
~> put [a] { nop }
▶ [a]
▶ <closure 0xc42004a480>
```

Like in the left hand of assignments, if you prefix the last argument with
`@`, it becomes a **rest argument**, and its value is a list containing all
the remaining arguments:

```elvish-transcript
~> f = [a @rest]{ put $a $rest }
~> $f lorem
▶ lorem
▶ []
~> $f lorem ipsum dolar sit
▶ lorem
▶ [ipsum dolar sit]
```

You can also declare options in the signature. The syntax is `&name=default`
(like a map pair), where `default` is the default value for the option:

```elvish-transcript
~> f = [&opt=default]{ echo "Value of $opt is "$opt }
~> $f
Value of $opt is default
~> $f &opt=foobar
Value of $opt is foobar
```

Options must have default values: Options should be **option**al.

If you call a function with too few arguments, too many arguments or unknown
options, an exception is thrown:

```elvish-transcript
~> [a]{ echo $a } foo bar
Exception: need 1 arguments, got 2
[tty], line 1: [a]{ echo $a } foo bar
~> [a b]{ echo $a $b } foo
Exception: need 2 arguments, got 1
[tty], line 1: [a b]{ echo $a $b } foo
~> [a b @rest]{ echo $a $b $rest } foo
Exception: need 2 or more arguments, got 1
[tty], line 1: [a b @rest]{ echo $a $b $rest } foo
~> [&k=v]{ echo $k } &k2=v2
Exception: unknown option k2
[tty], line 1: [&k=v]{ echo $k } &k2=v2
```


## Closure Semantics

User-defined functions are also known as "closures", because they have
[closure
semantics](https://en.wikipedia.org/wiki/Closure_(computer_programming)).

In the following example, the `make-adder` function outputs two functions,
both referring to a local variable `$n`. Closure semantics means that:

1.  Both functions can continue to refer to the `$n` variable after
    `make-adder` has returned.

2.  Multiple calls to the `make-adder` function generates distinct instances
    of the `$n` variables.

```elvish-transcript
~> fn make-adder {
     n = 0
     put { put $n } { n = (+ $n 1) }
   }
~> getter adder = (make-adder)
~> $getter # $getter outputs $n
▶ 0
~> $adder # $adder increments $n
~> $getter # $getter and $setter refer to the same $n
▶ 1
~> getter2 adder2 = (make-adder)
~> $getter2 # $getter2 and $getter refer to different $n
▶ 0
~> $getter
▶ 1
```

Variables that get "captured" in closures are called **upvalues**; this is why
the pseudo-namespace for variables in outer scopes is called `up:`. When
capturing upvalues, Elvish only captures the variables that are used. In the
following example, `$m` is not an upvalue of `$g` because it is not used:

```elvish-transcript
~> fn f { m = 2; n = 3; put { put $n } }
~> g = (f)
```

This effect is not currently observable, but will become so when namespaces
[become introspectable](https://github.com/elves/elvish/issues/492).


# Indexing

Indexing is done by putting one or more **index expressions** in brackets `[]`
after a value.

## List Indexing

Lists can be indexed with any of the following:

*   A non-negative integer, an offset counting from the beginning of the list.
    For example, `$li[0]` is the first element of `$li`.

*   A negative integer, an offset counting from the back of the list. For
    instance, `$li[-1]` is the last element `$li`.

*   A slice `$a:$b`, where both `$a` and `$b` are integers. The result is
    sublist of `$li[$a]` up to, but not including, `$li[$b]`. For instance,
    `$li[4:7]` equals `[$li[4] $li[5] $li[6]]`, while `$li[1:-1]` contains all
    elements from `$li` except the first and last one.

    Both integers may be omitted; `$a` defaults to 0 while `$b` defaults to the
    length of the list. For instance, `$li[:2]` is equivalent to `$li[0:2]`,
    `$li[2:]` is equivalent to `$li[2:(count $li)]`, and `$li[:]` makes a copy
    of `$li`. The last form is rarely useful, as lists are immutable.

    Note that the slice needs to be a **single** string, so there cannot be
    any spaces within the slice. For instance, `$li[2:10]` cannot be written
    as `$li[2: 10]`; the latter contains two indicies and is equivalent to
    `$li[2:] $li[10]` (see [Multiple Indicies](#multiple-indicies)).

*   Not yet implemented: The string `@`. The result is all the values in the
    list. Note that this is not the same as `:`: if `$li` has 10 elements,
    `$li[@]` evaluates to 10 values (all the elements in the list), while
    `$li[:]` evaluates to just one value (a copy of the list).

    When used on a variable like `$li`, it is equivalent to the explosion
    construct `$li[@]`. It is useful, however, when used on other constructs,
    like output capture or other

Examples:

```elvish-transcript
~> li = [lorem ipsum foo bar]
~> put $li[0]
▶ lorem
~> put $li[-1]
▶ bar
~> put $li[0:2]
▶ [lorem ipsum]
```

(Negative indicies and slicing are borrowed from Python.)

## String indexing

**NOTE**: String indexing will likely change.

Strings should always be UTF-8, and they can indexed by **byte indicies at
which codepoints start**, and indexing results in **the codepoint that starts
there**.  This is best explained with examples:

*   In the string `elv`, every codepoint is encoded with only one byte, so 0,
    1, 2 are all valid indices:

    ```elvish-transcript
    ~> put elv[0]
    ▶ e
    ~> put elv[1]
    ▶ l
    ~> put elv[2]
    ▶ v
    ```

*   In the string `世界`, each codepoint is encoded with three bytes. The first
    codepoint occupies byte 0 through 2, and the second occupies byte 3 through
    5\. Hence valid indicies are 0 and 3:

    ```elvish-transcript
    ~> put 世界[0]
    ▶ 世
    ~> put 世界[3]
    ▶ 界
    ```

Strings can also be indexed by slices.

(This idea of indexing codepoints by their byte positions is borrowed from
Julia.)

## Map indexing

Maps are simply indexed by their keys. There is no slice indexing, and `:` does
not have a special meaning. Examples:

```elvish-transcript
~> map = [&a=lorem &b=ipsum &a:b=haha]
~> echo $map[a]
lorem
~> echo $map[a:b]
haha
```

## Multiple Indices

If you put multiple values in the index, you get multiple values: `$li[x y z]`
is equivalent to `$li[x] $li[y] $li[z]`. This applies to all indexable values.
Examples:

```elvish-transcript
~> put elv[0 2 0:2]
▶ e
▶ v
▶ el
~> put [lorem ipsum foo bar][0 2 0:2]
▶ lorem
▶ foo
▶ [lorem ipsum]
~> put [&a=lorem &b=ipsum &a:b=haha][a a:b]
▶ lorem
▶ haha
```


# Output Capture

Output capture is formed by putting parentheses `()` around a [code
chunk](#code-chunk). It redirects the output of the chunk into an internal
pipe, and evaluates to all the values that have been output.

```elvish-transcript
~> + 1 10 100
▶ 111
~> x = (+ 1 10 100)
~> put $x
▶ 111
~> put lorem ipsum
▶ lorem
▶ ipsum
~> x y = (put lorem ipsum)
~> put $x
▶ lorem
~> put $y
▶ ipsum
```

If the chunk outputs bytes, Elvish strips the last newline (if any), and split them by newlines, and consider each line to be one string value:

```elvish-transcript
~> put (echo "a\nb")
▶ a
▶ b
```

**Note 1**. Only the last newline is ever removed, so empty lines are
preserved; `(echo "a\n")` evaluates to two values, `"a"` and `""`.

**Note 2**. One consequence of this mechanism is that you can not distinguish
outputs that lack a trailing newline from outputs that have one; `(echo what)`
evaluates to the same value as `(print what)`. If such a distinction is
needed, use [`slurp`](/ref/builtin.html#slurp) to preserve the original
bytes output.

If the chunk outputs both values and bytes, the values of output capture will
contain both value outputs and lines. However, the ordering between value
output and byte output might not agree with the order in which they happened:

```elvish-transcript
~> put (put a; echo b) # value order need not be the same as output order
▶ b
▶ a
```


# Exception Capture

Exception capture is formed by putting `?()` around a code chunk. It runs the chunk and evaluates to the exception it throws.

```elvish-transcript
~> fail bad
Exception: bad
Traceback:
  [interactive], line 1:
    fail bad
~> put ?(fail bad)
▶ ?(fail bad)
```

If there was no error, it evaluates to the special value `$ok`:

```elvish-transcript
~> nop
~> put ?(nop)
▶ $ok
```

Exceptions are booleanly false and `$ok` is booleanly true. This is useful in
`if` (introduced later):

```elvish-transcript
if ?(test -d ./a) {
  # ./a is a directory
}
```

Exception captures do not affect the output of the code chunk. You can combine
output capture and exception capture:

```elvish
output = (error = ?(commands-that-may-fail))
```


# Tilde Expansion

Tildes are special when they appear at the beginning of an expression (the
exact meaning of "expression" will be explained later). The string after it, up
to the first `/` or the end of the word, is taken as a user name; and they
together evaluate to the home directory of that user. If the user name is
empty, the current user is assumed.

In the following example, the home directory of the current user is `/home/xiaq`, while that of the root user is `/root`:

```elvish-transcript
~> put ~
▶ /home/xiaq
~> put ~root
▶ /root
~> put ~/xxx
▶ /home/xiaq/xxx
~> put ~root/xxx
▶ /root/xxx
```

Note that tildes are not special when they appear elsewhere in a word:

```elvish-transcript
~> put a~root
▶ a~root
```

If you need them to be, surround them with braces (the reason this works will be explained later):

```elvish-transcript
~> put a{~root}
▶ a/root
```


# Wildcard Patterns

**Wildcard patterns** are patterns containing **wildcards**, and they evaluate to all filenames they match.

We will use this directory tree in examples:

```
.x.conf
a.cc
ax.conf
foo.cc
d/
|__ .x.conf
|__ ax.conf
|__ y.cc
.d2/
|__ .x.conf
|__ ax.conf
```

Elvish supports the following wildcards:

*   `?` matches one arbitrary character except `/`. For example, `?.cc`
    matches `a.cc`;

*   `*` matches any number of arbitrary characters except `/`. For example,
    `*.cc` matches `a.cc` and `foo.cc`;

*   `**` matches any number of arbitrary characters including `/`. For example,
    `**.cc` matches `a.cc`, `foo.cc` and `b/y.cc`.

The following behaviors are default, although they can be altered by modifiers:

*   When the entire wildcard pattern has no match, an error is thrown.

*   None of the wildcards matches `.` at the beginning of filenames. For example:

    *   `?x.conf` does not match `.x.conf`;

    *   `d/*.conf` does not match `d/.x.conf`;

    *   `**.conf` does not match `d/.x.conf`.


## Modifiers

Wildcards can be **modified** using the same syntax as indexing. For instance,
in `*[match-hidden]` the `*` wildcard is modified with the `match-hidden`
modifier. Multiple matchers can be chained like `*[set:abc][range:0-9]`. There
are two kinds of modifiers:

**Global modifiers** apply to the whole pattern and can be placed after any
wildcard:

*   `nomatch-ok` tells Elvish not to throw an error when there is no match for
    the pattern. For instance, in the example directory `put bad*` will be an
    error, but `put bad*[nomatch-ok]` does exactly nothing.

*   `but:xxx` (where `xxx` is any filename) excludes the filename from the final
    result.

Although global modifiers affect the entire wildcard pattern, you can add it
after any wildcard, and the effect is the same. For example, `put
*/*[nomatch-ok].cpp` and `put *[nomatch-ok]/*.cpp` do the same thing.

On the other hand, you must add it after a wildcard, instead of after the
entire pattern: `put */*.cpp[nomatch-ok]` unfortunately does not do the correct
thing. (This will probably be fixed.)

**Local modifiers** only apply to the wildcard it immediately follows:

*   `match-hidden` tells the wildcard to match `.` at the beginning of
    filenames, e.g. `*[match-hidden].conf` matches `.x.conf` and `ax.conf`.

    Being a local modifier, it only applies to the wildcard it immediately
    follows. For instance, `*[match-hidden]/*.conf` matches `d/ax.conf` and
    `.d2/ax.conf`, but not `d/.x.conf` or `.d2/.x.conf`.

*   Character matchers restrict the characters to match:

    *   Character sets, like `set:aeoiu`;

    *   Character ranges like `range:a-z` (including `z`) or `range:a~z` (excluding `z`);

    *   Character classes: `control`, `digit`, `graphic`, `letter`, `lower`,
        `mark`, `number`, `print`, `punct`, `space`, `symbol`, `title`, and
        `upper`. See the Is* functions [here](https://godoc.org/unicode) for
        their definitions.

    Note the following caveats:

    *   Multiple matchers, they are OR'ed. For instance, ?[set:aeoiu][digit]
        matches `aeoiu` and digits.

    *   Dots at the beginning of filenames always require an explicit
        `match-hidden`, even if the matcher includes `.`. For example,
        `?[set:.a]x.conf` does **not** match `.x.conf`; you have to `?[set:.a
        match-hidden]x.conf`.

    *   Likewise, you always need to use `**` to match slashes, even if the
        matcher includes `/`. For example `*[set:abc/]` is the same as
        `*[set:abc]`.


# Compound Expression and Braced Lists

Writing several expressions together with no space in between will concatenate
them. This creates a **compound expression**, because it mimics the formation
of compound words in natural languages. Examples:

```elvish-transcript
~> put 'a'b"c" # compounding three string literals
▶ abc
~> v = value
~> put '$v is '$v # compounding one string literal with one string variable
▶ '$v is value'
```

Many constructs in Elvish can generate multiple values, like indexing with
multiple indices and output captures. Compounding multiple values with other
values generates all possible combinations:

```elvish-transcript
~> put (put a b)-(put 1 2)
▶ a-1
▶ a-2
▶ b-1
▶ b-2
```

Note the order of the generated values. The value that comes later changes faster.

**NOTE**: There is a perhaps a better way to explain the ordering, but you can think of the previous code as equivalent to this:

```elvish-transcript
for x [a b] {
  for y [1 2] {
    put $x-$y
  }
}
```

## Braced Lists

In practice, you never have to write `(put a b)`: you can use a braced list `{a,b}`:

```elvish-transcript
~> put {a,b}-{1,2}
▶ a-1
▶ a-2
▶ b-1
▶ b-2
```

Elements in braced lists can also be separated with whitespaces, or a combination of comma and whitespaces (the latter not recommended):

```elvish-transcript
~> put {a b , c,d}
▶ a
▶ b
▶ c
▶ d
```

(In future, the syntax might be made more strict.)

Braced list is merely a syntax for grouping multiple values. It is not a data
structure.


# Expression Structure and Precedence

Braced lists are evaluated before being compounded with other values. You can
use this to affect the order of evaluation. For instance, `put *.txt` gives you
all filenames that end with `.txt` in the current directory; while `put
{*}.txt` gives you all filenames in the current directory, appended with
`.txt`.

**TODO**: document evaluation order regarding tilde and wildcards.


# Ordinary Command

The **command** is probably the most important syntax construct in shell
languages, and Elvish is no exception. The word **command** itself, is
overloaded with meanings. In the terminology of this document, the term
**command** include the following:

*   An ordinary assignment, introduced above;

*   An ordinary command, introduced in this section;

*   A special command, introduced in the next section.

An **ordinary command** consists of a compulsory head, and any number of
arguments, options and redirections.

## Head

The **head** must appear first. It is an arbitrary word that determines what will be run. Examples:

```elvish-transcript
~> ls -l # the string ls is the head
(output omitted)
~> (put [@a]{ ls $@a }) -l
(same output)
```

The head must evaluate to one value. For instance, the following does not work:

```elvish-transcript
~> (put [@a]{ ls $@a } -l)
Exception: head of command must be a single value; got 2 values
[tty], line 1: (put [@a]{ ls $@a } -l)
```

The definition of barewords is relaxed for the head to include `<`, `>`, `*`
and `^`. These are all names of numeric builtins:

```elvish-transcript
~> < 3 5 # less-than
▶ $true
~> > 3 5 # greater-than
▶ $false
~> * 3 5 # multiplication
▶ 15
~> ^ 3 5 # power
▶ 243
```

## Arguments and Options

**Arguments** (args for short) and **options** (opts for short) can be supplied
to commands. Arguments are arbitrary words, while options have the same
syntax as map pairs. They are separated by inline whitespaces:

```elvish-transcript
~> echo &sep=, a b c # seq=, is an option; a b c are arguments
a,b,c
```


## Redirections

Redirections are used for modifying file descriptors (FD).

The most common form of redirections opens a file and associates it with an FD.
The form consists of an optional destination FD (like `2`), a redirection
operator (like `>`) and a filename (like `error.log`):

*   The **destination fd** determines which FD to modify. If absent, it is
    determined from the redirection operator. If present, there must be no
    space between the fd and the redirection operator. (Otherwise Elvish parses
    it as an argument.)

*   The **redirection operator** determines the mode to open the file, and the
    destination fd if it is not explicitly specified.

*   The **filename** names the file to open.

Possible redirection operators and their default FDs are:

*   `<` for reading. The default FD is 0 (stdin).

*   `>` for writing. The default FD is 1 (stdout).

*   `>>` for appending. The default FD is 1 (stdout).

*   `<>` for reading and writing. The default FD is 1 (stdout).

Examples:

```elvish-transcript
~> echo haha > log
~> cat log
haha
~> cat < log
haha
~> ls --bad-arg 2> error
Exception: ls exited with 2
Traceback:
  [interactive], line 1:
    ls --bad-arg 2> error
~> cat error
/bin/ls: unrecognized option '--bad-arg'
Try '/bin/ls --help' for more information.
```

Redirections can also be used for closing or duplicating FDs. Instead of
writing a filename, use `&n` (where `n` is a number) for duplicating, or `&-`
for closing. In this case, the redirection operator only determines the default
destination FD (and is totally irrevelant if a destination FD is specified).
Examples:

```elvish-transcript
~> ls >&- # close stdout
/bin/ls: write error: Bad file descriptor
Exception: ls exited with 2
Traceback:
  [interactive], line 1:
    ls >&-
```

If you have multiple related redirections, they are applied in the order they appear. For instance:

```elvish-transcript
~> fn f { echo out; echo err >&2 } # echoes "out" on stdout, "err" on stderr
~> f >log 2>&1 # use file "log" for stdout, then use (changed) stdout for stderr
~> cat log
out
err
```


## Ordering

Elvish does not impose any ordering of arguments, options and redirections:
they can intermix each other. The only requirement is that the head must come
first. This is different from POSIX shells, where redirections may appear
before the head. For instance, the following two both work in POSIX shell, but
only the former works in Elvish:


```sh
echo something > file
> file echo something # mistaken for the comparison builtin ">" in Elvish
```


# Special Commands

**Special commands** obey the same syntax rules as normal commands (i.e.
syntactically special commands can be treated the same as ordinary commands),
but have evaluation rules that are custom to each command. To explain this, we
use the following example:

```elvish-transcript
~> or ?(echo x) ?(echo y) ?(echo z)
x
▶ $ok
```

In the example, the `or` command first evaluates its first argument, which has the value `$ok` (a truish value) and the side effect of outputting `x`. Due to the custom evaluation rule of `or`, the rest of the arguments are not evaluated.

If `or` were a normal command, the code above is still syntactically correct. However, Elvish would then evaluate all its arguments, with the side effect of outputting `x`, `y` and `z`, before calling `or`.


## Deleting variable or element: `del`

The `del` special command can be used to either delete variables or elements
of lists and maps. Operands should be specified without a leading dollar sign,
like the left-hand side of assignments.

Example of deleting variable:

```elvish-transcript
~> x = 2
~> echo $x
2
~> del x
~> echo $x
Compilation error: variable $x not found
[tty], line 1: echo $x
```

Example of deleting element:

```elvish-transcript
~> m = [&k=v &k2=v2]
~> del m[k2]
~> put $m
▶ [&k=v]
~> l = [[&k=v &k2=v2]]
~> del l[0][k2]
~> put $l
▶ [[&k=v]]
```


## Logics: `and` and `or`

The `and` special command evaluates its arguments from left to right; as soon
as a booleanly false value is obtained, it outputs the value and stops. When
given no arguments, it outputs `$true`.

The `or` special command is the same except that it stops when a booleanly true
value is obtained. When given no arguments, it outpus `$false`.


## Condition: `if`

**TODO**: Document the syntax notation, and add more examples.

Syntax:

```elvish-transcript
if <condition> {
    <body>
} elif <condition> {
    <body>
} else {
    <else-body>
}
```

The `if` special command goes through the conditions one by one: as soon as one
evaluates to a booleanly true value, its corresponding body is executed. If
none of conditions are booleanly true and an else body is supplied, it is
executed.

The condition part is an expression, not a command like in other shells.

Tip: a combination of `if` and `?()` gives you a semantics close to other shells:

```elvish-transcript
if ?(test -d .git) {
    # do something
}
```

However, for Elvish's builtin predicates that output values instead of throw exceptions, the output capture construct `()` should be used.

**TODO**: add more examples.

## Conditional Loop: `while`

Syntax:

```elvish-transcript
while <condition> {
    <body>
} else {
    <else-body>
}
```

Execute the body as long as the condition evaluates to a booleanly true value.

The else body, if present, is executed if the body has never been executed
(i.e. the condition evaluates to a booleanly false value in the very
beginning).


## Iterative Loop: `for`

Syntax:

```elvish-transcript
for <var> <container> {
    <body>
} else {
    <body>
}
```

Iterate the container (e.g. a list). In each iteration, assign the variable to
an element of the container and execute the body.

The else body, if present, is executed if the body has never been executed
(i.e. the iteration value has no elements).


## Exception Control: `try`

(If you just want to capture the exception, you can use the more concise
exception capture construct `?()` instead.)

Syntax:

```elvish-transcript
try {
    <try-block>
} except except-varname {
    <except-block>
} else {
    <else-block>
} finally {
    <finally-block>
}
```

Only `try` and `try-block` are required. This control structure behaves as follows:

1.  The `try-block` is always executed first.

2.  If `except` is present and an exception occurs in `try-block`, it is
    caught and stored in `except-varname`, and `except-block` is then
    executed. Example:

    ```elvish-transcript
    ~> try { fail bad } except e { put $e }
    ▶ ?(fail bad)
    ```

    Note that if `except` is not present, exceptions thrown from `try` are not
    caught: for instance, `try { fail bad }` throws `bad`; it is equivalent to
    a plain `fail bad`.

    Note that the word after `except` names a variable, not a matching
    condition. Exception matching is not supported yet. For instance, you may
    want to only match exceptions that were created with `fail bad` with
    `except bad`, but in fact this creates a variable `$bad` that contains
    whatever exception was thrown.

3.  If no exception occurs and `else` is present, `else-block` is executed.
    Example:

    ```elvish-transcript
    ~> try { nop } else { echo well }
    well
    ```

4.  If `finally-block` is present, it is executed. Examples:

    ```elvish-transcript
    ~> try { fail bad } finally { echo final }
    final
    Exception: bad
    Traceback:
      [tty], line 1:
        try { fail bad } finally { echo final }
    ~> try { echo good } finally { echo final }
    good
    final
    ```

5.  If the exception was not caught (i.e. `except` is not present), it is
    rethrown.

Exceptions thrown in blocks other than `try-block` are not caught. If an
exception was thrown and either `except-block` or `finally-block` throws
another exception, the original exception is lost. Examples:

```elvish-transcript
~> try { fail bad } except e { fail worse }
Exception: worse
Traceback:
  [tty], line 1:
    try { fail bad } except e { fail worse }
~> try { fail bad } except e { fail worse } finally { fail worst }
Exception: worst
Traceback:
  [tty], line 1:
    try { fail bad } except e { fail worse } finally { fail worst }
```


## Function Definition: `fn`

Syntax:

```elvish-transcript
fn <name> <lambda>
```

Define a function with a given name. The function behaves in the same way to
the lambda used to define it, except that it "captures" `return`. In other
words, `return` will fall through lambdas not defined with `fn`, and continues
until it exits a function defined with `fn`:

```elvish-transcript
~> fn f {
     { echo a; return }
     echo b # will not execute
   }
~> f
a
~> {
     f
     echo c # executed, because f "captures" the return
   }
a
c
```

**TODO**: Find a better way to describe this. Hopefully the example is
illustrative enough, though.


# Command Resolution

When using a literal string as the head of a command, it is first **resolved**
during the compilation phase, using the following order:

1.  Special commands.

2.  Functions defined on any of the containing lexical scopes, with inner
    scopes looked up first. (This is exactly the same process as variable
    resolution).

3.  The `builtin:` namespace.

4.  Should all these steps fail, it is resolved to be an external command.
    Determination of the path of the external command does not happen in the
    resolution process; that happens during evaluation.

You can use the [resolve](/ref/builtin.html#resolve) command to see which
command Elvish resolves a string to.

During the evaluation phase, external commands are then subject to
**searching**. This can be observed with the
[search-external](/ref/builtin.html#search-builtin) command.


# Pipeline

A **pipeline** is formed by joining one or more commands together with the pipe
sign (`|`).

## IO Semantics

For each pair of adjacent commands `a | b`, the output of `a` is connected to
the input of `b`. Both the byte pipe and the value channel are connected, even
if one of them is not used.

Command redirections are applied before the connection happens. For instance,
the following writes `foo` to `a.txt` instead of the output:

```elvish-transcript
~> echo foo > a.txt | cat
~> cat a.txt
foo
```

## Execution Flow

All of the commands in a pipeline are executed in parallel, and the execution
of the pipeline finishes when all of its commands finish execution.

If one or more command in a pipeline throws an exception, the other commands
will continue to execute as normal. After all commands finish execution, an
exception is thrown, the value of which depends on the number of commands that
have thrown an exception:

*   If only one command has thrown an exception, that exception is rethrown.

*   If more than one commands have thrown exceptions, a "composite exception",
    containing information all exceptions involved, is thrown.

## Background Pipeline

Adding an ampersand `&` to the end of a pipeline will cause it to be executed
in the background. In this case, the rest of the code chunk will continue to
execute without waiting for the pipeline to finish. Exceptions thrown from the
background pipeline do not affect the code chunk that contains it.

When a background pipeline finishes, a message is printed to the terminal if
the shell is interactive.


# Code Chunk

A **code chunk** is formed by joining zero or more pipelines together,
separating them with either newlines or semicolons.

Pipelines in a code chunk are executed in sequence. If any pipeline throws an
exception, the execution of the whole code chunk stops, propagating that
exception.


# Exception and Flow Commands

Exceptions have similar semantics to those in Python or Java. They can be
thrown with the [fail](/ref/builtin.html#fail) command and caught with either
exception capture `?()` or the `try` special command.

If an external command exits with a non-zero status, Elvish treats that as an
exception.

Flow commands -- `break`, `continue` and `return` -- are ordinary builtin
commands that raise special "flow control" exceptions. The `for` and `while`
commands capture `break` and `continue`, while `fn` modifies its closure to
capture `return`.

One interesting implication is that since flow commands are just ordinary
commands you can build functions on top of them. For instance, this function
`break`s randomly:

```elvish
fn random-break {
  if eq (randint 2) 0 {
    break
  }
}
```

The function `random-break` can then be used in for-loops and while-loops.

Note that the `return` flow control exception is only captured by functions
defined with `fn`. It falls through ordinary lambdas:

```elvish
fn f {
  {
    # returns f, falling through the innermost lambda
    return
  }
}
```


# Namespaces and Modules

Namespace in Elvish helps prevent name collisions and is important for
building modules.

## Syntax

Prepend `namespace:` to command names and variable names to specify the
namespace. The following code

```elvish
e:echo $E:PATH
```

uses the `echo` command from the `e` namespace and the `PATH` variable from
the `E` namespace.

Names of namespaces can contain colons themselves. For instance, in `$x:y:z`
the namespace is `x:y`. More precisely, when parsing command names and
variable names, everything up to the last colon is considered to be the
namespace.

A convention used in articles is when referring to a namespace is to add a
trailing colon: for instance, `edit:` means "the namespace `edit`". This is
remniscent to the syntax but is not syntactically valid.

## Special Namespaces

The following namespaces have special meanings to the language:

*   `local:` and `up:` refer to lexical scopes, and have been documented above.

*   `e:` refers to externals. For instance, `e:ls` refers to the external
    command `ls`.

    Most of the time you can rely on the rules of [command
    resolution](#command-resolution) and do not need to use this explicitly,
    unless a function defined by you (or an Elvish builtin) shadows an
    external command.

*   `E:` refers to environment variables. For instance, `$E:USER` is the
    environment variable `USER`.

    This **is** always needed, because unlike command resolution, variable
    resolution does not fall back onto environment variables.

*   `builtin:` refers to builtin functions and variables.

    You don't need to use this explicitly unless you have defined names that
    shadows builtin counterparts.

## Pre-Imported Modules

Namespaces that are not special (i,e. one of the above) are also called
**modules**. Aside from these special namespaces, Elvish also comes with the
following modules:

*   `edit:` for accessing the Elvish editor. This module is available in
    interactive mode.

    See [reference](/ref/edit.html).

*   `re:` for regular expression facilities.
    This module is always available. See [reference](/ref/re.html).

*   `daemon:` for manipulating the daemon. This module is always available.

    This is not yet documented.

## User-Defined Modules

You can define your own modules with Elvishscript but putting them under
`~/.elvish/lib` and giving them a `.elv` extension. For instance, to define a
module named `a`, store it in `~/.elvish/lib/a.elv`:

```elvish-transcript
~> cat ~/.elvish/lib/a.elv
echo "mod a loading"
fn f {
  echo "f from mod a"
}
```

To import the module, use `use`:

```elvish-transcript
~> use a
mod a loading
~> a:f
f from mod a
```

The argument to `use` is called the **usespec** and will be explained in more
details below. In the simplest case, it is simply the module name.

Modules are evaluated in a seprate scope. That means that functions and
variables defined in the module does not pollute the default namespace, and
vice versa. For instance, if you define `ls` as a wrapper function in
`rc.elv`:

```elvish
fn ls [@a]{
    e:ls --color=auto $@a
}
```

That definition is not visible in module files: `ls` will still refer to the
external command `ls`, unless you shadow it in the very same module.

## Modules in Nested Directories

It is often useful to put modules under directories. When importing such
modules, you can control which (trailing) parts becomes the module name.

For instance, if you have the following content in `x/y/z.elv` (relative to `~/.elvish/lib`, of course):

```elvish
fn f {
    echo 'In a deeply nested module'
}
```

It is possible to import the module as `z`, `y:z`, or `x:y:z`:

```elvish-transcript
~> use x/y/z # imports as "z"
~> z:f
In a deeply nested module
~> use x/y:z # imports as "y:z"
~> y:z:f
In a deeply nested module
~> use x:y:z # imports as "x:y:z"
~> x:y:z:f
In a deeply nested module
```

To be precise:

*   The path of the module to import is derived by replacing all `:` with `/`
    and adding `.elv`. In the above example, all of `x/y/z`, `x/y:z` and
    `x:y:z` have the same path `x/y/z.elv` and refer to the same module

*   The part after the last `/` becomes the module name.

## Lexical Scoping of Imports

Namespace imports are also lexically scoped. For instance, if you `use` a
module within an inner scope, it is not available outside that scope:

```elvish
{
    use some-mod
    some-mod:some-func
}
some-mod:some-func # not valid
```

## Re-Importing

Modules are cached after one import. Subsequent imports do not re-execute the
module; they only serve the bring it into the current scope. Moreover, the
cache is keyed by the path of the module, not the name under which it is
imported. For instance, if you have the following in `~/.elvish/lib/a/b.elv`:

```elvish
echo importing
```

The following code only prints one `importing`:

```elvish
{ use a:b }
use a:b # only brings mod into the lexical scope
```

As does the following:

```elvish
use a/b
use a:b
```