github.com/outbrain/consul@v1.4.5/website/source/docs/commands/snapshot/agent.html.markdown.erb

---
layout: "docs"
page_title: "Commands: Snapshot Agent"
sidebar_current: "docs-commands-snapshot-agent"
---

# Consul Snapshot Agent

Command: `consul snapshot agent`

~> The [`agent`](/docs/commands/snapshot/agent.html) subcommand described here is
   available only in [Consul Enterprise](https://www.hashicorp.com/products/consul/)
   version 0.7.1 and later. All other [snapshot subcommands](/docs/commands/snapshot.html)
   are available in the open source version of Consul.

The `snapshot agent` subcommand starts a process that takes snapshots of the
state of the Consul servers and saves them locally, or pushes them to an
optional remote storage service. This subcommand is only available in Consul
Enterprise 0.7.1 and later.

The agent can be run as a long-running daemon process or in a one-shot mode
from a batch job, based on the [`-interval`](#interval) argument. Snapshotting
a remote datacenter is only available in one-shot mode.

As a long-running daemon, the agent will perform a leader election so multiple
processes can be run in a highly available fashion with automatic failover. The
agent will also register itself with Consul as a service, along with health
checks that show the agent is alive ("Consul Snapshot Agent Alive") and able to
take snapshots ("Consul Snapshot Agent Saving Snapshots"). The latter check is
only added on agents who have become a leader, so it's possible for operators to
tell which instances are alive and on standby and which instance has become
leader and starting saving snapshots.

As snapshots are saved, they will be reported in the log produced by the agent:

```
2016/11/16 21:21:13 [INFO] Snapshot agent running
2016/11/16 21:21:13 [INFO] Waiting to obtain leadership...
2016/11/16 21:21:13 [INFO] Obtained leadership
2016/11/16 21:21:13 [INFO] Saved snapshot 1479360073448728784
```

The number shown with the saved snapshot is its ID, which is based on a UNIX
timestamp with nanosecond resolution, so collisions are unlikely and IDs are
monotonically increasing with time. This makes it easy to locate the latest
snapshot, even if the log data isn't available. The snapshot ID always appears
in the file name when using local storage, or in the object key when using
remote storage.

Snapshots can be restored using the
[`consul snapshot restore`](/docs/commands/snapshot/restore.html) command, or
the [HTTP API](/api/snapshot.html).

If ACLs are enabled, a management token must be supplied in order to perform
snapshot operations.

## Usage

Usage: `consul snapshot agent [options]`

#### API Options

<%= partial "docs/commands/http_api_options_client" %>

#### Config File Options:

* `-config-dir` - Directory to look for JSON config files. Files will be read in
  alphabetical order and must end with the extension ".json". This won't
  recursively descend directories. This can be specified multiple times on the
  command line.

* `-config-file` - File to read JSON configuration from. Files must end with the
  extension ".json". This can be specified multiple times on the command line.

  Config files referenced using `-config-dir` and `-config-file` have the following
  format (shown populated with default values):

```javascript
  {
    "snapshot_agent": {
      "http_addr": "127.0.0.1:8500",
      "token": "",
      "datacenter": "",
      "ca_file": "",
      "ca_path": "",
      "cert_file": "",
      "key_file": "",
      "tls_server_name": "",
      "log": {
        "level": "INFO",
        "enable_syslog": false,
        "syslog_facility": "LOCAL0"
      },
      "snapshot": {
        "interval": "1h",
        "retain": 30,
        "stale": false,
        "service": "consul-snapshot",
        "deregister_after": "72h",
        "lock_key": "consul-snapshot/lock",
        "max_failures": 3
      },
      "local_storage": {
        "path": "."
      },
      "aws_storage": {
        "access_key_id": "",
        "secret_access_key": "",
        "s3_region": "",
        "s3_bucket": "",
        "s3_key_prefix": "consul-snapshot",
        "s3_server_side_encryption":false,
        "s3_static_snapshot_name":""
      }
    }
  }
```

All fields are optional, and config files without a `snapshot_agent` object will
be ignored. At least one config file needs to have a `snapshot_agent` object, or the
snapshot agent will fail to start. The Consul agent is set up to ignore any
`snapshot_agent` object, so it's safe to use common config directories for both agents
if desired.

#### Snapshot Options

* <a name="interval">`-interval`</a> - Interval at which to perform snapshots
  as a time with a unit suffix, which can be "s", "m", "h" for seconds, minutes,
  or hours. If 0 is provided, the agent will take a single snapshot and then exit,
  which is useful for running snapshots via batch jobs. Defaults to "1h"

* `-lock-key` - A prefix in Consul's KV store used to coordinate between
  different instances of the snapshot agent order to only have one active instance
  at a time. For highly available operation of the snapshot agent, simply run
  multiple instances. All instances must be configured with the same lock key in
  order to properly coordinate. Defaults to "consul-snapshot/lock".

* `-max-failures` - Number of snapshot failures after which the snapshot agent
  will give up leadership. In a highly available operation with multiple snapshot
  agents available, this gives another agent a chance to take over if an agent
  is experiencing issues, such as running out of disk space for snapshots.
  Defaults to 3.

* `-retain` - Number of snapshots to retain. After each snapshot is taken, the
  oldest snapshots will start to be deleted in order to retain at most this many
  snapshots. If this is set to 0, the agent will not perform this and snapshots
  will accumulate forever. Defaults to 30.


#### Agent Options

* `-deregister-after` - An interval, after which if the agent is unhealthy it will be
  automatically deregistered from Consul service discovery. This is a time with a
  unit suffix, which can be "s", "m", "h" for seconds, minutes, or hours. If 0 is
  provided, this will be disabled. Defaults to "72h".

* `-log-level` - Controls verbosity of snapshot agent logs. Valid options are
  "TRACE", "DEBUG", "INFO", "WARN", "ERR". Defaults to "INFO".

* `-service` - The service name to used when registering the agent with Consul.
  Registering helps monitor running agents and the leader registers an additional
  health check to monitor that snapshots are taking place. Defaults to
  "consul-snapshot".

* `-syslog` - This enables forwarding logs to syslog. Defaults to false.

* `-syslog-facility` - Sets the facility to use for forwarding logs to syslog.
  Defaults to "LOCAL0".

#### Local Storage Options

* `-local-path` - Location to store snapshots locally. The default behavior
  of the snapshot agent is to store snapshots locally in this directory. Defaults
  to "." to use the current working directory. If an alternate storage option is
  configured, then local storage will be disabled and this option will be ignored.

#### S3 Storage Options
Note that despite the AWS references, any S3-compatible endpoint can be specified with `-aws-s3-endpoint`.

* `-aws-access-key-id` and `-aws-secret-access-key` - These arguments supply
  authentication information for connecting to S3. These may also be supplied using
  the following alternative methods:<br>
  - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables
  - A credentials file (`~/.aws/credentials` or the file at the path specified by the
    `AWS_SHARED_CREDENTIALS_FILE` environment variable)
  - ECS task role metadata (container-specific)
  - EC2 instance role metadata

* `-aws-s3-bucket` - S3 bucket to use. Required for S3 storage, and setting this
  disables local storage. This should be only the bucket name without any
  part of the key prefix.

* `-aws-s3-key-prefix` - Prefix to use for snapshot files in S3. Defaults to
   "consul-snapshot".

* `-aws-s3-region` - S3 region to use. Required for S3 storage.

* `-aws-s3-endpoint` - Optional S3 endpoint to use. Can also be specified using the
  AWS_S3_ENDPOINT environment variable.

* `-aws-s3-server-side-encryption` - Enables saving snapshots to S3 using server side encryption with [Amazon S3-Managed Encryption Keys](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html)

* `-aws-s3-static-snapshot-name` - If this is given, all snapshots are saved with the same file name. The agent will not rotate or versionize snapshots, and will save them with the same name each time.
                                   Use this if you want to rely on [S3's versioning capabilities](http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) instead of the agent handling it for you.

* `-aws-s3-enable-kms` - Enables using [Amazon KMS](https://aws.amazon.com/kms/) for encrypting snapshots.

* `-aws-s3-kms-key` -  Optional Amazon KMS key to use, if this is not set the default KMS master key will be used. Set this if you want to manage key rotation yourself.

#### S3 Required Permissions

Different S3 permissions are required depending on the configuration of the snapshot agent. In particular extra permissions are required when
snapshot rotation is enabled. S3 storage snapshot rotation is enabled when the `retain` configuration is greater than 0 and when there is
no `aws-s3-static-snapshot-name` configured.

| Permission           | Resource                           | When you need it                                |
| -------------------- | ---------------------------------- | ----------------------------------------------- |
| `PutObject`          | `arn:aws:s3:::<bucket name>/<key>` | Required for all operations.                    |
| `DeleteObject`       | `arn:aws:s3:::<bucket name>/<key>` | Required only when snapshot rotation is enabled |
| `ListBucket`         | `arn:aws:s3:::<bucket name>`       | Required only when snapshot rotation is enabled |
| `ListBucketVersions` | `arn:aws:s3:::<bucket name>`       | Required only when snapshot rotation is enabled |

Within the table `<key>` refers to the the key used to store the snapshot. When `aws-s3-static-snapshot-name` is configured the `<key>` is simply the value of that configuration. Otherwise the `<key>` will be the `<aws-s3-key-prefix configuration>/consul-*.snap`.

The following example IAM policy document assumes that the `aws-s3-bucket` is `consul-data` with defaults for `aws-s3-key-prefix`, `aws-s3-static-snapshot-name` and `retain`:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::consul-data/consul-snapshots/consul-*.snap"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucketVersions",
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::consul-data"
    }
  ]
}
```

## Examples

Running the agent with no arguments will run a long-running daemon process that will
perform leader election for highly available operation, register itself with Consul
service discovery with health checks, take snapshots every hour, retain the last 30
snapshots, and save snapshots into the current working directory:

```
$ consul snapshot agent
```

To run a one-shot backup, set the backup interval to 0. This will run a single snapshot
and delete any old snapshots based on the retain settings, but it will not perform any
leader election or service registration:

```
$ consul snapshot agent -interval=0
```

Please see the [HTTP API](/api/snapshot.html) documentation for
more details about snapshot internals.