github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/how-tos/export-data-out-pachyderm/export-data-egress.md (about)

     1  # Export Your Data with `egress`
     2  
     3  The `egress` field in the Pachyderm [pipeline specification](../../../reference/pipeline_spec)
     4  enables you to push the results of a pipeline to an
     5  external datastore such as Amazon S3, Google Cloud Storage, or
     6  Azure Blob Storage. After the user code has finished running, but
     7  before the job is marked as successful, Pachyderm pushes the data
     8  to the specified destination.
     9  
    10  You can specify the following `egress` protocols for the
    11  corresponding storage:
    12  
    13  | Cloud Platform | Protocol | Description |
    14  | -------------- | -------- | ----------- |
    15  | Google Cloud <br>Storage | `gs://` | GCP uses the utility called `gsutil` to access GCP storage resources <br> from a CLI. This utility uses the `gs://` prefix to access these resources. <br>**Example:**<br> `gs://gs-bucket/gs-dir` |
    16  | Amazon S3 | `s3://` | The Amazon S3 storage protocol requires you to specify an `s3://`<br>prefix before the address of an Amazon resource. A valid address must <br>include an endpoint and a bucket, and, optionally, a directory in your <br>Amazon storage. <br>**Example:**<br> `s3://s3-endpoint/s3-bucket/s3-dir` |
    17  | Azure Blob <br>Storage | `wasb://` | Microsoft Windows Azure Storage Blob (WASB) is the default Azure <br>filesystem that outputs your data through `HDInsight`. To output your <br>data to Azure Blob Storage, use the ``wasb://`` prefix, the container name, <br>and your storage account in the path to your directory. <br>**Example:**<br>`wasb://default-container@storage-account/az-dir` |
    18  
    19  !!! example
    20      ```json
    21      "egress": {
    22         "URL": "s3://bucket/dir"
    23      },
    24      ```