github.com/pachyderm/pachyderm@v1.13.4/doc/docs/master/deploy-manage/manage/s3gateway/index.md (about)

     1  # Overview
     2  
     3  Pachyderm includes an S3 gateway that enables you to interact with PFS storage
     4  through an HTTP application programming interface (API) that imitates the
     5  Amazon S3 Storage API. Therefore, with Pachyderm S3 gateway, you can interact
     6  with Pachyderm through tools and libraries designed to work with object stores.
     7  For example, you can use these tools:
     8  
     9  * [MinIO](https://docs.min.io/docs/minio-client-complete-guide)
    10  * [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
    11  
    12  When you deploy `pachd`, the S3 gateway starts automatically.
    13  
    14  The S3 gateway has some limitations that are outlined below. If you need richer
    15  access, use the PFS gRPC interface instead, or one of the
    16  [client drivers](https://github.com/pachyderm/python-pachyderm).
    17  
    18  ## Authentication
    19  
    20  If auth is enabled on the Pachyderm cluster, credentials must be passed with
    21  each S3 gateway endpoint using AWS' signature v2 or v4 methods. Object store
    22  tools and libraries provide built-in support for these methods, but they do
    23  not work in the browser. When you use authentication, set the access and
    24  secret key to the same value. They are both the Pachyderm auth token used
    25  to issue the relevant PFS calls.
    26  
    27  If auth is disabled, you can still pass arbitrary credentials, but the
    28  secret key must match the access key.
    29  
    30  ## Buckets
    31  
    32  The S3 gateway presents each branch from every Pachyderm repository as
    33  an S3 bucket.
    34  For example, if you have a `master` branch in the `images` repository,
    35  an S3 tool sees `images@master` as the `master.images` S3 bucket.
    36  
    37  ## Versioning
    38  
    39  Most operations act on the `HEAD` of the given branch. However, if your object
    40  store library or tool supports versioning, you can get objects in non-`HEAD`
    41  commits by using the commit ID as the S3 object version ID.
    42  
    43  ## Port Forwarding
    44  
    45  If you do not have direct access to the Kubernetes cluster, you can use port
    46  forwarding instead. Simply run `pachctl port-forward`, which will allow you
    47  to access the s3 gateway through `localhost:30600`.
    48  
    49  However, the Kubernetes port forwarder incurs substantial overhead and
    50  does not recover well from broken connections. Therefore, connecting to the
    51  cluster directly is faster and more reliable.