github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.10.x/deploy-manage/manage/s3gateway/index.md (about)

     1  # Overview
     2  
     3  Pachyderm includes an S3 gateway that enables you to interact with PFS storage
     4  through an HTTP application programming interface (API) that imitates the
     5  Amazon S3 Storage API. Therefore, with Pachyderm S3 gateway, you can interact
     6  with Pachyderm through tools and libraries designed to work with object stores.
     7  For example, you can use these tools:
     8  
     9  * [MinIO](https://docs.min.io/docs/minio-client-complete-guide)
    10  * [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
    11  
    12  When you deploy `pachd`, the S3 gateway starts automatically.
    13  
    14  The S3 gateway has some limitations that are outlined below. If you need richer
    15  access, use the PFS gRPC interface instead, or one of the
    16  [client drivers](https://github.com/pachyderm/python-pachyderm).
    17  
    18  ## Authentication
    19  
    20  If auth is enabled on the Pachyderm cluster, credentials must be passed with
    21  each S3 gateway endpoint using AWS' signature v2 or v4 methods. Object store
    22  tools and libraries provide built-in support for these methods, but they do
    23  not work in the browser. When you use authentication, set the access and
    24  secret key to the same value. They are both the Pachyderm auth token used
    25  to issue the relevant PFS calls.
    26  
    27  If auth is not enabled on the Pachyderm cluster, access credentials are
    28  ignored. You can not pass an authorization header, or if you do, any
    29  values for the access and secret keys are ignored.
    30  
    31  ## Buckets
    32  
    33  The S3 gateway presents each branch from every Pachyderm repository as
    34  an S3 bucket.
    35  For example, if you have a `master` branch in the `images` repository,
    36  an S3 tool sees `images@master` as the `master.images` S3 bucket.
    37  
    38  ## Versioning
    39  
    40  Most operations act on the `HEAD` of the given branch. However, if your object
    41  store library or tool supports versioning, you can get objects in non-`HEAD`
    42  commits by using the commit ID as the S3 object version ID.
    43  
    44  ## Port Forwarding
    45  
    46  If you do not have direct access to the Kubernetes cluster, you can use port
    47  forwarding instead. Simply run `pachctl port-forward`, which will allow you
    48  to access the s3 gateway through `localhost:30600`.
    49  
    50  However, the Kubernetes port forwarder incurs substantial overhead and
    51  does not recover well from broken connections. Therefore, connecting to the
    52  cluster directly is faster and more reliable.