github.com/grafana/pyroscope@v1.18.0/docs/sources/configure-server/storage/configure-object-storage-backend.md (about)

     1  ---
     2  title: "Configure object storage backend"
     3  menuTitle: "Configure object storage"
     4  description: "Learn how to configure Pyroscope to use different object storage backend implementations."
     5  aliases:
     6    - /docs/phlare/latest/configure-server/configure-object-storage-backend/
     7    - ../configure-object-storage-backend/ # https://grafana.com/docs/pyroscope/latest/configure-server/configure-object-storage-backend/
     8  ---
     9  
    10  # Configure object storage backend
    11  
    12  Grafana Pyroscope can use different object storage services to persist blocks containing the profiles data.
    13  Blocks are flushed by ingesters [on disk](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-server/storage/configure-disk-storage/) first then are uploaded to the object store.
    14  
    15  The supported backends are:
    16  
    17  - [Amazon S3](https://aws.amazon.com/s3/) and compatible implementations like [MinIO](https://min.io/)
    18  - [Google Cloud Storage](https://cloud.google.com/storage)
    19  - [Azure Blob Storage](https://azure.microsoft.com/es-es/services/storage/blobs/)
    20  - [Swift (OpenStack Object Storage)](https://wiki.openstack.org/wiki/Swift)
    21  
    22  > Internally, Pyroscope uses [Thanos' object store client] library, so their stated limitations apply.
    23  
    24  [Thanos' object store client]: https://github.com/thanos-io/objstore#supported-providers-clients
    25  
    26  ## Amazon S3
    27  
    28  To use an AWS S3 or S3-compatible bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][aws_ref]. Apart from those parameters, it is also possible to supply configuration  parameters using [the well-known environment variables][aws_enf] of the AWS SDK.
    29  
    30  At a minimum, you will need to provide values for the `bucket_name`, `endpoint`, `access_key_id`, and `secret_access_key` keys.
    31  
    32  [aws_ref]: ../../reference-configuration-parameters/#s3_storage_backend
    33  [aws_enf]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
    34  
    35  ### Example using an AWS Bucket
    36  
    37  This how one would configure a bucket in the AWS region `eu-west-2`:
    38  
    39  ```yaml
    40  storage:
    41    backend: s3
    42    s3:
    43      bucket_name: #REPLACE_WITH_BUCKET_NAME
    44      region: eu-west-2
    45      endpoint: s3.eu-west-2.amazonaws.com
    46      access_key_id: #REPLACE_WITH_ACCESS_KEY
    47      secret_access_key: #REPLACE_WITH_SECRET_KEY
    48  ```
    49  
    50  ### Example using a S3 compatible Bucket
    51  
    52  This how one would configure a bucket on a locally running instance of [MinIO]:
    53  
    54  ```yaml
    55  storage:
    56    backend: s3
    57    s3:
    58      bucket_name: grafana-pyroscope-data
    59      endpoint: localhost:9000
    60      insecure: true
    61      access_key_id: grafana-pyroscope-data
    62      secret_access_key: grafana-pyroscope-data
    63  ```
    64  
    65  [MinIO]: https://min.io/docs/minio/container/index.html
    66  
    67  ## Google Cloud Storage
    68  
    69  To use a Google Cloud Storage (GCS) bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][gcs_ref].
    70  
    71  [gcs_ref]: ../../reference-configuration-parameters/#gcs_storage_backend
    72  
    73  At a minimum, you will need to provide a values for the `bucket_name` and a service account. To supply the service account there are two ways:
    74  
    75  * Use the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to locate your [application credentials](https://cloud.google.com/docs/authentication/production).
    76  * Provide the content the service account key within the `service_account` parameter.
    77  
    78  ### Example using a Google Cloud Storage bucket
    79  
    80  This how one would configure a GCS bucket using the `service_account` parameter:
    81  
    82  ```yaml
    83  storage:
    84    backend: gcs
    85    gcs:
    86      bucket_name: grafana-pyroscope-data
    87      service_account: |
    88        {
    89          "type": "service_account",
    90          "project_id": "PROJECT_ID",
    91          "private_key_id": "KEY_ID",
    92          "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n",
    93          "client_email": "SERVICE_ACCOUNT_EMAIL",
    94          "client_id": "CLIENT_ID",
    95          "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    96          "token_uri": "https://accounts.google.com/o/oauth2/token",
    97          "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    98          "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL"
    99        }
   100  ```
   101  
   102  ## Azure Blob Storage
   103  
   104  To use an Azure Blob Storage bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][azure_ref].
   105  
   106  [azure_ref]: ../../reference-configuration-parameters/#azure_storage_backend
   107  
   108  If `user_assigned_id` is used, authentication is done via user-assigned managed identity.
   109  
   110  [//TODO]: <> (Provide example with and without user-assigned managed identity)
   111  
   112  ## Swift (OpenStack Object Storage)
   113  
   114  To use a Swift (OpenStack Object Storage) bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][swift_ref].
   115  
   116  [swift_ref]: ../../reference-configuration-parameters/#swift_storage_backend
   117  
   118  >If the `name` of a user, project or tenant is used one must also specify its domain by ID or name. Various examples for OpenStack authentication can be found in the [official documentation](https://developer.openstack.org/api-ref/identity/v3/index.html?expanded=password-authentication-with-scoped-authorization-detail#password-authentication-with-unscoped-authorization).
   119  
   120  [//TODO]: <> (Provide example)