github.com/grafana/pyroscope@v1.18.0/docs/sources/configure-server/storage/configure-object-storage-backend.md (about) 1 --- 2 title: "Configure object storage backend" 3 menuTitle: "Configure object storage" 4 description: "Learn how to configure Pyroscope to use different object storage backend implementations." 5 aliases: 6 - /docs/phlare/latest/configure-server/configure-object-storage-backend/ 7 - ../configure-object-storage-backend/ # https://grafana.com/docs/pyroscope/latest/configure-server/configure-object-storage-backend/ 8 --- 9 10 # Configure object storage backend 11 12 Grafana Pyroscope can use different object storage services to persist blocks containing the profiles data. 13 Blocks are flushed by ingesters [on disk](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-server/storage/configure-disk-storage/) first then are uploaded to the object store. 14 15 The supported backends are: 16 17 - [Amazon S3](https://aws.amazon.com/s3/) and compatible implementations like [MinIO](https://min.io/) 18 - [Google Cloud Storage](https://cloud.google.com/storage) 19 - [Azure Blob Storage](https://azure.microsoft.com/es-es/services/storage/blobs/) 20 - [Swift (OpenStack Object Storage)](https://wiki.openstack.org/wiki/Swift) 21 22 > Internally, Pyroscope uses [Thanos' object store client] library, so their stated limitations apply. 23 24 [Thanos' object store client]: https://github.com/thanos-io/objstore#supported-providers-clients 25 26 ## Amazon S3 27 28 To use an AWS S3 or S3-compatible bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][aws_ref]. Apart from those parameters, it is also possible to supply configuration parameters using [the well-known environment variables][aws_enf] of the AWS SDK. 29 30 At a minimum, you will need to provide values for the `bucket_name`, `endpoint`, `access_key_id`, and `secret_access_key` keys. 31 32 [aws_ref]: ../../reference-configuration-parameters/#s3_storage_backend 33 [aws_enf]: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html 34 35 ### Example using an AWS Bucket 36 37 This how one would configure a bucket in the AWS region `eu-west-2`: 38 39 ```yaml 40 storage: 41 backend: s3 42 s3: 43 bucket_name: #REPLACE_WITH_BUCKET_NAME 44 region: eu-west-2 45 endpoint: s3.eu-west-2.amazonaws.com 46 access_key_id: #REPLACE_WITH_ACCESS_KEY 47 secret_access_key: #REPLACE_WITH_SECRET_KEY 48 ``` 49 50 ### Example using a S3 compatible Bucket 51 52 This how one would configure a bucket on a locally running instance of [MinIO]: 53 54 ```yaml 55 storage: 56 backend: s3 57 s3: 58 bucket_name: grafana-pyroscope-data 59 endpoint: localhost:9000 60 insecure: true 61 access_key_id: grafana-pyroscope-data 62 secret_access_key: grafana-pyroscope-data 63 ``` 64 65 [MinIO]: https://min.io/docs/minio/container/index.html 66 67 ## Google Cloud Storage 68 69 To use a Google Cloud Storage (GCS) bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][gcs_ref]. 70 71 [gcs_ref]: ../../reference-configuration-parameters/#gcs_storage_backend 72 73 At a minimum, you will need to provide a values for the `bucket_name` and a service account. To supply the service account there are two ways: 74 75 * Use the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to locate your [application credentials](https://cloud.google.com/docs/authentication/production). 76 * Provide the content the service account key within the `service_account` parameter. 77 78 ### Example using a Google Cloud Storage bucket 79 80 This how one would configure a GCS bucket using the `service_account` parameter: 81 82 ```yaml 83 storage: 84 backend: gcs 85 gcs: 86 bucket_name: grafana-pyroscope-data 87 service_account: | 88 { 89 "type": "service_account", 90 "project_id": "PROJECT_ID", 91 "private_key_id": "KEY_ID", 92 "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n", 93 "client_email": "SERVICE_ACCOUNT_EMAIL", 94 "client_id": "CLIENT_ID", 95 "auth_uri": "https://accounts.google.com/o/oauth2/auth", 96 "token_uri": "https://accounts.google.com/o/oauth2/token", 97 "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", 98 "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL" 99 } 100 ``` 101 102 ## Azure Blob Storage 103 104 To use an Azure Blob Storage bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][azure_ref]. 105 106 [azure_ref]: ../../reference-configuration-parameters/#azure_storage_backend 107 108 If `user_assigned_id` is used, authentication is done via user-assigned managed identity. 109 110 [//TODO]: <> (Provide example with and without user-assigned managed identity) 111 112 ## Swift (OpenStack Object Storage) 113 114 To use a Swift (OpenStack Object Storage) bucket for long term storage, you can find Pyroscope's configuration parameters [in the reference config][swift_ref]. 115 116 [swift_ref]: ../../reference-configuration-parameters/#swift_storage_backend 117 118 >If the `name` of a user, project or tenant is used one must also specify its domain by ID or name. Various examples for OpenStack authentication can be found in the [official documentation](https://developer.openstack.org/api-ref/identity/v3/index.html?expanded=password-authentication-with-scoped-authorization-detail#password-authentication-with-unscoped-authorization). 119 120 [//TODO]: <> (Provide example)