github.com/grafana/pyroscope@v1.18.0/docs/sources/configure-server/storage/configure-disk-storage.md (about)

     1  ---
     2  title: "Configure disk storage"
     3  menuTitle: "Configure disk storage"
     4  description: "Learn about the ingester component and how to configure disk storage for Pyroscope."
     5  aliases:
     6    - /docs/phlare/latest/configure-server/configure-disk-storage/
     7    - ../configure-disk-storage/ # https://grafana.com/docs/pyroscope/latest/configure-server/configure-disk-storage/
     8  ---
     9  
    10  # Configure disk storage
    11  
    12  The [ingester] component in Grafana Pyroscope processes the received profiling data.
    13  First it keeps the data organized in memory, in the so-called "head block".
    14  Once the size of the head block exceeds a threshold or the head block is older than
    15  `-pyroscopedb.max-block-duration` (by default 1 hour), the ingester writes
    16  the block to the local persistent disk.
    17  Refer to [block format] for more detail about the block's layout.
    18  
    19  Each of those blocks are identified by an [ULID] and stored
    20  within Pyroscope's data path `-pyroscopedb.data-path=` (by default
    21  `./data`).
    22  This directory is organized by the following:
    23  
    24  * `./<tenant-id>`: Each tenant has its own subdirectory with the following subdirectories:
    25     * `head/<block-id>`: Contains the current data still being written.
    26     * `local/<block-id>`: Contains the finished blocks, which are kept locally
    27  
    28  ## Object storage
    29  
    30  When [object storage is configured][object-store], finished blocks are
    31  uploaded to the object store bucket.
    32  
    33  ## High disk utilization
    34  
    35  To avoid losing the most recent data, Pyroscope removes the oldest blocks
    36  when it detects that the volume on which the data path is located is close to
    37  running out of disk space.
    38  The check for high disk utilization runs every
    39  `-pyroscopedb.retention-policy-enforcement-interval` when:
    40  
    41  * less than `-pyroscopedb.retention-policy-min-disk-available-percentage=0.05` of the total size of the volume is available and
    42  * the available disk space smaller then `-pyroscopedb.retention-policy-min-free-disk-gb=10`.
    43  
    44  The deletion is logged like this:
    45  
    46  ```
    47  level=warn caller=pyroscopedb.go:231 ts=2022-10-05T13:19:09.770693308Z msg="disk utilization is high, deleted oldest block" path=data/anonymous/local/01GDZYHKKKY2ANY6PCJJZGT1N8
    48  ```
    49  
    50  [block format]: ../../../reference-pyroscope-architecture/block-format/
    51  [object-store]: ../configure-object-storage-backend/
    52  [ULID]: https://github.com/ulid/spec
    53  [ingester]: ../../../reference-pyroscope-architecture/components/ingester/