github.com/grafana/pyroscope@v1.18.0/docs/sources/reference-pyroscope-architecture/block-format/_index.md (about)

     1  ---
     2  title: "Pyroscope Block format"
     3  menuTitle: "Block format"
     4  description: "Describe how Pyroscope's stores profiles on the durable object storage"
     5  weight: 50
     6  aliases:
     7    - /docs/phlare/latest/reference-phlare-architecture/block-format/
     8  ---
     9  
    10  # Pyroscope Block format
    11  
    12  This document describes how Pyroscope stores the data in its blocks. Each
    13  block belongs to a single tenant and is identified by a unique [ULID]. Within
    14  the block there are multiple files:
    15  
    16  [ULID]: https://github.com/ulid/spec
    17  
    18  * A metadata file `meta.json`, which contains information about what the block
    19    contains, like the time range of the profiling data.
    20  
    21  * A [TSDB index] `index.tsdb` mapping the external labels to the profiles
    22    stored in the profiles table.
    23  
    24  * `profiles.parquet` [parquet] table that contains profiles.
    25  
    26  * `symbols.symdb` that contains symbolic information for the profiles stored in the block.
    27  
    28  ## Data model
    29  
    30  The data model within the block is fairly aligned to Google's [proto definition][pprof] for the pprof wire format.
    31  
    32  
    33  Profile series labels contain additional information gathered at ingestion time and can
    34  be used to select certain profiles. They are comparable to Prometheus/Loki labels
    35  and typical label names are `namespace` and `pod` to describe which workload the profiles
    36  are coming from.
    37  
    38  Each profile ingested will be added into a new row in the profile table. If
    39  there are entries missing in the tables for the different models they are also
    40  inserted.
    41  
    42  [//source]:<> (https://https://mermaid.live/edit#pako:eNptU11P4zAQ_CuWn4HSlgvUjyicVImTTgTdC-ZhsTeJdYkdOfYJVOW_n_PhJFR9W8_Ozs6OkhMVRiJlVFTQtqmCwkLNNSFSWRROGU1eH_v30CeZs0oXy_sXNE0AyKmHCDmmhBGvtEvueqBbiM9GwKA2MafBY7rmE_KsNLaMvL33xblEgKbpn14P3i6NM3Jpf5xYnwLir7MgMHqKHo_pYOHiHb-tyVU1j2RoFbYvmH_3kUHdVOMhY3mmMoKzyOzk_J4_UPmzg7iOmXO65ZRcX5upivbZnC3XfSChrQfiwhhinrrEhM6oEENiS75cr3IamXq9a5UZ1zGcmRaPj41F4DVLH49a4mdYMIYx4ouZZS9bB8Q1vaI12hqUDJ_tECGnrsQaOWWhlJiDrxynXHeB6hsJDp-kcsZSlkPV4hUF70z2pQVlznqMpOnrn1mVAYlh6ETdV9P_I4VqXZAURueq6HFvqwCXzjUt22z69k2hXOk_boSpN62SJVhX_jskm2SXPMBuj8n9Hn7s91J8bA8P-e5um8v72-0OaNd1_wF4hit7)
    43  ![Data model of Pyroscope blocks](model.svg)
    44  
    45  [ULID]: https://github.com/ulid/spec
    46  [TSDB index]: https://ganeshvernekar.com/blog/prometheus-tsdb-persistent-block-and-its-index/
    47  [Parquet]: https://parquet.apache.org/docs/
    48  [parent pointer tree]: https://en.wikipedia.org/wiki/Parent_pointer_tree
    49  [pprof]: https://github.com/google/pprof/blob/main/proto/profile.proto