github.com/grafana/pyroscope@v1.18.0/.cursor/rules/pyroscope-general.mdc (about)

     1  ---
     2  description: General Pyroscope project context and architecture overview
     3  globs:
     4  alwaysApply: true
     5  ---
     6  
     7  # Pyroscope - Project Overview
     8  
     9  Pyroscope is a horizontally scalable, highly available, multi-tenant continuous profiling aggregation system.
    10  It stores and queries profiling data at scale, similar to how Prometheus works for metrics and Loki for logs.
    11  
    12  ## Key Characteristics
    13  - Written in **Go**
    14  - Microservices-based architecture inspired by Cortex/Mimir/Loki
    15  - Stores profiling data in object storage (S3, GCS, Azure, etc.)
    16  - Multi-tenant by design
    17  
    18  ## Architecture
    19  
    20  Pyroscope uses a **microservices architecture** where a single binary can run different components based on the `-target` parameter.
    21  
    22  ### V1 Components
    23  
    24  **Write Path:**
    25  - **Distributor**: Receives profile ingestion requests, validates, and forwards to ingesters
    26  - **Ingester**: Stores profiles in memory, periodically flushes to disk as blocks, periodically uploads blocks to long-term object storage
    27  - **Compactor**: Merges blocks and removes duplicates
    28  
    29  **Read Path:**
    30  - **Query Frontend**: Entry point for queries, handles query splitting and caching
    31  - **Query Scheduler**: Manages query queue and ensures fair execution across tenants
    32  - **Querier**: Executes queries by fetching data from ingesters and store-gateways
    33  - **Store Gateway**: Indexes and serves blocks from long-term object storage
    34  
    35  ### V2 Components
    36  
    37  **Write Path:**
    38  - **Distributor**: Receives profile ingestion requests, validates, and forwards to segment writers
    39  - **Segment Writer**: Writes block segments to long-term object storage and their metadata to metastore
    40  - **Metastore**: Maintains the block metadata index and coordinates the block compaction process
    41  - **Compaction Worker**: Merges small segments into larger blocks
    42  
    43  **Read Path:**
    44  - **Query Frontend**: Entry point for queries, creates the query plan and executes it against query backends
    45  - **Query Backend**: Executes queries and merges query responses
    46  
    47  ### Storage
    48  - **Block Format**: Profiles stored in Parquet tables, series data in a TSDB index, symbols in a custom format
    49  - **Multi-tenant**: Each tenant has isolated storage
    50  - **Object Storage**: Primary storage backend (S3, GCS, Azure, local filesystem)
    51  
    52  ## Repository Structure
    53  
    54  ```
    55  .
    56  ├── cmd/
    57  │   ├── pyroscope/           # Main server binary
    58  │   └── profilecli/          # CLI tool for profile operations
    59  ├── pkg/                     # Core Go packages
    60  │   ├── distributor/         # Distributor component
    61  │   ├── ingester/            # Ingester component
    62  │   ├── querier/             # Querier component
    63  │   ├── frontend/            # Query frontend component
    64  │   ├── compactor/           # Compactor component
    65  │   ├── metastore/           # Metadata component
    66  │   ├── phlaredb/            # V1 database storage engine
    67  │   ├── model/               # Data models and types
    68  │   ├── objstore/            # Object storage abstraction
    69  │   ├── api/                 # API definitions and handlers
    70  │   └── og/                  # Legacy code (original Pyroscope) - DO NOT USE
    71  ├── public/app/              # React/TypeScript frontend
    72  ├── api/                     # API definitions (protobuf, OpenAPI)
    73  ├── docs/                    # Documentation
    74  ├── operations/              # Deployment configs (jsonnet, helm)
    75  ├── examples/                # Example applications and SDKs
    76  └── tools/                   # Development and build tools
    77  ```
    78  
    79  ## Key Dependencies
    80  
    81  - **dskit**: Grafana's distributed systems toolkit (ring, services, middleware)
    82  - **connect**: RPC framework (gRPC-compatible)
    83  - **parquet-go**: Parquet file format implementation
    84  - **go-kit/log**: Structured logging
    85  - **prometheus/client_golang**: Metrics instrumentation
    86  - **opentracing-go**: Distributed tracing
    87  
    88  ## Critical Rules
    89  
    90  1. **Multi-tenancy is Critical**: Tenant isolation bugs are severe - test thoroughly
    91  2. **Performance Matters**: This system handles high-throughput profiling data
    92  3. **Favor Simplicity**: Pyroscope values simple, maintainable code over clever abstractions
    93  4. **Consistency with Grafana Labs Style**: Follow patterns from dskit, Mimir, Loki
    94  5. **Ask Before Large Refactors**: Propose significant architectural changes before implementing
    95  
    96  ## Things to Avoid
    97  
    98  1. **Don't** introduce dependencies on `pkg/og/` - this is legacy code being phased out
    99  2. **Don't** hardcode tenant IDs - always extract from context
   100  3. **Don't** create unbounded goroutines - use worker pools or semaphores
   101  4. **Don't** log PII or sensitive data
   102  5. **Don't** commit changes to `node_modules/` or generated code without source changes
   103  6. **Don't** modify frontend code (`public/app/`) - the frontend is not actively developed and changes are discouraged