github.com/grafana/pyroscope@v1.18.0/docs/sources/configure-server/configuring-memberlist.md (about)

     1  ---
     2  description: Learn how to configure Pyroscope memberlist.
     3  menuTitle: Configure memberlist
     4  title: Configure Pyroscope memberlist
     5  weight: 500
     6  aliases:
     7    - /docs/phlare/latest/operators-guide/configuring/configuring-memberlist/
     8    - /docs/phlare/latest/configure-server/configuring-memberlist/
     9  ---
    10  
    11  # Configure Pyroscope memberlist
    12  
    13  [Hash rings](../../reference-pyroscope-architecture/hash-ring/) are a distributed consistent hashing scheme and are widely used by Pyroscope for sharding and replication.
    14  Pyroscope only supports hash ring via the memberlist protocol.
    15  You can configure memberlist by either the CLI flag or its respective YAML [config option](../reference-configuration-parameters/#memberlist).
    16  
    17  ## Memberlist
    18  
    19  Pyroscope uses `memberlist` as the KV store backend.
    20  At startup, a Pyroscope instance connects to other Pyroscope replicas to join the cluster.
    21  A Pyroscope instance discovers the other replicas to join by resolving the addresses configured in `-memberlist.join`.
    22  The `-memberlist.join` CLI flag must resolve to other replicas in the cluster and can be specified multiple times.
    23  
    24  The `-memberlist.join` can be set to an address in the following formats:
    25  
    26  - `<ip>:<port>`
    27  - `<hostname>:<port>`
    28  - [DNS service discovery](#supported-discovery-modes)
    29  
    30  > **Note**: At a minimum, configure one or more addresses that resolve to a consistent subset of replicas (for example, all the ingesters).
    31  
    32  > **Note**: If you're running Pyroscope in Kubernetes, define a [headless Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/#headless-services) which resolves to the IP addresses of all Pyroscope pods, then set `-memberlist.join` to `dnssrv+<service name>.<namespace>.svc.cluster.local:<port>`.
    33  
    34  Pyroscope supports TLS for memberlist connections between its components.
    35  To see all supported configuration parameters, refer to [memberlist](../reference-configuration-parameters/#memberlist).
    36  
    37  #### Configuring the memberlist address and port
    38  
    39  By default, Pyroscope memberlist protocol listens on address `0.0.0.0` and port `7946`.
    40  If you run multiple Pyroscope processes on the same node or the port `7946` is not available, you can change the bind and advertise port by setting the following parameters:
    41  
    42  - `-memberlist.bind-addr`: IP address to listen on the local machine.
    43  - `-memberlist.bind-port`: Port to listen on the local machine.
    44  - `-memberlist.advertise-addr`: IP address to advertise to other Pyroscope replicas. The other replicas will connect to this IP to talk to the instance.
    45  - `-memberlist.advertise-port`: Port to advertise to other Pyroscope replicas. The other replicas will connect to this port to talk to the instance.
    46  
    47  ### Fine tuning memberlist changes propagation latency
    48  
    49  The `pyroscope_ring_oldest_member_timestamp` metric can be used to measure the propagation of hash ring changes.
    50  This metric tracks the oldest heartbeat timestamp across all instances in the ring.
    51  You can execute the following query to measure the age of the oldest heartbeat timestamp in the ring:
    52  
    53  ```promql
    54  max(time() - pyroscope_ring_oldest_member_timestamp{state="ACTIVE"})
    55  ```
    56  
    57  The measured age shouldn't be higher than the configured `<prefix>.heartbeat-period` plus a reasonable delta (for example, 15 seconds).
    58  If you experience a higher changes propagation latency, you can adjust the following settings:
    59  
    60  - Decrease `-memberlist.gossip-interval`
    61  - Increase `-memberlist.gossip-nodes`
    62  - Decrease `-memberlist.pullpush-interval`
    63  - Increase `-memberlist.retransmit-factor`
    64  
    65  ## About Pyroscope DNS service discovery
    66  
    67  Some clients in Pyroscope support service discovery via DNS to locate the addresses of backend servers to connect to. The following clients support service discovery via DNS:
    68  
    69  - [Memberlist KV store](../reference-configuration-parameters/#memberlist)
    70    - `-memberlist.join`
    71  
    72  ## Supported discovery modes
    73  
    74  DNS service discovery supports different discovery modes.
    75  You select a discovery mode by adding one of the following supported prefixes to the address:
    76  
    77  - **`dns+`**<br />
    78    The domain name after the prefix is looked up as an A/AAAA query. For example: `dns+memcached.local:11211`.
    79  - **`dnssrv+`**<br />
    80    The domain name after the prefix is looked up as a SRV query, and then each SRV record is resolved as an A/AAAA record. For example: `dnssrv+_memcached._tcp.memcached.namespace.svc.cluster.local`.
    81  - **`dnssrvnoa+`**<br />
    82    The domain name after the prefix is looked up as a SRV query, with no A/AAAA lookup made after that. For example: `dnssrvnoa+_memcached._tcp.memcached.namespace.svc.cluster.local`.
    83  
    84  If no prefix is provided, the provided IP or hostname is used without pre-resolving it.