github.com/MetalBlockchain/metalgo@v1.11.9/network/README.md (about)

     1  # Avalanche Networking
     2  
     3  ## Table of Contents
     4  
     5  - [Overview](#overview)
     6  - [Peers](#peers)
     7    - [Message Handling](#message-handling)
     8    - [Peer Handshake](#peer-handshake)
     9    - [Ping-Pong Messages](#ping-pong-messages)
    10  - [Peer Discovery](#peer-discovery)
    11      - [Inbound Connections](#inbound-connections)
    12      - [Outbound Connections](#outbound-connections)
    13        - [IP Authentication](#ip-authentication)
    14      - [Bootstrapping](#bootstrapping)
    15      - [PeerList Gossip](#peerlist-gossip)
    16        - [Bloom Filter](#bloom-filter)
    17        - [GetPeerList](#getpeerlist)
    18        - [PeerList](#peerlist)
    19  
    20  ## Overview
    21  
    22  Avalanche is a decentralized [p2p](https://en.wikipedia.org/wiki/Peer-to-peer) (peer-to-peer) network of nodes that work together to run the Avalanche blockchain protocol.
    23  
    24  The `network` package implements the networking layer of the protocol which allows a node to discover, connect to, and communicate with other peers.
    25  
    26  All connections are authenticated using [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security). However, there is no reliance on any certificate authorities. The `network` package identifies peers by the public key in the leaf certificate.
    27  
    28  ## Peers
    29  
    30  Peers are defined as members of the network that communicate with one another to participate in the Avalanche protocol.
    31  
    32  Peers communicate by enqueuing messages between one another. Each peer on either side of the connection asynchronously reads and writes messages to and from the remote peer. Messages include both application-level messages used to support the Avalanche protocol, as well as networking-level messages used to implement the peer-to-peer communication layer.
    33  
    34  ```mermaid
    35  sequenceDiagram
    36      actor Alice
    37      actor Bob
    38      loop 
    39          Alice->>Bob: Write outbound messages
    40          Bob->>Alice: Read incoming messages
    41      end
    42      loop
    43          Bob->>Alice: Write outbound messages
    44          Alice->>Bob: Read incoming messages
    45      end
    46  ```
    47  
    48  ### Message Handling
    49  
    50  All messages are prefixed with their length. Reading a message first reads the 4-byte message length from the connection. The rate-limiting logic then waits until there is sufficient capacity to read these bytes from the connection.
    51  
    52  A peer will then read the full message and attempt to parse it into either a networking message or an application message. If the message is malformed the connection is not dropped. The peer will simply continue to the next sent message.
    53  
    54  ### Peer Handshake
    55  
    56  Upon connection to a new peer, a handshake is performed between the node attempting to establish the outbound connection to the peer and the peer receiving the inbound connection.
    57  
    58  When attempting to establish the connection, the first message that the node sends is a `Handshake` message describing the compatibility of the nodes. If the `Handshake` message is successfully received and the peer decides that it wants a connection with this node, it replies with a `PeerList` message that contains metadata about other peers that allows a node to connect to them. See [Peerlist Gossip](#peerlist-gossip).
    59  
    60  As an example, nodes that are attempting to connect with an incompatible version of AvalancheGo or a significantly skewed local clock are rejected.
    61  
    62  ```mermaid
    63  sequenceDiagram
    64      actor Alice
    65      actor Bob
    66      Note over Alice,Bob: Connection Created
    67      par
    68          Alice->>Bob: AvalancheGo v1.0.0
    69      and
    70          Bob->>Alice: AvalancheGo v1.11.4
    71      end
    72      Note right of Bob: v1.0.0 is incompatible with v1.11.4.
    73      Note left of Alice: v1.11.4 could be compatible with v1.0.0!
    74      par
    75          Bob-->>Alice: Disconnect
    76      and
    77          Alice-XBob: Peerlist
    78      end
    79      Note over Alice,Bob: Handshake Failed
    80  ```
    81  
    82  Nodes that mutually desire the connection will both respond with `PeerList` messages and complete the handshake.
    83  
    84  ```mermaid
    85  sequenceDiagram
    86      actor Alice
    87      actor Bob
    88      Note over Alice,Bob: Connection Created
    89      par
    90          Alice->>Bob: AvalancheGo v1.11.0
    91      and
    92          Bob->>Alice: AvalancheGo v1.11.4
    93      end
    94      Note right of Bob: v1.11.0 is compatible with v1.11.4!
    95      Note left of Alice: v1.11.4 could be compatible with v1.11.0!
    96      par
    97          Bob->>Alice: Peerlist
    98      and
    99          Alice->>Bob: Peerlist
   100      end
   101      Note over Alice,Bob: Handshake Complete
   102  ```
   103  
   104  ### Ping-Pong Messages
   105  
   106  Peers periodically send `Ping` messages containing perceived uptime information. This information can be used to monitor how the node is considered to be performing by the network. It is expected for a node to reply to a `Ping` message with a `Pong` message.
   107  
   108  ```mermaid
   109  sequenceDiagram
   110      actor Alice
   111      actor Bob
   112      Note left of Alice: Send Ping
   113      Alice->>Bob: I think your uptime is 95%
   114      Note right of Bob: Send Pong
   115      Bob->>Alice: ACK
   116  ```
   117  
   118  ## Peer Discovery
   119  
   120  When starting an Avalanche node, a node needs to be able to initiate some process that eventually allows itself to become a participating member of the network. In traditional web2 systems, it's common to use a web service by hitting the service's DNS and being routed to an available server behind a load balancer. In decentralized p2p systems, however, connecting to a node is more complex as no single entity owns the network. [Avalanche consensus](https://docs.avax.network/overview/getting-started/avalanche-consensus) requires a node to repeatedly sample peers in the network, so each node needs some way of discovering and connecting to every other peer to participate in the protocol.
   121  
   122  ### Inbound Connections
   123  
   124  It is expected for Avalanche nodes to allow inbound connections. If a validator does not allow inbound connections, its observed uptime may be reduced.
   125  
   126  ### Outbound Connections
   127  
   128  Avalanche nodes that have identified the `IP:Port` pair of a node they want to connect to will initiate outbound connections to this `IP:Port` pair. If the connection is not able to complete the [Peer Handshake](#peer-handshake), the connection will be re-attempted with an [Exponential Backoff](https://en.wikipedia.org/wiki/Exponential_backoff).
   129  
   130  A node should initiate outbound connections to an `IP:Port` pair that is believed to belong to a node that is not connected and meets at least one of the following conditions:
   131  - The node is in the initial bootstrapper set.
   132  - The node is in the default bootstrapper set.
   133  - The node in the current Primary Network validator set.
   134  
   135  #### IP Authentication
   136  
   137  To ensure that outbound connections are being made to the correct `IP:Port` pair of a node, all `IP:Port` pairs sent by the network are signed by the node that is claiming ownership of the pair. To prevent replays of these messages, the signature is over the `Timestamp` in addition to the `IP:Port` pair.
   138  
   139  The `Timestamp` guarantees that nodes provided an `IP:Port` pair can track the most up-to-date `IP:Port` pair of a peer.
   140  
   141  ### Bootstrapping
   142  
   143  In Avalanche, nodes connect to an initial set (this is user-configurable) of bootstrap nodes.
   144  
   145  ### PeerList Gossip
   146  
   147  Once connected to an initial set of peers, a node can use these connections to discover additional peers.
   148  
   149  Peers are discovered by receiving [`PeerList`](#peerlist) messages during the [Peer Handshake](#peer-handshake). These messages quickly provide a node with knowledge of peers in the network. However, they offer no guarantee that the node will connect to and maintain connections with every peer in the network.
   150  
   151  To provide an eventual guarantee that all peers learn of one another, nodes periodically send a [`GetPeerList`](#getpeerlist) message to a randomly selected validator with the node's current [Bloom Filter](#bloom-filter) and `Salt`.
   152  
   153  #### Bloom Filter
   154  
   155  A [Bloom Filter](https://en.wikipedia.org/wiki/Bloom_filter) is used to track which nodes are known.
   156  
   157  The parameterization of the Bloom Filter is based on the number of desired peers.
   158  
   159  Entries in the Bloom Filter are determined by a locally calculated [`Salt`](https://en.wikipedia.org/wiki/Salt_(cryptography)) along with the `NodeID` and `Timestamp` of the most recently known `IP:Port`. The `Salt` is added to prevent griefing attacks where malicious nodes intentionally generate hash collisions with other virtuous nodes to reduce their connectivity.
   160  
   161  The Bloom Filter is reconstructed if there are more entries than expected to avoid increasing the false positive probability. It is also reconstructed periodically. When reconstructing the Bloom Filter, a new `Salt` is generated.
   162  
   163  To prevent a malicious node from arbitrarily filling this Bloom Filter, only `2` entries are added to the Bloom Filter per node. If a node's `IP:Port` pair changes once, it will immediately be added to the Bloom Filter. If a node's `IP:Port` pair changes more than once, it will only be added to the Bloom Filter after the Bloom Filter is reconstructed.
   164  
   165  #### GetPeerList
   166  
   167  A `GetPeerList` message contains the Bloom Filter of the currently known peers along with the `Salt` that was used to add entries to the Bloom Filter. Upon receipt of a `GetPeerList` message, a node is expected to respond with a `PeerList` message.
   168  
   169  #### PeerList
   170  
   171  `PeerList` messages are expected to contain `IP:Port` pairs that satisfy all of the following constraints:
   172  - The Bloom Filter sent when requesting the `PeerList` message does not contain the node claiming the `IP:Port` pair.
   173  - The node claiming the `IP:Port` pair is currently connected.
   174  - The `IP:Port` pair the node shared during the `Handshake` message is the node's most recently known `IP:Port` pair.
   175  - The node claiming the `IP:Port` pair is either in the default bootstrapper set or is a current Primary Network validator.
   176  
   177  #### Example PeerList Gossip
   178  
   179  The following diagram shows an example of `Alice` repeatedly learning about new peers from `Bob`.
   180  
   181  ```mermaid
   182  sequenceDiagram
   183      actor Alice
   184      actor Bob
   185      Note left of Alice: Initialize Bloom Filter
   186      Note left of Alice: Bloom: [0, 0, 0]
   187      Alice->>Bob: GetPeerList [0, 0, 0]
   188      Note right of Bob: Any peers can be sent.
   189      Bob->>Alice: PeerList - Peer-1
   190      Note left of Alice: Bloom: [1, 0, 0]
   191      Alice->>Bob: GetPeerList [1, 0, 0]
   192      Note right of Bob: Either Peer-2 or Peer-3 can be sent.
   193      Bob->>Alice: PeerList - Peer-3
   194      Note left of Alice: Bloom: [1, 0, 1]
   195      Alice->>Bob: GetPeerList [1, 0, 1]
   196      Note right of Bob: Only Peer-2 can be sent.
   197      Bob->>Alice: PeerList - Peer-2
   198      Note left of Alice: Bloom: [1, 1, 1]
   199      Alice->>Bob: GetPeerList [1, 1, 1]
   200      Note right of Bob: There are no more peers left to send!
   201      Bob->>Alice: PeerList - Empty
   202  ```