github.com/onflow/flow-go@v0.35.7-crescendo-preview.23-atree-inlining/network/p2p/scoring/README.md (about)

     1  # GossipSub App Specific Score 
     2  
     3  This package provides a scoring mechanism for peers in a GossipSub network by computing their application-specific scores.
     4  Application-specific score is part of the GossipSub scoring mechanism, which is used to determine the behavior of peers in the network from
     5  the perspective of their behavior at the application level (i.e., Flow protocol).
     6  The score is determined based on a combination of penalties and rewards related to various factors, such as spamming misbehaviors, staking status, and valid subscriptions.
     7  
     8  ## Key Components
     9  1. `GossipSubAppSpecificScoreRegistry`: This struct maintains the necessary information for determining a peer's score.
    10  2. `AppSpecificScoreFunc`: This function is exposed to GossipSub and calculates the application-specific score for a peer based on penalties and rewards.
    11  3. `stakingScore`: This function computes the staking score (reward/penalty) for a peer based on their identity and role.
    12  4. `subscriptionPenalty`: This function calculates the penalty for invalid subscriptions.
    13  5. `OnInvalidControlMessageNotification`: This method updates a peer's penalty when an invalid control message misbehavior is detected, e.g., spamming on a control message.
    14  
    15  ## Score Calculation
    16  The application-specific score for a peer is calculated as the sum of the following factors:
    17  
    18  1. Spam Penalty: A penalty applied when a peer conducts a spamming misbehavior (e.g., GRAFT, PRUNE, iHave, or iWant misbehaviors).
    19  2. Staking Penalty: A penalty applied for unknown peers with invalid Flow protocol identities. This ejects them from the GossipSub network. 
    20  3. Subscription Penalty: A penalty applied when a peer subscribes to a topic they are not allowed to, based on their role in the Flow network.
    21  4. Staking Reward: A reward applied to well-behaved staked peers (excluding access nodes at the moment) only if they have no penalties from spamming or invalid subscriptions.
    22  
    23  The score is updated every time a peer misbehaves, and the spam penalties decay over time using the default decay function, which applies a geometric decay to the peer's score.
    24  
    25  ### Usage
    26  To use the scoring mechanism, create a new `GossipSubAppSpecificScoreRegistry` with the desired configuration, and then obtain the `AppSpecificScoreFunc` to be passed to the GossipSub protocol.
    27  
    28  Example:
    29  ```go
    30  config := &GossipSubAppSpecificScoreRegistryConfig{
    31  	// ... configure the required components
    32  }
    33  registry := NewGossipSubAppSpecificScoreRegistry(config)
    34  appSpecificScoreFunc := registry.AppSpecificScoreFunc()
    35  
    36  // Use appSpecificScoreFunc as the score function for GossipSub
    37  ```
    38  
    39  The scoring mechanism can be easily integrated with the GossipSub protocol to ensure that well-behaved peers are prioritized, and misbehaving peers are penalized. See the `ScoreOption` below for more details.
    40  
    41  **Note**: This package was designed specifically for the Flow network and might require adjustments if used in other contexts.
    42  
    43  
    44  ## Score Option
    45  `ScoreOption` is a configuration object for the peer scoring system in the Flow network.
    46  It defines several scoring parameters and thresholds that determine the behavior of the network towards its peers.
    47  This includes rewarding well-behaving peers and penalizing misbehaving ones.
    48  
    49  **Note**: `ScoreOption` is passed to the GossipSub as a configuration option at the time of initialization.
    50  
    51  ### Usage
    52  To use the `ScoreOption`, you need to create a `ScoreOptionConfig` with the desired settings and then call `NewScoreOption` with that configuration.
    53  
    54  ```go
    55  config := NewScoreOptionConfig(logger)
    56  config.SetProvider(identityProvider)
    57  config.SetCacheSize(1000)
    58  config.SetCacheMetrics(metricsCollector)
    59  
    60  // Optional: Set custom app-specific scoring function
    61  config.SetAppSpecificScoreFunction(customAppSpecificScoreFunction)
    62  
    63  scoreOption := NewScoreOption(config)
    64  ```
    65  
    66  ### Scoring Parameters
    67  `ScoreOption` provides a set of default scoring parameters and thresholds that can be configured through the `ScoreOptionConfig`. These parameters include:
    68  
    69  1. `AppSpecificScoreWeight`: The weight of the application-specific score in the overall peer score calculation at the GossipSub.
    70  2. `GossipThreshold`: The threshold below which a peer's score will result in ignoring gossips to and from that peer.
    71  3. `PublishThreshold`: The threshold below which a peer's score will result in not propagating self-published messages to that peer.
    72  4. `GraylistThreshold`: The threshold below which a peer's score will result in ignoring incoming RPCs from that peer.
    73  5. `AcceptPXThreshold`: The threshold above which a peer's score will result in accepting PX information with a prune from that peer. PX stands for "Peer Exchange" in the context of libp2p's gossipsub protocol. When a peer sends a PRUNE control message to another peer, it can include a list of other peers as PX information. The purpose of this is to help the pruned peer find new peers to replace the ones that have been pruned from its mesh. When a node receives a PRUNE message containing PX information, it can decide whether to connect to the suggested peers based on its own criteria. In this package, the `DefaultAcceptPXThreshold` is used to determine if the originating peer's penalty score is good enough to accept the PX information. If the originating peer's penalty score exceeds the threshold, the node will consider connecting to the suggested peers.
    74  6. `OpportunisticGraftThreshold`: The threshold below which the median peer score in the mesh may result in selecting more peers with a higher score for opportunistic grafting.
    75  
    76  ### Flow Specific Scoring Parameters and Thresholds
    77  # GossipSub Scoring Parameters Explained
    78  1. `DefaultAppSpecificScoreWeight = 1`: This is the default weight for application-specific scoring. It basically tells us how important the application-specific score is in comparison to other scores.
    79  2. `MaxAppSpecificPenalty = -100` and `MinAppSpecificPenalty = -1`: These values define the range for application-specific penalties. A peer can have a maximum penalty of -100 and a minimum penalty of -1.
    80  3. `MaxAppSpecificReward = 100`: This is the maximum reward a peer can earn for good behavior.
    81  4. `DefaultStakedIdentityReward = MaxAppSpecificReward`: This reward is given to peers that contribute positively to the network (i.e., no misbehavior). It’s to encourage them and prioritize them in neighbor selection.
    82  5. `DefaultUnknownIdentityPenalty = MaxAppSpecificPenalty`: This penalty is given to a peer if it's not in the identity list. It's to discourage anonymity.
    83  6. `DefaultInvalidSubscriptionPenalty = MaxAppSpecificPenalty`: This penalty is for peers that subscribe to topics they are not authorized to subscribe to.
    84  7. `DefaultGossipThreshold = -99`: If a peer's penalty goes below this threshold, the peer is ignored for gossip. It means no gossip is sent to or received from that peer.
    85  8. `DefaultPublishThreshold = -99`: If a peer's penalty goes below this threshold, self-published messages will not be sent to this peer.
    86  9. `DefaultGraylistThreshold = -99`: If a peer's penalty goes below this threshold, it is graylisted. This means all incoming messages from this peer are ignored.
    87  10. `DefaultAcceptPXThreshold = 99`: This is a threshold for accepting peers. If a peer sends information and its score is above this threshold, the information is accepted.
    88  11. `DefaultOpportunisticGraftThreshold = MaxAppSpecificReward + 1`: This value is used to selectively connect to new peers if the median score of the current peers drops below this threshold.
    89  12. `defaultScoreCacheSize = 1000`: Sets the default size of the cache used to store the application-specific penalty of peers.
    90  13. `defaultDecayInterval = 1 * time.Minute`: Sets the default interval at which the score of a peer will be decayed.
    91  14. `defaultDecayToZero = 0.01`: This is a threshold below which a decayed score is reset to zero. It prevents the score from decaying to a very small value.
    92  15. `defaultTopicTimeInMeshQuantum` is a parameter in the GossipSub scoring system that represents a fixed time interval used to count the amount of time a peer stays in a topic mesh. It is set to 1 hour, meaning that for each hour a peer remains in a topic mesh, its time-in-mesh counter increases by 1, contributing to its availability score. This is to reward peers that stay in the mesh for longer durations and discourage those that frequently join and leave.
    93  16. `defaultTopicInvalidMessageDeliveriesWeight` is set to -1.0 and is used to penalize peers that send invalid messages by applying it to the square of the number of such messages. A message is considered invalid if it is not properly signed. A peer will be disconnected if it sends around 14 invalid messages within a gossipsub heartbeat interval.
    94  17. `defaultTopicInvalidMessageDeliveriesDecay` is a decay factor set to 0.99. It is used to reduce the number of invalid message deliveries counted against a peer by 1% at each heartbeat interval. This prevents the peer from being disconnected if it stops sending invalid messages. The heartbeat interval in the gossipsub scoring system is set to 1 minute by default.
    95  
    96  ## GossipSub Message Delivery Scoring
    97  This section provides an overview of the GossipSub message delivery scoring mechanism used in the Flow network. 
    98  It's designed to maintain an efficient, secure and stable peer-to-peer network by scoring each peer based on their message delivery performance. 
    99  The system ensures the reliability of message propagation by scoring peers, which discourages malicious behaviors and enhances overall network performance.
   100  
   101  ### Comprehensive System Overview
   102  The GossipSub message delivery scoring mechanism used in the Flow network is an integral component of its P2P communication model. 
   103  It is designed to monitor and incentivize appropriate network behaviors by attributing scores to peers based on their message delivery performance. 
   104  This scoring system is fundamental to ensure that messages are reliably propagated across the network, creating a robust P2P communication infrastructure.
   105  
   106  The scoring system is per topic, which means it tracks the efficiency of peers in delivering messages in each specific topic they are participating in. 
   107  These per-topic scores then contribute to an overall score for each peer, providing a comprehensive view of a peer's effectiveness within the network.
   108  In GossipSub, a crucial aspect of a peer's responsibility is to relay messages effectively to other nodes in the network. 
   109  The role of the scoring mechanism is to objectively assess a peer's efficiency in delivering these messages. 
   110  It takes into account several factors to determine the effectiveness of the peers.
   111  
   112  1. **Message Delivery Rate** - A peer's ability to deliver messages quickly is a vital metric. Slow delivery could lead to network lags and inefficiency.
   113  2. **Message Delivery Volume** - A peer's capacity to deliver a large number of messages accurately and consistently.
   114  3. **Continuity of Performance** - The scoring mechanism tracks not only the rate and volume of the messages but also the consistency in a peer's performance over time.
   115  4. **Prevention of Malicious Behaviors** - The scoring system also helps in mitigating potential network attacks such as spamming and message replay attacks.
   116  
   117  The system utilizes several parameters to maintain and adjust the scores of the peers:
   118  - `defaultTopicMeshMessageDeliveriesDecay`(value: 0.5): This parameter dictates how rapidly a peer's message delivery count decays with time. With a value of 0.5, it indicates a 50% decay at each decay interval. This mechanism ensures that past performances do not disproportionately impact the current score of the peer.
   119  - `defaultTopicMeshMessageDeliveriesCap` (value: 1000): This parameter sets an upper limit on the number of message deliveries that can contribute to the score of a peer in a topic. With a cap set at 1000, it prevents the score from being overly influenced by large volumes of message deliveries, providing a balanced assessment of peer performance.
   120  - `defaultTopicMeshMessageDeliveryThreshold` (value: 0.1 * `defaultTopicMeshMessageDeliveriesCap`): This threshold serves to identify under-performing peers. If a peer's message delivery count is below this threshold in a topic, the peer's score is penalized. This encourages peers to maintain a minimum level of performance.
   121  - `defaultTopicMeshMessageDeliveriesWeight`  (value: -0.05 * `MaxAppSpecificReward` / (`defaultTopicMeshMessageDeliveryThreshold` ^ 2) = 5^-4): This weight is applied when penalizing under-performing peers. The penalty is proportional to the square of the difference between the actual message deliveries and the threshold, multiplied by this weight.
   122  - `defaultMeshMessageDeliveriesWindow` (value: `defaultDecayInterval` = 1 minute): This parameter defines the time window within which a message delivery is counted towards the score. This window is set to the decay interval, preventing replay attacks and counting only unique message deliveries.
   123  - `defaultMeshMessageDeliveriesActivation` (value: 2 * `defaultDecayInterval` = 2 minutes): This time interval is the grace period before the scoring system starts tracking a new peer's performance. It accounts for the time it takes for a new peer to fully integrate into the network.
   124  
   125  By continually updating and adjusting the scores of peers based on these parameters, the GossipSub message delivery scoring mechanism ensures a robust, efficient, and secure P2P network.
   126  
   127  ### Examples
   128  
   129  #### Scenario 1: Peer A Delivers Messages Within Cap and Above Threshold
   130  Let's assume a Peer A that consistently delivers 500 messages per decay interval. This is within the `defaultTopicMeshMessageDeliveriesCap` (1000) and above the `defaultTopicMeshMessageDeliveryThreshold` (100).
   131  As Peer A's deliveries are above the threshold and within the cap, its score will not be penalized. Instead, it will be maintained, promoting healthy network participation.
   132  
   133  #### Scenario 2: Peer B Delivers Messages Below Threshold
   134  Now, assume Peer B delivers 50 messages per decay interval, below the `defaultTopicMeshMessageDeliveryThreshold` (100).
   135  In this case, the score of Peer B will be penalized because its delivery rate is below the threshold. The penalty is calculated as `-|w| * (actual - threshold)^2`, where `w` is the weight (`defaultTopicMeshMessageDeliveriesWeight`), `actual` is the actual messages delivered (50), and `threshold` is the delivery threshold (100).
   136  
   137  #### Scenario 3: Peer C Delivers Messages Exceeding the Cap
   138  Consider Peer C, which delivers 1500 messages per decay interval, exceeding the `defaultTopicMeshMessageDeliveriesCap` (1000).
   139  In this case, even though Peer C is highly active, its score will not increase further once it hits the cap (1000). This is to avoid overemphasis on high delivery counts, which could skew the scoring system.
   140  
   141  #### Scenario 4: Peer D Joins a Topic Mesh
   142  When a new Peer D joins a topic mesh, it will be given a grace period of `defaultMeshMessageDeliveriesActivation` (2 decay intervals) before its message delivery performance is tracked. This grace period allows the peer to set up and begin receiving messages from the network.
   143  Remember, the parameters and scenarios described here aim to maintain a stable, efficient, and secure peer-to-peer network by carefully tracking and scoring each peer's message delivery performance.
   144  
   145  #### Scenario 5: Message Delivery Decay
   146  To better understand how the message delivery decay (`defaultTopicMeshMessageDeliveriesDecay`) works in the GossipSub protocol, let's examine a hypothetical scenario.
   147  Let's say we have a peer named `Peer A` who is actively participating in `Topic X`. `Peer A` has successfully delivered 800 messages in `Topic X` over a given time period.
   148  **Initial State**: At this point, `Peer A`'s message delivery count for `Topic X` is 800. Now, the decay interval elapses without `Peer A` delivering any new messages in `Topic X`.
   149  **After One Decay Interval**: Given that our `defaultTopicMeshMessageDeliveriesDecay` value is 0.5, after one decay interval, `Peer A`'s message delivery count for `Topic X` will decay by 50%. Therefore, `Peer A`'s count is now:
   150  
   151      800 (previous message count) * 0.5 (decay factor) = 400
   152  
   153  **After Two Decay Intervals**
   154  If `Peer A` still hasn't delivered any new messages in `Topic X` during the next decay interval, the decay is applied again, further reducing the message delivery count:
   155  
   156      400 (current message count) * 0.5 (decay factor) = 200
   157  And this process will continue at every decay interval, halving `Peer A`'s message delivery count for `Topic X` until `Peer A` delivers new messages in `Topic X` or the count reaches zero.
   158  This decay process ensures that a peer cannot rest on its past deliveries; it must continually contribute to the network to maintain its score. 
   159  It helps maintain a lively and dynamic network environment, incentivizing constant active participation from all peers.
   160  
   161  ### Scenario 6: Replay Attack
   162  The `defaultMeshMessageDeliveriesWindow` and `defaultMeshMessageDeliveriesActivation` parameters play a crucial role in preventing replay attacks in the GossipSub protocol. Let's illustrate this with an example.
   163  Consider a scenario where we have three peers: `Peer A`, `Peer B`, and `Peer C`. All three peers are active participants in `Topic X`.
   164  **Initial State**: At Time = 0: `Peer A` generates and broadcasts a new message `M` in `Topic X`. `Peer B` and `Peer C` receive this message from `Peer A` and update their message caches accordingly.
   165  **After Few Seconds**: At Time = 30 seconds: `Peer B`, with malicious intent, tries to rebroadcast the same message `M` back into `Topic X`. 
   166  Given that our `defaultMeshMessageDeliveriesWindow` value is equal to the decay interval (let's assume 1 minute), `Peer C` would have seen the original message `M` from `Peer A` less than one minute ago.
   167  This is within the `defaultMeshMessageDeliveriesWindow`. Because `Peer A` (the original sender) is different from `Peer B` (the current sender), this delivery will be counted towards `Peer B`'s message delivery score in `Topic X`.
   168  **After One Minute**: At Time = 61 seconds: `Peer B` tries to rebroadcast the same message `M` again. 
   169  Now, more than a minute has passed since `Peer C` first saw the message `M` from `Peer A`. This is outside the `defaultMeshMessageDeliveriesWindow`. 
   170  Therefore, the message `M` from `Peer B` will not count towards `Peer B`'s message delivery score in `Topic X` and `Peer B` still needs to fill up its threshold of message delivery in order not to be penalized for under-performing. 
   171  This effectively discouraging replay attacks of messages older than the `defaultMeshMessageDeliveriesWindow`.
   172  This mechanism, combined with other parameters, helps maintain the security and efficiency of the network by discouraging harmful behaviors such as message replay attacks.
   173  
   174  ## Mitigating iHave Broken Promises Attacks in GossipSub Protocol
   175  ### What is an iHave Broken Promise Attack?
   176  In the GossipSub protocol, peers gossip information about new messages to a subset of random peers (out of their local mesh) in the form of an "iHave" message which basically tells the receiving peer what messages the sender has. 
   177  The receiving peer then replies with an "iWant" message, requesting for the messages it doesn't have. Note that for the peers in local mesh the actual new messages are sent instead of an "iHave" message (i.e., eager push). However,
   178  iHave-iWant protocol is part of a complementary mechanism to ensure that the information is disseminated to the entire network in a timely manner (i.e., lazy pull).
   179  
   180  An "iHave Broken Promise" attack occurs when a peer advertises many "iHave" for a message but doesn't respond to the "iWant" requests for those messages.
   181  This not only hinders the effective dissemination of information but can also strain the network with redundant requests. Hence, we stratify it as a spam behavior mounting a DoS attack on the network.
   182  
   183  ### Detecting iHave Broken Promise Attacks
   184  Detecting iHave Broken Promise Attacks is done by the GossipSub itself. On each incoming RPC from a remote node, the local GossipSub node checks if the RPC contains an iHave message. It then samples one (and only one) iHave message
   185  randomly out of the entire set of iHave messages piggybacked on the incoming RPC. If the sampled iHave message is not literally addressed with the actual message, the local GossipSub node considers this as an iHave broken promise and
   186  increases the behavior penalty counter for that remote node. Hence, incrementing the behavior penalty counter for a remote peer is done per RPC containing at least one iHave broken promise and not per iHave message.
   187  Note that the behavior penalty counter also keeps track of GRAFT flood attacks that are done by a remote peer when it advertises many GRAFTs while it is on a PRUNE backoff by the local node. Mitigating iHave broken promise attacks also
   188  mitigates GRAFT flood attacks.
   189  
   190  ### Configuring GossipSub Parameters
   191  In order to mitigate the iHave broken promises attacks, GossipSub expects the application layer (i.e., Flow protocol) to properly configure the relevant scoring parameters, notably: 
   192  
   193  - `BehaviourPenaltyThreshold` is set to `defaultBehaviourPenaltyThreshold`, i.e., `10`.
   194  - `BehaviourPenaltyWeight` is set to `defaultBehaviourPenaltyWeight`, i.e., `0.01` * `MaxAppSpecificPenalty`
   195  - `BehaviourPenaltyDecay` is set to `defaultBehaviourPenaltyDecay`, i.e., `0.99`.
   196  
   197  #### 1. `defaultBehaviourPenaltyThreshold`
   198  This parameter sets the threshold for when the behavior of a peer is considered bad. Misbehavior is defined as advertising an iHave without responding to the iWants (iHave broken promises), and attempting on GRAFT when the peer is considered for a PRUNE backoff.
   199  If a remote peer sends an RPC that advertises at least one iHave for a message but doesn't respond to the iWant requests for that message within the next `3 seconds`, the peer misbehavior counter is incremented by `1`. This threshold is set to `10`, meaning that we at most tolerate 10 such RPCs containing iHave broken promises. After this, the peer is penalized for every excess RPC containing iHave broken promises. The counter decays by (`0.99`) every decay interval (defaultDecayInterval) i.e., every minute.
   200  
   201  #### 2. `defaultBehaviourPenaltyWeight`
   202  This is the weight applied as a penalty when a peer's misbehavior goes beyond the `defaultBehaviourPenaltyThreshold`. 
   203  The penalty is applied to the square of the difference between the misbehavior counter and the threshold, i.e., -|w| * (misbehavior counter - threshold)^2, where `|w|` is the absolute value of the `defaultBehaviourPenaltyWeight`. 
   204  Note that `defaultBehaviourPenaltyWeight` is a negative value, meaning that the penalty is applied in the opposite direction of the misbehavior counter. For sake of illustration, we use the notion of `-|w|` to denote that a negative penalty is applied.
   205  We set  `defaultBehaviourPenaltyWeight` to `0.01 * MaxAppSpecificPenalty`, meaning a peer misbehaving `10` times more than the threshold (i.e., `10 + 10`) will lose its entire `MaxAppSpecificReward`, which is a reward given to all staked nodes in Flow blockchain.
   206  This also means that a peer misbehaving `sqrt(2) * 10` times more than the threshold will cause the peer score to be dropped below the `MaxAppSpecificPenalty`, which is also below the `GraylistThreshold`, and the peer will be graylisted (i.e., all incoming and outgoing GossipSub RPCs from and to that peer will be rejected). 
   207  This means the peer is temporarily disconnected from the network, preventing it from causing further harm.
   208  
   209  #### 3. defaultBehaviourPenaltyDecay
   210  This is the decay interval for the misbehavior counter of a peer. This counter is decayed by the `defaultBehaviourPenaltyDecay` parameter (e.g., `0.99`) per decay interval, which is currently every 1 minute.
   211  This parameter helps to gradually reduce the effect of past misbehaviors and provides a chance for penalized nodes to rejoin the network. A very slow decay rate can help identify and isolate persistent offenders, while also allowing potentially honest nodes that had transient issues to regain their standing in the network.
   212  The duration a peer remains graylisted is governed by the choice of `defaultBehaviourPenaltyWeight` and the decay parameters. 
   213  Based on the given configuration, a peer which has misbehaved on `sqrt(2) * 10` RPCs more than the threshold will get graylisted (disconnected at GossipSub level).
   214  With the decay interval set to 1 minute and decay value of 0.99, a graylisted peer due to broken promises would be expected to be reconnected in about 50 minutes. 
   215  This is calculated by solving for `x` in the equation `(0.99)^x * (sqrt(2) * 10)^2 * MaxAppSpecificPenalty > GraylistThreshold`. 
   216  Simplifying, we find `x` to be approximately `527` decay intervals, or roughly `527` minutes. 
   217  This is the estimated time it would take for a severely misbehaving peer to have its penalty decayed enough to exceed the `GraylistThreshold` and thus be reconnected to the network.
   218  
   219  ### Example Scenarios
   220  **Scenario 1: Misbehaving Below Threshold**
   221  In this scenario, consider peer `B` that has recently joined the network and is taking part in GossipSub. 
   222  This peer advertises to peer `A` many `iHave` messages over an RPC. But when other peer `A` requests these message with `iWant`s it fails to deliver the message within 3 seconds. 
   223  This action constitutes an _iHave broken promise_ for a single RPC and peer `A` increases the local behavior penalty counter of peer `B` by 1.
   224  If the peer `B` commits this misbehavior infrequently, such that the total number of these RPCs does not exceed the `defaultBehaviourPenaltyThreshold` (set to 10 in our configuration), 
   225  the misbehavior counter for this peer will increment by 1 for each RPC and decays by `1%` evey decay interval (1 minute), but no additional penalty will be applied. 
   226  The misbehavior counter decays by a factor of `defaultBehaviourPenaltyDecay` (0.99) every minute, allowing the peer to recover from these minor infractions without significant disruption.
   227  
   228  **Scenario 2: Misbehaving Above Threshold But Below Graylisting**
   229  Now consider that peer `B` frequently sends RPCs advertising many `iHaves`  to peer `A` but fails to deliver the promised messages. 
   230  If the number of these misbehaviors exceeds our threshold (10 in our configuration), the peer `B` is now penalized by the local GossipSub mechanism of peer `A`. 
   231  The amount of the penalty is determined by the `defaultBehaviourPenaltyWeight` (set to 0.01 * MaxAppSpecificPenalty) applied to the square of the difference between the misbehavior counter and the threshold.
   232  This penalty will progressively affect the peer's score, deteriorating its reputation in the local GossipSub scoring system of node `A`, but does not yet result in disconnection or graylisting. 
   233  The peer has a chance to amend its behavior before crossing into graylisting territory through stop misbehaving and letting the score to decay.
   234  When peer `B` has a deteriorated score at node `A`, it will be less likely to be selected by node `A` as its local mesh peer (i.e., to directly receive new messages from node `A`), and is deprived of the opportunity to receive new messages earlier through node `A`.
   235  
   236  **Scenario 3: Graylisting**
   237  Now assume that peer `B` peer has been continually misbehaving, with RPCs including iHave broken promises exceeding `sqrt(2) * 10` the threshold. 
   238  At this point, the peer's score drops below the `GraylistThreshold` due to the `defaultBehaviorPenaltyWeight` applied to the excess misbehavior. 
   239  The peer is then graylisted by peer `A`, i.e., peer `A` rejects all incoming RPCs to and from peer `B` at GossipSub level. 
   240  In our configuration, peer `B` will stay disconnected for at least `527` decay intervals or approximately `527` minutes. 
   241  This gives a strong disincentive for the peer to continue this behavior and also gives it time to recover and eventually be reconnected to the network.
   242  
   243  ## Customization
   244  The scoring mechanism can be easily customized to suit the needs of the Flow network. This includes changing the scoring parameters, thresholds, and the scoring function itself.
   245  You can customize the scoring parameters and thresholds by using the various setter methods provided in the `ScoreOptionConfig` object. Additionally, you can provide a custom app-specific scoring function through the `SetAppSpecificScoreFunction` method.
   246  
   247  **Note**: Usage of non-default app-specific scoring function is not recommended unless you are familiar with the scoring mechanism and the Flow network. It may result in _routing attack vulnerabilities_. It is **always safer** to use the default scoring function unless you know what you are doing.
   248  
   249  Example of setting custom app-specific scoring function:
   250  ```go
   251  config.SetAppSpecificScoreFunction(customAppSpecificScoreFunction)
   252  ```
   253  
   254  ## Peer Scoring System Integration
   255  The peer scoring system is integrated with the GossipSub protocol through the `ScoreOption` configuration option. 
   256  This option is passed to the GossipSub at the time of initialization.
   257  `ScoreOption` can be used to build scoring options for GossipSub protocol with the desired scoring parameters and thresholds.
   258  ```go
   259  flowPubSubOption := scoreOption.BuildFlowPubSubScoreOption()
   260  gossipSubOption := scoreOption.BuildGossipSubScoreOption()
   261  ```
   262  
   263  # Caching Application Specific Score
   264  ![app-specific-score-cache.png](app-specific-score-cache.png)
   265  The application-specific score of a peer is part of its overall score in the GossipSub protocol. In contrast to the rest of the GossipSub score of the peer that is computed
   266  internally by the GossipSub protocol, the application-specific score of a peer is computed externally by the application, i.e., the Flow protocol-level semantics.
   267  As the figure above illustrates, GossipSub's peer scoring mechanism invokes the application-specific scoring function on a peer id upon receiving a gossip message from that peer.
   268  This means that the application-specific score of a peer is computed every time a gossip message is received from that peer. 
   269  This can be computationally expensive, especially when the network is large and the number of gossip messages is high. 
   270  As shown by the figure above, each time the application-specific score of a peer is computed, the score is computed from scratch by
   271  computing the spam penalty, staking score and subscription penalty. Each of these computations involves a cache lookup and a computation.
   272  Hence, a single computation of the application-specific score of a peer involves 3 cache lookups and 3 computations.
   273  As the application-specific score of a peer is not expected to change frequently, we can cache the score of a peer and reuse it for a certain period of time. 
   274  This can significantly reduce the computational overhead of the scoring mechanism. 
   275  By caching the application-specific score of a peer, we can reduce the number of cache lookups and computations from 3 to 1 per computation of the application-specific score of a peer, which
   276  results in a 66% reduction in the computational overhead of the scoring mechanism.
   277  The caching mechanism is implemented in the `GossipSubAppSpecificScoreRegistry` struct. Each time the application-specific score of a peer is requested by the GossipSub protocol, the registry
   278  checks if the score of the peer is cached. If the score is cached, the cached score is returned. Otherwise, a score of zero is returned, and a request for the application-specific score of the peer is 
   279  queued to the `appScoreUpdateWorkerPool` to be computed asynchronously. Once the score is computed, it is cached and the score is updated in the `appScoreCache`. 
   280  Each score record in the cache is associated with a TTL (time-to-live) value, which is the duration for which the score is valid. 
   281  When the retrieved score is expired, the expired score is still returned to the GossipSub protocol, but the score is updated asynchronously in the background by submitting a request to the `appScoreUpdateWorkerPool`.
   282  The app-specific score configuration values are configurable through the following parameters in the `default-config.yaml` file:
   283  ```yaml
   284      scoring-parameters:
   285        app-specific-score:
   286          # number of workers that asynchronously update the app specific score requests when they are expired.
   287          score-update-worker-num: 5
   288          # size of the queue used by the worker pool for the app specific score update requests. The queue is used to buffer the app specific score update requests
   289          # before they are processed by the worker pool. The queue size must be larger than total number of peers in the network.
   290          # The queue is deduplicated based on the peer ids ensuring that there is only one app specific score update request per peer in the queue.
   291          score-update-request-queue-size: 10_000
   292          # score ttl is the time to live for the app specific score. Once the score is expired; a new request will be sent to the app specific score provider to update the score.
   293          # until the score is updated, the previous score will be used.
   294          score-ttl: 1m
   295  ```