github.com/MetalBlockchain/metalgo@v1.11.9/chains/atomic/README.md (about)

     1  # Shared Memory
     2  
     3  Shared memory creates a way for blockchains in the Avalanche Ecosystem to communicate with each other by using a shared database to create a bidirectional communication channel between any two blockchains in the same subnet.
     4  
     5  ## Shared Database
     6  
     7  ### Using Shared Base Database
     8  
     9  AvalancheGo uses a single base database (typically leveldb). This database is partitioned using the `prefixdb` package, so that each recipient of a `prefixdb` can treat it as if it were its own unique database.
    10  
    11  Each blockchain in the Avalanche Ecosystem has its own `prefixdb` passed in via `vm.Initialize(...)`, which means that the `prefixdb`s that are given to each blockchain share the same base level database.
    12  
    13  Shared Memory, which also uses the same underlying database, leverages this fact to support the ability to combine a database batch performed on shared memory with any other batches that are performed on the same underlying database.
    14  
    15  This allows VMs the ability to perform some database operations on their own database, and commit them atomically with operations that need to be applied to shared memory.
    16  
    17  ### Creating a Unique Shared Database
    18  
    19  Shared Memory creates a unique sharedID for any pair of blockchains by ordering the two blockchainIDs, marshalling it to a byte array, and taking a hash of the resulting byte array.
    20  
    21  This sharedID is used to create a `prefixdb` on top of shared memory's database. The sharedID prefixed database is the shared database used to create a bidirectional communication channel.
    22  
    23  The shared database is split up into two `state` objects one for each blockchain in the pair.
    24  
    25  Shared Memory can then build the interface to send a message from ChainA to ChainB, which can then only be read and deleted by ChainB. Specifically, Shared Memory exposes the following interface:
    26  
    27  ```go
    28  type SharedMemory interface {
    29      // Get fetches the values corresponding to [keys] that have been sent from
    30      // [peerChainID]
    31      Get(peerChainID ids.ID, keys [][]byte) (values [][]byte, err error)
    32      // Indexed returns a paginated result of values that possess any of the
    33      // given traits and were sent from [peerChainID].
    34      Indexed(
    35          peerChainID ids.ID,
    36          traits [][]byte,
    37          startTrait,
    38          startKey []byte,
    39          limit int,
    40      ) (
    41          values [][]byte,
    42          lastTrait,
    43          lastKey []byte,
    44          err error,
    45      )
    46      // Apply performs the requested set of operations by atomically applying
    47      // [requests] to their respective chainID keys in the map along with the
    48      // batches on the underlying DB.
    49      //
    50      // Invariant: The underlying database of [batches] must be the same as the
    51      //            underlying database for SharedMemory.
    52      Apply(requests map[ids.ID]*Requests, batches ...database.Batch) error
    53  }
    54  ```
    55  
    56  When ChainA calls `Apply`, the `requests` map keys are the chainIDs on which to perform requests. If ChainA wants to send a Put and Remove request on ChainB, then it will be broken down as follows:
    57  
    58  Shared Memory grabs the shared database between ChainA and ChainB and it will first perform the Remove request. ChainA can only remove messages that were sent from ChainB, so the Remove request will be processed by Removing the specified message from ChainA's state.
    59  
    60  The Put operation needs to be sent to ChainB, so any Put request will be added to the state of ChainB.
    61  
    62  `Get` and `Indexed` will both grab the same shared database and then look at the state of ChainA to read the messages that have been sent to ChainA from ChainB.
    63  
    64  This setup ensures that the SharedMemory interface passed in to each VM can only send messages to destination chains and can only read and remove messages that were delivered to it by another source chain.
    65  
    66  ### Atomic Elements
    67  
    68  Atomic Elements contain a Key, Value, and a list of traits, all of which are byte arrays:
    69  
    70  ```go
    71  type Element struct {
    72      Key    []byte   `serialize:"true"`
    73      Value  []byte   `serialize:"true"`
    74      Traits [][]byte `serialize:"true"`
    75  }
    76  ```
    77  
    78  A Put operation on shared memory contains a list of Elements to be sent to the destination chain. Each Element is then indexed into the state of the recipient chain.
    79  
    80  The Shared Memory State is divided into a value database and index database as mentioned above. The value database contains a one-to-one mapping of `Key -> Value`, to support efficient `Get` requests for the keys. The index database contains a one-to-many mapping from a `Trait` to the `Keys` that possess that `Trait`.
    81  
    82  Therefore, a Put operation performs the following actions to maintain these mappings:
    83  
    84  - Add `Key -> Value` to the value database
    85  - Add `Trait -> Key` for each Trait in the element to the one-to-many mapping in the index database
    86  
    87  ### Accessing the Shared Database
    88  
    89  Shared Memory creates a shared resource across multiple chains which operate in parallel. This means that Shared Memory must provide concurrency control. Therefore, when grabbing a shared memory database, we use the functions `GetSharedDatabase` and `ReleaseSharedDatabase`, which must be called in conjunction with each other.
    90  
    91  Under the hood, memory creates a shared lock when `makeLock(sharedID)` is called. This returns the lock without grabbing it and tracks the number of callers that have requested access to the shared database. After `makeLock(sharedID)` is called, `releaseLock(sharedID)` is called in `memory.go` to return the same lock and decrement the count of callers that have access to it.
    92  
    93  `Lock()` and `Unlock()` are not called within `makeLock(sharedID)` and `releaseLock(sharedID)`, it's up to the caller of these functions to grab and release the returned lock.
    94  
    95  Returning the lock instead of grabbing it within the function, ensures that only the thread calling `GetSharedDatabase` will block. We grab the lock outside of `makeLock` to avoid grabbing a shared lock while holding onto the lock within `memory.go`, which allows access to the maintained maps of shared locks.
    96  
    97  ## Using Shared Memory for Cross-Chain Communication
    98  
    99  Shared Memory enables generic cross-chain communication. Here we'll go through the lifecycle of a message through shared memory that is used to move assets from ChainA to ChainB.
   100  
   101  ### Issue an export transaction to ChainA
   102  
   103  ChainA will verify this transaction is valid within the block containing the transaction. This verification will ensure that it pays an appropriate fee, ensure that the transaction is well formed, and check to ensure that the destination chain, ChainB, is on the same subnet as ChainA. After the block containing this transaction has been verified, it will be issued to consensus. It's important to note that the message to the shared memory of ChainB is added when this transaction is accepted by the VM. This ensures that an import transaction on ChainB is only valid when the atomic UTXO has been finalized on ChainA.
   104  
   105  ### API service uses Indexed to return the set of UTXOs owned by a set of addresses
   106  
   107  A user that wants to issue an import transaction may need to look up the UTXOs that they control. Atomic UTXOs use the traits field to include the set of addresses that own them. This allows a VM's API service to use the `Indexed` call to look up all of the UTXOs owned by a set of addresses and return them to the user. The user can then form an import transaction that spends the given atomic UTXOs.
   108  
   109  ### Issue an import transaction to ChainB
   110  
   111  The user issues an import transaction to ChainB, which specifies the atomic UTXOs that it spends from ChainA. This transaction is verified within the block that it was issued in. ChainB will check basic correctness of the transaction as well as confirming that the atomic UTXOs that it needs to spend are valid. This check will contain at least the following checks:
   112  
   113  - Confirm the UTXO is present in shared memory from the `sourceChain`
   114  - Confirm that no blocks in processing between the block containing this tx and the last accepted block attempt to spend the same UTXO
   115  - Confirm that the `sourceChain` is an eligible source, which means at least that it is on the same subnet
   116  
   117  Once the block is accepted, then the atomic UTXOs spent by the transaction will be consumed and removed from shared memory. It's important to note that because we remove UTXOs from shared memory when the transaction is accepted, we need to verify that any processing ancestor of the block we're verifying does not conflict with the atomic UTXOs that are being spent by this block. It is not sufficient to check that the atomic UTXO we want to spend is present in shared memory when the block is verified because there may be another block that has not yet been accepted, which attempts to spend the same atomic UTXO.
   118  
   119  For example, there could be a chain of blocks that looks like the following:
   120  
   121  ```text
   122  L    (last accepted block)
   123  |
   124  B1   (spends atomic UTXOA)
   125  |
   126  B2   (spends atomic UTXOA)
   127  ```
   128  
   129  If B1 is processing (has been verified and issued to consensus, but not accepted yet) when block B2 is verified, then ChainB may look at shared memory and see that `UTXOA` is present in shared memory. However, because its parent also attempts to spend it, block B2 obviously conflicts with B1 and is invalid.
   130  
   131  ## Generic Communication
   132  
   133  Shared memory provides the interface for generic communication across blockchains on the same subnet. Cross-chain transactions moving assets between chains is just the first example. The same primitive can be used to send generic messages between blockchains on top of shared memory, but the basic principles of how it works and how to use it correctly remain the same.