github.com/gnolang/gno@v0.0.0-20240520182011-228e9d0192ce/examples/gno.land/p/demo/gnorkle/README.md (about)

     1  # gnorkle
     2  
     3  gnorkle is an attempt at a generalized oracle implementation. The gnorkle `Instance` definitions lives in the `gnorkle` directory. It can be effectively initialized by using the `gnorkle.NewInstance` constructor. The owner of the instance is then able to add feeds with or without whitelists, manage the instance's own whitelist, remove feeds, get the latest feed values, and handle incoming message requests that result in some type of action taking place.
     4  
     5  An example of gnorkle in action can be found in the `examples/gno.land/r/gnoland/ghverify` realm. Any realm that wishes to integrate the minimum oracle functionality should include functions that relay incoming messages to the oracle instance's message handler and that can produce the latest feed values.
     6  
     7  ## Feeds
     8  
     9  Feeds that are added to an oracle must implement the `gnorkle.Feed` interface. A feed can be thought of as a mechanism that takes in off-chain data and produces data to be consumed on-chain. The three important components of a feed are:
    10  - Tasks: a feed is composed of one or more `feed.Task` instances. These tasks ultimately instruct an agent what it needs to do to provide data to a feed for ingestion.
    11  - Ingester: a feed should have a `gnorkle.Ingester` instance that handles incoming data from agents. The ingester's role is to perform the correct action on the incoming data, like adding it to an aggregate or storing it for later evaluation.
    12  - Storage: a feed should have a `gnorkle.Storage` instance that manages the state of the data a feed publishes -- data to be consumed on-chain.
    13  
    14  A single oracle instance may be composed of many feeds, each of which has a unique ID.
    15  
    16  Here is an example of what differences may exist amongst feed implementations:
    17  - Static: a static feed is one that only needs to produce a value once. It ingests values and then publishes the result. Once a single value is published, the state of the feed becomes immutable.
    18  	- Example: a realm wants to integrate football match results for its users. It may embed a static oracle that allows it to publish the match results to the chain. A static feed is a good choice for this because the match results will never change.
    19  - Continuous: a continuous feed can accept and ingest data, continously adding and changing its own internal state based on the data received. It can then publish values on demand based on its current state.
    20  	- Example: a realm wants to provide a verifiable random function. It embeds an oracle that defines tasks and whitelists a select group of trusted agents to provide data that gets combined to produce a random value. A continuous feed is a good choice for this because data may be accepted continously and there is no single static result.
    21  - Periodic:  periodic feed may give all whitelisted agents the opportunity to send data for ingestion within a bounded period of time. After this window closes, the results can be committed and a value is pubished. The process then begins again for the next period.
    22  	- Example: a realm wants to provide weather information to its users. It may choose a group of trusted agents to publish weather data for each half hour interval. This periodic feed can finalize the weather data for each postal code at the end of each interval using the aggregation function defined by the owner of the oracle.
    23  
    24  The only feed currently implemented is the `feeds/static.Feed` type.
    25  
    26  ## Tasks
    27  
    28  It's not hard to be a task -- just implement the one method `feed.Task` interface. On-chain task definitions should not do anything other than store data and be able to marshal that data to JSON. Of course, it is also useful if the task marshal's a `type` field as part of the JSON object so an agent is able to know what type of task it is dealing with and what data to expect in the payload.
    29  
    30  ### Example use case
    31  Imagine there is a public API out there. The oracle defines a task with a type of `HTTPGET`. The agent interacting with the oracle knows how to extract the task's type and extract the rest of the data to complete the task
    32  ```go
    33  type apiGet struct {
    34  	taskType   string
    35  	url        string
    36  }
    37  ```
    38  So the data might look something like:
    39  ```json
    40  apiGet {
    41  	"task_type": "HTTPGET",
    42  	"url": "http://example.com/api/latest"
    43  }
    44  ```
    45  The agent can use this data to make the request and publish the results to the oracle. Tasks can have structures as complex or simple as is required.
    46  
    47  ## Ingesters
    48  
    49  An ingester's primary role is to receive data provided by agents and figure out what to do with it. Ingesters must implement the `gnorkle.Ingester` interface. There are currently two message function types that an ingester may want to handle, `message.FuncTypeIngest` and `message.FuncTypeCommit`. The former message type should result in the ingester accumulating data in its own data store, while the latter should use what it has in its data store to publish a feed value to the `gnorkle.Storage` instance provided to it.
    50  
    51  The only ingester currently implemented is the `ingesters/single.ValueIngester` type.
    52  
    53  ### Example use case
    54  The `Ingester` interface has two main methods that must be implemented, `Ingest` and `CommitValue`, as defined here:
    55  ```go
    56  type Ingester interface {
    57  	Type() ingester.Type
    58  	Ingest(value, providerAddress string) (canAutoCommit bool)
    59  	CommitValue(storage Storage, providerAddress string)
    60  }
    61  ```
    62  Consider an oracle that provides a price feed. This price feed would need an ingester capable of ingesting incoming values and producing a final result at the end of a given amount of time. So we may have something that looks like:
    63  ```go
    64  type MultiValueIngester struct {
    65  	agentAddresses []string
    66  	prices         []uint64
    67  }
    68  
    69  func (i *MultiValueIngester) Ingest(value, providerAddress string) bool {
    70  	price, err := strconv.ParseUint(value, 10, 64)
    71  	if err != nil {
    72  		panic("invalid value type")
    73  	}
    74  
    75  	i.agentAddresses = append(i.agentAddresses, providerAddress)
    76  	i.prices = append(i.prices, price)
    77  
    78  	// This value cannot be autocommitted to storage because the ingester expects
    79  	// multiple values and it doesn't know if this is the final value.
    80  	return false
    81  }
    82  
    83  func (i *MultiValueIngester) CommitValue(storage Storage, providerAddress string) {
    84  	priceAggregate := i.aggregatePrices()
    85  	storage.Put(priceAggregate)
    86  
    87  	// Reset to prepare for the next price period.
    88  	i.agentAddresses = []string{}
    89  	i.prices = []uint64{}
    90  }
    91  ```
    92  An ingester is highly customizable and should be used to do any necessary aggregations.
    93  
    94  ## Storage
    95  
    96  Storage types are responsible for storing values produced by a feed's ingester. A storage type must implement `gnorkle.Storage`. This type should be able add values to the storage, retrieve the latest value, and retrieve a set of historical values. It is probably a good idea to make the storage bounded.
    97  
    98  The only storage currently implemented is the `storage/simple.Storage` type.
    99  
   100  ### Example use case
   101  In most cases the storage implementation will be a key value store, so its use case is fairly generic. The `Put` method is used to store finalized data and the `GetLatest` method can be used by consumers of the data. `Storage` is an interface in order to allow for memory management throughout the lifetime of the oracle writing data to it. The `GetHistory` method can be used, if desired, to keep a fixed historical window of published data points.
   102  ```go
   103  type Storage interface {
   104  	Put(value string)
   105  	GetLatest() feed.Value
   106  	GetHistory() []feed.Value
   107  }
   108  ```
   109  
   110  ## Whitelists
   111  
   112  Whitelists are optional but they can be set on both the oracle instance and the feed levels. The absence of a whitelist definition indicates that ALL addresses should be considered to be whitelisted. Otherwise, the presence of a defined whitelist indicates the callers address MUST be in the whitelist in order for the request to succeed. A feed whitelist has precedence over the oracle instance's whitelist. If a feed has a whitelist and the caller is not on it, the call fails. If a feed doesn't have a whitelist but the instance does and the caller is not on it, the call fails. If neither have a whitelist, the call succeeds. The whitlist logic mostly lives in `gnorkle/whitelist.gno` while the only current `gnorkle.Whitelist` implementation is the `agent.Whitelist` type. The whitelist is not owned by the feeds they are associated with in order to not further pollute the `gnorkle.Feed` interface.