github.com/tetratelabs/proxy-wasm-go-sdk@v0.23.1-0.20240517021853-021aa9cf78e8/doc/OVERVIEW.md (about)

     1  # Introduction
     2  
     3  Proxy-Wasm project's key objective is to bring the extensibility to network proxies with any programming language in a flexible way.
     4  
     5  This Proxy-Wasm Go SDK is the SDK for extending network proxies (e.g. Envoyproxy) on top of the [Proxy-Wasm ABI specification](https://github.com/proxy-wasm/spec) with Go programming language, and the Proxy-Wasm ABI defines the interface between network proxies and Wasm virtual machines running inside network proxies.
     6  
     7  With this SDK, everyone can produce Proxy-Wasm spec compatible Wasm binaries easily *without knowing Proxy-Wasm ABI specification* which is low-level and overwhelming to anyone who doesn't have the expertise in this field. Instead, developers rely on the Go API of this SDK to perform what they want to do to extend network proxies.
     8  
     9  This document explains the things you should know when writing programs with this SDK for your custom plugins.
    10  
    11  **Note that this document assumes that you are using Envoyproxy, and relies on its implementation detail**. Therefore some statement may not apply to other network proxies such as [mosn](https://github.com/mosn).
    12  
    13  # TinyGo vs the official Go compiler
    14  
    15  This SDK relies on the TinyGo, a compiler implementation of Go programming language specification. So first of all, we answer the question "Why not the official Go?".
    16  
    17  There are several reasons why we cannot use the official Go compiler. Tl;dr is that as of this writing, the official compiler cannot produce Wasm binary which can run outside web browsers, and therefore cannot produce Proxy-Wasm compatible binaries.
    18  
    19  For those who are insterested in the detail, please refer to the related issues in the Go repository: 
    20  - https://github.com/golang/go/issues/25612
    21  - https://github.com/golang/go/issues/31105
    22  - https://github.com/golang/go/issues/38248
    23  
    24  # Wasm VM, Plugin and Envoy configuration
    25  
    26  ## Terminology
    27  
    28  *Wasm virtual machine (Wasm VM)* or simply *VM* means instances of loaded programs. In Envoy, VMs are usually created in *each thread* and isolated from each other. Therefore, your program will be copied up to the number of threads created by Envoy, and loaded on each of these virtual machines.
    29  
    30  *Plugin* means the basic unit of your configuration for extending your network proxies. Proxy-Wasm specification allows for having **multiple plugins in a single VM**. In other words, a VM may be used by multiple plugins of network proxies. With this SDK, there are three types of plugins that you can configure in Envoy; *Http Filter*, *Network(Tcp) Filter*, and *Wasm Service*. Given that, you can write programs which can be run as, for example, Network Filter and Http Filter at the same time.
    31  
    32  *Http Filter* is a type of plugins to handle Http protocol such as operating on Http request headers, body, trailers, etc. It uses a VM in worker threads which handle traffic.
    33  
    34  *Network Filter* is a type of plugins to handle Tcp protocol such as operating on Tcp data frame, connection establishment, etc. It uses a VM in worker threads which handle traffic.
    35  
    36  *Wasm Service* is a type of plugins running in a singleton VM (i.e. only one instance exists in the Envoy main thread). It is mainly used for doing some extra work in parallel to Network or Http Filters such as aggregating metrics, logs, etc. Sometimes, such a singleton VM itself is also called *Wasm Service*.
    37  
    38  <img src="./images/terminology.png" width="1000">
    39  
    40  ## Envoy configuration
    41  
    42  Among all the types of plugins, we share the configuration for Envoy something like
    43  
    44  ```yaml
    45  vm_config:
    46    vm_id: "foo"
    47    runtime: "envoy.wasm.runtime.v8"
    48    configuration:
    49      "@type": type.googleapis.com/google.protobuf.StringValue
    50      value: '{"my-vm-env": "dev"}'
    51    code:
    52      local:
    53        filename: "example.wasm"
    54  configuration:
    55    "@type": type.googleapis.com/google.protobuf.StringValue
    56    value: '{"my-plugin-config": "bar"}'
    57  ```
    58  
    59  where
    60  
    61  | Field | Description |
    62  | --- | --- |
    63  | `vm_config` | configures specific Wasm VM on which this plugin runs |
    64  | `vm_config.vm_id` | used for semantic isolation towards Cross-VM communications. Please refer to [Cross-VM communications](#cross-vm-communications) section for detail.|
    65  | `vm_config.runtime` | specifies the Wasm runtime type. Usually set to `envoy.wasm.runtime.v8`. |
    66  | `vm_config.configuration` | arbitray configuration data used for setting up the VM. |
    67  | `vm_config.code` | location of a Wasm binary |
    68  | `configuration` | corresponds to each *Plugin* instance (which we call `PluginContext` explained below) inside the Wasm VM. |
    69  
    70  The important thing is that **giving exactly the same `vm_config` field for multiple plugins ends up sharing one Wasm VM among them**. 
    71  That means you can use a single Wasm VM for multiple Http filters, or maybe Http and Tcp filters per thread (See [example config](#sharing-one-vm-among-multiple-plugins-per-thread) for detail). This is useful in terms of memory/cpu resource efficiency, startup latency, etc.
    72  
    73  The full API definition is [here](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/wasm/v3/wasm.proto#envoy-v3-api-msg-extensions-wasm-v3-pluginconfig) and this is what we call *plugin config* here and elsewhere.
    74  
    75  Now here's some example configurations in Envoy for each plugin type. Note that how your Wasm VM is created by Envoy depends on these types. 
    76  
    77  ### Http Filter
    78  
    79  If a plugin config is given at `envoy.filter.http.wasm`, you can run your program as *Http Filter* plugin so that it can operate on Http events.
    80  
    81  ```yaml
    82  http_filters:
    83  - name: envoy.filters.http.wasm
    84    typed_config:
    85      "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
    86      config:
    87        vm_config: { ... }
    88        # ... plugin config follows
    89  - name: envoy.filters.http.router
    90  ```
    91  
    92  In this case, Wasm VMs are created on *each worker thread* in Envoy, and each VM is responsible for operating on each Http streams on a listener handled by a corresponding worker thread. Note that the way VMs and Plugins get created is exactly the same as [*Network Filter*](#network-filter), and the only difference is that *Plugins* only operate on Http streams rather than Tcp streams.
    93  
    94  See [example.yaml](../examples/http_headers/envoy.yaml) for a full example.
    95  
    96  ### Network Filter
    97  
    98  If a plugin config is given at `envoy.filter.network.wasm`, you can run your program as *Network Filter* plugin so that it can operate on Tcp events.
    99  
   100  ```yaml
   101  filter_chains:
   102  - filters:
   103      - name: envoy.filters.network.wasm
   104        typed_config:
   105          "@type": type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm
   106          config:
   107            vm_config: { ... }
   108            # ... plugin config follows
   109      - name: envoy.tcp_proxy
   110  ```
   111  
   112  In this case, Wasm VMs are created on *each worker thread* in Envoy, and each VM is responsible for operating on each Tcp streams on a listener handled by a corresponding worker thread. Note that the way VMs and Plugins get created is exactly the same as [*Http Filter*](#http-filter), and the only difference is that *Plugins* only operate on Tcp streams rather than Http streams.
   113  
   114  See [example.yaml](../examples/network/envoy.yaml) for a full example.
   115  
   116  ### Wasm Service
   117  
   118  If a plugin config is given at `envoy.bootstrap.wasm`, you can run your program as *Wasm Service* plugin.
   119  
   120  ```yaml
   121  bootstrap_extensions:
   122  - name: envoy.bootstrap.wasm
   123    typed_config:
   124      "@type": type.googleapis.com/envoy.extensions.wasm.v3.WasmService
   125      singleton: true
   126      config:
   127        vm_config: { ... }
   128        # ... plugin config follows
   129  ```
   130  
   131  The top `singleton` field is normally set `true`. In this way, only one VM for this configuration exists across all the threads of Envoy process, and runs on the Envoy's main thread so that it won't block any worker threads.
   132  
   133  See [example.yaml](../examples/shared_queue/envoy.yaml) for a full example.
   134  
   135  ### Sharing one VM among multiple plugins per thread
   136  
   137  As we explained, we can share one VM across multiple plugins. Here's an example yaml for such configuration: 
   138  
   139  ```yaml
   140  static_resources:
   141    listeners:
   142      - name: http-header-operation
   143        address:
   144          socket_address:
   145            address: 0.0.0.0
   146            port_value: 18000
   147        filter_chains:
   148          - filters:
   149              - name: envoy.http_connection_manager
   150                typed_config:
   151                  # ....
   152                  http_filters:
   153                    - name: envoy.filters.http.wasm
   154                      typed_config:
   155                        "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
   156                        config:
   157                          configuration:
   158                            "@type": type.googleapis.com/google.protobuf.StringValue
   159                            value: "http-header-operation"
   160                          vm_config:
   161                            vm_id: "my-vm-id"
   162                            runtime: "envoy.wasm.runtime.v8"
   163                            configuration:
   164                              "@type": type.googleapis.com/google.protobuf.StringValue
   165                              value: "my-vm-configuration"
   166                            code:
   167                              local:
   168                                filename: "all-in-one.wasm"
   169                    - name: envoy.filters.http.router
   170  
   171      - name: http-body-operation
   172        address:
   173          socket_address:
   174            address: 0.0.0.0
   175            port_value: 18001
   176        filter_chains:
   177          - filters:
   178              - name: envoy.http_connection_manager
   179                typed_config:
   180                  # ....
   181                  http_filters:
   182                    - name: envoy.filters.http.wasm
   183                      typed_config:
   184                        "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
   185                        config:
   186                          configuration:
   187                            "@type": type.googleapis.com/google.protobuf.StringValue
   188                            value: "http-body-operation"
   189                          vm_config:
   190                            vm_id: "my-vm-id"
   191                            runtime: "envoy.wasm.runtime.v8"
   192                            configuration:
   193                              "@type": type.googleapis.com/google.protobuf.StringValue
   194                              value: "my-vm-configuration"
   195                            code:
   196                              local:
   197                                filename: "all-in-one.wasm"
   198                    - name: envoy.filters.http.router
   199  
   200      - name: tcp-total-data-size-counter
   201        address:
   202          socket_address:
   203            address: 0.0.0.0
   204            port_value: 18002
   205        filter_chains:
   206          - filters:
   207              - name: envoy.filters.network.wasm
   208                typed_config:
   209                  "@type": type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm
   210                  config:
   211                    configuration:
   212                      "@type": type.googleapis.com/google.protobuf.StringValue
   213                      value: "tcp-total-data-size-counter"
   214                      vm_config:
   215                        vm_id: "my-vm-id"
   216                        runtime: "envoy.wasm.runtime.v8"
   217                        configuration:
   218                          "@type": type.googleapis.com/google.protobuf.StringValue
   219                          value: "my-vm-configuration"
   220                        code:
   221                          local:
   222                            filename: "all-in-one.wasm"
   223              - name: envoy.tcp_proxy
   224                typed_config: # ...
   225  ```
   226  
   227  You see that `vm_config` fields are all the same on Http filter chains on 18000 and 18001 listeners plus a Network filter chain on 18002. That means one Wasm VM is used by multiple plugins in Envoy per worker thread in this case. In other words, all of `vm_config.vm_id`, `vm_config.runtime` `vm_config.configuration`, and `vm_config.code` must be same in order to reuse the same VMs.
   228  
   229  As a result, **Three** [`PluginContext`](#contexts) will be created per Wasm VM and each of them corresponds to each of the above filter configurations (the top `configuration` fields at 18000, 18001, and 18002 respectively).
   230  
   231  See [example.yaml](../examples/shared_queue/envoy.yaml) for a full example.
   232  
   233  # Proxy-Wasm Go SDK API
   234  
   235  So far we have explaind the concepts and *plugin configs*. Now we are ready to dive into the API of this SDK.
   236  
   237  ## *Contexts*
   238  
   239  *Contexts* are collection of interfaces in Proxy-Wasm Go SDK, and all of them are mapped to the concepts  explained above. They are defined in [types](../proxywasm/types) package, and developers are supposed to implement these interfaces in order to extend network proxies.
   240  
   241  There are four types of contexts: `VMContext`, `PluginContext`, `TcpContext` and `HttpContext`. Their relationship and how they are mapped to the concepts above can be described as the following diagram:
   242  
   243  ```
   244                      Wasm Virtual Machine
   245                        (.vm_config.code)
   246  ┌────────────────────────────────────────────────────────────────┐
   247  │  Your program (.vm_config.code)                TcpContext      │
   248  │          │                                  ╱ (Tcp stream)     │
   249  │          │ 1: 1                            ╱                   │
   250  │          │         1: N                   ╱ 1: N               │
   251  │      VMContext  ──────────  PluginContext                      │
   252  │                                (Plugin)   ╲ 1: N               │
   253  │                                            ╲                   │
   254  │                                             ╲  HttpContext     │
   255  │                                               (Http stream)    │
   256  └────────────────────────────────────────────────────────────────┘
   257  ```
   258  
   259  To summarize,
   260  
   261  1) `VMContext` corresponds to each `.vm_config.code`, and only one `VMContext` exists in each VM.
   262  2) `VMContext` is the parent of *PluginContexts*, and is responsible for creating arbitrary number of `PluginContext`s.
   263  3) `PluginContext` corresponds to a *Plugin* instance. That means, a `PluginContext` corresponds to a *Http Filter* or *Network Filter* or maybe *Wasm Service*, configured via `.configuration` field in the *plugin config*.
   264  4) `PluginContext` is the parent of `TcpContext` and `HttpContext`, and is responsible for creating arbitrary number of these contexts when it is configured at *Http Filter* or *Network Filter*.
   265  5) `TcpContext` is responsible for handling each Tcp stream.
   266  6) `HttpContext` is responsible for handling each Http stream.
   267  
   268  So all you have to do is implement `VMContext` and `PluginContext`. And if you want to plug in to *Http Filter* or *Network Filter*, then implement `HttpContext` or `TcpContext` respectively.
   269  
   270  Let's look at some of these interfaces. First we see `VMContext` is defined as follows:
   271  
   272  ```go
   273  type VMContext interface {
   274  	// OnVMStart is called after the VM is created and main function is called.
   275  	// During this call, GetVMConfiguration hostcall is available and can be used to
   276  	// retrieve the configuration set at vm_config.configuration.
   277  	// This is mainly used for doing Wasm VM-wise initialization.
   278  	OnVMStart(vmConfigurationSize int) OnVMStartStatus
   279  
   280  	// NewPluginContext is used for creating PluginContext for each plugin configurations.
   281  	NewPluginContext(contextID uint32) PluginContext
   282  }
   283  ```
   284  
   285  As you expect, `VMContext` is responsible for creating `PluginContext` via `NewPluginContext` method. In addition, `OnVMStart` is called at the startup phase of the VM, and you can retrieve the value of `.vm_config.configuration` via `GetVMConfiguration` [hostcall API](#hostcall-api). This way you can do the VM-wise plugin-independent initialization and control the behavior of `VMContext`.
   286  
   287  Next is `PluginContext` and it it defined as (here we omit some of the methods for simplicity)
   288  
   289  ```go
   290  type PluginContext interface {
   291  	// OnPluginStart is called on all plugin contexts (after OnVmStart if this is the VM context).
   292  	// During this call, GetPluginConfiguration is available and can be used to
   293  	// retrieve the configuration set at config.configuration in envoy.yaml
   294  	OnPluginStart(pluginConfigurationSize int) OnPluginStartStatus
   295  
   296  	// The following functions are used for creating contexts on streams,
   297  	// and developers *must* implement either of them corresponding to
   298  	// extension points. For example, if you configure this plugin context is running
   299  	// at Http filters, then NewHttpContext must be implemented. Same goes for
   300  	// Tcp filters.
   301  	//
   302  	// NewTcpContext is used for creating TcpContext for each Tcp streams.
   303  	NewTcpContext(contextID uint32) TcpContext
   304  	// NewHttpContext is used for creating HttpContext for each Http streams.
   305  	NewHttpContext(contextID uint32) HttpContext
   306  }
   307  ```
   308  
   309  Just like `VMContext`, `PluginContext` has `OnPluginStart` method which is called on the plugin creation in network proxies. During that call, the top level `.configuratin` field's value in the *plugin config* can be retrieved via `GetPluginConfiguration` [hostcall API](#hostcall-api). This way developers can inform a `PluginContext` how it should behave, for example, specifying a `PluginContext` should behave as a Http Filter and which custom headers it should insert as a request headers, etc. 
   310  
   311  Also note that `PluginContext` has `NewTcpContext` and `NewHttpContext` methods which are called when creating these contexts in response to each Http or Tcp streams in network proxies.
   312  
   313  The definition of `HttpContext` and `TcpContext` is fairly straightforward so please refer to [context.go](../proxywasm/types/context.go) for detail.
   314  
   315  ## Hostcall API
   316  
   317  Hostcall API provides a variety of ways to interact with network proxies from your program, and it is defined at [hostcall.go](../proxywasm/hostcall.go) in [proxywasm](../proxywasm) package. For example, `GetHttpRequestHeaders` API can be used for accessing Http request headers by `HttpContext`. The other example is that `LogInfo` API and it can be used for emitting strings as logs in Envoy.
   318  
   319  Please refer to [hostcall.go](../proxywasm/hostcall.go) for all the available hostcalls, and the documentation is given at the function definitions.
   320  
   321  ## Entrypoint
   322  
   323  When Envoy creates VMs, it calls `main` function of your program at startup phase before it tries to create `VMContext` inside VMs. Therefore you must pass your own implementation of `VMContext` in `main` function.
   324  
   325  [proxywasm](../proxywasm) package's `SetVMContext` function is the entrypoint used for that purpose. That being said, your `main` function should look like the following:
   326  
   327  ```go
   328  func main() {
   329  	proxywasm.SetVMContext(&myVMContext{})
   330  }
   331  
   332  type myVMContext struct { .... }
   333  
   334  var _ types.VMContext = &myVMContext{}
   335  
   336  // Implementations follow...
   337  ```
   338  
   339  # Cross-VM communications
   340  
   341  Given that VMs are created in the thread-local way, sometimes we may want to communicate with other VMs. For example, aggregating data or stats, caching data, etc.
   342  
   343  There are two concepts for Cross-VM communications called *Shared Data* and *Shared Queue*.
   344  
   345  We also recommend you watch [this talk](https://www.youtube.com/watch?v=XdWmm_mtVXI&t=1168s) for introduction.
   346  
   347  ## *Shared Data (Shared KVS)*
   348  
   349  What if you want to have global request counters across all the Wasm VMs running in multiple worker threads? Or what if you want to cache some data that should be used by all of your Wasm VMs? Then *Shared Data* or equivalently *Shared KVS* will come into play.
   350  
   351  *Shared Data* is basically a key-value store that is shared across all the VMs (i.e. cross-VM or cross-threads). A shared-data KVS is created per [`vm_id`](#envoy-configuration) specified in the `vm_config`. That means you can share a key-value store across all Wasm VMs not necessarily with the same binary (`vm_config.code`). The only requirement is having the same `vm_id`.
   352  
   353  <img src="./images/shared_data.png" width="600">
   354  
   355  In the diagram above, you see that VMs with "vm_id=foo" share the same shared data storage even though they have different binary (hello.wasm and bye.wasm).
   356  
   357  Here's the shared data related API of this Go SDK in [hostcall.go](../proxywasm/hostcall.go):
   358  
   359  ```go
   360  // GetSharedData is used for retrieving the value for given "key".
   361  // Returned "cas" is be used for SetSharedData on that key for
   362  // thread-safe updates.
   363  func GetSharedData(key string) (value []byte, cas uint32, err error)
   364  
   365  // SetSharedData is used for setting key-value pairs in the shared data storage
   366  // which is defined per "vm_config.vm_id" in the hosts.
   367  //
   368  // ErrorStatusCasMismatch will be returned when a given CAS value is mismatched
   369  // with the current value. That indicates that other Wasm VMs has already succeeded
   370  // to set a value on the same key and the current CAS for the key is incremented.
   371  // Having retry logic in the face of this error is recommended.
   372  //
   373  // Setting cas = 0 will never return ErrorStatusCasMismatch and always succeed, but
   374  // it is not thread-safe, i.e. maybe another VM has already set the value
   375  // and the value you see is already different from the one stored by the time
   376  // when you call this function.
   377  func SetSharedData(key string, value []byte, cas uint32) error
   378  ```
   379  
   380  
   381  The API is straightforward, but the important part is its thread-safety and cross-VM-safety with "cas" or [Compare-And-Swap](https://en.wikipedia.org/wiki/Compare-and-swap) value.
   382  
   383  Please refer to [an example](../examples/shared_data) for demonstration.
   384  
   385  ## *Shared Queue*
   386  
   387  What if you want to aggregate metrics across all the Wasm VMs in parallel to request/response processing? Or what if you want to push some cross-VM aggregated information to a remote server? Now *Shared Queue* is here for you.
   388  
   389  *Shared Queue* is a FIFO(First-In-First-Out) queue created for a pair of `vm_id` and the name of the queue. And a *queue id* is assigned uniquely to the pair (vm_id, name) which is used for enqueue/dequeue operations.
   390  
   391  As you expect, the operations such as "enqueue" and "dequeue" have thread-safety and cross-VM-safety. Let's look at the *Shared Queue* related API in [hostcall.go](../proxywasm/hostcall.go):
   392  
   393  ```golang
   394  // DequeueSharedQueue dequeues an data from the shared queue of the given queueID.
   395  // In order to get queue id for a target queue, use "ResolveSharedQueue" first.
   396  func DequeueSharedQueue(queueID uint32) ([]byte, error)
   397  
   398  // RegisterSharedQueue registers the shared queue on this plugin context.
   399  // "Register" means that OnQueueReady is called for this plugin context whenever a new item is enqueued on that queueID.
   400  // Only available for types.PluginContext. The returned queueID can be used for Enqueue/DequeueSharedQueue.
   401  // Note that "name" must be unique across all Wasm VMs which share a same "vm_id".
   402  // That means you can use "vm_id" can be used for separating shared queue namespace.
   403  //
   404  // Only after RegisterSharedQueue is called, ResolveSharedQueue("this vm_id", "name") succeeds
   405  // to retrive queueID by other VMs.
   406  func RegisterSharedQueue(name string) (queueID uint32, err error)
   407  
   408  // EnqueueSharedQueue enqueues an data to the shared queue of the given queueID.
   409  // In order to get queue id for a target queue, use "ResolveSharedQueue" first.
   410  func EnqueueSharedQueue(queueID uint32, data []byte) error
   411  
   412  // ResolveSharedQueue acquires the queueID for the given vm_id and queue name.
   413  // The returned queueID can be used for Enqueue/DequeueSharedQueue.
   414  func ResolveSharedQueue(vmID, queueName string) (queueID uint32, err error)
   415  ```
   416  
   417  Basically `RegisterSharedQueue` and `DequeueSharedQueue` are used by "consumer" of the queue 
   418  while `ResolveSharedQueue` and `EnqueueSharedQueue` are for "producer" of queue items. Note that
   419  
   420  - `RegisterSharedQueue` is used for creating a shared queue for `name` and `vm_id` of the caller. That means if you want to use a queue, then this must be called by a VM beforehand.  This can be called by `PluginContext`, and therefore we can think of "comsumers" = `PluginContext`s.
   421  - `ResolveSharedQueue` is used for getting the *queue id* for `name` and `vm_id`. Usually this is used by VMs that doesn't call `ResolveSharedQueue` but rather are supposed to enqueue items. This is for "producer".
   422  
   423  and both of these calls return a queue id, and it is used for `DequeueSharedQueue` and `EnqueueSharedQueue`.
   424  
   425  However, from the consumer's point of view, how can a consumer (= `PluginContext`) be notified when a queue is enqueued with an item? This is why we have the `OnQueueReady(queueID uint32)` interface in `PluginContext`. This method is called whenever an item is enqueued in a queue registered by that `PluginContext`.
   426  
   427  Also it is highly recommended that shared queues should be created by a singleton *Wasm Service*, i.e. on the Envoy's main thread. Otherwise `OnQueueReady` is called on worker threads which blocks their processing of Http or Tcp streams.
   428  
   429  The following diagram is an illustrative usage of shared queues:
   430  
   431  <img src="./images/shared_queue.png" width="800">
   432  
   433  `my-singleton.wasm` is loaded as a singleton VM with `vm_id=foo` where two *Wasm Service* are created which correspond to `PluginContext 1` and `PluginContext 2` in the VM. Each of these plugin contexts calls `RegisterQueue` with "http" and "tcp" names, and that results in creating two corresponding queues. On the other hand, in the worker threads, two types of Wasm VMs are created per thread. They are processing Tcp streams and Http streams, and enqueue some data into the corresponding queues respectively. As we explained above, enqueueing data into a queue ends up calling the `OnQueueReady` method of a `PluginContext` which called `RegisterQueue` for that queue. In this example, enqueueing data into the queue with queue id=2 invokes `OnQueueReady(2)` of `PluginContext 2` in the singleton VM.
   434  
   435  Please refer to [an example](../examples/shared_queue) for demonstration.
   436  
   437  # Unit tests with testing framework
   438  
   439  This SDK contains the testing framework for unit testing Proxy-Wasm programs without actually running network proxies and with the official Go test toolchain. [proxytest](../proxywasm/proxytest) package implements the Envoy proxy emulator. That is, you can run tests just like you do when writing native programs:
   440  
   441  ```
   442  go test ./...
   443  ```
   444  
   445  Please refer to `main_test.go` files under [examples](../examples) directory for demonstrations.
   446  
   447  # Limitations and Considerations
   448  
   449  Here's what users should know when writing plugins with Proxy-Wasm Go SDK and Proxy-Wasm in general.
   450  
   451  ## Some of existing libraries not available
   452  
   453  Some of existing libraries are not available (importable but runtime panic / non-importable). There are several reasons for this:
   454  1. TinyGo's WASI target does not support some of syscall.
   455  2. TinyGo does not implement all of reflect package.
   456  3. [Proxy-Wasm C++ host](https://github.com/proxy-wasm/proxy-wasm-cpp-host) has not supported some of WASI APIs yet 
   457  4. Some language features are not available in TinyGo or Proxy-Wasm: examples include `recover` and `goroutine`.
   458  
   459  These issues will be mitigated as TinyGo and Proxy-Wasm evolve.
   460  
   461  ## Performance overhead due to Garbage Collection
   462  
   463  There's performance overhead of using Go/TinyGo due to GC, although, optimistically speaking, we could say that the overhead of GC is small enough compared to the other operations in the proxy.
   464  
   465  Internally, `runtime.GC` is called whenever the heap runs out (see 
   466  [1](https://tinygo.org/lang-support/#garbage-collection),
   467  [2](https://github.com/tinygo-org/tinygo/blob/v0.14.1/src/runtime/gc_conservative.go#L218-L239)) in TinyGo.
   468  
   469  TinyGo allows us to disable GC, but we cannot do that since internally we need to use maps (implicitly causes allocation) for saving the Virtual Machine's state. Theoretically, we can implement our own GC algorithms tailored for proxy-wasm through `alloc(uintptr)` [interface](https://github.com/tinygo-org/tinygo/blob/v0.14.1/src/runtime/gc_none.go#L13) with `-gc=none` option. This is a future TODO.
   470  
   471  ## `recover` not implemented
   472  
   473  `recover` is not implemented (https://github.com/tinygo-org/tinygo/issues/891) in TinyGo, and there's no way to prevent the Wasm virtual machine from aborting. Also that means that codes rely on `recover` won't work as expected.
   474  
   475  ## Goroutine support
   476  
   477  In TinyGo, Goroutine is implemented through LLVM's coroutine (see [this blog post](https://aykevl.nl/2019/02/tinygo-goroutines)).
   478  
   479  In Envoy, Wasm modules are run in the event driven manner, and therefore the "scheduler" is not executed once the main function exits.
   480  That means you cannot have the expected behavior of Goroutine as in ordinary host environments.
   481  
   482  The question "How to deal with Goroutine in a thread local Wasm VM executed in the event drive manner" has yet to be answered.
   483  
   484  We strongly recommend that you implement the `OnTick` function for any asynchronous task instead of using Goroutine.