github.com/apache/beam/sdks/v2@v2.48.2/go/examples/wasm/README.md (about)

     1  <!--
     2      Licensed to the Apache Software Foundation (ASF) under one
     3      or more contributor license agreements.  See the NOTICE file
     4      distributed with this work for additional information
     5      regarding copyright ownership.  The ASF licenses this file
     6      to you under the Apache License, Version 2.0 (the
     7      "License"); you may not use this file except in compliance
     8      with the License.  You may obtain a copy of the License at
     9  
    10        http://www.apache.org/licenses/LICENSE-2.0
    11  
    12      Unless required by applicable law or agreed to in writing,
    13      software distributed under the License is distributed on an
    14      "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    15      KIND, either express or implied.  See the License for the
    16      specific language governing permissions and limitations
    17      under the License.
    18  -->
    19  # Overview
    20  
    21  wasm is an **_EXPERIMENTAL_** simple example that loads and executes a wasm file function.
    22  greet.wasm, Cargo.toml and greet.rs were copied from the example provided by the wazero library:
    23  https://github.com/tetratelabs/wazero/blob/v1.0.0-pre.3/examples/allocation/rust/greet.go
    24  
    25  # Usage
    26  
    27  To run this example in various runners, the following assumes:
    28  
    29  ```
    30  OUTPUT=<path to output i.e. /tmp/example/output>
    31  git clone https://github.com/apache/beam
    32  BEAM_HOME=$(pwd)/beam
    33  cd $BEAM_HOME/sdks
    34  ```
    35  
    36  ## Local Runner execution
    37  
    38  To execute this example on the local runner:
    39  
    40  ```shell
    41  go run ./go/examples/wasm --output=$OUTPUT
    42  ```
    43  
    44  ## Flink Portable Runner
    45  
    46  The following describes how to execute this example on the flink portable runner.
    47  
    48  ### REQUIREMENTS:
    49  
    50  - [Docker](https://docker.io); MacOS users may consider alternative: https://github.com/abiosoft/colima
    51  - Google Cloud Storage (https://cloud.google.com/storage) or S3 (https://aws.amazon.com/s3/) bucket;
    52  NOTE this example was only tested on Google Cloud Storage
    53  
    54  #### 0. Set OUTPUT to a Cloud storage bucket path
    55  
    56  The example below shows Google Cloud Storage.
    57  ```
    58  OUTPUT=gs://<my bucket>/greet
    59  ```
    60  
    61  #### 1. Find the latest flink runner version
    62  
    63  ```shell
    64  cd $BEAM_HOME
    65  ./gradlew :runners:flink:properties --property flink_versions
    66  ```
    67  
    68  Expected output should include the following, from which you acquire the latest flink runner version.
    69  
    70  ```shell
    71  'flink_versions: 1.12,1.13,1.14,1.15,1.16'
    72  ```
    73  
    74  #### 2. Set to the latest flink runner version i.e. 1.16
    75  
    76  ```shell
    77  FLINK_VERSION=1.16
    78  ```
    79  
    80  #### 3. In a separate terminal, start the flink runner (It should take a few minutes on the first execution)
    81  ```shell
    82  cd $BEAM_HOME
    83  ./gradlew :runners:flink:$FLINK_VERSION:job-server:runShadow
    84  ```
    85  
    86  Note the JobService host and port from the output, similar to:
    87  
    88  ```shell
    89  INFO: JobService started on localhost:8099
    90  ```
    91  
    92  #### 4. Set the JOB_SERVICE variable from the aforementioned output
    93  
    94  ```shell
    95  JOB_SERVICE=localhost:8099
    96  ```
    97  
    98  #### 5. Execute this example using the portable flink runner
    99  
   100  ```shell
   101  cd $BEAM_HOME/sdks
   102  go run ./go/examples/wasm --runner=universal --endpoint=$JOB_SERVICE --output=$OUTPUT --environment_config=apache/beam_go_sdk:latest
   103  ```
   104  
   105  ## Dataflow Runner
   106  
   107  The following describes how to execute this example on Dataflow.
   108  
   109  ### REQUIREMENTS:
   110  
   111  - Google Cloud Storage (https://cloud.google.com/storage) bucket
   112  
   113  #### 1. Set OUTPUT to a Cloud storage bucket path
   114  
   115  The example below shows Google Cloud Storage.
   116  
   117  ```shell
   118  OUTPUT=gs://<my bucket>/greet
   119  ```
   120  
   121  #### 2. Set additional variables
   122  
   123  ```shell
   124  PROJECT=<project id>
   125  REGION=<region>
   126  STAGING=gs://<my bucket>/staging
   127  NETWORK=<network>
   128  SUBNETWORK=regions/$REGION/subnetworks/<subnetwork>
   129  ```
   130  
   131  #### 3. Execute this example using Dataflow
   132  
   133  ```
   134  cd $BEAM_HOME/sdks
   135  go run ./go/examples/wasm --runner=dataflow --output=$OUTPUT --environment_config=apache/beam_go_sdk:latest \
   136      --project=$PROJECT --region=$REGION --network=$NETWORK --subnetwork=$SUBNETWORK --staging_location=$STAGING
   137  ```