github.com/Jeffail/benthos/v3@v3.65.0/website/blog/2019-05-27-compiling-benthos-to-wasm.md (about)

     1  ---
     2  title: "Compiling Benthos to Web Assembly"
     3  author: "Ashley Jeffs"
     4  author_url: https://github.com/Jeffail
     5  author_image_url: /img/ash.jpg
     6  description: "Don't worry about why"
     7  keywords: [
     8  	"benthos",
     9  	"go",
    10  	"golang",
    11  	"web assembly",
    12  	"wasm",
    13  	"gowasm",
    14  ]
    15  tags: [ "Benthos Lab" ]
    16  ---
    17  
    18  Web assembly won't fix seasons 7 and 8, but it's still pretty cool. At a
    19  [Meltwater hackathon](https://underthehood.meltwater.com/blog/2019/06/17/benthos-lab-a-case-study-of-hackathon-innovation/) I had a project in mind (details soon to
    20  follow) that would benefit hugely from Benthos running directly in the browser.
    21  I therefore set out to compile it in wasm, this is my short and sweet journey.
    22  
    23  <!--truncate-->
    24  
    25  ## The Build
    26  
    27  The first thing I did and the first thing you ought to do if you are targeting
    28  wasm yourself is skim through [this section of the Go wiki][wasm-go-wiki].
    29  
    30  In short, I wrote a Go file:
    31  
    32  ``` go
    33  package main
    34  
    35  import (
    36  	"syscall/js"
    37  
    38  	"github.com/Jeffail/benthos/lib/config"
    39  	"gopkg.in/yaml.v3"
    40  )
    41  
    42  func normalise(this js.Value, args []js.Value) interface{} {
    43  	var configStr string
    44  	if len(args) > 0 {
    45  		configStr = args[0].String()
    46  	}
    47  
    48  	conf := config.New()
    49  
    50  	// Ignoring errors for brevity
    51  	yaml.Unmarshal([]byte(configStr), &conf)
    52  
    53  	sanit, _ := conf.Sanitised()
    54  	sanitBytes, _ := yaml.Marshal(sanit)
    55  
    56  	return string(sanitBytes)
    57  }
    58  
    59  func main() {
    60  	c := make(chan struct{}, 0)
    61  	js.Global().Set("benthosNormaliseConfig", js.FuncOf(normalise))
    62  	<-c
    63  }
    64  ```
    65  
    66  And compiled it:
    67  
    68  ``` sh
    69  GOOS=js GOARCH=wasm go build -o main.wasm
    70  ```
    71  
    72  I was pretty sure that this would be the end of the road for me. Benthos uses a
    73  vast swathe of dependencies for its various connectors and so I was sure that I
    74  would be immobilised with errors. However, to my surprise there were only three
    75  (formatted for brevity):
    76  
    77  ``` text
    78  lib/util/disk/check.go:29:11: undefined: syscall.Statfs_t
    79  github.com/edsrzf/mmap-go@v1.0.0/mmap.go:77:9: undefined: mmap
    80  github.com/lib/pq@v1.0.0/conn.go:321:13: undefined: userCurrent
    81  ```
    82  
    83  Which involved some calls for a buffer implementation using a memory-mapped file
    84  library and the PostgreSQL driver for the SQL package. The errors themselves are
    85  basically "this thing doesn't exist in Web Assembly", which usually means the
    86  library has a feature behind build constraints but doesn't support wasm yet.
    87  
    88  The solution for these problems in my case was as simple as to not to do the
    89  call, and perhaps document that the feature doesn't work with a wasm build.
    90  
    91  Obviously, we only want to disable these calls specifically when targeting wasm.
    92  In Go that's easy, stick a cheeky
    93  [build constraint on there][go-build-constraint]. Here's the actual commit:
    94  [9903b3d5d8519fcf7ecbce94c336e7f054a75942][wasm-commit], note that you can't
    95  just constrain the feature, you also need to add an empty stub that has the
    96  opposite constraint in order to satisfy your build.
    97  
    98  ## Executing Go From JavaScript
    99  
   100  The [Go Wiki][wasm-go-wiki] shows you how to actually execute your wasm build
   101  and I won't repeat it here, but I followed the steps and it was pretty straight
   102  forward.
   103  
   104  There was, however, one issue I came across. Some functions that I was calling
   105  from JavaScript were causing my wasm runtime to panic and stop. The functions
   106  all had channel blocking in common, something like this:
   107  
   108  ``` go
   109  func ashHasACoolBlog(this js.Value, args []js.Value) interface{} {
   110  	someChan <- args[0].String()
   111  	return <-someOtherChanIHateNamingThings
   112  }
   113  ```
   114  
   115  The function would sometimes execute successfully. Other times, specifically for
   116  longer running calls, I would get a deadlock panic:
   117  
   118  ``` text
   119  fatal error: all goroutines are asleep - deadlock! wasm_exec.js:47:6
   120  wasm_exec.js:47:6
   121  goroutine 1 [chan receive]: wasm_exec.js:47:6
   122  main.main() wasm_exec.js:47:6
   123  	/home/ash/tmp/wasm/main.go:20 +0x7
   124  ```
   125  
   126  Which was odd as they would be occasions where I would not expect a real
   127  deadlock. I then found the relevant docs in the [`syscall/js`][syscall-js-func]
   128  package:
   129  
   130  > Blocking operations in the wrapped function will block the event loop. As a
   131  > consequence, if one wrapped function blocks, other wrapped funcs will not be
   132  > processed. A blocking function should therefore explicitly start a new
   133  > goroutine.
   134  
   135  The consequences of blocking sound pretty harmless here, but in reality it
   136  seemed to be the cause of my deadlock crash. I assume the odd error message is a
   137  result of some nuanced mechanics within the wasm runtime.
   138  
   139  I didn't investigate this crash any further as I was a lazy idiot back in those
   140  dark days. I simply stopped writing blocking functions, and instead spawned
   141  goroutines everywhere like they were losers at a Nickelback concert:
   142  
   143  ``` go
   144  func iJustWantToClarify(this js.Value, args []js.Value) interface{} {
   145  	go func() {
   146  		someChan <- args[0].String()
   147  		otherThing := <-someOtherChanIHateNamingThings
   148  
   149  		js.Global().Get("thatActually").Set(
   150  			"textContent",
   151  			"I quite enjoy and respect Knickelback as artists... " + otherThing,
   152  		)
   153  	}()
   154  	return nil
   155  }
   156  ```
   157  
   158  ## Other Issues
   159  
   160  There weren't any. 
   161  
   162  ## Final Words
   163  
   164  It took a day for me to get a working application together and soon I'll be
   165  blogging about the resulting product. Web assembly with Go is dope.
   166  
   167  Kudos to both the W3C and the Go team for taking their time to build something
   168  to completion without rushing the conclusion. Yes, I'm still bitter about Game
   169  of Thrones.
   170  
   171  [meltwater]: https://underthehood.meltwater.com/blog/2019/06/17/benthos-lab-a-case-study-of-hackathon-innovation/
   172  [Benthos]: https://www.benthos.dev/
   173  [wasm-go-wiki]: https://github.com/golang/go/wiki/WebAssembly
   174  [syscall-js-func]: https://godoc.org/syscall/js#Func
   175  [go-build-constraint]: https://golang.org/pkg/go/build/#hdr-Build_Constraints
   176  [wasm-commit]: https://github.com/Jeffail/benthos/commit/9903b3d5d8519fcf7ecbce94c336e7f054a75942#diff-146b6fd87106d7f70f56facf7b1e7d98