github.com/eh-steve/goloader@v0.0.0-20240111193454-90ff3cfdae39/README.md (about)

     1  # Goloader/JIT Compiler for Go
     2  
     3  [![Build Status](https://github.com/eh-steve/goloader/actions/workflows/go.yml/badge.svg)](https://github.com/eh-steve/goloader/actions/workflows/go.yml)
     4  
     5  The `goloader/jit` package can compile and load Go code from text, file, folder or remote package (including code with
     6  package imports).
     7  
     8  It automatically resolves package dependencies recursively, and provides a type safe way of interacting with the built
     9  functions.
    10  
    11  Forked from [dearplain](https://github.com/dearplain/goloader) and [pkujhd](https://github.com/pkujhd/goloader).
    12  
    13  # Usage
    14  
    15  ## Build
    16  
    17  **Make sure you're using go >= 1.18.**
    18  
    19  First, execute the following command. This is because Goloader relies on the internal package, which is forbidden by the
    20  Go compiler.
    21  
    22  ```
    23  cp -r $GOROOT/src/cmd/internal $GOROOT/src/cmd/objfile
    24  ```
    25  
    26  ## Go compiler patch
    27  
    28  To allow the loader to know the types of exported functions, this package will attempt to patch the Go compiler (gc) to
    29  emit these if not already patched.
    30  
    31  The effect of the patch can be found in [`jit/gc.patch`](https://github.com/eh-steve/goloader/blob/master/jit/gc.patch).
    32  
    33  ```bash
    34  go install github.com/eh-steve/goloader/jit/patchgc@latest
    35  # You may need to run patchgc as sudo if your $GOROOT is owned by root
    36  # (alternatively `chown -R $USER:$USER $GOROOT`)
    37  patchgc
    38  ```
    39  
    40  ## Example Usage
    41  
    42  ```go
    43  package main
    44  
    45  import (
    46  	"fmt"
    47  	"github.com/eh-steve/goloader/jit"
    48  )
    49  
    50  func main() {
    51  	conf := jit.BuildConfig{
    52  		KeepTempFiles:   false,          // Files are copied/written to a temp dir to ensure it is writable. This retains the temporary copies
    53  		ExtraBuildFlags: []string{"-x"}, // Flags passed to go build command
    54  		BuildEnv:        nil,            // Env vars to set for go build toolchain
    55  		TmpDir:          "",             // To control where temporary files are copied
    56  		DebugLog:        true,           //
    57  	}
    58  
    59  	loadable, err := jit.BuildGoFiles(conf, "./path/to/file1.go", "/path/to/file2.go")
    60  	if err != nil {
    61  		panic(err)
    62  	}
    63  	// or
    64  	loadable, err = jit.BuildGoPackage(conf, "./path/to/package")
    65  	if err != nil {
    66  		panic(err)
    67  	}
    68  	// or
    69  	loadable, err = jit.BuildGoPackageRemote(conf, "github.com/some/package/v4", "latest")
    70  	if err != nil {
    71  		panic(err)
    72  	}
    73  	// or
    74  	loadable, err = jit.BuildGoText(conf, `
    75  package mypackage
    76  
    77  import "encoding/json"
    78  
    79  func MyFunc(input []byte) (interface{}, error) {
    80  	var output interface{}
    81  	err := json.Unmarshal(input, &output)
    82  	return output, err
    83  }
    84  `)
    85  
    86  	if err != nil {
    87  		panic(err)
    88  	}
    89  
    90  	module, err := loadable.Load()
    91  	// module.SymbolsByPkg is a map[string]map[string]interface{} of all packages and their exported functions and global vars
    92  	symbols := module.SymbolsByPkg[loadable.ImportPath]
    93  	if err != nil {
    94  		panic(err)
    95  	}
    96  	defer func() {
    97  		err = module.Unload()
    98  		if err != nil {
    99  			panic(err)
   100  		}
   101  	}()
   102  	switch f := symbols["MyFunc"].(type) {
   103  	case func([]byte) (interface{}, error):
   104  		result, err := f([]byte(`{"k":"v"}`))
   105  		if err != nil {
   106  			panic(err)
   107  		}
   108  		fmt.Println(result)
   109  	default:
   110  		panic("Function signature was not what was expected")
   111  	}
   112  }
   113  
   114  ```
   115  
   116  ## How does it work?
   117  
   118  Goloader works like a linker, it relocates the addresses of symbols in an object file, generates runnable code, and then
   119  reuses the runtime functions and the type pointers of the loader where available.
   120  
   121  Goloader provides some information to the runtime and garbage collector of Go, which allows it to work correctly with
   122  them.
   123  
   124  Please note that Goloader is not a scripting engine. It reads the archives emitted from the Go compiler and makes them
   125  runnable. All features of Go are supported, and run just as fast and lightweight as native Go code.
   126  
   127  ## Comparison with `plugin`
   128  
   129  Plugin:
   130  
   131  * Can't load plugins not built with exact same versions of packages that host binary
   132    uses (`plugin was built with a different version of package`) - this makes them basically unusable in most large
   133    projects
   134  * Introduces dependency on `libdl`/CGo (and doesn't work on Windows)
   135  * Prevents linker deadcode elimination for unreachable methods (increases host binary size with unused methods)
   136  * Can't be unloaded/dynamically updated
   137  * Duplicates a lot of the go runtime (large binary sizes)
   138  
   139  Goloader:
   140  
   141  * Can build/load any packages (somewhat unsafely - it attempts to verify that types across JIT packages and host
   142    packages match, but doesn't do the same checks for function signatures)
   143  * Pure Go - no dependency on `libdl`/Cgo
   144  * Patches host itabs containing unreachable methods instead of preventing linker deadcode elimination
   145  * Can be unloaded, and objects from one version of a JIT package can be converted at runtime to those from another
   146    version, to allow dynamic adjustment of functions/methods without losing state
   147  * Reuses the runtime from the host binary (much smaller binaries)
   148  
   149  Goloader supports pprof tool (yes, you can see code loaded by Goloader in pprof), but does not (yet) support debugging
   150  with `delve`.
   151  
   152  ## OS/Arch Compatibility
   153  
   154  JIT compiler tested/passing on:
   155  
   156  | **OS/Arch**        | amd64/+CGo         | arm64/+CGo         | amd64/-CGo         | arm64/-CGo         |
   157  |--------------------|--------------------|--------------------|--------------------|--------------------|
   158  | Linux/go-1.20.3    | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   159  | Darwin/go-1.20.3   | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   160  | Windows/go-1.20.3  | :heavy_check_mark: | :interrobang:      | :heavy_check_mark: | :interrobang:      |
   161  | Linux/go-1.19.4    | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   162  | Darwin/go-1.19.4   | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   163  | Windows/go-1.19.4  | :heavy_check_mark: | :interrobang:      | :heavy_check_mark: | :interrobang:      |
   164  | Linux/go-1.18.8    | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   165  | Darwin/go-1.18.8   | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
   166  | Windows/go-1.18.8  | :x:                | :interrobang:      | :heavy_check_mark: | :interrobang:      |
   167  
   168  ## Warning
   169  
   170  Don't use "-s -w" compile argument, It strips symbol table.