github.com/batchcorp/thrift-iterator@v0.0.0-20220918180557-4c4a158fc6e9/README.md (about)

     1  # thrifter
     2  
     3  decode/encode thrift message without IDL
     4  
     5  Why?
     6  
     7  * because IDL generated model is ugly and inflexible, it is seldom used in application directly. instead we define another model, which leads to bad performance.
     8    * bytes need to be copied twice 
     9    * more objects to gc
    10  * thrift proxy can not know all possible IDL in advance, in scenarios like api gateway, we need to decode/encode in a generic way to modify embedded header.
    11  * official thrift library for go is slow, verified in several benchmarks. It is even slower than [json-iterator](https://github.com/json-iterator/go)
    12  
    13  # works like encoding/json
    14  
    15  `encoding/json` has a super simple api to encode/decode json.
    16  thrifter mimic the same api.
    17  
    18  ```go
    19  import "github.com/thrift-iterator/go"
    20  // marshal to thrift
    21  thriftEncodedBytes, err := thrifter.Marshal([]int{1, 2, 3})
    22  // unmarshal back
    23  var val []int
    24  err = thrifter.Unmarshal(thriftEncodedBytes, &val)
    25  ```
    26  
    27  even struct data binding is supported
    28  
    29  ```go
    30  import "github.com/thrift-iterator/go"
    31  
    32  type NewOrderRequest struct {
    33      Lines []NewOrderLine `thrift:",1"`
    34  }
    35  
    36  type NewOrderLine struct {
    37      ProductId string `thrift:",1"`
    38      Quantity int `thrift:",2"`
    39  }
    40  
    41  // marshal to thrift
    42  thriftEncodedBytes, err := thrifter.Marshal(NewOrderRequest{
    43  	Lines: []NewOrderLine{
    44  		{"apple", 1},
    45  		{"orange", 2},
    46  	}
    47  })
    48  // unmarshal back
    49  var val NewOrderRequest
    50  err = thrifter.Unmarshal(thriftEncodedBytes, &val)
    51  ```
    52  
    53  # without IDL
    54  
    55  you do not need to define IDL. you do not need to use static code generation.
    56  you do not event need to define struct.
    57  
    58  ```go
    59  import "github.com/thrift-iterator/go"
    60  import "github.com/thrift-iterator/go/general"
    61  
    62  var msg general.Message
    63  err := thrifter.Unmarshal(thriftEncodedBytes, &msg)
    64  // the RPC call method name, type is string
    65  fmt.Println(msg.MessageName)
    66  // the RPC call arguments, type is general.Struct
    67  fmt.Println(msg.Arguments)
    68  ```
    69  
    70  what is `general.Struct`, it is defined as a map
    71  
    72  ```go
    73  type FieldId int16
    74  type Struct map[FieldId]interface{}
    75  ```
    76  
    77  we can extract out specific argument from deeply nested arguments using one line
    78  
    79  ```go
    80  productId := msg.MessageArgs.Get(
    81  	protocol.FieldId(1), // lines of request
    82  	0, // the first line
    83  	protocol.FieldId(1), // product id
    84  ).(string)
    85  ```
    86  
    87  You can unmarshal any thrift bytes into general objects. And you can marshal them back.
    88  
    89  # Partial decoding
    90  
    91  fully decoding into a go struct consumes substantial resources. 
    92  thrifter provide option to do partial decoding. You can modify part of the 
    93  message, with untouched parts in `[]byte` form.
    94  
    95  ```go
    96  import "github.com/thrift-iterator/go"
    97  import "github.com/thrift-iterator/go/protocol"
    98  import "github.com/thrift-iterator/go/raw"
    99  
   100  // partial decoding
   101  decoder := thrifter.NewDecoder(reader)
   102  var msgHeader protocol.MessageHeader
   103  decoder.Decode(&msgHeader)
   104  var msgArgs raw.Struct
   105  decoder.Decode(&msgArgs)
   106  
   107  // modify...
   108  
   109  // encode back
   110  encoder := thrifter.NewEncoder(writer)
   111  encoder.Encode(msgHeader)
   112  encoder.Encode(msgArgs)
   113  ```
   114  
   115  the definition of `raw.Struct` is 
   116  
   117  ```go
   118  type StructField struct {
   119  	Buffer []byte
   120  	Type protocol.TType
   121  }
   122  
   123  type Struct map[protocol.FieldId]StructField
   124  ```
   125  
   126  # Performance
   127  
   128  thrifter does not compromise performance. 
   129  
   130  gogoprotobuf
   131  
   132  ```
   133  5000000	       366 ns/op	     144 B/op	      12 allocs/op
   134  ```
   135  
   136  thrift 
   137  
   138  ```
   139  1000000	      1549 ns/op	     528 B/op	       9 allocs/op
   140  ```
   141  
   142  thrifter by static codegen
   143  
   144  ```
   145  5000000	       389 ns/op	     192 B/op	       6 allocs/op
   146  ```
   147  
   148  thrifter by reflection
   149  
   150  ```
   151  2000000	       585 ns/op	     192 B/op	       6 allocs/op
   152  ```
   153  
   154  You can see the reflection implementation is not bad, much faster than the 
   155  static code generated by thrift original implementation.
   156  
   157  To have best performance, you can choose to use static code generation. The api
   158  is unchanged, just need to add extra static codegen in your build steps, and include
   159  the generated code in your package. The runtime will automatically use the 
   160  generated encoder/decoder instead of reflection.
   161  
   162  For example of static codegen, checkout [https://github.com/thrift-iterator/go/blob/master/test/api/init.go](https://github.com/thrift-iterator/go/blob/master/test/api/init.go)
   163  
   164  # Sync IDL and Go Struct
   165  
   166  Keep IDL and your object model is challenging. We do not always like the code 
   167  generated from thrift IDL. But manually keeping the IDL and model in sync is
   168  tedious and error prone. 
   169  
   170  A separate toolchain to manipulate thrift IDL file, and keeping them bidirectionally in sync
   171  will be provided in another project.
   172