github.com/patricebensoussan/go/codec@v1.2.99/README.md (about) 1 # Package Documentation for github.com/ugorji/go/codec 2 3 Package codec provides a High Performance, Feature-Rich Idiomatic Go 1.4+ 4 codec/encoding library for binc, msgpack, cbor, json. 5 6 Supported Serialization formats are: 7 8 - msgpack: https://github.com/msgpack/msgpack 9 - binc: http://github.com/ugorji/binc 10 - cbor: http://cbor.io http://tools.ietf.org/html/rfc7049 11 - json: http://json.org http://tools.ietf.org/html/rfc7159 12 - simple: 13 14 This package will carefully use 'package unsafe' for performance reasons in 15 specific places. You can build without unsafe use by passing the safe or 16 appengine tag i.e. 'go install -tags=codec.safe ...'. 17 18 This library works with both the standard `gc` and the `gccgo` compilers. 19 20 For detailed usage information, read the primer at 21 http://ugorji.net/blog/go-codec-primer . 22 23 The idiomatic Go support is as seen in other encoding packages in the 24 standard library (ie json, xml, gob, etc). 25 26 Rich Feature Set includes: 27 28 - Simple but extremely powerful and feature-rich API 29 - Support for go 1.4 and above, while selectively using newer APIs for later releases 30 - Excellent code coverage ( > 90% ) 31 - Very High Performance. 32 Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X. 33 - Careful selected use of 'unsafe' for targeted performance gains. 34 - 100% safe mode supported, where 'unsafe' is not used at all. 35 - Lock-free (sans mutex) concurrency for scaling to 100's of cores 36 - In-place updates during decode, with option to zero value in maps and slices prior to decode 37 - Coerce types where appropriate 38 e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc 39 - Corner Cases: 40 Overflows, nil maps/slices, nil values in streams are handled correctly 41 - Standard field renaming via tags 42 - Support for omitting empty fields during an encoding 43 - Encoding from any value and decoding into pointer to any value 44 (struct, slice, map, primitives, pointers, interface{}, etc) 45 - Extensions to support efficient encoding/decoding of any named types 46 - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces 47 - Support using existence of `IsZero() bool` to determine if a value is a zero value. 48 Analogous to time.Time.IsZero() bool. 49 - Decoding without a schema (into a interface{}). 50 Includes Options to configure what specific map or slice type to use 51 when decoding an encoded list or map into a nil interface{} 52 - Mapping a non-interface type to an interface, so we can decode appropriately 53 into any interface type with a correctly configured non-interface value. 54 - Encode a struct as an array, and decode struct from an array in the data stream 55 - Option to encode struct keys as numbers (instead of strings) 56 (to support structured streams with fields encoded as numeric codes) 57 - Comprehensive support for anonymous fields 58 - Fast (no-reflection) encoding/decoding of common maps and slices 59 - Code-generation for faster performance, supported in go 1.6+ 60 - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats 61 - Support indefinite-length formats to enable true streaming 62 (for formats which support it e.g. json, cbor) 63 - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes. 64 This mostly applies to maps, where iteration order is non-deterministic. 65 - NIL in data stream decoded as zero value 66 - Never silently skip data when decoding. 67 User decides whether to return an error or silently skip data when keys or indexes 68 in the data stream do not map to fields in the struct. 69 - Detect and error when encoding a cyclic reference (instead of stack overflow shutdown) 70 - Encode/Decode from/to chan types (for iterative streaming support) 71 - Drop-in replacement for encoding/json. `json:` key in struct tag supported. 72 - Provides a RPC Server and Client Codec for net/rpc communication protocol. 73 - Handle unique idiosyncrasies of codecs e.g. 74 - For messagepack, configure how ambiguities in handling raw bytes are resolved 75 - For messagepack, provide rpc server/client codec to support 76 msgpack-rpc protocol defined at: 77 https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md 78 79 80 ## Extension Support 81 82 Users can register a function to handle the encoding or decoding of their 83 custom types. 84 85 There are no restrictions on what the custom type can be. Some examples: 86 87 ```go 88 type BisSet []int 89 type BitSet64 uint64 90 type UUID string 91 type MyStructWithUnexportedFields struct { a int; b bool; c []int; } 92 type GifImage struct { ... } 93 ``` 94 95 As an illustration, MyStructWithUnexportedFields would normally be encoded 96 as an empty map because it has no exported fields, while UUID would be 97 encoded as a string. However, with extension support, you can encode any of 98 these however you like. 99 100 There is also seamless support provided for registering an extension (with a 101 tag) but letting the encoding mechanism default to the standard way. 102 103 104 ## Custom Encoding and Decoding 105 106 This package maintains symmetry in the encoding and decoding halfs. We 107 determine how to encode or decode by walking this decision tree 108 109 - is there an extension registered for the type? 110 - is type a codec.Selfer? 111 - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler? 112 - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler? 113 - is format text-based, and type an encoding.TextMarshaler and TextUnmarshaler? 114 - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc 115 116 This symmetry is important to reduce chances of issues happening because the 117 encoding and decoding sides are out of sync e.g. decoded via very specific 118 encoding.TextUnmarshaler but encoded via kind-specific generalized mode. 119 120 Consequently, if a type only defines one-half of the symmetry (e.g. it 121 implements UnmarshalJSON() but not MarshalJSON() ), then that type doesn't 122 satisfy the check and we will continue walking down the decision tree. 123 124 125 ## RPC 126 127 RPC Client and Server Codecs are implemented, so the codecs can be used with 128 the standard net/rpc package. 129 130 131 ## Usage 132 133 The Handle is SAFE for concurrent READ, but NOT SAFE for concurrent 134 modification. 135 136 The Encoder and Decoder are NOT safe for concurrent use. 137 138 Consequently, the usage model is basically: 139 140 - Create and initialize the Handle before any use. 141 Once created, DO NOT modify it. 142 - Multiple Encoders or Decoders can now use the Handle concurrently. 143 They only read information off the Handle (never write). 144 - However, each Encoder or Decoder MUST not be used concurrently 145 - To re-use an Encoder/Decoder, call Reset(...) on it first. 146 This allows you use state maintained on the Encoder/Decoder. 147 148 Sample usage model: 149 150 ```go 151 // create and configure Handle 152 var ( 153 bh codec.BincHandle 154 mh codec.MsgpackHandle 155 ch codec.CborHandle 156 ) 157 158 mh.MapType = reflect.TypeOf(map[string]interface{}(nil)) 159 160 // configure extensions 161 // e.g. for msgpack, define functions and enable Time support for tag 1 162 // mh.SetExt(reflect.TypeOf(time.Time{}), 1, myExt) 163 164 // create and use decoder/encoder 165 var ( 166 r io.Reader 167 w io.Writer 168 b []byte 169 h = &bh // or mh to use msgpack 170 ) 171 172 dec = codec.NewDecoder(r, h) 173 dec = codec.NewDecoderBytes(b, h) 174 err = dec.Decode(&v) 175 176 enc = codec.NewEncoder(w, h) 177 enc = codec.NewEncoderBytes(&b, h) 178 err = enc.Encode(v) 179 180 //RPC Server 181 go func() { 182 for { 183 conn, err := listener.Accept() 184 rpcCodec := codec.GoRpc.ServerCodec(conn, h) 185 //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h) 186 rpc.ServeCodec(rpcCodec) 187 } 188 }() 189 190 //RPC Communication (client side) 191 conn, err = net.Dial("tcp", "localhost:5555") 192 rpcCodec := codec.GoRpc.ClientCodec(conn, h) 193 //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h) 194 client := rpc.NewClientWithCodec(rpcCodec) 195 ``` 196 197 198 ## Running Tests 199 200 To run tests, use the following: 201 202 ``` 203 go test 204 ``` 205 206 To run the full suite of tests, use the following: 207 208 ``` 209 go test -tags alltests -run Suite 210 ``` 211 212 You can run the tag 'codec.safe' to run tests or build in safe mode. e.g. 213 214 ``` 215 go test -tags codec.safe -run Json 216 go test -tags "alltests codec.safe" -run Suite 217 ``` 218 219 ## Running Benchmarks 220 221 ``` 222 cd bench 223 go test -bench . -benchmem -benchtime 1s 224 ``` 225 226 Please see http://github.com/ugorji/go-codec-bench . 227 228 229 ## Caveats 230 231 Struct fields matching the following are ignored during encoding and 232 decoding 233 234 - struct tag value set to - 235 - func, complex numbers, unsafe pointers 236 - unexported and not embedded 237 - unexported and embedded and not struct kind 238 - unexported and embedded pointers (from go1.10) 239 240 Every other field in a struct will be encoded/decoded. 241 242 Embedded fields are encoded as if they exist in the top-level struct, with 243 some caveats. See Encode documentation. 244 245 ## Exported Package API 246 247 ```go 248 const CborStreamBytes byte = 0x5f ... 249 const GenVersion = 25 250 var SelfExt = &extFailWrapper{} 251 var GoRpc goRpc 252 var MsgpackSpecRpc msgpackSpecRpc 253 func GenHelper() (g genHelper) 254 type BasicHandle struct{ ... } 255 type BincHandle struct{ ... } 256 type BytesExt interface{ ... } 257 type CborHandle struct{ ... } 258 type DecodeOptions struct{ ... } 259 type Decoder struct{ ... } 260 func NewDecoder(r io.Reader, h Handle) *Decoder 261 func NewDecoderBytes(in []byte, h Handle) *Decoder 262 func NewDecoderString(s string, h Handle) *Decoder 263 type EncodeOptions struct{ ... } 264 type Encoder struct{ ... } 265 func NewEncoder(w io.Writer, h Handle) *Encoder 266 func NewEncoderBytes(out *[]byte, h Handle) *Encoder 267 type Ext interface{ ... } 268 type Handle interface{ ... } 269 type InterfaceExt interface{ ... } 270 type JsonHandle struct{ ... } 271 type MapBySlice interface{ ... } 272 type MissingFielder interface{ ... } 273 type MsgpackHandle struct{ ... } 274 type MsgpackSpecRpcMultiArgs []interface{} 275 type RPCOptions struct{ ... } 276 type Raw []byte 277 type RawExt struct{ ... } 278 type Rpc interface{ ... } 279 type Selfer interface{ ... } 280 type SimpleHandle struct{ ... } 281 type TypeInfos struct{ ... } 282 func NewTypeInfos(tags []string) *TypeInfos 283 ```