github.com/patricebensoussan/go/codec@v1.2.99/doc.go (about)

     1  // Copyright (c) 2012-2020 Ugorji Nwoke. All rights reserved.
     2  // Use of this source code is governed by a MIT license found in the LICENSE file.
     3  
     4  /*
     5  Package codec provides a
     6  High Performance, Feature-Rich Idiomatic Go 1.4+ codec/encoding library
     7  for binc, msgpack, cbor, json.
     8  
     9  Supported Serialization formats are:
    10  
    11    - msgpack: https://github.com/msgpack/msgpack
    12    - binc:    http://github.com/ugorji/binc
    13    - cbor:    http://cbor.io http://tools.ietf.org/html/rfc7049
    14    - json:    http://json.org http://tools.ietf.org/html/rfc7159
    15    - simple:
    16  
    17  This package will carefully use 'package unsafe' for performance reasons in specific places.
    18  You can build without unsafe use by passing the safe or appengine tag
    19  i.e. 'go install -tags=codec.safe ...'.
    20  
    21  This library works with both the standard `gc` and the `gccgo` compilers.
    22  
    23  For detailed usage information, read the primer at http://ugorji.net/blog/go-codec-primer .
    24  
    25  The idiomatic Go support is as seen in other encoding packages in
    26  the standard library (ie json, xml, gob, etc).
    27  
    28  Rich Feature Set includes:
    29  
    30    - Simple but extremely powerful and feature-rich API
    31    - Support for go 1.4 and above, while selectively using newer APIs for later releases
    32    - Excellent code coverage ( > 90% )
    33    - Very High Performance.
    34      Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X.
    35    - Careful selected use of 'unsafe' for targeted performance gains.
    36    - 100% safe mode supported, where 'unsafe' is not used at all.
    37    - Lock-free (sans mutex) concurrency for scaling to 100's of cores
    38    - In-place updates during decode, with option to zero value in maps and slices prior to decode
    39    - Coerce types where appropriate
    40      e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc
    41    - Corner Cases:
    42      Overflows, nil maps/slices, nil values in streams are handled correctly
    43    - Standard field renaming via tags
    44    - Support for omitting empty fields during an encoding
    45    - Encoding from any value and decoding into pointer to any value
    46      (struct, slice, map, primitives, pointers, interface{}, etc)
    47    - Extensions to support efficient encoding/decoding of any named types
    48    - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces
    49    - Support using existence of `IsZero() bool` to determine if a value is a zero value.
    50      Analogous to time.Time.IsZero() bool.
    51    - Decoding without a schema (into a interface{}).
    52      Includes Options to configure what specific map or slice type to use
    53      when decoding an encoded list or map into a nil interface{}
    54    - Mapping a non-interface type to an interface, so we can decode appropriately
    55      into any interface type with a correctly configured non-interface value.
    56    - Encode a struct as an array, and decode struct from an array in the data stream
    57    - Option to encode struct keys as numbers (instead of strings)
    58      (to support structured streams with fields encoded as numeric codes)
    59    - Comprehensive support for anonymous fields
    60    - Fast (no-reflection) encoding/decoding of common maps and slices
    61    - Code-generation for faster performance, supported in go 1.6+
    62    - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats
    63    - Support indefinite-length formats to enable true streaming
    64      (for formats which support it e.g. json, cbor)
    65    - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes.
    66      This mostly applies to maps, where iteration order is non-deterministic.
    67    - NIL in data stream decoded as zero value
    68    - Never silently skip data when decoding.
    69      User decides whether to return an error or silently skip data when keys or indexes
    70      in the data stream do not map to fields in the struct.
    71    - Detect and error when encoding a cyclic reference (instead of stack overflow shutdown)
    72    - Encode/Decode from/to chan types (for iterative streaming support)
    73    - Drop-in replacement for encoding/json. `json:` key in struct tag supported.
    74    - Provides a RPC Server and Client Codec for net/rpc communication protocol.
    75    - Handle unique idiosyncrasies of codecs e.g.
    76      - For messagepack, configure how ambiguities in handling raw bytes are resolved
    77      - For messagepack, provide rpc server/client codec to support
    78        msgpack-rpc protocol defined at:
    79        https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
    80  
    81  Extension Support
    82  
    83  Users can register a function to handle the encoding or decoding of
    84  their custom types.
    85  
    86  There are no restrictions on what the custom type can be. Some examples:
    87  
    88      type BisSet   []int
    89      type BitSet64 uint64
    90      type UUID     string
    91      type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
    92      type GifImage struct { ... }
    93  
    94  As an illustration, MyStructWithUnexportedFields would normally be
    95  encoded as an empty map because it has no exported fields, while UUID
    96  would be encoded as a string. However, with extension support, you can
    97  encode any of these however you like.
    98  
    99  There is also seamless support provided for registering an extension (with a tag)
   100  but letting the encoding mechanism default to the standard way.
   101  
   102  Custom Encoding and Decoding
   103  
   104  This package maintains symmetry in the encoding and decoding halfs.
   105  We determine how to encode or decode by walking this decision tree
   106  
   107    - is there an extension registered for the type?
   108    - is type a codec.Selfer?
   109    - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler?
   110    - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler?
   111    - is format text-based, and type an encoding.TextMarshaler and TextUnmarshaler?
   112    - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc
   113  
   114  This symmetry is important to reduce chances of issues happening because the
   115  encoding and decoding sides are out of sync e.g. decoded via very specific
   116  encoding.TextUnmarshaler but encoded via kind-specific generalized mode.
   117  
   118  Consequently, if a type only defines one-half of the symmetry
   119  (e.g. it implements UnmarshalJSON() but not MarshalJSON() ),
   120  then that type doesn't satisfy the check and we will continue walking down the
   121  decision tree.
   122  
   123  RPC
   124  
   125  RPC Client and Server Codecs are implemented, so the codecs can be used
   126  with the standard net/rpc package.
   127  
   128  Usage
   129  
   130  The Handle is SAFE for concurrent READ, but NOT SAFE for concurrent modification.
   131  
   132  The Encoder and Decoder are NOT safe for concurrent use.
   133  
   134  Consequently, the usage model is basically:
   135  
   136      - Create and initialize the Handle before any use.
   137        Once created, DO NOT modify it.
   138      - Multiple Encoders or Decoders can now use the Handle concurrently.
   139        They only read information off the Handle (never write).
   140      - However, each Encoder or Decoder MUST not be used concurrently
   141      - To re-use an Encoder/Decoder, call Reset(...) on it first.
   142        This allows you use state maintained on the Encoder/Decoder.
   143  
   144  Sample usage model:
   145  
   146      // create and configure Handle
   147      var (
   148        bh codec.BincHandle
   149        mh codec.MsgpackHandle
   150        ch codec.CborHandle
   151      )
   152  
   153      mh.MapType = reflect.TypeOf(map[string]interface{}(nil))
   154  
   155      // configure extensions
   156      // e.g. for msgpack, define functions and enable Time support for tag 1
   157      // mh.SetExt(reflect.TypeOf(time.Time{}), 1, myExt)
   158  
   159      // create and use decoder/encoder
   160      var (
   161        r io.Reader
   162        w io.Writer
   163        b []byte
   164        h = &bh // or mh to use msgpack
   165      )
   166  
   167      dec = codec.NewDecoder(r, h)
   168      dec = codec.NewDecoderBytes(b, h)
   169      err = dec.Decode(&v)
   170  
   171      enc = codec.NewEncoder(w, h)
   172      enc = codec.NewEncoderBytes(&b, h)
   173      err = enc.Encode(v)
   174  
   175      //RPC Server
   176      go func() {
   177          for {
   178              conn, err := listener.Accept()
   179              rpcCodec := codec.GoRpc.ServerCodec(conn, h)
   180              //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
   181              rpc.ServeCodec(rpcCodec)
   182          }
   183      }()
   184  
   185      //RPC Communication (client side)
   186      conn, err = net.Dial("tcp", "localhost:5555")
   187      rpcCodec := codec.GoRpc.ClientCodec(conn, h)
   188      //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
   189      client := rpc.NewClientWithCodec(rpcCodec)
   190  
   191  Running Tests
   192  
   193  To run tests, use the following:
   194  
   195      go test
   196  
   197  To run the full suite of tests, use the following:
   198  
   199      go test -tags alltests -run Suite
   200  
   201  You can run the tag 'codec.safe' to run tests or build in safe mode. e.g.
   202  
   203      go test -tags codec.safe -run Json
   204      go test -tags "alltests codec.safe" -run Suite
   205  
   206  Running Benchmarks
   207  
   208      cd bench
   209      go test -bench . -benchmem -benchtime 1s
   210  
   211  Please see http://github.com/ugorji/go-codec-bench .
   212  
   213  Caveats
   214  
   215  Struct fields matching the following are ignored during encoding and decoding
   216      - struct tag value set to -
   217      - func, complex numbers, unsafe pointers
   218      - unexported and not embedded
   219      - unexported and embedded and not struct kind
   220      - unexported and embedded pointers (from go1.10)
   221  
   222  Every other field in a struct will be encoded/decoded.
   223  
   224  Embedded fields are encoded as if they exist in the top-level struct,
   225  with some caveats. See Encode documentation.
   226  
   227  */
   228  package codec