github.com/graybobo/golang.org-package-offline-cache@v0.0.0-20200626051047-6608995c132f/x/blog/content/json-and-go.article (about)

     1  JSON and Go
     2  25 Jan 2011
     3  Tags: json, technical
     4  
     5  Andrew Gerrand
     6  
     7  * Introduction
     8  
     9  JSON (JavaScript Object Notation) is a simple data interchange format. Syntactically it resembles the objects and lists of JavaScript. It is most commonly used for communication between web back-ends and JavaScript programs running in the browser, but it is used in many other places, too. Its home page, [[http://json.org][json.org]], provides a wonderfully clear and concise definition of the standard.
    10  
    11  With the [[http://golang.org/pkg/encoding/json/][json package]] it's a snap to read and write JSON data from your Go programs.
    12  
    13  * Encoding
    14  
    15  To encode JSON data we use the [[http://golang.org/pkg/encoding/json/#Marshal][`Marshal`]] function.
    16  
    17  	func Marshal(v interface{}) ([]byte, error)
    18  
    19  Given the Go data structure, `Message`,
    20  
    21  	type Message struct {
    22  	    Name string
    23  	    Body string
    24  	    Time int64
    25  	}
    26  
    27  and an instance of `Message`
    28  
    29  	    m := Message{"Alice", "Hello", 1294706395881547000}
    30  
    31  we can marshal a JSON-encoded version of m using `json.Marshal`:
    32  
    33  	    b, err := json.Marshal(m)
    34  
    35  If all is well, `err` will be `nil` and `b` will be a `[]byte` containing this JSON data:
    36  
    37  	b == []byte(`{"Name":"Alice","Body":"Hello","Time":1294706395881547000}`)
    38  
    39  Only data structures that can be represented as valid JSON will be encoded:
    40  
    41  - JSON objects only support strings as keys; to encode a Go map type it must be of the form `map[string]T` (where `T` is any Go type supported by the json package).
    42  
    43  - Channel, complex, and function types cannot be encoded.
    44  
    45  - Cyclic data structures are not supported; they will cause `Marshal` to go into an infinite loop.
    46  
    47  - Pointers will be encoded as the values they point to (or 'null' if the pointer is `nil`).
    48  
    49  The json package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the the exported fields of a struct will be present in the JSON output.
    50  
    51  * Decoding
    52  
    53  To decode JSON data we use the [[http://golang.org/pkg/encoding/json/#Unmarshal][`Unmarshal`]] function.
    54  
    55  	func Unmarshal(data []byte, v interface{}) error
    56  
    57  We must first create a place where the decoded data will be stored
    58  
    59  	    var m Message
    60  
    61  and call `json.Unmarshal`, passing it a `[]byte` of JSON data and a pointer to `m`
    62  
    63  	    err := json.Unmarshal(b, &m)
    64  
    65  If `b` contains valid JSON that fits in `m`, after the call `err` will be `nil` and the data from `b` will have been stored in the struct `m`, as if by an assignment like:
    66  
    67  	    m = Message{
    68  	        Name: "Alice",
    69  	        Body: "Hello",
    70  	        Time: 1294706395881547000,
    71  	    }
    72  
    73  How does `Unmarshal` identify the fields in which to store the decoded data? For a given JSON key `"Foo"`, `Unmarshal` will look through the destination struct's fields to find (in order of preference):
    74  
    75  - An exported field with a tag of `"Foo"` (see the [[http://golang.org/ref/spec#Struct_types][Go spec]] for more on struct tags),
    76  
    77  - An exported field named `"Foo"`, or
    78  
    79  - An exported field named `"FOO"` or `"FoO"` or some other case-insensitive match of `"Foo"`.
    80  
    81  What happens when the structure of the JSON data doesn't exactly match the Go type?
    82  
    83  	    b := []byte(`{"Name":"Bob","Food":"Pickle"}`)
    84  	    var m Message
    85  	    err := json.Unmarshal(b, &m)
    86  
    87  `Unmarshal` will decode only the fields that it can find in the destination type.  In this case, only the Name field of m will be populated, and the Food field will be ignored. This behavior is particularly useful when you wish to pick only a few specific fields out of a large JSON blob. It also means that any unexported fields in the destination struct will be unaffected by `Unmarshal`.
    88  
    89  But what if you don't know the structure of your JSON data beforehand?
    90  
    91  * Generic JSON with interface{}
    92  
    93  The `interface{}` (empty interface) type describes an interface with zero methods.  Every Go type implements at least zero methods and therefore satisfies the empty interface.
    94  
    95  The empty interface serves as a general container type:
    96  
    97  	    var i interface{}
    98  	    i = "a string"
    99  	    i = 2011
   100  	    i = 2.777
   101  
   102  A type assertion accesses the underlying concrete type:
   103  
   104  	    r := i.(float64)
   105  	    fmt.Println("the circle's area", math.Pi*r*r)
   106  
   107  Or, if the underlying type is unknown, a type switch determines the type:
   108  
   109  	    switch v := i.(type) {
   110  	    case int:
   111  	        fmt.Println("twice i is", v*2)
   112  	    case float64:
   113  	        fmt.Println("the reciprocal of i is", 1/v)
   114  	    case string:
   115  	        h := len(v) / 2
   116  	        fmt.Println("i swapped by halves is", v[h:]+v[:h])
   117  	    default:
   118  	        // i isn't one of the types above
   119  	    }
   120  
   121  The json package uses `map[string]interface{}` and
   122  `[]interface{}` values to store arbitrary JSON objects and arrays;
   123  it will happily unmarshal any valid JSON blob into a plain
   124  `interface{}` value.  The default concrete Go types are:
   125  
   126  - `bool` for JSON booleans,
   127  
   128  - `float64` for JSON numbers,
   129  
   130  - `string` for JSON strings, and
   131  
   132  - `nil` for JSON null.
   133  
   134  * Decoding arbitrary data
   135  
   136  Consider this JSON data, stored in the variable `b`:
   137  
   138  	    b := []byte(`{"Name":"Wednesday","Age":6,"Parents":["Gomez","Morticia"]}`)
   139  
   140  Without knowing this data's structure, we can decode it into an `interface{}` value with `Unmarshal`:
   141  
   142  	    var f interface{}
   143  	    err := json.Unmarshal(b, &f)
   144  
   145  At this point the Go value in `f` would be a map whose keys are strings and whose values are themselves stored as empty interface values:
   146  
   147  	    f = map[string]interface{}{
   148  	        "Name": "Wednesday",
   149  	        "Age":  6,
   150  	        "Parents": []interface{}{
   151  	            "Gomez",
   152  	            "Morticia",
   153  	        },
   154  	    }
   155  
   156  To access this data we can use a type assertion to access `f`'s underlying `map[string]interface{}`:
   157  
   158  	    m := f.(map[string]interface{})
   159  
   160  We can then iterate through the map with a range statement and use a type switch to access its values as their concrete types:
   161  
   162  	    for k, v := range m {
   163  	        switch vv := v.(type) {
   164  	        case string:
   165  	            fmt.Println(k, "is string", vv)
   166  	        case int:
   167  	            fmt.Println(k, "is int", vv)
   168  	        case []interface{}:
   169  	            fmt.Println(k, "is an array:")
   170  	            for i, u := range vv {
   171  	                fmt.Println(i, u)
   172  	            }
   173  	        default:
   174  	            fmt.Println(k, "is of a type I don't know how to handle")
   175  	        }
   176  	    }
   177  
   178  In this way you can work with unknown JSON data while still enjoying the benefits of type safety.
   179  
   180  * Reference Types
   181  
   182  Let's define a Go type to contain the data from the previous example:
   183  
   184  	type FamilyMember struct {
   185  	    Name    string
   186  	    Age     int
   187  	    Parents []string
   188  	}
   189  
   190  	    var m FamilyMember
   191  	    err := json.Unmarshal(b, &m)
   192  
   193  Unmarshaling that data into a `FamilyMember` value works as expected, but if we look closely we can see a remarkable thing has happened. With the var statement we allocated a `FamilyMember` struct, and then provided a pointer to that value to `Unmarshal`, but at that time the `Parents` field was a `nil` slice value. To populate the `Parents` field, `Unmarshal` allocated a new slice behind the scenes. This is typical of how `Unmarshal` works with the supported reference types (pointers, slices, and maps).
   194  
   195  Consider unmarshaling into this data structure:
   196  
   197  	type Foo struct {
   198  	    Bar *Bar
   199  	}
   200  
   201  If there were a `Bar` field in the JSON object, `Unmarshal` would allocate a new `Bar` and populate it. If not, `Bar` would be left as a `nil` pointer.
   202  
   203  From this a useful pattern arises: if you have an application that receives a few distinct message types, you might define "receiver" structure like
   204  
   205  	type IncomingMessage struct {
   206  	    Cmd *Command
   207  	    Msg *Message
   208  	}
   209  
   210  and the sending party can populate the `Cmd` field and/or the `Msg` field of the top-level JSON object, depending on the type of message they want to communicate. `Unmarshal`, when decoding the JSON into an `IncomingMessage` struct, will only allocate the data structures present in the JSON data. To know which messages to process, the programmer need simply test that either `Cmd` or `Msg` is not `nil`.
   211  
   212  * Streaming Encoders and Decoders
   213  
   214  The json package provides `Decoder` and `Encoder` types to support the common operation of reading and writing streams of JSON data. The `NewDecoder` and `NewEncoder` functions wrap the [[http://golang.org/pkg/io/#Reader][`io.Reader`]] and [[http://golang.org/pkg/io/#Writer][`io.Writer`]] interface types.
   215  
   216  	func NewDecoder(r io.Reader) *Decoder
   217  	func NewEncoder(w io.Writer) *Encoder
   218  
   219  Here's an example program that reads a series of JSON objects from standard input, removes all but the `Name` field from each object, and then writes the objects to standard output:
   220  
   221  	package main
   222  
   223  	import (
   224  	    "encoding/json"
   225  	    "log"
   226  	    "os"
   227  	)
   228  
   229  	func main() {
   230  	    dec := json.NewDecoder(os.Stdin)
   231  	    enc := json.NewEncoder(os.Stdout)
   232  	    for {
   233  	        var v map[string]interface{}
   234  	        if err := dec.Decode(&v); err != nil {
   235  	            log.Println(err)
   236  	            return
   237  	        }
   238  	        for k := range v {
   239  	            if k != "Name" {
   240  	                delete(v, k)
   241  	            }
   242  	        }
   243  	        if err := enc.Encode(&v); err != nil {
   244  	            log.Println(err)
   245  	        }
   246  	    }
   247  	}
   248  
   249  Due to the ubiquity of Readers and Writers, these `Encoder` and `Decoder` types can be used in a broad range of scenarios, such as reading and writing to HTTP connections, WebSockets, or files.
   250  
   251  * References
   252  
   253  For more information see the [[http://golang.org/pkg/encoding/json/][json package documentation]]. For an example usage of json see the source files of the [[http://golang.org/pkg/net/rpc/jsonrpc/][jsonrpc package]].