github.com/graybobo/golang.org-package-offline-cache@v0.0.0-20200626051047-6608995c132f/x/blog/content/json-and-go.article (about) 1 JSON and Go 2 25 Jan 2011 3 Tags: json, technical 4 5 Andrew Gerrand 6 7 * Introduction 8 9 JSON (JavaScript Object Notation) is a simple data interchange format. Syntactically it resembles the objects and lists of JavaScript. It is most commonly used for communication between web back-ends and JavaScript programs running in the browser, but it is used in many other places, too. Its home page, [[http://json.org][json.org]], provides a wonderfully clear and concise definition of the standard. 10 11 With the [[http://golang.org/pkg/encoding/json/][json package]] it's a snap to read and write JSON data from your Go programs. 12 13 * Encoding 14 15 To encode JSON data we use the [[http://golang.org/pkg/encoding/json/#Marshal][`Marshal`]] function. 16 17 func Marshal(v interface{}) ([]byte, error) 18 19 Given the Go data structure, `Message`, 20 21 type Message struct { 22 Name string 23 Body string 24 Time int64 25 } 26 27 and an instance of `Message` 28 29 m := Message{"Alice", "Hello", 1294706395881547000} 30 31 we can marshal a JSON-encoded version of m using `json.Marshal`: 32 33 b, err := json.Marshal(m) 34 35 If all is well, `err` will be `nil` and `b` will be a `[]byte` containing this JSON data: 36 37 b == []byte(`{"Name":"Alice","Body":"Hello","Time":1294706395881547000}`) 38 39 Only data structures that can be represented as valid JSON will be encoded: 40 41 - JSON objects only support strings as keys; to encode a Go map type it must be of the form `map[string]T` (where `T` is any Go type supported by the json package). 42 43 - Channel, complex, and function types cannot be encoded. 44 45 - Cyclic data structures are not supported; they will cause `Marshal` to go into an infinite loop. 46 47 - Pointers will be encoded as the values they point to (or 'null' if the pointer is `nil`). 48 49 The json package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the the exported fields of a struct will be present in the JSON output. 50 51 * Decoding 52 53 To decode JSON data we use the [[http://golang.org/pkg/encoding/json/#Unmarshal][`Unmarshal`]] function. 54 55 func Unmarshal(data []byte, v interface{}) error 56 57 We must first create a place where the decoded data will be stored 58 59 var m Message 60 61 and call `json.Unmarshal`, passing it a `[]byte` of JSON data and a pointer to `m` 62 63 err := json.Unmarshal(b, &m) 64 65 If `b` contains valid JSON that fits in `m`, after the call `err` will be `nil` and the data from `b` will have been stored in the struct `m`, as if by an assignment like: 66 67 m = Message{ 68 Name: "Alice", 69 Body: "Hello", 70 Time: 1294706395881547000, 71 } 72 73 How does `Unmarshal` identify the fields in which to store the decoded data? For a given JSON key `"Foo"`, `Unmarshal` will look through the destination struct's fields to find (in order of preference): 74 75 - An exported field with a tag of `"Foo"` (see the [[http://golang.org/ref/spec#Struct_types][Go spec]] for more on struct tags), 76 77 - An exported field named `"Foo"`, or 78 79 - An exported field named `"FOO"` or `"FoO"` or some other case-insensitive match of `"Foo"`. 80 81 What happens when the structure of the JSON data doesn't exactly match the Go type? 82 83 b := []byte(`{"Name":"Bob","Food":"Pickle"}`) 84 var m Message 85 err := json.Unmarshal(b, &m) 86 87 `Unmarshal` will decode only the fields that it can find in the destination type. In this case, only the Name field of m will be populated, and the Food field will be ignored. This behavior is particularly useful when you wish to pick only a few specific fields out of a large JSON blob. It also means that any unexported fields in the destination struct will be unaffected by `Unmarshal`. 88 89 But what if you don't know the structure of your JSON data beforehand? 90 91 * Generic JSON with interface{} 92 93 The `interface{}` (empty interface) type describes an interface with zero methods. Every Go type implements at least zero methods and therefore satisfies the empty interface. 94 95 The empty interface serves as a general container type: 96 97 var i interface{} 98 i = "a string" 99 i = 2011 100 i = 2.777 101 102 A type assertion accesses the underlying concrete type: 103 104 r := i.(float64) 105 fmt.Println("the circle's area", math.Pi*r*r) 106 107 Or, if the underlying type is unknown, a type switch determines the type: 108 109 switch v := i.(type) { 110 case int: 111 fmt.Println("twice i is", v*2) 112 case float64: 113 fmt.Println("the reciprocal of i is", 1/v) 114 case string: 115 h := len(v) / 2 116 fmt.Println("i swapped by halves is", v[h:]+v[:h]) 117 default: 118 // i isn't one of the types above 119 } 120 121 The json package uses `map[string]interface{}` and 122 `[]interface{}` values to store arbitrary JSON objects and arrays; 123 it will happily unmarshal any valid JSON blob into a plain 124 `interface{}` value. The default concrete Go types are: 125 126 - `bool` for JSON booleans, 127 128 - `float64` for JSON numbers, 129 130 - `string` for JSON strings, and 131 132 - `nil` for JSON null. 133 134 * Decoding arbitrary data 135 136 Consider this JSON data, stored in the variable `b`: 137 138 b := []byte(`{"Name":"Wednesday","Age":6,"Parents":["Gomez","Morticia"]}`) 139 140 Without knowing this data's structure, we can decode it into an `interface{}` value with `Unmarshal`: 141 142 var f interface{} 143 err := json.Unmarshal(b, &f) 144 145 At this point the Go value in `f` would be a map whose keys are strings and whose values are themselves stored as empty interface values: 146 147 f = map[string]interface{}{ 148 "Name": "Wednesday", 149 "Age": 6, 150 "Parents": []interface{}{ 151 "Gomez", 152 "Morticia", 153 }, 154 } 155 156 To access this data we can use a type assertion to access `f`'s underlying `map[string]interface{}`: 157 158 m := f.(map[string]interface{}) 159 160 We can then iterate through the map with a range statement and use a type switch to access its values as their concrete types: 161 162 for k, v := range m { 163 switch vv := v.(type) { 164 case string: 165 fmt.Println(k, "is string", vv) 166 case int: 167 fmt.Println(k, "is int", vv) 168 case []interface{}: 169 fmt.Println(k, "is an array:") 170 for i, u := range vv { 171 fmt.Println(i, u) 172 } 173 default: 174 fmt.Println(k, "is of a type I don't know how to handle") 175 } 176 } 177 178 In this way you can work with unknown JSON data while still enjoying the benefits of type safety. 179 180 * Reference Types 181 182 Let's define a Go type to contain the data from the previous example: 183 184 type FamilyMember struct { 185 Name string 186 Age int 187 Parents []string 188 } 189 190 var m FamilyMember 191 err := json.Unmarshal(b, &m) 192 193 Unmarshaling that data into a `FamilyMember` value works as expected, but if we look closely we can see a remarkable thing has happened. With the var statement we allocated a `FamilyMember` struct, and then provided a pointer to that value to `Unmarshal`, but at that time the `Parents` field was a `nil` slice value. To populate the `Parents` field, `Unmarshal` allocated a new slice behind the scenes. This is typical of how `Unmarshal` works with the supported reference types (pointers, slices, and maps). 194 195 Consider unmarshaling into this data structure: 196 197 type Foo struct { 198 Bar *Bar 199 } 200 201 If there were a `Bar` field in the JSON object, `Unmarshal` would allocate a new `Bar` and populate it. If not, `Bar` would be left as a `nil` pointer. 202 203 From this a useful pattern arises: if you have an application that receives a few distinct message types, you might define "receiver" structure like 204 205 type IncomingMessage struct { 206 Cmd *Command 207 Msg *Message 208 } 209 210 and the sending party can populate the `Cmd` field and/or the `Msg` field of the top-level JSON object, depending on the type of message they want to communicate. `Unmarshal`, when decoding the JSON into an `IncomingMessage` struct, will only allocate the data structures present in the JSON data. To know which messages to process, the programmer need simply test that either `Cmd` or `Msg` is not `nil`. 211 212 * Streaming Encoders and Decoders 213 214 The json package provides `Decoder` and `Encoder` types to support the common operation of reading and writing streams of JSON data. The `NewDecoder` and `NewEncoder` functions wrap the [[http://golang.org/pkg/io/#Reader][`io.Reader`]] and [[http://golang.org/pkg/io/#Writer][`io.Writer`]] interface types. 215 216 func NewDecoder(r io.Reader) *Decoder 217 func NewEncoder(w io.Writer) *Encoder 218 219 Here's an example program that reads a series of JSON objects from standard input, removes all but the `Name` field from each object, and then writes the objects to standard output: 220 221 package main 222 223 import ( 224 "encoding/json" 225 "log" 226 "os" 227 ) 228 229 func main() { 230 dec := json.NewDecoder(os.Stdin) 231 enc := json.NewEncoder(os.Stdout) 232 for { 233 var v map[string]interface{} 234 if err := dec.Decode(&v); err != nil { 235 log.Println(err) 236 return 237 } 238 for k := range v { 239 if k != "Name" { 240 delete(v, k) 241 } 242 } 243 if err := enc.Encode(&v); err != nil { 244 log.Println(err) 245 } 246 } 247 } 248 249 Due to the ubiquity of Readers and Writers, these `Encoder` and `Decoder` types can be used in a broad range of scenarios, such as reading and writing to HTTP connections, WebSockets, or files. 250 251 * References 252 253 For more information see the [[http://golang.org/pkg/encoding/json/][json package documentation]]. For an example usage of json see the source files of the [[http://golang.org/pkg/net/rpc/jsonrpc/][jsonrpc package]].