gitee.com/quant1x/gox@v1.7.6/fastjson/README.md (about) 1 [](https://travis-ci.org/valyala/fastjson) 2 [](http://godoc.org/github.com/valyala/fastjson) 3 [](https://goreportcard.com/report/github.com/valyala/fastjson) 4 [](https://codecov.io/gh/valyala/fastjson) 5 6 # fastjson - fast JSON parser and validator for Go 7 8 ## Features 9 10 * Fast. As usual, up to 15x faster than the standard [encoding/json](https://golang.org/pkg/encoding/json/). 11 See [benchmarks](#benchmarks). 12 * Parses arbitrary JSON without schema, reflection, struct magic and code generation contrary 13 to [easyjson](https://github.com/mailru/easyjson). 14 * Provides simple [API](http://godoc.org/github.com/valyala/fastjson). 15 * Outperforms [jsonparser](https://github.com/buger/jsonparser) and [gjson](https://github.com/tidwall/gjson) 16 when accessing multiple unrelated fields, since `fastjson` parses the input JSON only once. 17 * Validates the parsed JSON unlike [jsonparser](https://github.com/buger/jsonparser) 18 and [gjson](https://github.com/tidwall/gjson). 19 * May quickly extract a part of the original JSON with `Value.Get(...).MarshalTo` and modify it 20 with [Del](https://godoc.org/github.com/valyala/fastjson#Value.Del) 21 and [Set](https://godoc.org/github.com/valyala/fastjson#Value.Set) functions. 22 * May parse array containing values with distinct types (aka non-homogenous types). For instance, `fastjson` easily 23 parses the following JSON array `[123, "foo", [456], {"k": "v"}, null]`. 24 * `fastjson` preserves the original order of object items when calling 25 [Object.Visit](https://godoc.org/github.com/valyala/fastjson#Object.Visit). 26 27 ## Known limitations 28 29 * Requies extra care to work with - references to certain objects recursively returned 30 by [Parser](https://godoc.org/github.com/valyala/fastjson#Parser) 31 must be released before the next call to [Parse](https://godoc.org/github.com/valyala/fastjson#Parser.Parse). 32 Otherwise the program may work improperly. The same applies to objects returned 33 by [Arena](https://godoc.org/github.com/valyala/fastjson#Arena). Adhere recommendations 34 from [docs](https://godoc.org/github.com/valyala/fastjson). 35 * Cannot parse JSON from `io.Reader`. There is [Scanner](https://godoc.org/github.com/valyala/fastjson#Scanner) 36 for parsing stream of JSON values from a string. 37 38 ## Usage 39 40 One-liner accessing a single field: 41 42 ```go 43 s := []byte(`{"foo": [123, "bar"]}`) 44 fmt.Printf("foo.0=%d\n", fastjson.GetInt(s, "foo", "0")) 45 46 // Output: 47 // foo.0=123 48 ``` 49 50 Accessing multiple fields with error handling: 51 52 ```go 53 var p fastjson.Parser 54 v, err := p.Parse(`{ 55 "str": "bar", 56 "int": 123, 57 "float": 1.23, 58 "bool": true, 59 "arr": [1, "foo", {}] 60 }`) 61 if err != nil { 62 log.Fatal(err) 63 } 64 fmt.Printf("foo=%s\n", v.GetStringBytes("str")) 65 fmt.Printf("int=%d\n", v.GetInt("int")) 66 fmt.Printf("float=%f\n", v.GetFloat64("float")) 67 fmt.Printf("bool=%v\n", v.GetBool("bool")) 68 fmt.Printf("arr.1=%s\n", v.GetStringBytes("arr", "1")) 69 70 // Output: 71 // foo=bar 72 // int=123 73 // float=1.230000 74 // bool=true 75 // arr.1=foo 76 ``` 77 78 See also [examples](https://godoc.org/github.com/valyala/fastjson#pkg-examples). 79 80 ## Security 81 82 * `fastjson` shouldn't crash or panic when parsing input strings specially crafted by an attacker. It must return error 83 on invalid input JSON. 84 * `fastjson` requires up to `sizeof(Value) * len(inputJSON)` bytes of memory for parsing `inputJSON` string. Limit the 85 maximum size of the `inputJSON` 86 before parsing it in order to limit the maximum memory usage. 87 88 ## Performance optimization tips 89 90 * Re-use [Parser](https://godoc.org/github.com/valyala/fastjson#Parser) 91 and [Scanner](https://godoc.org/github.com/valyala/fastjson#Scanner) 92 for parsing many JSONs. This reduces memory allocations overhead. 93 [ParserPool](https://godoc.org/github.com/valyala/fastjson#ParserPool) may be useful in this case. 94 * Prefer calling `Value.Get*` on the value returned from [Parser](https://godoc.org/github.com/valyala/fastjson#Parser) 95 instead of calling `Get*` one-liners when multiple fields must be obtained from JSON, since each `Get*` one-liner 96 re-parses the input JSON again. 97 * Prefer calling once [Value.Get](https://godoc.org/github.com/valyala/fastjson#Value.Get) 98 for common prefix paths and then calling `Value.Get*` on the returned value for distinct suffix paths. 99 * Prefer iterating over array returned from [Value.GetArray](https://godoc.org/github.com/valyala/fastjson#Object.Visit) 100 with a range loop instead of calling `Value.Get*` for each array item. 101 102 ## Benchmarks 103 104 Go 1.12 has been used for benchmarking. 105 106 Legend: 107 108 * `small` - parse [small.json](testdata/small.json) (190 bytes). 109 * `medium` - parse [medium.json](testdata/medium.json) (2.3KB). 110 * `large` - parse [large.json](testdata/large.json) (28KB). 111 * `canada` - parse [canada.json](testdata/canada.json) (2.2MB). 112 * `citm` - parse [citm_catalog.json](testdata/citm_catalog.json) (1.7MB). 113 * `twitter` - parse [twitter.json](testdata/twitter.json) (617KB). 114 115 * `stdjson-map` - parse into a `map[string]interface{}` using `encoding/json`. 116 * `stdjson-struct` - parse into a struct containing a subset of fields of the parsed JSON, using `encoding/json`. 117 * `stdjson-empty-struct` - parse into an empty struct using `encoding/json`. This is the fastest possible solution 118 for `encoding/json`, may be used for json validation. See also benchmark results for json validation. 119 * `fastjson` - parse using `fastjson` without fields access. 120 * `fastjson-get` - parse using `fastjson` with fields access similar to `stdjson-struct`. 121 122 ``` 123 $ GOMAXPROCS=1 go test github.com/valyala/fastjson -bench='Parse$' 124 goos: linux 125 goarch: amd64 126 pkg: github.com/valyala/fastjson 127 BenchmarkParse/small/stdjson-map 200000 7305 ns/op 26.01 MB/s 960 B/op 51 allocs/op 128 BenchmarkParse/small/stdjson-struct 500000 3431 ns/op 55.37 MB/s 224 B/op 4 allocs/op 129 BenchmarkParse/small/stdjson-empty-struct 500000 2273 ns/op 83.58 MB/s 168 B/op 2 allocs/op 130 BenchmarkParse/small/fastjson 5000000 347 ns/op 547.53 MB/s 0 B/op 0 allocs/op 131 BenchmarkParse/small/fastjson-get 2000000 620 ns/op 306.39 MB/s 0 B/op 0 allocs/op 132 BenchmarkParse/medium/stdjson-map 30000 40672 ns/op 57.26 MB/s 10196 B/op 208 allocs/op 133 BenchmarkParse/medium/stdjson-struct 30000 47792 ns/op 48.73 MB/s 9174 B/op 258 allocs/op 134 BenchmarkParse/medium/stdjson-empty-struct 100000 22096 ns/op 105.40 MB/s 280 B/op 5 allocs/op 135 BenchmarkParse/medium/fastjson 500000 3025 ns/op 769.90 MB/s 0 B/op 0 allocs/op 136 BenchmarkParse/medium/fastjson-get 500000 3211 ns/op 725.20 MB/s 0 B/op 0 allocs/op 137 BenchmarkParse/large/stdjson-map 2000 614079 ns/op 45.79 MB/s 210734 B/op 2785 allocs/op 138 BenchmarkParse/large/stdjson-struct 5000 298554 ns/op 94.18 MB/s 15616 B/op 353 allocs/op 139 BenchmarkParse/large/stdjson-empty-struct 5000 268577 ns/op 104.69 MB/s 280 B/op 5 allocs/op 140 BenchmarkParse/large/fastjson 50000 35210 ns/op 798.56 MB/s 5 B/op 0 allocs/op 141 BenchmarkParse/large/fastjson-get 50000 35171 ns/op 799.46 MB/s 5 B/op 0 allocs/op 142 BenchmarkParse/canada/stdjson-map 20 68147307 ns/op 33.03 MB/s 12260502 B/op 392539 allocs/op 143 BenchmarkParse/canada/stdjson-struct 20 68044518 ns/op 33.08 MB/s 12260123 B/op 392534 allocs/op 144 BenchmarkParse/canada/stdjson-empty-struct 100 17709250 ns/op 127.11 MB/s 280 B/op 5 allocs/op 145 BenchmarkParse/canada/fastjson 300 4182404 ns/op 538.22 MB/s 254902 B/op 381 allocs/op 146 BenchmarkParse/canada/fastjson-get 300 4274744 ns/op 526.60 MB/s 254902 B/op 381 allocs/op 147 BenchmarkParse/citm/stdjson-map 50 27772612 ns/op 62.19 MB/s 5214163 B/op 95402 allocs/op 148 BenchmarkParse/citm/stdjson-struct 100 14936191 ns/op 115.64 MB/s 1989 B/op 75 allocs/op 149 BenchmarkParse/citm/stdjson-empty-struct 100 14946034 ns/op 115.56 MB/s 280 B/op 5 allocs/op 150 BenchmarkParse/citm/fastjson 1000 1879714 ns/op 918.87 MB/s 17628 B/op 30 allocs/op 151 BenchmarkParse/citm/fastjson-get 1000 1881598 ns/op 917.94 MB/s 17628 B/op 30 allocs/op 152 BenchmarkParse/twitter/stdjson-map 100 11289146 ns/op 55.94 MB/s 2187878 B/op 31266 allocs/op 153 BenchmarkParse/twitter/stdjson-struct 300 5779442 ns/op 109.27 MB/s 408 B/op 6 allocs/op 154 BenchmarkParse/twitter/stdjson-empty-struct 300 5738504 ns/op 110.05 MB/s 408 B/op 6 allocs/op 155 BenchmarkParse/twitter/fastjson 2000 774042 ns/op 815.86 MB/s 2541 B/op 2 allocs/op 156 BenchmarkParse/twitter/fastjson-get 2000 777833 ns/op 811.89 MB/s 2541 B/op 2 allocs/op 157 ``` 158 159 Benchmark results for json validation: 160 161 ``` 162 $ GOMAXPROCS=1 go test github.com/valyala/fastjson -bench='Validate$' 163 goos: linux 164 goarch: amd64 165 pkg: github.com/valyala/fastjson 166 BenchmarkValidate/small/stdjson 2000000 955 ns/op 198.83 MB/s 72 B/op 2 allocs/op 167 BenchmarkValidate/small/fastjson 5000000 384 ns/op 493.60 MB/s 0 B/op 0 allocs/op 168 BenchmarkValidate/medium/stdjson 200000 10799 ns/op 215.66 MB/s 184 B/op 5 allocs/op 169 BenchmarkValidate/medium/fastjson 300000 3809 ns/op 611.30 MB/s 0 B/op 0 allocs/op 170 BenchmarkValidate/large/stdjson 10000 133064 ns/op 211.31 MB/s 184 B/op 5 allocs/op 171 BenchmarkValidate/large/fastjson 30000 45268 ns/op 621.14 MB/s 0 B/op 0 allocs/op 172 BenchmarkValidate/canada/stdjson 200 8470904 ns/op 265.74 MB/s 184 B/op 5 allocs/op 173 BenchmarkValidate/canada/fastjson 500 2973377 ns/op 757.07 MB/s 0 B/op 0 allocs/op 174 BenchmarkValidate/citm/stdjson 200 7273172 ns/op 237.48 MB/s 184 B/op 5 allocs/op 175 BenchmarkValidate/citm/fastjson 1000 1684430 ns/op 1025.39 MB/s 0 B/op 0 allocs/op 176 BenchmarkValidate/twitter/stdjson 500 2849439 ns/op 221.63 MB/s 312 B/op 6 allocs/op 177 BenchmarkValidate/twitter/fastjson 2000 1036796 ns/op 609.10 MB/s 0 B/op 0 allocs/op 178 ``` 179 180 ## FAQ 181 182 * Q: _There are a ton of other high-perf packages for JSON parsing in Go. Why creating yet another package?_ 183 A: Because other packages require either rigid JSON schema via struct magic and code generation or perform poorly when 184 multiple unrelated fields must be obtained from the parsed JSON. Additionally, `fastjson` provides 185 nicer [API](http://godoc.org/github.com/valyala/fastjson). 186 187 * Q: _What is the main purpose for `fastjson`?_ 188 A: High-perf JSON parsing 189 for [RTB](https://www.iab.com/wp-content/uploads/2015/05/OpenRTB_API_Specification_Version_2_3_1.pdf) 190 and other [JSON-RPC](https://en.wikipedia.org/wiki/JSON-RPC) services. 191 192 * Q: _Why fastjson doesn't provide fast marshaling (serialization)?_ 193 A: Actually it provides some sort of marshaling - 194 see [Value.MarshalTo](https://godoc.org/github.com/valyala/fastjson#Value.MarshalTo). But I'd recommend 195 using [quicktemplate](https://github.com/valyala/quicktemplate#use-cases) 196 for high-performance JSON marshaling :) 197 198 * Q: _`fastjson` crashes my program!_ 199 A: There is high probability of improper use. 200 * Make sure you don't hold references to objects recursively returned by `Parser` / `Scanner` 201 beyond the next `Parser.Parse` / `Scanner.Next` call if such restriction is mentioned 202 in [docs](https://github.com/valyala/fastjson/issues/new). 203 * Make sure you don't access `fastjson` objects from concurrently running goroutines if such restriction is 204 mentioned in [docs](https://github.com/valyala/fastjson/issues/new). 205 * Build and run your program with [-race](https://golang.org/doc/articles/race_detector.html) flag. Make sure the 206 race detector detects zero races. 207 * If your program continue crashing after fixing issues mentioned 208 above, [file a bug](https://github.com/valyala/fastjson/issues/new).