github.com/bingoohuang/gg@v0.0.0-20240325092523-45da7dee9335/pkg/yaml/README.md (about) 1 # YAML support for the Go language 2 3 1. Keep synchronized with [original](https://github.com/goccy/go-yaml) to `864ce75@2021-08-25T11:36:45+09:00` 4 5 ## Enhanced features 6 7 1. Labeled Decoder 8 2. Type Decoder 9 3. KeyNaming strategies when encoding or decoding 10 11 ### Labeled Decoder 12 13 ```go 14 type Config struct { 15 Size int64 `yaml:",label=size"` 16 } 17 18 func decodeSize(node ast.Node, typ reflect.Type) (reflect.Value, error) { 19 if v, ok := node.(*ast.StringNode); ok { 20 if v, err := man.ParseBytes(v.Value); err != nil { 21 return reflect.Value{}, err 22 } else { 23 return yaml.CastUint64(v, typ) 24 } 25 } 26 return reflect.Value{}, yaml.ErrContinue 27 } 28 29 func TestDecoderLabel(t *testing.T) { 30 c := Config{} 31 decodeOption := yaml.LabelDecoder("size", decodeSize) 32 decoder := yaml.NewDecoder(strings.NewReader(`size: 10MiB`), decodeOption) 33 err := decoder.Decode(&c) 34 assert.Equal(t, Config{Size: 10 * 1024 * 1024}, c) 35 36 decoder = yaml.NewDecoder(strings.NewReader(`size: 10,485,761`), decodeOption) 37 c.Size = 0 38 err = decoder.Decode(&c) 39 assert.Equal(t, Config{Size: 10485761}, c) 40 } 41 ``` 42 43 ### Type Decoder 44 45 ```go 46 47 type Duration struct { 48 Dur time.Duration 49 } 50 51 func decodeDuration(node ast.Node, typ reflect.Type) (reflect.Value, error) { 52 if v, ok := node.(*ast.StringNode); ok { 53 d, err := time.ParseDuration(v.Value) 54 return reflect.ValueOf(d), err 55 } 56 return reflect.Value{}, yaml.ErrContinue 57 } 58 59 func TestDecoderDuration(t *testing.T) { 60 decodeOption := yaml.TypeDecoder(reflect.TypeOf((*time.Duration)(nil)).Elem(), decodeDuration) 61 decoder := yaml.NewDecoder(strings.NewReader(`dur: 10s`), decodeOption) 62 c := Duration{} 63 err := decoder.Decode(&c) 64 assert.Equal(t, Duration{Dur: 10 * time.Second}, c) 65 66 decoder = yaml.NewDecoder(strings.NewReader(`dur: 111`), decodeOption) 67 err = decoder.Decode(&c) 68 assert.Equal(t, Duration{Dur: 111}, c) 69 } 70 ``` 71 72 ### KeyNaming strategies when decoding 73 74 ```go 75 type UseFieldName struct { 76 Size int64 77 } 78 79 func TestUseFieldName(t *testing.T) { 80 c := UseFieldName{} 81 decoder := yaml.NewDecoder(strings.NewReader(`Size: 10`)) 82 err := decoder.Decode(&c) 83 assert.Equal(t, UseFieldName{Size: 10}, c) 84 85 var buf bytes.Buffer 86 err = yaml.NewEncoder(&buf, yaml.KeyNaming(yaml.KeyNamingRaw)).Encode(c) 87 assert.Equal(t, "Size: 10\n", buf.String()) 88 } 89 90 ``` 91 92 <img width="300px" src="https://user-images.githubusercontent.com/209884/67159116-64d94b80-f37b-11e9-9b28-f8379636a43c.png"></img> 93 94 ## Why a new library? 95 96 As of this writing, there already exists a de facto standard library for YAML processing for 97 Go: [https://github.com/go-yaml/yaml](https://github.com/go-yaml/yaml). However we feel that some features are lacking, 98 namely: 99 100 - Pretty format for error notifications 101 - Direct manipulation of YAML abstract syntax tree 102 - Support for `Anchor` and `Alias` when marshaling 103 - Allow referencing elements declared in another file via anchors 104 105 ## Features 106 107 - Pretty format for error notifications 108 - Supports `Scanner` or `Lexer` or `Parser` as public API 109 - Supports `Anchor` and `Alias` to Marshaler 110 - Allow referencing elements declared in another file via anchors 111 - Extract value or AST by YAMLPath ( YAMLPath is like a JSONPath ) 112 113 ## Installation 114 115 ```sh 116 go get -u github.com/bingoohuang/gg/pkg/yaml 117 ``` 118 119 # Synopsis 120 121 ## 1. Simple Encode/Decode 122 123 Has an interface like `go-yaml/yaml` using `reflect` 124 125 ```go 126 var v struct { 127 A int 128 B string 129 } 130 v.A = 1 131 v.B = "hello" 132 bytes, err := yaml.Marshal(v) 133 if err != nil { 134 //... 135 } 136 fmt.Println(string(bytes)) // "a: 1\nb: hello\n" 137 ``` 138 139 ```go 140 yml := ` 141 %YAML 1.2 142 --- 143 a: 1 144 b: c 145 ` 146 var v struct { 147 A int 148 B string 149 } 150 if err := yaml.Unmarshal([]byte(yml), &v); err != nil { 151 //... 152 } 153 ``` 154 155 To control marshal/unmarshal behavior, you can use the `yaml` tag. 156 157 ```go 158 yml := `--- 159 foo: 1 160 bar: c 161 ` 162 var v struct { 163 A int `yaml:"foo"` 164 B string `yaml:"bar"` 165 } 166 if err := yaml.Unmarshal([]byte(yml), &v); err != nil { 167 //... 168 } 169 ``` 170 171 For convenience, we also accept the `json` tag. Note that not all options from the `json` tag will have significance 172 when parsing YAML documents. If both tags exist, `yaml` tag will take precedence. 173 174 ```go 175 yml := `--- 176 foo: 1 177 bar: c 178 ` 179 var v struct { 180 A int `json:"foo"` 181 B string `json:"bar"` 182 } 183 if err := yaml.Unmarshal([]byte(yml), &v); err != nil { 184 //... 185 } 186 ``` 187 188 For custom marshal/unmarshaling, implement either `Bytes` or `Interface` variant of marshaler/unmarshaler. The 189 difference is that while `BytesMarshaler`/`BytesUnmarshaler` behaves 190 like [`encoding/json`](https://pkg.go.dev/encoding/json) and `InterfaceMarshaler`/`InterfaceUnmarshaler` behaves 191 like [`gopkg.in/yaml.v2`](https://pkg.go.dev/gopkg.in/yaml.v2). 192 193 Semantically both are the same, but they differ in performance. Because indentation matters in YAML, you cannot simply 194 accept a valid YAML fragment from a Marshaler, and expect it to work when it is attached to the parent container's 195 serialized form. Therefore when we receive use the `BytesMarshaler`, which returns `[]byte`, we must decode it once to 196 figure out how to make it work in the given context. If you use the `InterfaceMarshaler`, we can skip the decoding. 197 198 If you are repeatedly marshaling complex objects, the latter is always better performance wise. But if you are, for 199 example, just providing a choice between a config file format that is read only once, the former is probably easier to 200 code. 201 202 ## 2. Reference elements declared in another file 203 204 `testdata` directory contains `anchor.yml` file: 205 206 ```shell 207 ├── testdata 208 └── anchor.yml 209 ``` 210 211 And `anchor.yml` is defined as follows: 212 213 ```yaml 214 a: &a 215 b: 1 216 c: hello 217 ``` 218 219 Then, if `yaml.ReferenceDirs("testdata")` option is passed to `yaml.Decoder`, 220 `Decoder` tries to find the anchor definition from YAML files the under `testdata` directory. 221 222 ```go 223 buf := bytes.NewBufferString("a: *a\n") 224 dec := yaml.NewDecoder(buf, yaml.ReferenceDirs("testdata")) 225 var v struct { 226 A struct { 227 B int 228 C string 229 } 230 } 231 if err := dec.Decode(&v); err != nil { 232 //... 233 } 234 fmt.Printf("%+v\n", v) // {A:{B:1 C:hello}} 235 ``` 236 237 ## 3. Encode with `Anchor` and `Alias` 238 239 ### 3.1. Explicitly declared `Anchor` name and `Alias` name 240 241 If you want to use `anchor` or `alias`, you can define it as a struct tag. 242 243 ```go 244 type T struct { 245 A int 246 B string 247 } 248 var v struct { 249 C *T `yaml:"c,anchor=x"` 250 D *T `yaml:"d,alias=x"` 251 } 252 v.C = &T{A: 1, B: "hello"} 253 v.D = v.C 254 bytes, err := yaml.Marshal(v) 255 if err != nil { 256 panic(err) 257 } 258 fmt.Println(string(bytes)) 259 /* 260 c: &x 261 a: 1 262 b: hello 263 d: *x 264 */ 265 ``` 266 267 ### 3.2. Implicitly declared `Anchor` and `Alias` names 268 269 If you do not explicitly declare the anchor name, the default behavior is to use the equivalent 270 of `strings.ToLower($FieldName)` as the name of the anchor. 271 272 If you do not explicitly declare the alias name AND the value is a pointer to another element, we look up the anchor 273 name by finding out which anchor field the value is assigned to by looking up its pointer address. 274 275 ```go 276 type T struct { 277 I int 278 S string 279 } 280 var v struct { 281 A *T `yaml:"a,anchor"` 282 B *T `yaml:"b,anchor"` 283 C *T `yaml:"c,alias"` 284 D *T `yaml:"d,alias"` 285 } 286 v.A = &T{I: 1, S: "hello"} 287 v.B = &T{I: 2, S: "world"} 288 v.C = v.A // C has same pointer address to A 289 v.D = v.B // D has same pointer address to B 290 bytes, err := yaml.Marshal(v) 291 if err != nil { 292 //... 293 } 294 fmt.Println(string(bytes)) 295 /* 296 a: &a 297 i: 1 298 s: hello 299 b: &b 300 i: 2 301 s: world 302 c: *a 303 d: *b 304 */ 305 ``` 306 307 ### 3.3 MergeKey and Alias 308 309 Merge key and alias ( `<<: *alias` ) can be used by embedding a structure with the `inline,alias` tag. 310 311 ```go 312 type Person struct { 313 *Person `yaml:",omitempty,inline,alias"` // embed Person type for default value 314 Name string `yaml:",omitempty"` 315 Age int `yaml:",omitempty"` 316 } 317 defaultPerson := &Person{ 318 Name: "John Smith", 319 Age: 20, 320 } 321 people := []*Person{ 322 { 323 Person: defaultPerson, // assign default value 324 Name: "Ken", // override Name property 325 Age: 10, // override Age property 326 }, 327 { 328 Person: defaultPerson, // assign default value only 329 }, 330 } 331 var doc struct { 332 Default *Person `yaml:"default,anchor"` 333 People []*Person `yaml:"people"` 334 } 335 doc.Default = defaultPerson 336 doc.People = people 337 bytes, err := yaml.Marshal(doc) 338 if err != nil { 339 //... 340 } 341 fmt.Println(string(bytes)) 342 /* 343 default: &default 344 name: John Smith 345 age: 20 346 people: 347 - <<: *default 348 name: Ken 349 age: 10 350 - <<: *default 351 */ 352 ``` 353 354 ## 4. Pretty Formatted Errors 355 356 Error values produced during parsing have two extra features over regular error values. 357 358 First, by default, they contain extra information on the location of the error from the source YAML document, to make it 359 easier to find the error location. 360 361 Second, the error messages can optionally be colorized. 362 363 If you would like to control exactly how the output looks like, consider using `yaml.FormatError`, which accepts two 364 boolean values to control turning these features on or off. 365 366 <img src="https://user-images.githubusercontent.com/209884/67358124-587f0980-f59a-11e9-96fc-7205aab77695.png"></img> 367 368 ## 5. Use YAMLPath 369 370 ```go 371 yml := ` 372 store: 373 book: 374 - author: john 375 price: 10 376 - author: ken 377 price: 12 378 bicycle: 379 color: red 380 price: 19.95 381 ` 382 path, err := yaml.PathString("$.store.book[*].author") 383 if err != nil { 384 //... 385 } 386 var authors []string 387 if err := path.Read(strings.NewReader(yml), &authors); err != nil { 388 //... 389 } 390 fmt.Println(authors) 391 // [john ken] 392 ``` 393 394 ### 5.1 Print customized error with YAML source code 395 396 ```go 397 package main 398 399 import ( 400 "fmt" 401 402 "github.com/bingoohuang/go-yaml" 403 ) 404 405 func main() { 406 yml := ` 407 a: 1 408 b: "hello" 409 ` 410 var v struct { 411 A int 412 B string 413 } 414 if err := yaml.Unmarshal([]byte(yml), &v); err != nil { 415 panic(err) 416 } 417 if v.A != 2 { 418 // output error with YAML source 419 path, err := yaml.PathString("$.a") 420 if err != nil { 421 panic(err) 422 } 423 source, err := path.AnnotateSource([]byte(yml), true) 424 if err != nil { 425 panic(err) 426 } 427 fmt.Printf("a value expected 2 but actual %d:\n%s\n", v.A, string(source)) 428 } 429 } 430 ``` 431 432 output result is the following: 433 434 <img src="https://user-images.githubusercontent.com/209884/84148813-7aca8680-aa9a-11ea-8fc9-37dece2ebdac.png"></img> 435 436 # Tools 437 438 ## ycat 439 440 print yaml file with color 441 442 <img width="713" alt="ycat" src="https://user-images.githubusercontent.com/209884/66986084-19b00600-f0f9-11e9-9f0e-1f91eb072fe0.png"> 443 444 ### Installation 445 446 ```sh 447 go get -u github.com/bingoohuang/go-yaml/cmd/ycat 448 ``` 449 450 # License 451 452 MIT