github.com/relloyd/viper@v1.3.2-0.20190808092755-8ae060f4ca38/README.md (about) 1 ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png) 2 3 Go configuration with fangs! 4 5 Many Go projects are built using Viper including: 6 7 * [Hugo](http://gohugo.io) 8 * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) 9 * [Imgur’s Incus](https://github.com/Imgur/incus) 10 * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack) 11 * [Docker Notary](https://github.com/docker/Notary) 12 * [BloomApi](https://www.bloomapi.com/) 13 * [doctl](https://github.com/digitalocean/doctl) 14 * [Clairctl](https://github.com/jgsqware/clairctl) 15 16 [![Build Status](https://travis-ci.org/spf13/viper.svg)](https://travis-ci.org/spf13/viper) [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GoDoc](https://godoc.org/github.com/spf13/viper?status.svg)](https://godoc.org/github.com/spf13/viper) 17 18 19 ## What is Viper? 20 21 Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed 22 to work within an application, and can handle all types of configuration needs 23 and formats. It supports: 24 25 * setting defaults 26 * reading from JSON, TOML, YAML, HCL, and Java properties config files 27 * live watching and re-reading of config files (optional) 28 * reading from environment variables 29 * reading from remote config systems (etcd or Consul), and watching changes 30 * reading from command line flags 31 * reading from buffer 32 * setting explicit values 33 34 Viper can be thought of as a registry for all of your applications 35 configuration needs. 36 37 ## Why Viper? 38 39 When building a modern application, you don’t want to worry about 40 configuration file formats; you want to focus on building awesome software. 41 Viper is here to help with that. 42 43 Viper does the following for you: 44 45 1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats. 46 2. Provide a mechanism to set default values for your different 47 configuration options. 48 3. Provide a mechanism to set override values for options specified through 49 command line flags. 50 4. Provide an alias system to easily rename parameters without breaking existing 51 code. 52 5. Make it easy to tell the difference between when a user has provided a 53 command line or config file which is the same as the default. 54 55 Viper uses the following precedence order. Each item takes precedence over the 56 item below it: 57 58 * explicit call to Set 59 * flag 60 * env 61 * config 62 * key/value store 63 * default 64 65 Viper configuration keys are case insensitive by default. They can be made case 66 sensitive with `viper.SetKeyCaseSensitivity(true)`. 67 68 ## Putting Values into Viper 69 70 ### Establishing Defaults 71 72 A good configuration system will support default values. A default value is not 73 required for a key, but it’s useful in the event that a key hasn’t been set via 74 config file, environment variable, remote configuration or flag. 75 76 Examples: 77 78 ```go 79 viper.SetDefault("ContentDir", "content") 80 viper.SetDefault("LayoutDir", "layouts") 81 viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"}) 82 ``` 83 84 ### Reading Config Files 85 86 Viper requires minimal configuration so it knows where to look for config files. 87 Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but 88 currently a single Viper instance only supports a single configuration file. 89 Viper does not default to any configuration search paths leaving defaults decision 90 to an application. 91 92 Here is an example of how to use Viper to search for and read a configuration file. 93 None of the specific paths are required, but at least one path should be provided 94 where a configuration file is expected. 95 96 ```go 97 viper.SetConfigName("config") // name of config file (without extension) 98 viper.AddConfigPath("/etc/appname/") // path to look for the config file in 99 viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths 100 viper.AddConfigPath(".") // optionally look for config in the working directory 101 err := viper.ReadInConfig() // Find and read the config file 102 if err != nil { // Handle errors reading the config file 103 panic(fmt.Errorf("Fatal error config file: %s \n", err)) 104 } 105 ``` 106 107 ### Watching and re-reading config files 108 109 Viper supports the ability to have your application live read a config file while running. 110 111 Gone are the days of needing to restart a server to have a config take effect, 112 viper powered applications can read an update to a config file while running and 113 not miss a beat. 114 115 Simply tell the viper instance to watchConfig. 116 Optionally you can provide a function for Viper to run each time a change occurs. 117 118 **Make sure you add all of the configPaths prior to calling `WatchConfig()`** 119 120 ```go 121 viper.WatchConfig() 122 viper.OnConfigChange(func(e fsnotify.Event) { 123 fmt.Println("Config file changed:", e.Name) 124 }) 125 ``` 126 127 ### Reading Config from io.Reader 128 129 Viper predefines many configuration sources such as files, environment 130 variables, flags, and remote K/V store, but you are not bound to them. You can 131 also implement your own required configuration source and feed it to viper. 132 133 ```go 134 viper.SetConfigType("yaml") // or viper.SetConfigType("YAML") 135 136 // any approach to require this configuration into your program. 137 var yamlExample = []byte(` 138 Hacker: true 139 name: steve 140 hobbies: 141 - skateboarding 142 - snowboarding 143 - go 144 clothing: 145 jacket: leather 146 trousers: denim 147 age: 35 148 eyes : brown 149 beard: true 150 `) 151 152 viper.ReadConfig(bytes.NewBuffer(yamlExample)) 153 154 viper.Get("name") // this would be "steve" 155 ``` 156 157 ### Setting Overrides 158 159 These could be from a command line flag, or from your own application logic. 160 161 ```go 162 viper.Set("Verbose", true) 163 viper.Set("LogFile", LogFile) 164 ``` 165 166 ### Registering and Using Aliases 167 168 Aliases permit a single value to be referenced by multiple keys 169 170 ```go 171 viper.RegisterAlias("loud", "Verbose") 172 173 viper.Set("verbose", true) // same result as next line 174 viper.Set("loud", true) // same result as prior line 175 176 viper.GetBool("loud") // true 177 viper.GetBool("verbose") // true 178 ``` 179 180 ### Working with Environment Variables 181 182 Viper has full support for environment variables. This enables 12 factor 183 applications out of the box. There are five methods that exist to aid working 184 with ENV: 185 186 * `AutomaticEnv()` 187 * `BindEnv(string...) : error` 188 * `SetEnvPrefix(string)` 189 * `SetEnvKeyReplacer(string...) *strings.Replacer` 190 * `AllowEmptyEnvVar(bool)` 191 192 _When working with ENV variables, it’s important to recognize that Viper 193 treats ENV variables as case sensitive._ 194 195 Viper provides a mechanism to try to ensure that ENV variables are unique. By 196 using `SetEnvPrefix`, you can tell Viper to use a prefix while reading from 197 the environment variables. Both `BindEnv` and `AutomaticEnv` will use this 198 prefix. 199 200 `BindEnv` takes one or two parameters. The first parameter is the key name, the 201 second is the name of the environment variable. The name of the environment 202 variable is case sensitive. If the ENV variable name is not provided, then 203 Viper will automatically assume that the key name matches the ENV variable name, 204 but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV 205 variable name, it **does not** automatically add the prefix. 206 207 One important thing to recognize when working with ENV variables is that the 208 value will be read each time it is accessed. Viper does not fix the value when 209 the `BindEnv` is called. 210 211 `AutomaticEnv` is a powerful helper especially when combined with 212 `SetEnvPrefix`. When called, Viper will check for an environment variable any 213 time a `viper.Get` request is made. It will apply the following rules. It will 214 check for a environment variable with a name matching the key uppercased and 215 prefixed with the `EnvPrefix` if set. 216 217 `SetEnvKeyReplacer` allows you to use a `strings.Replacer` object to rewrite Env 218 keys to an extent. This is useful if you want to use `-` or something in your 219 `Get()` calls, but want your environmental variables to use `_` delimiters. An 220 example of using it can be found in `viper_test.go`. 221 222 By default empty environment variables are considered unset and will fall back to 223 the next configuration source. To treat empty environment variables as set, use 224 the `AllowEmptyEnv` method. 225 226 #### Env example 227 228 ```go 229 SetEnvPrefix("spf") // will be uppercased automatically 230 BindEnv("id") 231 232 os.Setenv("SPF_ID", "13") // typically done outside of the app 233 234 id := Get("id") // 13 235 ``` 236 237 ### Working with Flags 238 239 Viper has the ability to bind to flags. Specifically, Viper supports `Pflags` 240 as used in the [Cobra](https://github.com/spf13/cobra) library. 241 242 Like `BindEnv`, the value is not set when the binding method is called, but when 243 it is accessed. This means you can bind as early as you want, even in an 244 `init()` function. 245 246 For individual flags, the `BindPFlag()` method provides this functionality. 247 248 Example: 249 250 ```go 251 serverCmd.Flags().Int("port", 1138, "Port to run Application server on") 252 viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) 253 ``` 254 255 You can also bind an existing set of pflags (pflag.FlagSet): 256 257 Example: 258 259 ```go 260 pflag.Int("flagname", 1234, "help message for flagname") 261 262 pflag.Parse() 263 viper.BindPFlags(pflag.CommandLine) 264 265 i := viper.GetInt("flagname") // retrieve values from viper instead of pflag 266 ``` 267 268 The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude 269 the use of other packages that use the [flag](https://golang.org/pkg/flag/) 270 package from the standard library. The pflag package can handle the flags 271 defined for the flag package by importing these flags. This is accomplished 272 by a calling a convenience function provided by the pflag package called 273 AddGoFlagSet(). 274 275 Example: 276 277 ```go 278 package main 279 280 import ( 281 "flag" 282 "github.com/spf13/pflag" 283 ) 284 285 func main() { 286 287 // using standard library "flag" package 288 flag.Int("flagname", 1234, "help message for flagname") 289 290 pflag.CommandLine.AddGoFlagSet(flag.CommandLine) 291 pflag.Parse() 292 viper.BindPFlags(pflag.CommandLine) 293 294 i := viper.GetInt("flagname") // retrieve value from viper 295 296 ... 297 } 298 ``` 299 300 #### Flag interfaces 301 302 Viper provides two Go interfaces to bind other flag systems if you don’t use `Pflags`. 303 304 `FlagValue` represents a single flag. This is a very simple example on how to implement this interface: 305 306 ```go 307 type myFlag struct {} 308 func (f myFlag) HasChanged() bool { return false } 309 func (f myFlag) Name() string { return "my-flag-name" } 310 func (f myFlag) ValueString() string { return "my-flag-value" } 311 func (f myFlag) ValueType() string { return "string" } 312 ``` 313 314 Once your flag implements this interface, you can simply tell Viper to bind it: 315 316 ```go 317 viper.BindFlagValue("my-flag-name", myFlag{}) 318 ``` 319 320 `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface: 321 322 ```go 323 type myFlagSet struct { 324 flags []myFlag 325 } 326 327 func (f myFlagSet) VisitAll(fn func(FlagValue)) { 328 for _, flag := range flags { 329 fn(flag) 330 } 331 } 332 ``` 333 334 Once your flag set implements this interface, you can simply tell Viper to bind it: 335 336 ```go 337 fSet := myFlagSet{ 338 flags: []myFlag{myFlag{}, myFlag{}}, 339 } 340 viper.BindFlagValues("my-flags", fSet) 341 ``` 342 343 ### Remote Key/Value Store Support 344 345 To enable remote support in Viper, do a blank import of the `viper/remote` 346 package: 347 348 `import _ "github.com/spf13/viper/remote"` 349 350 Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path 351 in a Key/Value store such as etcd or Consul. These values take precedence over 352 default values, but are overridden by configuration values retrieved from disk, 353 flags, or environment variables. 354 355 Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve 356 configuration from the K/V store, which means that you can store your 357 configuration values encrypted and have them automatically decrypted if you have 358 the correct gpg keyring. Encryption is optional. 359 360 You can use remote configuration in conjunction with local configuration, or 361 independently of it. 362 363 `crypt` has a command-line helper that you can use to put configurations in your 364 K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. 365 366 ```bash 367 $ go get github.com/xordataexchange/crypt/bin/crypt 368 $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json 369 ``` 370 371 Confirm that your value was set: 372 373 ```bash 374 $ crypt get -plaintext /config/hugo.json 375 ``` 376 377 See the `crypt` documentation for examples of how to set encrypted values, or 378 how to use Consul. 379 380 ### Remote Key/Value Store Example - Unencrypted 381 382 #### etcd 383 ```go 384 viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json") 385 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 386 err := viper.ReadRemoteConfig() 387 ``` 388 389 #### Consul 390 You need to set a key to Consul key/value storage with JSON value containing your desired config. 391 For example, create a Consul key/value store key `MY_CONSUL_KEY` with value: 392 393 ```json 394 { 395 "port": 8080, 396 "hostname": "myhostname.com" 397 } 398 ``` 399 400 ```go 401 viper.AddRemoteProvider("consul", "localhost:8500", "MY_CONSUL_KEY") 402 viper.SetConfigType("json") // Need to explicitly set this to json 403 err := viper.ReadRemoteConfig() 404 405 fmt.Println(viper.Get("port")) // 8080 406 fmt.Println(viper.Get("hostname")) // myhostname.com 407 ``` 408 409 ### Remote Key/Value Store Example - Encrypted 410 411 ```go 412 viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") 413 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 414 err := viper.ReadRemoteConfig() 415 ``` 416 417 ### Watching Changes in etcd - Unencrypted 418 419 ```go 420 // alternatively, you can create a new viper instance. 421 var runtime_viper = viper.New() 422 423 runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml") 424 runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 425 426 // read from remote config the first time. 427 err := runtime_viper.ReadRemoteConfig() 428 429 // unmarshal config 430 runtime_viper.Unmarshal(&runtime_conf) 431 432 // open a goroutine to watch remote changes forever 433 go func(){ 434 for { 435 time.Sleep(time.Second * 5) // delay after each request 436 437 // currently, only tested with etcd support 438 err := runtime_viper.WatchRemoteConfig() 439 if err != nil { 440 log.Errorf("unable to read remote config: %v", err) 441 continue 442 } 443 444 // unmarshal new config into our runtime config struct. you can also use channel 445 // to implement a signal to notify the system of the changes 446 runtime_viper.Unmarshal(&runtime_conf) 447 } 448 }() 449 ``` 450 451 ## Getting Values From Viper 452 453 In Viper, there are a few ways to get a value depending on the value’s type. 454 The following functions and methods exist: 455 456 * `Get(key string) : interface{}` 457 * `GetBool(key string) : bool` 458 * `GetFloat64(key string) : float64` 459 * `GetInt(key string) : int` 460 * `GetString(key string) : string` 461 * `GetStringMap(key string) : map[string]interface{}` 462 * `GetStringMapString(key string) : map[string]string` 463 * `GetStringSlice(key string) : []string` 464 * `GetTime(key string) : time.Time` 465 * `GetDuration(key string) : time.Duration` 466 * `IsSet(key string) : bool` 467 * `AllSettings() : map[string]interface{}` 468 469 One important thing to recognize is that each Get function will return a zero 470 value if it’s not found. To check if a given key exists, the `IsSet()` method 471 has been provided. 472 473 Example: 474 ```go 475 viper.GetString("logfile") // case-insensitive Setting & Getting 476 if viper.GetBool("verbose") { 477 fmt.Println("verbose enabled") 478 } 479 ``` 480 ### Accessing nested keys 481 482 The accessor methods also accept formatted paths to deeply nested keys. For 483 example, if the following JSON file is loaded: 484 485 ```json 486 { 487 "host": { 488 "address": "localhost", 489 "port": 5799 490 }, 491 "datastore": { 492 "metric": { 493 "host": "127.0.0.1", 494 "port": 3099 495 }, 496 "warehouse": { 497 "host": "198.0.0.1", 498 "port": 2112 499 } 500 } 501 } 502 503 ``` 504 505 Viper can access a nested field by passing a `.` delimited path of keys: 506 507 ```go 508 GetString("datastore.metric.host") // (returns "127.0.0.1") 509 ``` 510 511 This obeys the precedence rules established above; the search for the path 512 will cascade through the remaining configuration registries until found. 513 514 For example, given this configuration file, both `datastore.metric.host` and 515 `datastore.metric.port` are already defined (and may be overridden). If in addition 516 `datastore.metric.protocol` was defined in the defaults, Viper would also find it. 517 518 However, if `datastore.metric` was overridden (by a flag, an environment variable, 519 the `Set()` method, …) with an immediate value, then all sub-keys of 520 `datastore.metric` become undefined, they are “shadowed” by the higher-priority 521 configuration level. 522 523 Lastly, if there exists a key that matches the delimited key path, its value 524 will be returned instead. E.g. 525 526 ```json 527 { 528 "datastore.metric.host": "0.0.0.0", 529 "host": { 530 "address": "localhost", 531 "port": 5799 532 }, 533 "datastore": { 534 "metric": { 535 "host": "127.0.0.1", 536 "port": 3099 537 }, 538 "warehouse": { 539 "host": "198.0.0.1", 540 "port": 2112 541 } 542 } 543 } 544 545 GetString("datastore.metric.host") // returns "0.0.0.0" 546 ``` 547 548 ### Extract sub-tree 549 550 Extract sub-tree from Viper. 551 552 For example, `viper` represents: 553 554 ```json 555 app: 556 cache1: 557 max-items: 100 558 item-size: 64 559 cache2: 560 max-items: 200 561 item-size: 80 562 ``` 563 564 After executing: 565 566 ```go 567 subv := viper.Sub("app.cache1") 568 ``` 569 570 `subv` represents: 571 572 ```json 573 max-items: 100 574 item-size: 64 575 ``` 576 577 Suppose we have: 578 579 ```go 580 func NewCache(cfg *Viper) *Cache {...} 581 ``` 582 583 which creates a cache based on config information formatted as `subv`. 584 Now it’s easy to create these 2 caches separately as: 585 586 ```go 587 cfg1 := viper.Sub("app.cache1") 588 cache1 := NewCache(cfg1) 589 590 cfg2 := viper.Sub("app.cache2") 591 cache2 := NewCache(cfg2) 592 ``` 593 594 ### Unmarshaling 595 596 You also have the option of Unmarshaling all or a specific value to a struct, map, 597 etc. 598 599 There are two methods to do this: 600 601 * `Unmarshal(rawVal interface{}) : error` 602 * `UnmarshalKey(key string, rawVal interface{}) : error` 603 604 Example: 605 606 ```go 607 type config struct { 608 Port int 609 Name string 610 PathMap string `mapstructure:"path_map"` 611 } 612 613 var C config 614 615 err := Unmarshal(&C) 616 if err != nil { 617 t.Fatalf("unable to decode into struct, %v", err) 618 } 619 ``` 620 621 ### Marshalling to string 622 623 You may need to marhsal all the settings held in viper into a string rather than write them to a file. 624 You can use your favorite format's marshaller with the config returned by `AllSettings()`. 625 626 ```go 627 import ( 628 yaml "gopkg.in/yaml.v2" 629 // ... 630 ) 631 632 func yamlStringSettings() string { 633 c := viper.AllSettings() 634 bs, err := yaml.Marshal(c) 635 if err != nil { 636 t.Fatalf("unable to marshal config to YAML: %v", err) 637 } 638 return string(bs) 639 } 640 ``` 641 642 ## Viper or Vipers? 643 644 Viper comes ready to use out of the box. There is no configuration or 645 initialization needed to begin using Viper. Since most applications will want 646 to use a single central repository for their configuration, the viper package 647 provides this. It is similar to a singleton. 648 649 In all of the examples above, they demonstrate using viper in its singleton 650 style approach. 651 652 ### Working with multiple vipers 653 654 You can also create many different vipers for use in your application. Each will 655 have its own unique set of configurations and values. Each can read from a 656 different config file, key value store, etc. All of the functions that viper 657 package supports are mirrored as methods on a viper. 658 659 Example: 660 661 ```go 662 x := viper.New() 663 y := viper.New() 664 665 x.SetDefault("ContentDir", "content") 666 y.SetDefault("ContentDir", "foobar") 667 668 //... 669 ``` 670 671 When working with multiple vipers, it is up to the user to keep track of the 672 different vipers. 673 674 ## Q & A 675 676 Q: Why not INI files? 677 678 A: Ini files are pretty awful. There’s no standard format, and they are hard to 679 validate. Viper is designed to work with JSON, TOML or YAML files. If someone 680 really wants to add this feature, I’d be happy to merge it. It’s easy to specify 681 which formats your application will permit. 682 683 Q: Why is it called “Viper”? 684 685 A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) 686 to [Cobra](https://github.com/spf13/cobra). While both can operate completely 687 independently, together they make a powerful pair to handle much of your 688 application foundation needs. 689 690 Q: Why is it called “Cobra”? 691 692 A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)?