github.com/dhillondeep/viper@v1.3.3-0.20190507190913-f3f73ccde32e/README.md (about) 1 ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png) 2 3 Go configuration with fangs! 4 5 Many Go projects are built using Viper including: 6 7 * [Hugo](http://gohugo.io) 8 * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) 9 * [Imgur’s Incus](https://github.com/Imgur/incus) 10 * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack) 11 * [Docker Notary](https://github.com/docker/Notary) 12 * [BloomApi](https://www.bloomapi.com/) 13 * [doctl](https://github.com/digitalocean/doctl) 14 * [Clairctl](https://github.com/jgsqware/clairctl) 15 16 [![Build Status](https://travis-ci.org/spf13/viper.svg)](https://travis-ci.org/spf13/viper) [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GoDoc](https://godoc.org/github.com/spf13/viper?status.svg)](https://godoc.org/github.com/spf13/viper) 17 18 19 ## What is Viper? 20 21 Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed 22 to work within an application, and can handle all types of configuration needs 23 and formats. It supports: 24 25 * setting defaults 26 * reading from JSON, TOML, YAML, HCL, and Java properties config files 27 * live watching and re-reading of config files (optional) 28 * reading from environment variables 29 * reading from remote config systems (etcd or Consul), and watching changes 30 * reading from command line flags 31 * reading from buffer 32 * setting explicit values 33 34 Viper can be thought of as a registry for all of your applications 35 configuration needs. 36 37 ## Why Viper? 38 39 When building a modern application, you don’t want to worry about 40 configuration file formats; you want to focus on building awesome software. 41 Viper is here to help with that. 42 43 Viper does the following for you: 44 45 1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats. 46 2. Provide a mechanism to set default values for your different 47 configuration options. 48 3. Provide a mechanism to set override values for options specified through 49 command line flags. 50 4. Provide an alias system to easily rename parameters without breaking existing 51 code. 52 5. Make it easy to tell the difference between when a user has provided a 53 command line or config file which is the same as the default. 54 55 Viper uses the following precedence order. Each item takes precedence over the 56 item below it: 57 58 * explicit call to Set 59 * flag 60 * env 61 * config 62 * key/value store 63 * default 64 65 Viper configuration keys are case insensitive. 66 67 ## Putting Values into Viper 68 69 ### Establishing Defaults 70 71 A good configuration system will support default values. A default value is not 72 required for a key, but it’s useful in the event that a key hasn’t been set via 73 config file, environment variable, remote configuration or flag. 74 75 Examples: 76 77 ```go 78 viper.SetDefault("ContentDir", "content") 79 viper.SetDefault("LayoutDir", "layouts") 80 viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"}) 81 ``` 82 83 ### Reading Config Files 84 85 Viper requires minimal configuration so it knows where to look for config files. 86 Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but 87 currently a single Viper instance only supports a single configuration file. 88 Viper does not default to any configuration search paths leaving defaults decision 89 to an application. 90 91 Here is an example of how to use Viper to search for and read a configuration file. 92 None of the specific paths are required, but at least one path should be provided 93 where a configuration file is expected. 94 95 ```go 96 viper.SetConfigName("config") // name of config file (without extension) 97 viper.AddConfigPath("/etc/appname/") // path to look for the config file in 98 viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths 99 viper.AddConfigPath(".") // optionally look for config in the working directory 100 err := viper.ReadInConfig() // Find and read the config file 101 if err != nil { // Handle errors reading the config file 102 panic(fmt.Errorf("Fatal error config file: %s \n", err)) 103 } 104 ``` 105 106 ### Watching and re-reading config files 107 108 Viper supports the ability to have your application live read a config file while running. 109 110 Gone are the days of needing to restart a server to have a config take effect, 111 viper powered applications can read an update to a config file while running and 112 not miss a beat. 113 114 Simply tell the viper instance to watchConfig. 115 Optionally you can provide a function for Viper to run each time a change occurs. 116 117 **Make sure you add all of the configPaths prior to calling `WatchConfig()`** 118 119 ```go 120 viper.WatchConfig() 121 viper.OnConfigChange(func(e fsnotify.Event) { 122 fmt.Println("Config file changed:", e.Name) 123 }) 124 ``` 125 126 ### Reading Config from io.Reader 127 128 Viper predefines many configuration sources such as files, environment 129 variables, flags, and remote K/V store, but you are not bound to them. You can 130 also implement your own required configuration source and feed it to viper. 131 132 ```go 133 viper.SetConfigType("yaml") // or viper.SetConfigType("YAML") 134 135 // any approach to require this configuration into your program. 136 var yamlExample = []byte(` 137 Hacker: true 138 name: steve 139 hobbies: 140 - skateboarding 141 - snowboarding 142 - go 143 clothing: 144 jacket: leather 145 trousers: denim 146 age: 35 147 eyes : brown 148 beard: true 149 `) 150 151 viper.ReadConfig(bytes.NewBuffer(yamlExample)) 152 153 viper.Get("name") // this would be "steve" 154 ``` 155 156 ### Setting Overrides 157 158 These could be from a command line flag, or from your own application logic. 159 160 ```go 161 viper.Set("Verbose", true) 162 viper.Set("LogFile", LogFile) 163 ``` 164 165 ### Registering and Using Aliases 166 167 Aliases permit a single value to be referenced by multiple keys 168 169 ```go 170 viper.RegisterAlias("loud", "Verbose") 171 172 viper.Set("verbose", true) // same result as next line 173 viper.Set("loud", true) // same result as prior line 174 175 viper.GetBool("loud") // true 176 viper.GetBool("verbose") // true 177 ``` 178 179 ### Working with Environment Variables 180 181 Viper has full support for environment variables. This enables 12 factor 182 applications out of the box. There are five methods that exist to aid working 183 with ENV: 184 185 * `AutomaticEnv()` 186 * `BindEnv(string...) : error` 187 * `SetEnvPrefix(string)` 188 * `SetEnvKeyReplacer(string...) *strings.Replacer` 189 * `AllowEmptyEnvVar(bool)` 190 191 _When working with ENV variables, it’s important to recognize that Viper 192 treats ENV variables as case sensitive._ 193 194 Viper provides a mechanism to try to ensure that ENV variables are unique. By 195 using `SetEnvPrefix`, you can tell Viper to use a prefix while reading from 196 the environment variables. Both `BindEnv` and `AutomaticEnv` will use this 197 prefix. 198 199 `BindEnv` takes one or two parameters. The first parameter is the key name, the 200 second is the name of the environment variable. The name of the environment 201 variable is case sensitive. If the ENV variable name is not provided, then 202 Viper will automatically assume that the key name matches the ENV variable name, 203 but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV 204 variable name, it **does not** automatically add the prefix. 205 206 One important thing to recognize when working with ENV variables is that the 207 value will be read each time it is accessed. Viper does not fix the value when 208 the `BindEnv` is called. 209 210 `AutomaticEnv` is a powerful helper especially when combined with 211 `SetEnvPrefix`. When called, Viper will check for an environment variable any 212 time a `viper.Get` request is made. It will apply the following rules. It will 213 check for a environment variable with a name matching the key uppercased and 214 prefixed with the `EnvPrefix` if set. 215 216 `SetEnvKeyReplacer` allows you to use a `strings.Replacer` object to rewrite Env 217 keys to an extent. This is useful if you want to use `-` or something in your 218 `Get()` calls, but want your environmental variables to use `_` delimiters. An 219 example of using it can be found in `viper_test.go`. 220 221 By default empty environment variables are considered unset and will fall back to 222 the next configuration source. To treat empty environment variables as set, use 223 the `AllowEmptyEnv` method. 224 225 #### Env example 226 227 ```go 228 SetEnvPrefix("spf") // will be uppercased automatically 229 BindEnv("id") 230 231 os.Setenv("SPF_ID", "13") // typically done outside of the app 232 233 id := Get("id") // 13 234 ``` 235 236 ### Working with Flags 237 238 Viper has the ability to bind to flags. Specifically, Viper supports `Pflags` 239 as used in the [Cobra](https://github.com/spf13/cobra) library. 240 241 Like `BindEnv`, the value is not set when the binding method is called, but when 242 it is accessed. This means you can bind as early as you want, even in an 243 `init()` function. 244 245 For individual flags, the `BindPFlag()` method provides this functionality. 246 247 Example: 248 249 ```go 250 serverCmd.Flags().Int("port", 1138, "Port to run Application server on") 251 viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) 252 ``` 253 254 You can also bind an existing set of pflags (pflag.FlagSet): 255 256 Example: 257 258 ```go 259 pflag.Int("flagname", 1234, "help message for flagname") 260 261 pflag.Parse() 262 viper.BindPFlags(pflag.CommandLine) 263 264 i := viper.GetInt("flagname") // retrieve values from viper instead of pflag 265 ``` 266 267 The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude 268 the use of other packages that use the [flag](https://golang.org/pkg/flag/) 269 package from the standard library. The pflag package can handle the flags 270 defined for the flag package by importing these flags. This is accomplished 271 by a calling a convenience function provided by the pflag package called 272 AddGoFlagSet(). 273 274 Example: 275 276 ```go 277 package main 278 279 import ( 280 "flag" 281 "github.com/spf13/pflag" 282 ) 283 284 func main() { 285 286 // using standard library "flag" package 287 flag.Int("flagname", 1234, "help message for flagname") 288 289 pflag.CommandLine.AddGoFlagSet(flag.CommandLine) 290 pflag.Parse() 291 viper.BindPFlags(pflag.CommandLine) 292 293 i := viper.GetInt("flagname") // retrieve value from viper 294 295 ... 296 } 297 ``` 298 299 #### Flag interfaces 300 301 Viper provides two Go interfaces to bind other flag systems if you don’t use `Pflags`. 302 303 `FlagValue` represents a single flag. This is a very simple example on how to implement this interface: 304 305 ```go 306 type myFlag struct {} 307 func (f myFlag) HasChanged() bool { return false } 308 func (f myFlag) Name() string { return "my-flag-name" } 309 func (f myFlag) ValueString() string { return "my-flag-value" } 310 func (f myFlag) ValueType() string { return "string" } 311 ``` 312 313 Once your flag implements this interface, you can simply tell Viper to bind it: 314 315 ```go 316 viper.BindFlagValue("my-flag-name", myFlag{}) 317 ``` 318 319 `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface: 320 321 ```go 322 type myFlagSet struct { 323 flags []myFlag 324 } 325 326 func (f myFlagSet) VisitAll(fn func(FlagValue)) { 327 for _, flag := range flags { 328 fn(flag) 329 } 330 } 331 ``` 332 333 Once your flag set implements this interface, you can simply tell Viper to bind it: 334 335 ```go 336 fSet := myFlagSet{ 337 flags: []myFlag{myFlag{}, myFlag{}}, 338 } 339 viper.BindFlagValues("my-flags", fSet) 340 ``` 341 342 ### Remote Key/Value Store Support 343 344 To enable remote support in Viper, do a blank import of the `viper/remote` 345 package: 346 347 `import _ "github.com/spf13/viper/remote"` 348 349 Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path 350 in a Key/Value store such as etcd or Consul. These values take precedence over 351 default values, but are overridden by configuration values retrieved from disk, 352 flags, or environment variables. 353 354 Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve 355 configuration from the K/V store, which means that you can store your 356 configuration values encrypted and have them automatically decrypted if you have 357 the correct gpg keyring. Encryption is optional. 358 359 You can use remote configuration in conjunction with local configuration, or 360 independently of it. 361 362 `crypt` has a command-line helper that you can use to put configurations in your 363 K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. 364 365 ```bash 366 $ go get github.com/xordataexchange/crypt/bin/crypt 367 $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json 368 ``` 369 370 Confirm that your value was set: 371 372 ```bash 373 $ crypt get -plaintext /config/hugo.json 374 ``` 375 376 See the `crypt` documentation for examples of how to set encrypted values, or 377 how to use Consul. 378 379 ### Remote Key/Value Store Example - Unencrypted 380 381 #### etcd 382 ```go 383 viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json") 384 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 385 err := viper.ReadRemoteConfig() 386 ``` 387 388 #### Consul 389 You need to set a key to Consul key/value storage with JSON value containing your desired config. 390 For example, create a Consul key/value store key `MY_CONSUL_KEY` with value: 391 392 ```json 393 { 394 "port": 8080, 395 "hostname": "myhostname.com" 396 } 397 ``` 398 399 ```go 400 viper.AddRemoteProvider("consul", "localhost:8500", "MY_CONSUL_KEY") 401 viper.SetConfigType("json") // Need to explicitly set this to json 402 err := viper.ReadRemoteConfig() 403 404 fmt.Println(viper.Get("port")) // 8080 405 fmt.Println(viper.Get("hostname")) // myhostname.com 406 ``` 407 408 ### Remote Key/Value Store Example - Encrypted 409 410 ```go 411 viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") 412 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 413 err := viper.ReadRemoteConfig() 414 ``` 415 416 ### Watching Changes in etcd - Unencrypted 417 418 ```go 419 // alternatively, you can create a new viper instance. 420 var runtime_viper = viper.New() 421 422 runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml") 423 runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 424 425 // read from remote config the first time. 426 err := runtime_viper.ReadRemoteConfig() 427 428 // unmarshal config 429 runtime_viper.Unmarshal(&runtime_conf) 430 431 // open a goroutine to watch remote changes forever 432 go func(){ 433 for { 434 time.Sleep(time.Second * 5) // delay after each request 435 436 // currently, only tested with etcd support 437 err := runtime_viper.WatchRemoteConfig() 438 if err != nil { 439 log.Errorf("unable to read remote config: %v", err) 440 continue 441 } 442 443 // unmarshal new config into our runtime config struct. you can also use channel 444 // to implement a signal to notify the system of the changes 445 runtime_viper.Unmarshal(&runtime_conf) 446 } 447 }() 448 ``` 449 450 ## Getting Values From Viper 451 452 In Viper, there are a few ways to get a value depending on the value’s type. 453 The following functions and methods exist: 454 455 * `Get(key string) : interface{}` 456 * `GetBool(key string) : bool` 457 * `GetFloat64(key string) : float64` 458 * `GetInt(key string) : int` 459 * `GetString(key string) : string` 460 * `GetStringMap(key string) : map[string]interface{}` 461 * `GetStringMapString(key string) : map[string]string` 462 * `GetStringSlice(key string) : []string` 463 * `GetTime(key string) : time.Time` 464 * `GetDuration(key string) : time.Duration` 465 * `IsSet(key string) : bool` 466 * `AllSettings() : map[string]interface{}` 467 468 One important thing to recognize is that each Get function will return a zero 469 value if it’s not found. To check if a given key exists, the `IsSet()` method 470 has been provided. 471 472 Example: 473 ```go 474 viper.GetString("logfile") // case-insensitive Setting & Getting 475 if viper.GetBool("verbose") { 476 fmt.Println("verbose enabled") 477 } 478 ``` 479 ### Accessing nested keys 480 481 The accessor methods also accept formatted paths to deeply nested keys. For 482 example, if the following JSON file is loaded: 483 484 ```json 485 { 486 "host": { 487 "address": "localhost", 488 "port": 5799 489 }, 490 "datastore": { 491 "metric": { 492 "host": "127.0.0.1", 493 "port": 3099 494 }, 495 "warehouse": { 496 "host": "198.0.0.1", 497 "port": 2112 498 } 499 } 500 } 501 502 ``` 503 504 Viper can access a nested field by passing a `.` delimited path of keys: 505 506 ```go 507 GetString("datastore.metric.host") // (returns "127.0.0.1") 508 ``` 509 510 This obeys the precedence rules established above; the search for the path 511 will cascade through the remaining configuration registries until found. 512 513 For example, given this configuration file, both `datastore.metric.host` and 514 `datastore.metric.port` are already defined (and may be overridden). If in addition 515 `datastore.metric.protocol` was defined in the defaults, Viper would also find it. 516 517 However, if `datastore.metric` was overridden (by a flag, an environment variable, 518 the `Set()` method, …) with an immediate value, then all sub-keys of 519 `datastore.metric` become undefined, they are “shadowed” by the higher-priority 520 configuration level. 521 522 Lastly, if there exists a key that matches the delimited key path, its value 523 will be returned instead. E.g. 524 525 ```json 526 { 527 "datastore.metric.host": "0.0.0.0", 528 "host": { 529 "address": "localhost", 530 "port": 5799 531 }, 532 "datastore": { 533 "metric": { 534 "host": "127.0.0.1", 535 "port": 3099 536 }, 537 "warehouse": { 538 "host": "198.0.0.1", 539 "port": 2112 540 } 541 } 542 } 543 544 GetString("datastore.metric.host") // returns "0.0.0.0" 545 ``` 546 547 ### Extract sub-tree 548 549 Extract sub-tree from Viper. 550 551 For example, `viper` represents: 552 553 ```json 554 app: 555 cache1: 556 max-items: 100 557 item-size: 64 558 cache2: 559 max-items: 200 560 item-size: 80 561 ``` 562 563 After executing: 564 565 ```go 566 subv := viper.Sub("app.cache1") 567 ``` 568 569 `subv` represents: 570 571 ```json 572 max-items: 100 573 item-size: 64 574 ``` 575 576 Suppose we have: 577 578 ```go 579 func NewCache(cfg *Viper) *Cache {...} 580 ``` 581 582 which creates a cache based on config information formatted as `subv`. 583 Now it’s easy to create these 2 caches separately as: 584 585 ```go 586 cfg1 := viper.Sub("app.cache1") 587 cache1 := NewCache(cfg1) 588 589 cfg2 := viper.Sub("app.cache2") 590 cache2 := NewCache(cfg2) 591 ``` 592 593 ### Unmarshaling 594 595 You also have the option of Unmarshaling all or a specific value to a struct, map, 596 etc. 597 598 There are two methods to do this: 599 600 * `Unmarshal(rawVal interface{}) : error` 601 * `UnmarshalKey(key string, rawVal interface{}) : error` 602 603 Example: 604 605 ```go 606 type config struct { 607 Port int 608 Name string 609 PathMap string `mapstructure:"path_map"` 610 } 611 612 var C config 613 614 err := Unmarshal(&C) 615 if err != nil { 616 t.Fatalf("unable to decode into struct, %v", err) 617 } 618 ``` 619 620 ### Marshalling to string 621 622 You may need to marhsal all the settings held in viper into a string rather than write them to a file. 623 You can use your favorite format's marshaller with the config returned by `AllSettings()`. 624 625 ```go 626 import ( 627 yaml "gopkg.in/yaml.v2" 628 // ... 629 ) 630 631 func yamlStringSettings() string { 632 c := viper.AllSettings() 633 bs, err := yaml.Marshal(c) 634 if err != nil { 635 t.Fatalf("unable to marshal config to YAML: %v", err) 636 } 637 return string(bs) 638 } 639 ``` 640 641 ## Viper or Vipers? 642 643 Viper comes ready to use out of the box. There is no configuration or 644 initialization needed to begin using Viper. Since most applications will want 645 to use a single central repository for their configuration, the viper package 646 provides this. It is similar to a singleton. 647 648 In all of the examples above, they demonstrate using viper in its singleton 649 style approach. 650 651 ### Working with multiple vipers 652 653 You can also create many different vipers for use in your application. Each will 654 have its own unique set of configurations and values. Each can read from a 655 different config file, key value store, etc. All of the functions that viper 656 package supports are mirrored as methods on a viper. 657 658 Example: 659 660 ```go 661 x := viper.New() 662 y := viper.New() 663 664 x.SetDefault("ContentDir", "content") 665 y.SetDefault("ContentDir", "foobar") 666 667 //... 668 ``` 669 670 When working with multiple vipers, it is up to the user to keep track of the 671 different vipers. 672 673 ## Q & A 674 675 Q: Why not INI files? 676 677 A: Ini files are pretty awful. There’s no standard format, and they are hard to 678 validate. Viper is designed to work with JSON, TOML or YAML files. If someone 679 really wants to add this feature, I’d be happy to merge it. It’s easy to specify 680 which formats your application will permit. 681 682 Q: Why is it called “Viper”? 683 684 A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) 685 to [Cobra](https://github.com/spf13/cobra). While both can operate completely 686 independently, together they make a powerful pair to handle much of your 687 application foundation needs. 688 689 Q: Why is it called “Cobra”? 690 691 A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)?