github.com/biasys/viper@v1.0.0/README.md (about) 1 ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png) 2 3 Go configuration with fangs! 4 5 Many Go projects are built using Viper including: 6 7 * [Hugo](http://gohugo.io) 8 * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) 9 * [Imgur’s Incus](https://github.com/Imgur/incus) 10 * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack) 11 * [Docker Notary](https://github.com/docker/Notary) 12 * [BloomApi](https://www.bloomapi.com/) 13 * [doctl](https://github.com/digitalocean/doctl) 14 * [Clairctl](https://github.com/jgsqware/clairctl) 15 16 [![Build Status](https://travis-ci.org/spf13/viper.svg)](https://travis-ci.org/spf13/viper) [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GoDoc](https://godoc.org/github.com/spf13/viper?status.svg)](https://godoc.org/github.com/spf13/viper) 17 18 19 ## What is Viper? 20 21 Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed 22 to work within an application, and can handle all types of configuration needs 23 and formats. It supports: 24 25 * setting defaults 26 * reading from JSON, TOML, YAML, HCL, and Java properties config files 27 * live watching and re-reading of config files (optional) 28 * reading from environment variables 29 * reading from remote config systems (etcd or Consul), and watching changes 30 * reading from command line flags 31 * reading from buffer 32 * setting explicit values 33 34 Viper can be thought of as a registry for all of your applications 35 configuration needs. 36 37 ## Why Viper? 38 39 When building a modern application, you don’t want to worry about 40 configuration file formats; you want to focus on building awesome software. 41 Viper is here to help with that. 42 43 Viper does the following for you: 44 45 1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats. 46 2. Provide a mechanism to set default values for your different 47 configuration options. 48 3. Provide a mechanism to set override values for options specified through 49 command line flags. 50 4. Provide an alias system to easily rename parameters without breaking existing 51 code. 52 5. Make it easy to tell the difference between when a user has provided a 53 command line or config file which is the same as the default. 54 55 Viper uses the following precedence order. Each item takes precedence over the 56 item below it: 57 58 * explicit call to Set 59 * flag 60 * env 61 * config 62 * key/value store 63 * default 64 65 Viper configuration keys are case insensitive. 66 67 ## Putting Values into Viper 68 69 ### Establishing Defaults 70 71 A good configuration system will support default values. A default value is not 72 required for a key, but it’s useful in the event that a key hasn’t been set via 73 config file, environment variable, remote configuration or flag. 74 75 Examples: 76 77 ```go 78 viper.SetDefault("ContentDir", "content") 79 viper.SetDefault("LayoutDir", "layouts") 80 viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"}) 81 ``` 82 83 ### Reading Config Files 84 85 Viper requires minimal configuration so it knows where to look for config files. 86 Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but 87 currently a single Viper instance only supports a single configuration file. 88 Viper does not default to any configuration search paths leaving defaults decision 89 to an application. 90 91 Here is an example of how to use Viper to search for and read a configuration file. 92 None of the specific paths are required, but at least one path should be provided 93 where a configuration file is expected. 94 95 ```go 96 viper.SetConfigName("config") // name of config file (without extension) 97 viper.AddConfigPath("/etc/appname/") // path to look for the config file in 98 viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths 99 viper.AddConfigPath(".") // optionally look for config in the working directory 100 err := viper.ReadInConfig() // Find and read the config file 101 if err != nil { // Handle errors reading the config file 102 panic(fmt.Errorf("Fatal error config file: %s \n", err)) 103 } 104 ``` 105 106 ### Watching and re-reading config files 107 108 Viper supports the ability to have your application live read a config file while running. 109 110 Gone are the days of needing to restart a server to have a config take effect, 111 viper powered applications can read an update to a config file while running and 112 not miss a beat. 113 114 Simply tell the viper instance to watchConfig. 115 Optionally you can provide a function for Viper to run each time a change occurs. 116 117 **Make sure you add all of the configPaths prior to calling `WatchConfig()`** 118 119 ```go 120 viper.WatchConfig() 121 viper.OnConfigChange(func(e fsnotify.Event) { 122 fmt.Println("Config file changed:", e.Name) 123 }) 124 ``` 125 126 ### Reading Config from io.Reader 127 128 Viper predefines many configuration sources such as files, environment 129 variables, flags, and remote K/V store, but you are not bound to them. You can 130 also implement your own required configuration source and feed it to viper. 131 132 ```go 133 viper.SetConfigType("yaml") // or viper.SetConfigType("YAML") 134 135 // any approach to require this configuration into your program. 136 var yamlExample = []byte(` 137 Hacker: true 138 name: steve 139 hobbies: 140 - skateboarding 141 - snowboarding 142 - go 143 clothing: 144 jacket: leather 145 trousers: denim 146 age: 35 147 eyes : brown 148 beard: true 149 `) 150 151 viper.ReadConfig(bytes.NewBuffer(yamlExample)) 152 153 viper.Get("name") // this would be "steve" 154 ``` 155 156 ### Setting Overrides 157 158 These could be from a command line flag, or from your own application logic. 159 160 ```go 161 viper.Set("Verbose", true) 162 viper.Set("LogFile", LogFile) 163 ``` 164 165 ### Registering and Using Aliases 166 167 Aliases permit a single value to be referenced by multiple keys 168 169 ```go 170 viper.RegisterAlias("loud", "Verbose") 171 172 viper.Set("verbose", true) // same result as next line 173 viper.Set("loud", true) // same result as prior line 174 175 viper.GetBool("loud") // true 176 viper.GetBool("verbose") // true 177 ``` 178 179 ### Working with Environment Variables 180 181 Viper has full support for environment variables. This enables 12 factor 182 applications out of the box. There are four methods that exist to aid working 183 with ENV: 184 185 * `AutomaticEnv()` 186 * `BindEnv(string...) : error` 187 * `SetEnvPrefix(string)` 188 * `SetEnvReplacer(string...) *strings.Replacer` 189 190 _When working with ENV variables, it’s important to recognize that Viper 191 treats ENV variables as case sensitive._ 192 193 Viper provides a mechanism to try to ensure that ENV variables are unique. By 194 using `SetEnvPrefix`, you can tell Viper to use add a prefix while reading from 195 the environment variables. Both `BindEnv` and `AutomaticEnv` will use this 196 prefix. 197 198 `BindEnv` takes one or two parameters. The first parameter is the key name, the 199 second is the name of the environment variable. The name of the environment 200 variable is case sensitive. If the ENV variable name is not provided, then 201 Viper will automatically assume that the key name matches the ENV variable name, 202 but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV 203 variable name, it **does not** automatically add the prefix. 204 205 One important thing to recognize when working with ENV variables is that the 206 value will be read each time it is accessed. Viper does not fix the value when 207 the `BindEnv` is called. 208 209 `AutomaticEnv` is a powerful helper especially when combined with 210 `SetEnvPrefix`. When called, Viper will check for an environment variable any 211 time a `viper.Get` request is made. It will apply the following rules. It will 212 check for a environment variable with a name matching the key uppercased and 213 prefixed with the `EnvPrefix` if set. 214 215 `SetEnvReplacer` allows you to use a `strings.Replacer` object to rewrite Env 216 keys to an extent. This is useful if you want to use `-` or something in your 217 `Get()` calls, but want your environmental variables to use `_` delimiters. An 218 example of using it can be found in `viper_test.go`. 219 220 #### Env example 221 222 ```go 223 SetEnvPrefix("spf") // will be uppercased automatically 224 BindEnv("id") 225 226 os.Setenv("SPF_ID", "13") // typically done outside of the app 227 228 id := Get("id") // 13 229 ``` 230 231 ### Working with Flags 232 233 Viper has the ability to bind to flags. Specifically, Viper supports `Pflags` 234 as used in the [Cobra](https://github.com/spf13/cobra) library. 235 236 Like `BindEnv`, the value is not set when the binding method is called, but when 237 it is accessed. This means you can bind as early as you want, even in an 238 `init()` function. 239 240 For individual flags, the `BindPFlag()` method provides this functionality. 241 242 Example: 243 244 ```go 245 serverCmd.Flags().Int("port", 1138, "Port to run Application server on") 246 viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) 247 ``` 248 249 You can also bind an existing set of pflags (pflag.FlagSet): 250 251 Example: 252 253 ```go 254 pflag.Int("flagname", 1234, "help message for flagname") 255 256 pflag.Parse() 257 viper.BindPFlags(pflag.CommandLine) 258 259 i := viper.GetInt("flagname") // retrieve values from viper instead of pflag 260 ``` 261 262 The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude 263 the use of other packages that use the [flag](https://golang.org/pkg/flag/) 264 package from the standard library. The pflag package can handle the flags 265 defined for the flag package by importing these flags. This is accomplished 266 by a calling a convenience function provided by the pflag package called 267 AddGoFlagSet(). 268 269 Example: 270 271 ```go 272 package main 273 274 import ( 275 "flag" 276 "github.com/spf13/pflag" 277 ) 278 279 func main() { 280 281 // using standard library "flag" package 282 flag.Int("flagname", 1234, "help message for flagname") 283 284 pflag.CommandLine.AddGoFlagSet(flag.CommandLine) 285 pflag.Parse() 286 viper.BindPFlags(pflag.CommandLine) 287 288 i := viper.GetInt("flagname") // retrieve value from viper 289 290 ... 291 } 292 ``` 293 294 #### Flag interfaces 295 296 Viper provides two Go interfaces to bind other flag systems if you don’t use `Pflags`. 297 298 `FlagValue` represents a single flag. This is a very simple example on how to implement this interface: 299 300 ```go 301 type myFlag struct {} 302 func (f myFlag) HasChanged() bool { return false } 303 func (f myFlag) Name() string { return "my-flag-name" } 304 func (f myFlag) ValueString() string { return "my-flag-value" } 305 func (f myFlag) ValueType() string { return "string" } 306 ``` 307 308 Once your flag implements this interface, you can simply tell Viper to bind it: 309 310 ```go 311 viper.BindFlagValue("my-flag-name", myFlag{}) 312 ``` 313 314 `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface: 315 316 ```go 317 type myFlagSet struct { 318 flags []myFlag 319 } 320 321 func (f myFlagSet) VisitAll(fn func(FlagValue)) { 322 for _, flag := range flags { 323 fn(flag) 324 } 325 } 326 ``` 327 328 Once your flag set implements this interface, you can simply tell Viper to bind it: 329 330 ```go 331 fSet := myFlagSet{ 332 flags: []myFlag{myFlag{}, myFlag{}}, 333 } 334 viper.BindFlagValues("my-flags", fSet) 335 ``` 336 337 ### Remote Key/Value Store Support 338 339 To enable remote support in Viper, do a blank import of the `viper/remote` 340 package: 341 342 `import _ "github.com/spf13/viper/remote"` 343 344 Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path 345 in a Key/Value store such as etcd or Consul. These values take precedence over 346 default values, but are overridden by configuration values retrieved from disk, 347 flags, or environment variables. 348 349 Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve 350 configuration from the K/V store, which means that you can store your 351 configuration values encrypted and have them automatically decrypted if you have 352 the correct gpg keyring. Encryption is optional. 353 354 You can use remote configuration in conjunction with local configuration, or 355 independently of it. 356 357 `crypt` has a command-line helper that you can use to put configurations in your 358 K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. 359 360 ```bash 361 $ go get github.com/xordataexchange/crypt/bin/crypt 362 $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json 363 ``` 364 365 Confirm that your value was set: 366 367 ```bash 368 $ crypt get -plaintext /config/hugo.json 369 ``` 370 371 See the `crypt` documentation for examples of how to set encrypted values, or 372 how to use Consul. 373 374 ### Remote Key/Value Store Example - Unencrypted 375 376 ```go 377 viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json") 378 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 379 err := viper.ReadRemoteConfig() 380 ``` 381 382 ### Remote Key/Value Store Example - Encrypted 383 384 ```go 385 viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") 386 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 387 err := viper.ReadRemoteConfig() 388 ``` 389 390 ### Watching Changes in etcd - Unencrypted 391 392 ```go 393 // alternatively, you can create a new viper instance. 394 var runtime_viper = viper.New() 395 396 runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml") 397 runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop" 398 399 // read from remote config the first time. 400 err := runtime_viper.ReadRemoteConfig() 401 402 // unmarshal config 403 runtime_viper.Unmarshal(&runtime_conf) 404 405 // open a goroutine to watch remote changes forever 406 go func(){ 407 for { 408 time.Sleep(time.Second * 5) // delay after each request 409 410 // currently, only tested with etcd support 411 err := runtime_viper.WatchRemoteConfig() 412 if err != nil { 413 log.Errorf("unable to read remote config: %v", err) 414 continue 415 } 416 417 // unmarshal new config into our runtime config struct. you can also use channel 418 // to implement a signal to notify the system of the changes 419 runtime_viper.Unmarshal(&runtime_conf) 420 } 421 }() 422 ``` 423 424 ## Getting Values From Viper 425 426 In Viper, there are a few ways to get a value depending on the value’s type. 427 The following functions and methods exist: 428 429 * `Get(key string) : interface{}` 430 * `GetBool(key string) : bool` 431 * `GetFloat64(key string) : float64` 432 * `GetInt(key string) : int` 433 * `GetString(key string) : string` 434 * `GetStringMap(key string) : map[string]interface{}` 435 * `GetStringMapString(key string) : map[string]string` 436 * `GetStringSlice(key string) : []string` 437 * `GetTime(key string) : time.Time` 438 * `GetDuration(key string) : time.Duration` 439 * `IsSet(key string) : bool` 440 441 One important thing to recognize is that each Get function will return a zero 442 value if it’s not found. To check if a given key exists, the `IsSet()` method 443 has been provided. 444 445 Example: 446 ```go 447 viper.GetString("logfile") // case-insensitive Setting & Getting 448 if viper.GetBool("verbose") { 449 fmt.Println("verbose enabled") 450 } 451 ``` 452 ### Accessing nested keys 453 454 The accessor methods also accept formatted paths to deeply nested keys. For 455 example, if the following JSON file is loaded: 456 457 ```json 458 { 459 "host": { 460 "address": "localhost", 461 "port": 5799 462 }, 463 "datastore": { 464 "metric": { 465 "host": "127.0.0.1", 466 "port": 3099 467 }, 468 "warehouse": { 469 "host": "198.0.0.1", 470 "port": 2112 471 } 472 } 473 } 474 475 ``` 476 477 Viper can access a nested field by passing a `.` delimited path of keys: 478 479 ```go 480 GetString("datastore.metric.host") // (returns "127.0.0.1") 481 ``` 482 483 This obeys the precedence rules established above; the search for the path 484 will cascade through the remaining configuration registries until found. 485 486 For example, given this configuration file, both `datastore.metric.host` and 487 `datastore.metric.port` are already defined (and may be overridden). If in addition 488 `datastore.metric.protocol` was defined in the defaults, Viper would also find it. 489 490 However, if `datastore.metric` was overridden (by a flag, an environment variable, 491 the `Set()` method, …) with an immediate value, then all sub-keys of 492 `datastore.metric` become undefined, they are “shadowed” by the higher-priority 493 configuration level. 494 495 Lastly, if there exists a key that matches the delimited key path, its value 496 will be returned instead. E.g. 497 498 ```json 499 { 500 "datastore.metric.host": "0.0.0.0", 501 "host": { 502 "address": "localhost", 503 "port": 5799 504 }, 505 "datastore": { 506 "metric": { 507 "host": "127.0.0.1", 508 "port": 3099 509 }, 510 "warehouse": { 511 "host": "198.0.0.1", 512 "port": 2112 513 } 514 } 515 } 516 517 GetString("datastore.metric.host") // returns "0.0.0.0" 518 ``` 519 520 ### Extract sub-tree 521 522 Extract sub-tree from Viper. 523 524 For example, `viper` represents: 525 526 ```json 527 app: 528 cache1: 529 max-items: 100 530 item-size: 64 531 cache2: 532 max-items: 200 533 item-size: 80 534 ``` 535 536 After executing: 537 538 ```go 539 subv := viper.Sub("app.cache1") 540 ``` 541 542 `subv` represents: 543 544 ```json 545 max-items: 100 546 item-size: 64 547 ``` 548 549 Suppose we have: 550 551 ```go 552 func NewCache(cfg *Viper) *Cache {...} 553 ``` 554 555 which creates a cache based on config information formatted as `subv`. 556 Now it’s easy to create these 2 caches separately as: 557 558 ```go 559 cfg1 := viper.Sub("app.cache1") 560 cache1 := NewCache(cfg1) 561 562 cfg2 := viper.Sub("app.cache2") 563 cache2 := NewCache(cfg2) 564 ``` 565 566 ### Unmarshaling 567 568 You also have the option of Unmarshaling all or a specific value to a struct, map, 569 etc. 570 571 There are two methods to do this: 572 573 * `Unmarshal(rawVal interface{}) : error` 574 * `UnmarshalKey(key string, rawVal interface{}) : error` 575 576 Example: 577 578 ```go 579 type config struct { 580 Port int 581 Name string 582 PathMap string `mapstructure:"path_map"` 583 } 584 585 var C config 586 587 err := Unmarshal(&C) 588 if err != nil { 589 t.Fatalf("unable to decode into struct, %v", err) 590 } 591 ``` 592 593 ## Viper or Vipers? 594 595 Viper comes ready to use out of the box. There is no configuration or 596 initialization needed to begin using Viper. Since most applications will want 597 to use a single central repository for their configuration, the viper package 598 provides this. It is similar to a singleton. 599 600 In all of the examples above, they demonstrate using viper in its singleton 601 style approach. 602 603 ### Working with multiple vipers 604 605 You can also create many different vipers for use in your application. Each will 606 have its own unique set of configurations and values. Each can read from a 607 different config file, key value store, etc. All of the functions that viper 608 package supports are mirrored as methods on a viper. 609 610 Example: 611 612 ```go 613 x := viper.New() 614 y := viper.New() 615 616 x.SetDefault("ContentDir", "content") 617 y.SetDefault("ContentDir", "foobar") 618 619 //... 620 ``` 621 622 When working with multiple vipers, it is up to the user to keep track of the 623 different vipers. 624 625 ## Q & A 626 627 Q: Why not INI files? 628 629 A: Ini files are pretty awful. There’s no standard format, and they are hard to 630 validate. Viper is designed to work with JSON, TOML or YAML files. If someone 631 really wants to add this feature, I’d be happy to merge it. It’s easy to specify 632 which formats your application will permit. 633 634 Q: Why is it called “Viper”? 635 636 A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) 637 to [Cobra](https://github.com/spf13/cobra). While both can operate completely 638 independently, together they make a powerful pair to handle much of your 639 application foundation needs. 640 641 Q: Why is it called “Cobra”? 642 643 A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)?