github.com/DataDog/viper@v1.13.3/README.md (about)

     1  ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png)
     2  
     3  Go configuration with fangs!
     4  
     5  Many Go projects are built using Viper including:
     6  
     7  * [Hugo](http://gohugo.io)
     8  * [EMC RexRay](http://rexray.readthedocs.org/en/stable/)
     9  * [Imgur’s Incus](https://github.com/Imgur/incus)
    10  * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack)
    11  * [Docker Notary](https://github.com/docker/Notary)
    12  * [BloomApi](https://www.bloomapi.com/)
    13  * [doctl](https://github.com/digitalocean/doctl)
    14  * [Clairctl](https://github.com/jgsqware/clairctl)
    15  
    16  [![Build Status](https://travis-ci.org/spf13/viper.svg)](https://travis-ci.org/spf13/viper) [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GoDoc](https://godoc.org/github.com/spf13/viper?status.svg)](https://godoc.org/github.com/spf13/viper)
    17  
    18  
    19  ## What is Viper?
    20  
    21  Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed
    22  to work within an application, and can handle all types of configuration needs
    23  and formats. It supports:
    24  
    25  * setting defaults
    26  * reading from JSON, TOML, YAML, HCL, and Java properties config files
    27  * live watching and re-reading of config files (optional)
    28  * reading from environment variables
    29  * reading from remote config systems (etcd or Consul), and watching changes
    30  * reading from command line flags
    31  * reading from buffer
    32  * setting explicit values
    33  
    34  Viper can be thought of as a registry for all of your applications
    35  configuration needs.
    36  
    37  ## Why Viper?
    38  
    39  When building a modern application, you don’t want to worry about
    40  configuration file formats; you want to focus on building awesome software.
    41  Viper is here to help with that.
    42  
    43  Viper does the following for you:
    44  
    45  1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats.
    46  2. Provide a mechanism to set default values for your different
    47     configuration options.
    48  3. Provide a mechanism to set override values for options specified through
    49     command line flags.
    50  4. Provide an alias system to easily rename parameters without breaking existing
    51     code.
    52  5. Make it easy to tell the difference between when a user has provided a
    53     command line or config file which is the same as the default.
    54  
    55  Viper uses the following precedence order. Each item takes precedence over the
    56  item below it:
    57  
    58   * explicit call to Set
    59   * flag
    60   * env
    61   * config
    62   * key/value store
    63   * default
    64  
    65  Viper configuration keys are case insensitive.
    66  
    67  ## Putting Values into Viper
    68  
    69  ### Establishing Defaults
    70  
    71  A good configuration system will support default values. A default value is not
    72  required for a key, but it’s useful in the event that a key hasn’t been set via
    73  config file, environment variable, remote configuration or flag.
    74  
    75  Examples:
    76  
    77  ```go
    78  viper.SetDefault("ContentDir", "content")
    79  viper.SetDefault("LayoutDir", "layouts")
    80  viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"})
    81  ```
    82  
    83  ### Reading Config Files
    84  
    85  Viper requires minimal configuration so it knows where to look for config files.
    86  Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but
    87  currently a single Viper instance only supports a single configuration file.
    88  Viper does not default to any configuration search paths leaving defaults decision
    89  to an application.
    90  
    91  Here is an example of how to use Viper to search for and read a configuration file.
    92  None of the specific paths are required, but at least one path should be provided
    93  where a configuration file is expected.
    94  
    95  ```go
    96  viper.SetConfigName("config") // name of config file (without extension)
    97  viper.AddConfigPath("/etc/appname/")   // path to look for the config file in
    98  viper.AddConfigPath("$HOME/.appname")  // call multiple times to add many search paths
    99  viper.AddConfigPath(".")               // optionally look for config in the working directory
   100  err := viper.ReadInConfig() // Find and read the config file
   101  if err != nil { // Handle errors reading the config file
   102  	panic(fmt.Errorf("Fatal error config file: %s \n", err))
   103  }
   104  ```
   105  
   106  ### Watching and re-reading config files
   107  
   108  Viper supports the ability to have your application live read a config file while running.
   109  
   110  Gone are the days of needing to restart a server to have a config take effect,
   111  viper powered applications can read an update to a config file while running and
   112  not miss a beat.
   113  
   114  Simply tell the viper instance to watchConfig.
   115  Optionally you can provide a function for Viper to run each time a change occurs.
   116  
   117  **Make sure you add all of the configPaths prior to calling `WatchConfig()`**
   118  
   119  ```go
   120  viper.WatchConfig()
   121  viper.OnConfigChange(func(e fsnotify.Event) {
   122  	fmt.Println("Config file changed:", e.Name)
   123  })
   124  ```
   125  
   126  ### Reading Config from io.Reader
   127  
   128  Viper predefines many configuration sources such as files, environment
   129  variables, flags, and remote K/V store, but you are not bound to them. You can
   130  also implement your own required configuration source and feed it to viper.
   131  
   132  ```go
   133  viper.SetConfigType("yaml") // or viper.SetConfigType("YAML")
   134  
   135  // any approach to require this configuration into your program.
   136  var yamlExample = []byte(`
   137  Hacker: true
   138  name: steve
   139  hobbies:
   140  - skateboarding
   141  - snowboarding
   142  - go
   143  clothing:
   144    jacket: leather
   145    trousers: denim
   146  age: 35
   147  eyes : brown
   148  beard: true
   149  `)
   150  
   151  viper.ReadConfig(bytes.NewBuffer(yamlExample))
   152  
   153  viper.Get("name") // this would be "steve"
   154  ```
   155  
   156  ### Setting Overrides
   157  
   158  These could be from a command line flag, or from your own application logic.
   159  
   160  ```go
   161  viper.Set("Verbose", true)
   162  viper.Set("LogFile", LogFile)
   163  ```
   164  
   165  ### Registering and Using Aliases
   166  
   167  Aliases permit a single value to be referenced by multiple keys
   168  
   169  ```go
   170  viper.RegisterAlias("loud", "Verbose")
   171  
   172  viper.Set("verbose", true) // same result as next line
   173  viper.Set("loud", true)   // same result as prior line
   174  
   175  viper.GetBool("loud") // true
   176  viper.GetBool("verbose") // true
   177  ```
   178  
   179  ### Working with Environment Variables
   180  
   181  Viper has full support for environment variables. This enables 12 factor
   182  applications out of the box. There are five methods that exist to aid working
   183  with ENV:
   184  
   185   * `AutomaticEnv()`
   186   * `BindEnv(string...) : error`
   187   * `SetEnvPrefix(string)`
   188   * `SetEnvKeyReplacer(string...) *strings.Replacer`
   189    * `AllowEmptyEnvVar(bool)`
   190  
   191  _When working with ENV variables, it’s important to recognize that Viper
   192  treats ENV variables as case sensitive._
   193  
   194  Viper provides a mechanism to try to ensure that ENV variables are unique. By
   195  using `SetEnvPrefix`, you can tell Viper to use a prefix while reading from
   196  the environment variables. Both `BindEnv` and `AutomaticEnv` will use this
   197  prefix.
   198  
   199  `BindEnv` takes one or two parameters. The first parameter is the key name, the
   200  second is the name of the environment variable. The name of the environment
   201  variable is case sensitive. If the ENV variable name is not provided, then
   202  Viper will automatically assume that the key name matches the ENV variable name,
   203  but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV
   204  variable name, it **does not** automatically add the prefix.
   205  
   206  One important thing to recognize when working with ENV variables is that the
   207  value will be read each time it is accessed. Viper does not fix the value when
   208  the `BindEnv` is called.
   209  
   210  `AutomaticEnv` is a powerful helper especially when combined with
   211  `SetEnvPrefix`. When called, Viper will check for an environment variable any
   212  time a `viper.Get` request is made. It will apply the following rules. It will
   213  check for a environment variable with a name matching the key uppercased and
   214  prefixed with the `EnvPrefix` if set.
   215  
   216  `SetEnvKeyReplacer` allows you to use a `strings.Replacer` object to rewrite Env
   217  keys to an extent. This is useful if you want to use `-` or something in your
   218  `Get()` calls, but want your environmental variables to use `_` delimiters. An
   219  example of using it can be found in `viper_test.go`.
   220  
   221  By default empty environment variables are considered unset and will fall back to
   222  the next configuration source. To treat empty environment variables as set, use
   223  the `AllowEmptyEnv` method.
   224  
   225  #### Env example
   226  
   227  ```go
   228  SetEnvPrefix("spf") // will be uppercased automatically
   229  BindEnv("id")
   230  
   231  os.Setenv("SPF_ID", "13") // typically done outside of the app
   232  
   233  id := Get("id") // 13
   234  ```
   235  
   236  ### Working with Flags
   237  
   238  Viper has the ability to bind to flags. Specifically, Viper supports `Pflags`
   239  as used in the [Cobra](https://github.com/spf13/cobra) library.
   240  
   241  Like `BindEnv`, the value is not set when the binding method is called, but when
   242  it is accessed. This means you can bind as early as you want, even in an
   243  `init()` function.
   244  
   245  For individual flags, the `BindPFlag()` method provides this functionality.
   246  
   247  Example:
   248  
   249  ```go
   250  serverCmd.Flags().Int("port", 1138, "Port to run Application server on")
   251  viper.BindPFlag("port", serverCmd.Flags().Lookup("port"))
   252  ```
   253  
   254  You can also bind an existing set of pflags (pflag.FlagSet):
   255  
   256  Example:
   257  
   258  ```go
   259  pflag.Int("flagname", 1234, "help message for flagname")
   260  
   261  pflag.Parse()
   262  viper.BindPFlags(pflag.CommandLine)
   263  
   264  i := viper.GetInt("flagname") // retrieve values from viper instead of pflag
   265  ```
   266  
   267  The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude
   268  the use of other packages that use the [flag](https://golang.org/pkg/flag/)
   269  package from the standard library. The pflag package can handle the flags
   270  defined for the flag package by importing these flags. This is accomplished
   271  by a calling a convenience function provided by the pflag package called
   272  AddGoFlagSet().
   273  
   274  Example:
   275  
   276  ```go
   277  package main
   278  
   279  import (
   280  	"flag"
   281  	"github.com/spf13/pflag"
   282  )
   283  
   284  func main() {
   285  
   286  	// using standard library "flag" package
   287  	flag.Int("flagname", 1234, "help message for flagname")
   288  
   289  	pflag.CommandLine.AddGoFlagSet(flag.CommandLine)
   290  	pflag.Parse()
   291  	viper.BindPFlags(pflag.CommandLine)
   292  
   293  	i := viper.GetInt("flagname") // retrieve value from viper
   294  
   295  	...
   296  }
   297  ```
   298  
   299  #### Flag interfaces
   300  
   301  Viper provides two Go interfaces to bind other flag systems if you don’t use `Pflags`.
   302  
   303  `FlagValue` represents a single flag. This is a very simple example on how to implement this interface:
   304  
   305  ```go
   306  type myFlag struct {}
   307  func (f myFlag) HasChanged() bool { return false }
   308  func (f myFlag) Name() string { return "my-flag-name" }
   309  func (f myFlag) ValueString() string { return "my-flag-value" }
   310  func (f myFlag) ValueType() string { return "string" }
   311  ```
   312  
   313  Once your flag implements this interface, you can simply tell Viper to bind it:
   314  
   315  ```go
   316  viper.BindFlagValue("my-flag-name", myFlag{})
   317  ```
   318  
   319  `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface:
   320  
   321  ```go
   322  type myFlagSet struct {
   323  	flags []myFlag
   324  }
   325  
   326  func (f myFlagSet) VisitAll(fn func(FlagValue)) {
   327  	for _, flag := range flags {
   328  		fn(flag)
   329  	}
   330  }
   331  ```
   332  
   333  Once your flag set implements this interface, you can simply tell Viper to bind it:
   334  
   335  ```go
   336  fSet := myFlagSet{
   337  	flags: []myFlag{myFlag{}, myFlag{}},
   338  }
   339  viper.BindFlagValues("my-flags", fSet)
   340  ```
   341  
   342  ### Remote Key/Value Store Support
   343  
   344  To enable remote support in Viper, do a blank import of the `viper/remote`
   345  package:
   346  
   347  `import _ "github.com/spf13/viper/remote"`
   348  
   349  Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path
   350  in a Key/Value store such as etcd or Consul.  These values take precedence over
   351  default values, but are overridden by configuration values retrieved from disk,
   352  flags, or environment variables.
   353  
   354  Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve
   355  configuration from the K/V store, which means that you can store your
   356  configuration values encrypted and have them automatically decrypted if you have
   357  the correct gpg keyring.  Encryption is optional.
   358  
   359  You can use remote configuration in conjunction with local configuration, or
   360  independently of it.
   361  
   362  `crypt` has a command-line helper that you can use to put configurations in your
   363  K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001.
   364  
   365  ```bash
   366  $ go get github.com/xordataexchange/crypt/bin/crypt
   367  $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json
   368  ```
   369  
   370  Confirm that your value was set:
   371  
   372  ```bash
   373  $ crypt get -plaintext /config/hugo.json
   374  ```
   375  
   376  See the `crypt` documentation for examples of how to set encrypted values, or
   377  how to use Consul.
   378  
   379  ### Remote Key/Value Store Example - Unencrypted
   380  
   381  #### etcd
   382  ```go
   383  viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json")
   384  viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   385  err := viper.ReadRemoteConfig()
   386  ```
   387  
   388  #### Consul
   389  You need to set a key to Consul key/value storage with JSON value containing your desired config.  
   390  For example, create a Consul key/value store key `MY_CONSUL_KEY` with value:
   391  
   392  ```json
   393  {
   394      "port": 8080,
   395      "hostname": "myhostname.com"
   396  }
   397  ```
   398  
   399  ```go
   400  viper.AddRemoteProvider("consul", "localhost:8500", "MY_CONSUL_KEY")
   401  viper.SetConfigType("json") // Need to explicitly set this to json
   402  err := viper.ReadRemoteConfig()
   403  
   404  fmt.Println(viper.Get("port")) // 8080
   405  fmt.Println(viper.Get("hostname")) // myhostname.com
   406  ```
   407  
   408  ### Remote Key/Value Store Example - Encrypted
   409  
   410  ```go
   411  viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg")
   412  viper.SetConfigType("json") // because there is no file extension in a stream of bytes,  supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   413  err := viper.ReadRemoteConfig()
   414  ```
   415  
   416  ### Watching Changes in etcd - Unencrypted
   417  
   418  ```go
   419  // alternatively, you can create a new viper instance.
   420  var runtime_viper = viper.New()
   421  
   422  runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml")
   423  runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   424  
   425  // read from remote config the first time.
   426  err := runtime_viper.ReadRemoteConfig()
   427  
   428  // unmarshal config
   429  runtime_viper.Unmarshal(&runtime_conf)
   430  
   431  // open a goroutine to watch remote changes forever
   432  go func(){
   433  	for {
   434  	    time.Sleep(time.Second * 5) // delay after each request
   435  
   436  	    // currently, only tested with etcd support
   437  	    err := runtime_viper.WatchRemoteConfig()
   438  	    if err != nil {
   439  	        log.Errorf("unable to read remote config: %v", err)
   440  	        continue
   441  	    }
   442  
   443  	    // unmarshal new config into our runtime config struct. you can also use channel
   444  	    // to implement a signal to notify the system of the changes
   445  	    runtime_viper.Unmarshal(&runtime_conf)
   446  	}
   447  }()
   448  ```
   449  
   450  ## Getting Values From Viper
   451  
   452  In Viper, there are a few ways to get a value depending on the value’s type.
   453  The following functions and methods exist:
   454  
   455   * `Get(key string) : interface{}`
   456   * `GetBool(key string) : bool`
   457   * `GetFloat64(key string) : float64`
   458   * `GetInt(key string) : int`
   459   * `GetString(key string) : string`
   460   * `GetStringMap(key string) : map[string]interface{}`
   461   * `GetStringMapString(key string) : map[string]string`
   462   * `GetStringSlice(key string) : []string`
   463   * `GetTime(key string) : time.Time`
   464   * `GetDuration(key string) : time.Duration`
   465   * `IsSet(key string) : bool`
   466   * `AllSettings() : map[string]interface{}`
   467  
   468  One important thing to recognize is that each Get function will return a zero
   469  value if it’s not found. To check if a given key exists, the `IsSet()` method
   470  has been provided.
   471  
   472  Example:
   473  ```go
   474  viper.GetString("logfile") // case-insensitive Setting & Getting
   475  if viper.GetBool("verbose") {
   476      fmt.Println("verbose enabled")
   477  }
   478  ```
   479  ### Accessing nested keys
   480  
   481  The accessor methods also accept formatted paths to deeply nested keys. For
   482  example, if the following JSON file is loaded:
   483  
   484  ```json
   485  {
   486      "host": {
   487          "address": "localhost",
   488          "port": 5799
   489      },
   490      "datastore": {
   491          "metric": {
   492              "host": "127.0.0.1",
   493              "port": 3099
   494          },
   495          "warehouse": {
   496              "host": "198.0.0.1",
   497              "port": 2112
   498          }
   499      }
   500  }
   501  
   502  ```
   503  
   504  Viper can access a nested field by passing a `.` delimited path of keys:
   505  
   506  ```go
   507  GetString("datastore.metric.host") // (returns "127.0.0.1")
   508  ```
   509  
   510  This obeys the precedence rules established above; the search for the path
   511  will cascade through the remaining configuration registries until found.
   512  
   513  For example, given this configuration file, both `datastore.metric.host` and
   514  `datastore.metric.port` are already defined (and may be overridden). If in addition
   515  `datastore.metric.protocol` was defined in the defaults, Viper would also find it.
   516  
   517  However, if `datastore.metric` was overridden (by a flag, an environment variable,
   518  the `Set()` method, …) with an immediate value, then all sub-keys of
   519  `datastore.metric` become undefined, they are “shadowed” by the higher-priority
   520  configuration level.
   521  
   522  Lastly, if there exists a key that matches the delimited key path, its value
   523  will be returned instead. E.g.
   524  
   525  ```json
   526  {
   527      "datastore.metric.host": "0.0.0.0",
   528      "host": {
   529          "address": "localhost",
   530          "port": 5799
   531      },
   532      "datastore": {
   533          "metric": {
   534              "host": "127.0.0.1",
   535              "port": 3099
   536          },
   537          "warehouse": {
   538              "host": "198.0.0.1",
   539              "port": 2112
   540          }
   541      }
   542  }
   543  
   544  GetString("datastore.metric.host") // returns "0.0.0.0"
   545  ```
   546  
   547  ### Extract sub-tree
   548  
   549  Extract sub-tree from Viper.
   550  
   551  For example, `viper` represents:
   552  
   553  ```json
   554  app:
   555    cache1:
   556      max-items: 100
   557      item-size: 64
   558    cache2:
   559      max-items: 200
   560      item-size: 80
   561  ```
   562  
   563  After executing:
   564  
   565  ```go
   566  subv := viper.Sub("app.cache1")
   567  ```
   568  
   569  `subv` represents:
   570  
   571  ```json
   572  max-items: 100
   573  item-size: 64
   574  ```
   575  
   576  Suppose we have:
   577  
   578  ```go
   579  func NewCache(cfg *Viper) *Cache {...}
   580  ```
   581  
   582  which creates a cache based on config information formatted as `subv`.
   583  Now it’s easy to create these 2 caches separately as:
   584  
   585  ```go
   586  cfg1 := viper.Sub("app.cache1")
   587  cache1 := NewCache(cfg1)
   588  
   589  cfg2 := viper.Sub("app.cache2")
   590  cache2 := NewCache(cfg2)
   591  ```
   592  
   593  ### Unmarshaling
   594  
   595  You also have the option of Unmarshaling all or a specific value to a struct, map,
   596  etc.
   597  
   598  There are two methods to do this:
   599  
   600   * `Unmarshal(rawVal interface{}) : error`
   601   * `UnmarshalKey(key string, rawVal interface{}) : error`
   602  
   603  Example:
   604  
   605  ```go
   606  type config struct {
   607  	Port int
   608  	Name string
   609  	PathMap string `mapstructure:"path_map"`
   610  }
   611  
   612  var C config
   613  
   614  err := Unmarshal(&C)
   615  if err != nil {
   616  	t.Fatalf("unable to decode into struct, %v", err)
   617  }
   618  ```
   619  
   620  ### Marshalling to string
   621  
   622  You may need to marhsal all the settings held in viper into a string rather than write them to a file. 
   623  You can use your favorite format's marshaller with the config returned by `AllSettings()`.
   624  
   625  ```go
   626  import (
   627      yaml "gopkg.in/yaml.v2"
   628      // ...
   629  ) 
   630  
   631  func yamlStringSettings() string {
   632      c := viper.AllSettings()
   633  	bs, err := yaml.Marshal(c)
   634  	if err != nil {
   635          t.Fatalf("unable to marshal config to YAML: %v", err)
   636      }
   637  	return string(bs)
   638  }
   639  ```
   640  
   641  ## Viper or Vipers?
   642  
   643  Viper comes ready to use out of the box. There is no configuration or
   644  initialization needed to begin using Viper. Since most applications will want
   645  to use a single central repository for their configuration, the viper package
   646  provides this. It is similar to a singleton.
   647  
   648  In all of the examples above, they demonstrate using viper in its singleton
   649  style approach.
   650  
   651  ### Working with multiple vipers
   652  
   653  You can also create many different vipers for use in your application. Each will
   654  have its own unique set of configurations and values. Each can read from a
   655  different config file, key value store, etc. All of the functions that viper
   656  package supports are mirrored as methods on a viper.
   657  
   658  Example:
   659  
   660  ```go
   661  x := viper.New()
   662  y := viper.New()
   663  
   664  x.SetDefault("ContentDir", "content")
   665  y.SetDefault("ContentDir", "foobar")
   666  
   667  //...
   668  ```
   669  
   670  When working with multiple vipers, it is up to the user to keep track of the
   671  different vipers.
   672  
   673  ## Q & A
   674  
   675  Q: Why not INI files?
   676  
   677  A: Ini files are pretty awful. There’s no standard format, and they are hard to
   678  validate. Viper is designed to work with JSON, TOML or YAML files. If someone
   679  really wants to add this feature, I’d be happy to merge it. It’s easy to specify
   680  which formats your application will permit.
   681  
   682  Q: Why is it called “Viper”?
   683  
   684  A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe))
   685  to [Cobra](https://github.com/spf13/cobra). While both can operate completely
   686  independently, together they make a powerful pair to handle much of your
   687  application foundation needs.
   688  
   689  Q: Why is it called “Cobra”?
   690  
   691  A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)?