github.com/samt42/viper@v0.0.0-20190213113551-4b317a1ea64b/README.md (about)

     1  ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png)
     2  
     3  Go configuration with fangs!
     4  
     5  Many Go projects are built using Viper including:
     6  
     7  * [Hugo](http://gohugo.io)
     8  * [EMC RexRay](http://rexray.readthedocs.org/en/stable/)
     9  * [Imgur’s Incus](https://github.com/Imgur/incus)
    10  * [Nanobox](https://github.com/nanobox-io/nanobox)/[Nanopack](https://github.com/nanopack)
    11  * [Docker Notary](https://github.com/docker/Notary)
    12  * [BloomApi](https://www.bloomapi.com/)
    13  * [doctl](https://github.com/digitalocean/doctl)
    14  * [Clairctl](https://github.com/jgsqware/clairctl)
    15  
    16  [![Build Status](https://travis-ci.org/spf13/viper.svg)](https://travis-ci.org/spf13/viper) [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![GoDoc](https://godoc.org/github.com/samt42/viper?status.svg)](https://godoc.org/github.com/samt42/viper)
    17  
    18  
    19  ## What is Viper?
    20  
    21  Viper is a complete configuration solution for Go applications including 12-Factor apps. It is designed
    22  to work within an application, and can handle all types of configuration needs
    23  and formats. It supports:
    24  
    25  * setting defaults
    26  * reading from JSON, TOML, YAML, HCL, and Java properties config files
    27  * live watching and re-reading of config files (optional)
    28  * reading from environment variables
    29  * reading from remote config systems (etcd or Consul), and watching changes
    30  * reading from command line flags
    31  * reading from buffer
    32  * setting explicit values
    33  
    34  Viper can be thought of as a registry for all of your applications
    35  configuration needs.
    36  
    37  ## Why Viper?
    38  
    39  When building a modern application, you don’t want to worry about
    40  configuration file formats; you want to focus on building awesome software.
    41  Viper is here to help with that.
    42  
    43  Viper does the following for you:
    44  
    45  1. Find, load, and unmarshal a configuration file in JSON, TOML, YAML, HCL, or Java properties formats.
    46  2. Provide a mechanism to set default values for your different
    47     configuration options.
    48  3. Provide a mechanism to set override values for options specified through
    49     command line flags.
    50  4. Provide an alias system to easily rename parameters without breaking existing
    51     code.
    52  5. Make it easy to tell the difference between when a user has provided a
    53     command line or config file which is the same as the default.
    54  
    55  Viper uses the following precedence order. Each item takes precedence over the
    56  item below it:
    57  
    58   * explicit call to Set
    59   * flag
    60   * env
    61   * config
    62   * key/value store
    63   * default
    64  
    65  Viper configuration keys are case insensitive.
    66  
    67  ## Putting Values into Viper
    68  
    69  ### Establishing Defaults
    70  
    71  A good configuration system will support default values. A default value is not
    72  required for a key, but it’s useful in the event that a key hasn’t been set via
    73  config file, environment variable, remote configuration or flag.
    74  
    75  Examples:
    76  
    77  ```go
    78  viper.SetDefault("ContentDir", "content")
    79  viper.SetDefault("LayoutDir", "layouts")
    80  viper.SetDefault("Taxonomies", map[string]string{"tag": "tags", "category": "categories"})
    81  ```
    82  
    83  ### Reading Config Files
    84  
    85  Viper requires minimal configuration so it knows where to look for config files.
    86  Viper supports JSON, TOML, YAML, HCL, and Java Properties files. Viper can search multiple paths, but
    87  currently a single Viper instance only supports a single configuration file.
    88  Viper does not default to any configuration search paths leaving defaults decision
    89  to an application.
    90  
    91  Here is an example of how to use Viper to search for and read a configuration file.
    92  None of the specific paths are required, but at least one path should be provided
    93  where a configuration file is expected.
    94  
    95  ```go
    96  viper.SetConfigName("config") // name of config file (without extension)
    97  viper.AddConfigPath("/etc/appname/")   // path to look for the config file in
    98  viper.AddConfigPath("$HOME/.appname")  // call multiple times to add many search paths
    99  viper.AddConfigPath(".")               // optionally look for config in the working directory
   100  err := viper.ReadInConfig() // Find and read the config file
   101  if err != nil { // Handle errors reading the config file
   102  	panic(fmt.Errorf("Fatal error config file: %s \n", err))
   103  }
   104  ```
   105  
   106  ### Watching and re-reading config files
   107  
   108  Viper supports the ability to have your application live read a config file while running.
   109  
   110  Gone are the days of needing to restart a server to have a config take effect,
   111  viper powered applications can read an update to a config file while running and
   112  not miss a beat.
   113  
   114  Simply tell the viper instance to watchConfig.
   115  Optionally you can provide a function for Viper to run each time a change occurs.
   116  
   117  **Make sure you add all of the configPaths prior to calling `WatchConfig()`**
   118  
   119  ```go
   120  viper.WatchConfig()
   121  viper.OnConfigChange(func(e fsnotify.Event) {
   122  	fmt.Println("Config file changed:", e.Name)
   123  })
   124  ```
   125  
   126  ### Reading Config from io.Reader
   127  
   128  Viper predefines many configuration sources such as files, environment
   129  variables, flags, and remote K/V store, but you are not bound to them. You can
   130  also implement your own required configuration source and feed it to viper.
   131  
   132  ```go
   133  viper.SetConfigType("yaml") // or viper.SetConfigType("YAML")
   134  
   135  // any approach to require this configuration into your program.
   136  var yamlExample = []byte(`
   137  Hacker: true
   138  name: steve
   139  hobbies:
   140  - skateboarding
   141  - snowboarding
   142  - go
   143  clothing:
   144    jacket: leather
   145    trousers: denim
   146  age: 35
   147  eyes : brown
   148  beard: true
   149  `)
   150  
   151  viper.ReadConfig(bytes.NewBuffer(yamlExample))
   152  
   153  viper.Get("name") // this would be "steve"
   154  ```
   155  
   156  ### Setting Overrides
   157  
   158  These could be from a command line flag, or from your own application logic.
   159  
   160  ```go
   161  viper.Set("Verbose", true)
   162  viper.Set("LogFile", LogFile)
   163  ```
   164  
   165  ### Registering and Using Aliases
   166  
   167  Aliases permit a single value to be referenced by multiple keys
   168  
   169  ```go
   170  viper.RegisterAlias("loud", "Verbose")
   171  
   172  viper.Set("verbose", true) // same result as next line
   173  viper.Set("loud", true)   // same result as prior line
   174  
   175  viper.GetBool("loud") // true
   176  viper.GetBool("verbose") // true
   177  ```
   178  
   179  ### Working with Environment Variables
   180  
   181  Viper has full support for environment variables. This enables 12 factor
   182  applications out of the box. There are four methods that exist to aid working
   183  with ENV:
   184  
   185   * `AutomaticEnv()`
   186   * `BindEnv(string...) : error`
   187   * `SetEnvPrefix(string)`
   188   * `SetEnvReplacer(string...) *strings.Replacer`
   189  
   190  _When working with ENV variables, it’s important to recognize that Viper
   191  treats ENV variables as case sensitive._
   192  
   193  Viper provides a mechanism to try to ensure that ENV variables are unique. By
   194  using `SetEnvPrefix`, you can tell Viper to use add a prefix while reading from
   195  the environment variables. Both `BindEnv` and `AutomaticEnv` will use this
   196  prefix.
   197  
   198  `BindEnv` takes one or two parameters. The first parameter is the key name, the
   199  second is the name of the environment variable. The name of the environment
   200  variable is case sensitive. If the ENV variable name is not provided, then
   201  Viper will automatically assume that the key name matches the ENV variable name,
   202  but the ENV variable is IN ALL CAPS. When you explicitly provide the ENV
   203  variable name, it **does not** automatically add the prefix.
   204  
   205  One important thing to recognize when working with ENV variables is that the
   206  value will be read each time it is accessed. Viper does not fix the value when
   207  the `BindEnv` is called.
   208  
   209  `AutomaticEnv` is a powerful helper especially when combined with
   210  `SetEnvPrefix`. When called, Viper will check for an environment variable any
   211  time a `viper.Get` request is made. It will apply the following rules. It will
   212  check for a environment variable with a name matching the key uppercased and
   213  prefixed with the `EnvPrefix` if set.
   214  
   215  `SetEnvReplacer` allows you to use a `strings.Replacer` object to rewrite Env
   216  keys to an extent. This is useful if you want to use `-` or something in your
   217  `Get()` calls, but want your environmental variables to use `_` delimiters. An
   218  example of using it can be found in `viper_test.go`.
   219  
   220  #### Env example
   221  
   222  ```go
   223  SetEnvPrefix("spf") // will be uppercased automatically
   224  BindEnv("id")
   225  
   226  os.Setenv("SPF_ID", "13") // typically done outside of the app
   227  
   228  id := Get("id") // 13
   229  ```
   230  
   231  ### Working with Flags
   232  
   233  Viper has the ability to bind to flags. Specifically, Viper supports `Pflags`
   234  as used in the [Cobra](https://github.com/spf13/cobra) library.
   235  
   236  Like `BindEnv`, the value is not set when the binding method is called, but when
   237  it is accessed. This means you can bind as early as you want, even in an
   238  `init()` function.
   239  
   240  For individual flags, the `BindPFlag()` method provides this functionality.
   241  
   242  Example:
   243  
   244  ```go
   245  serverCmd.Flags().Int("port", 1138, "Port to run Application server on")
   246  viper.BindPFlag("port", serverCmd.Flags().Lookup("port"))
   247  ```
   248  
   249  You can also bind an existing set of pflags (pflag.FlagSet):
   250  
   251  Example:
   252  
   253  ```go
   254  pflag.Int("flagname", 1234, "help message for flagname")
   255  
   256  pflag.Parse()
   257  viper.BindPFlags(pflag.CommandLine)
   258  
   259  i := viper.GetInt("flagname") // retrieve values from viper instead of pflag
   260  ```
   261  
   262  The use of [pflag](https://github.com/spf13/pflag/) in Viper does not preclude
   263  the use of other packages that use the [flag](https://golang.org/pkg/flag/)
   264  package from the standard library. The pflag package can handle the flags
   265  defined for the flag package by importing these flags. This is accomplished
   266  by a calling a convenience function provided by the pflag package called
   267  AddGoFlagSet().
   268  
   269  Example:
   270  
   271  ```go
   272  package main
   273  
   274  import (
   275  	"flag"
   276  	"github.com/spf13/pflag"
   277  )
   278  
   279  func main() {
   280  
   281  	// using standard library "flag" package
   282  	flag.Int("flagname", 1234, "help message for flagname")
   283  
   284  	pflag.CommandLine.AddGoFlagSet(flag.CommandLine)
   285  	pflag.Parse()
   286  	viper.BindPFlags(pflag.CommandLine)
   287  
   288  	i := viper.GetInt("flagname") // retrieve value from viper
   289  
   290  	...
   291  }
   292  ```
   293  
   294  #### Flag interfaces
   295  
   296  Viper provides two Go interfaces to bind other flag systems if you don’t use `Pflags`.
   297  
   298  `FlagValue` represents a single flag. This is a very simple example on how to implement this interface:
   299  
   300  ```go
   301  type myFlag struct {}
   302  func (f myFlag) HasChanged() bool { return false }
   303  func (f myFlag) Name() string { return "my-flag-name" }
   304  func (f myFlag) ValueString() string { return "my-flag-value" }
   305  func (f myFlag) ValueType() string { return "string" }
   306  ```
   307  
   308  Once your flag implements this interface, you can simply tell Viper to bind it:
   309  
   310  ```go
   311  viper.BindFlagValue("my-flag-name", myFlag{})
   312  ```
   313  
   314  `FlagValueSet` represents a group of flags. This is a very simple example on how to implement this interface:
   315  
   316  ```go
   317  type myFlagSet struct {
   318  	flags []myFlag
   319  }
   320  
   321  func (f myFlagSet) VisitAll(fn func(FlagValue)) {
   322  	for _, flag := range flags {
   323  		fn(flag)
   324  	}
   325  }
   326  ```
   327  
   328  Once your flag set implements this interface, you can simply tell Viper to bind it:
   329  
   330  ```go
   331  fSet := myFlagSet{
   332  	flags: []myFlag{myFlag{}, myFlag{}},
   333  }
   334  viper.BindFlagValues("my-flags", fSet)
   335  ```
   336  
   337  ### Remote Key/Value Store Support
   338  
   339  To enable remote support in Viper, do a blank import of the `viper/remote`
   340  package:
   341  
   342  `import _ "github.com/samt42/viper/remote"`
   343  
   344  Viper will read a config string (as JSON, TOML, YAML or HCL) retrieved from a path
   345  in a Key/Value store such as etcd or Consul.  These values take precedence over
   346  default values, but are overridden by configuration values retrieved from disk,
   347  flags, or environment variables.
   348  
   349  Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve
   350  configuration from the K/V store, which means that you can store your
   351  configuration values encrypted and have them automatically decrypted if you have
   352  the correct gpg keyring.  Encryption is optional.
   353  
   354  You can use remote configuration in conjunction with local configuration, or
   355  independently of it.
   356  
   357  `crypt` has a command-line helper that you can use to put configurations in your
   358  K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001.
   359  
   360  ```bash
   361  $ go get github.com/xordataexchange/crypt/bin/crypt
   362  $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json
   363  ```
   364  
   365  Confirm that your value was set:
   366  
   367  ```bash
   368  $ crypt get -plaintext /config/hugo.json
   369  ```
   370  
   371  See the `crypt` documentation for examples of how to set encrypted values, or
   372  how to use Consul.
   373  
   374  ### Remote Key/Value Store Example - Unencrypted
   375  
   376  ```go
   377  viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001","/config/hugo.json")
   378  viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   379  err := viper.ReadRemoteConfig()
   380  ```
   381  
   382  ### Remote Key/Value Store Example - Encrypted
   383  
   384  ```go
   385  viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg")
   386  viper.SetConfigType("json") // because there is no file extension in a stream of bytes,  supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   387  err := viper.ReadRemoteConfig()
   388  ```
   389  
   390  ### Watching Changes in etcd - Unencrypted
   391  
   392  ```go
   393  // alternatively, you can create a new viper instance.
   394  var runtime_viper = viper.New()
   395  
   396  runtime_viper.AddRemoteProvider("etcd", "http://127.0.0.1:4001", "/config/hugo.yml")
   397  runtime_viper.SetConfigType("yaml") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop"
   398  
   399  // read from remote config the first time.
   400  err := runtime_viper.ReadRemoteConfig()
   401  
   402  // unmarshal config
   403  runtime_viper.Unmarshal(&runtime_conf)
   404  
   405  // open a goroutine to watch remote changes forever
   406  go func(){
   407  	for {
   408  	    time.Sleep(time.Second * 5) // delay after each request
   409  
   410  	    // currently, only tested with etcd support
   411  	    err := runtime_viper.WatchRemoteConfig()
   412  	    if err != nil {
   413  	        log.Errorf("unable to read remote config: %v", err)
   414  	        continue
   415  	    }
   416  
   417  	    // unmarshal new config into our runtime config struct. you can also use channel
   418  	    // to implement a signal to notify the system of the changes
   419  	    runtime_viper.Unmarshal(&runtime_conf)
   420  	}
   421  }()
   422  ```
   423  
   424  ## Getting Values From Viper
   425  
   426  In Viper, there are a few ways to get a value depending on the value’s type.
   427  The following functions and methods exist:
   428  
   429   * `Get(key string) : interface{}`
   430   * `GetBool(key string) : bool`
   431   * `GetFloat64(key string) : float64`
   432   * `GetInt(key string) : int`
   433   * `GetString(key string) : string`
   434   * `GetStringMap(key string) : map[string]interface{}`
   435   * `GetStringMapString(key string) : map[string]string`
   436   * `GetStringSlice(key string) : []string`
   437   * `GetTime(key string) : time.Time`
   438   * `GetDuration(key string) : time.Duration`
   439   * `IsSet(key string) : bool`
   440  
   441  One important thing to recognize is that each Get function will return a zero
   442  value if it’s not found. To check if a given key exists, the `IsSet()` method
   443  has been provided.
   444  
   445  Example:
   446  ```go
   447  viper.GetString("logfile") // case-insensitive Setting & Getting
   448  if viper.GetBool("verbose") {
   449      fmt.Println("verbose enabled")
   450  }
   451  ```
   452  ### Accessing nested keys
   453  
   454  The accessor methods also accept formatted paths to deeply nested keys. For
   455  example, if the following JSON file is loaded:
   456  
   457  ```json
   458  {
   459      "host": {
   460          "address": "localhost",
   461          "port": 5799
   462      },
   463      "datastore": {
   464          "metric": {
   465              "host": "127.0.0.1",
   466              "port": 3099
   467          },
   468          "warehouse": {
   469              "host": "198.0.0.1",
   470              "port": 2112
   471          }
   472      }
   473  }
   474  
   475  ```
   476  
   477  Viper can access a nested field by passing a `.` delimited path of keys:
   478  
   479  ```go
   480  GetString("datastore.metric.host") // (returns "127.0.0.1")
   481  ```
   482  
   483  This obeys the precedence rules established above; the search for the path
   484  will cascade through the remaining configuration registries until found.
   485  
   486  For example, given this configuration file, both `datastore.metric.host` and
   487  `datastore.metric.port` are already defined (and may be overridden). If in addition
   488  `datastore.metric.protocol` was defined in the defaults, Viper would also find it.
   489  
   490  However, if `datastore.metric` was overridden (by a flag, an environment variable,
   491  the `Set()` method, …) with an immediate value, then all sub-keys of
   492  `datastore.metric` become undefined, they are “shadowed” by the higher-priority
   493  configuration level.
   494  
   495  Lastly, if there exists a key that matches the delimited key path, its value
   496  will be returned instead. E.g.
   497  
   498  ```json
   499  {
   500      "datastore.metric.host": "0.0.0.0",
   501      "host": {
   502          "address": "localhost",
   503          "port": 5799
   504      },
   505      "datastore": {
   506          "metric": {
   507              "host": "127.0.0.1",
   508              "port": 3099
   509          },
   510          "warehouse": {
   511              "host": "198.0.0.1",
   512              "port": 2112
   513          }
   514      }
   515  }
   516  
   517  GetString("datastore.metric.host") // returns "0.0.0.0"
   518  ```
   519  
   520  ### Extract sub-tree
   521  
   522  Extract sub-tree from Viper.
   523  
   524  For example, `viper` represents:
   525  
   526  ```json
   527  app:
   528    cache1:
   529      max-items: 100
   530      item-size: 64
   531    cache2:
   532      max-items: 200
   533      item-size: 80
   534  ```
   535  
   536  After executing:
   537  
   538  ```go
   539  subv := viper.Sub("app.cache1")
   540  ```
   541  
   542  `subv` represents:
   543  
   544  ```json
   545  max-items: 100
   546  item-size: 64
   547  ```
   548  
   549  Suppose we have:
   550  
   551  ```go
   552  func NewCache(cfg *Viper) *Cache {...}
   553  ```
   554  
   555  which creates a cache based on config information formatted as `subv`.
   556  Now it’s easy to create these 2 caches separately as:
   557  
   558  ```go
   559  cfg1 := viper.Sub("app.cache1")
   560  cache1 := NewCache(cfg1)
   561  
   562  cfg2 := viper.Sub("app.cache2")
   563  cache2 := NewCache(cfg2)
   564  ```
   565  
   566  ### Unmarshaling
   567  
   568  You also have the option of Unmarshaling all or a specific value to a struct, map,
   569  etc.
   570  
   571  There are two methods to do this:
   572  
   573   * `Unmarshal(rawVal interface{}) : error`
   574   * `UnmarshalKey(key string, rawVal interface{}) : error`
   575  
   576  Example:
   577  
   578  ```go
   579  type config struct {
   580  	Port int
   581  	Name string
   582  	PathMap string `mapstructure:"path_map"`
   583  }
   584  
   585  var C config
   586  
   587  err := Unmarshal(&C)
   588  if err != nil {
   589  	t.Fatalf("unable to decode into struct, %v", err)
   590  }
   591  ```
   592  
   593  ## Viper or Vipers?
   594  
   595  Viper comes ready to use out of the box. There is no configuration or
   596  initialization needed to begin using Viper. Since most applications will want
   597  to use a single central repository for their configuration, the viper package
   598  provides this. It is similar to a singleton.
   599  
   600  In all of the examples above, they demonstrate using viper in its singleton
   601  style approach.
   602  
   603  ### Working with multiple vipers
   604  
   605  You can also create many different vipers for use in your application. Each will
   606  have its own unique set of configurations and values. Each can read from a
   607  different config file, key value store, etc. All of the functions that viper
   608  package supports are mirrored as methods on a viper.
   609  
   610  Example:
   611  
   612  ```go
   613  x := viper.New()
   614  y := viper.New()
   615  
   616  x.SetDefault("ContentDir", "content")
   617  y.SetDefault("ContentDir", "foobar")
   618  
   619  //...
   620  ```
   621  
   622  When working with multiple vipers, it is up to the user to keep track of the
   623  different vipers.
   624  
   625  ## Q & A
   626  
   627  Q: Why not INI files?
   628  
   629  A: Ini files are pretty awful. There’s no standard format, and they are hard to
   630  validate. Viper is designed to work with JSON, TOML or YAML files. If someone
   631  really wants to add this feature, I’d be happy to merge it. It’s easy to specify
   632  which formats your application will permit.
   633  
   634  Q: Why is it called “Viper”?
   635  
   636  A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe))
   637  to [Cobra](https://github.com/spf13/cobra). While both can operate completely
   638  independently, together they make a powerful pair to handle much of your
   639  application foundation needs.
   640  
   641  Q: Why is it called “Cobra”?
   642  
   643  A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)?