Go learning notes (82) - viper of go third-party library (parsing configuration file, hot update configuration file)

Posted by roy on Wed, 08 Dec 2021 01:45:39 +0100

1. viper features

viper is a complete configuration solution for Go applications. It is designed to work in applications and can handle all types of configuration requirements and formats. The supported features and functions are as follows:

  • Set defaults
  • Read configuration files for JSON, TOML, YAML, HCL, envfile, and Java properties
  • Monitor configuration file changes and hot load configuration files
  • Read from environment variable
  • Read the configuration (etcd / consumer) from the remote configuration center and monitor the changes
  • Read from command line flags
  • Read from buffer
  • Setting the value of configuration item directly is supported

Priority order of viper reading configuration file:

  • Value set by viper.Set()
  • Command line flag
  • environment variable
  • configuration file
  • Configuration center etcd / consumer
  • Default value

Note: viper's configuration keys are case insensitive.

2. Installation

Official website address: https://github.com/spf13/viper Installation command

go get github.com/spf13/viper

3. Register the configuration with viper

3.1 creating default values

viper.SetDefault("Name", "wohu")
viper.SetDefault("Gender", "male")
viper.SetDefault("City", map[string]string{"country": "China", "Province": "Beijing"})

3.2 reading values from configuration files

viper does not the search path of the configuration file by default, and puts the search decision of the configuration file path in the user program.

viper.SetConfigName("config") // The name of the configuration file. Note that there is no extension
viper.SetConfigType("yaml") // This field is required if the name of the configuration file does not contain an extension
viper.AddConfigPath("/etc/appname/")   // Path to configuration file
viper.AddConfigPath("$HOME/.appname")  // The path to add multiple profiles is called multiple times
viper.AddConfigPath(".")               // Find the configuration file in the current working directory
err := viper.ReadInConfig() // Find and read configuration files
if err != nil { 
	panic(fmt.Errorf("Fatal error config file: %w \n", err))
}

3.3 save viper value to configuration file

viper.WriteConfig()

Write the current configuration to the path previously configured through viper.AddConfigPath() and viper.SetConfigName(). If there is a corresponding configuration file in this directory, it will be overwritten. If the corresponding path is not found, an error will be reported.

viper.SafeWriteConfig() // The first difference is that the existing files will not be overwritten
viper.WriteConfigAs("/path/to/my/.config") // The existing file will be overwritten
viper.SafeWriteConfigAs("/path/to/my/.config")  // Existing files will not be overwritten
viper.SafeWriteConfigAs("/path/to/my/.other_config")

3.4 monitoring and hot loading configuration files

viper supports the ability of applications to read configuration files in real time while running. Make sure to add all configPaths before calling WatchConfig().

viper.OnConfigChange(func(e fsnotify.Event) {
	fmt.Println("Config file changed:", e.Name)
})
viper.WatchConfig()

3.5 read configuration from io.Reader

viper.SetConfigType("yaml") // or viper.SetConfigType("YAML")

// any approach to require this configuration into your program.
var yamlExample = []byte(`
Hacker: true
name: steve
hobbies:
- skateboarding
- snowboarding
- go
clothing:
  jacket: leather
  trousers: denim
age: 35
eyes : brown
beard: true
`)

viper.ReadConfig(bytes.NewBuffer(yamlExample))

viper.Get("name") // this would be "steve"

4. Read configuration from viper

4.1 method of reading single value

In viper, there are several ways to get a value, depending on the type of value. The following functions and methods exist.

Get(key string) : interface{}
GetBool(key string) : bool
GetFloat64(key string) : float64
GetInt(key string) : int
GetIntSlice(key string) : []int
GetString(key string) : string
GetStringMap(key string) : map[string]interface{}
GetStringMapString(key string) : map[string]string
GetStringSlice(key string) : []string
GetTime(key string) : time.Time
GetDuration(key string) : time.Duration
IsSet(key string) : bool
AllSettings() : map[string]interface{}

Note: if not found, each Get function will return a value of 0. To check whether a given key exists, an IsSet() method has been provided.

4.1 read nested configuration

{
    "host": {
        "address": "localhost",
        "port": 5799
    },
    "datastore": {
        "metric": {
            "host": "127.0.0.1",
            "port": 3099
        },
        "warehouse": {
            "host": "198.0.0.1",
            "port": 2112
        }
    }
}

viper can access a nested field by passing a key value path bounded by.

GetString("datastore.metric.host") // (returns "127.0.0.1")

viper can access array indexes by using numbers in the path, such as

{
    "host": {
        "address": "localhost",
        "ports": [
            5799,
            6029
        ]
    },
    "datastore": {
        "metric": {
            "host": "127.0.0.1",
            "port": 3099
        },
        "warehouse": {
            "host": "198.0.0.1",
            "port": 2112
        }
    }
}

GetInt("host.ports.1") // returns 6029

If there is a key matching the delimited key path, its value will be returned. For example.

{
    "datastore.metric.host": "0.0.0.0",
    "host": {
        "address": "localhost",
        "port": 5799
    },
    "datastore": {
        "metric": {
            "host": "127.0.0.1",
            "port": 3099
        },
        "warehouse": {
            "host": "198.0.0.1",
            "port": 2112
        }
    }
}

GetString("datastore.metric.host") // returns "0.0.0.0"

5. Use examples

The code structure is as follows:

.
├── conf
│   └── config.yaml
├── config
│   └── config.go
├── go.mod
├── go.sum
├── main.go
└── README.md

2 directories, 6 files

config.yaml content

name: demo
host: 127.0.0.1:3306
username: root
password: root         

config.go content:

package config

import (
	"log"

	"github.com/fsnotify/fsnotify"
	"github.com/spf13/viper"
)

type Config struct {
	Name     string
	Host     string
	Username string
	Password string
}

func Init() (*Config, error) {
	viper.AddConfigPath("conf")   // Set profile path
	viper.SetConfigName("config") // Set profile name
	viper.SetConfigType("yaml")   // Set the configuration file type format to YAML

	// Initialize profile
	if err := viper.ReadInConfig(); err != nil { // viper parsing configuration file
		return &Config{}, err
	}
	// Monitor the configuration file changes and hot load the program, that is, the latest configuration can be loaded without restarting the program process
	viper.WatchConfig()
	viper.OnConfigChange(func(e fsnotify.Event) {
		log.Printf("Config file changed: %s", e.Name)
	})

	c := &Config{
		Name:     viper.GetString("name"),
		Host:     viper.GetString("host"),
		Username: viper.GetString("username"),
		Password: viper.GetString("password"),
	}

	return c, nil
}

main.go content:

package main

import (
	"fmt"
	"time"
	"webserver/config"

	"github.com/spf13/viper"
)

func main() {

	// init config
	_, err := config.Init()
	if err != nil {
		fmt.Println(err)
	}
	// Note: the configuration can only be read again through viper.Get method after init, otherwise it will not take effect
	for {
		cfg := &config.Config{
			Name:     viper.GetString("name"),
			Host:     viper.GetString("host"),
			Username: viper.GetString("username"),
			Password: viper.GetString("password"),
		}
		fmt.Println(cfg.Name)
		time.Sleep(4 * time.Second)
	}

}

After running main.go, modify the configuration file to view the print:

$ go run main.go 
123
123
2021/12/03 14:05:45 Config file changed: /home/wohu/goProject/webserver/conf/config.yaml
2021/12/03 14:05:45 Config file changed: /home/wohu/goProject/webserver/conf/config.yaml
345
345
345
2021/12/03 14:05:56 Config file changed: /home/wohu/goProject/webserver/conf/config.yaml
2021/12/03 14:05:56 Config file changed: /home/wohu/goProject/webserver/conf/config.yaml
demo
demo

Topics: Go