Golang:JSON, int64 and interface

Posted by Toby on Tue, 01 Feb 2022 23:43:28 +0100

Let's take a look first: About JSON number

Cross reference relationship between golang data type and json type:

bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

There is no int64 type in json. When deserializing, if you directly use json Unmarshal may lose accuracy
You can refer to this article: json deserialization number precision lost

Principle:
When parsing JSON, it is actually a traversal of the JSON string. In fact, all the traversed values are of type [] byte, and then the field type of the object is parsed according to the recognition target, or the content conversion format of [] byte data is recognized.

For example, if the data is parsed to int, convert [] byte to int;
If it is resolved to interface {}, it can only be converted through the type of [] byte,
And the numbers will be processed into float64. There is a problem that the accuracy will be lost.
When parsed through Number, the value will be directly saved as a string type.

Correct practice:

func DecodeMyStruct(jsonStr string) (*MyStruct, error) {
	var regular *MyStruct
	decoder := jsoniter.NewDecoder(strings.NewReader(jsonStr))
	//When using float. json() to convert a floating-point number into a large number, it is not allowed to use float. json() Marshal
	decoder.UseNumber() 
	err := decoder.Decode(&regular)
	if err != nil {
		return nil, err
	}
	return &regular, nil
}

MyStruct is the type we need to deserialize, which can be struct, slice, map, etc

If you don't know what type to deserialize, it can be map[string]interface {}, which can be used with assertions,

For example, in the following scenario, we need to compare the structure of two json inversely sequenced into map[string]interface {},
Judge: whether filterAttrMap contains all key value pairs of conditionMap.

func CompareRecursiveV2(ctx context.Context, filterAttrMap, conditionMap map[string]interface{}) bool {
	for conditionKey, conditionValue := range conditionMap {
		filterValue, filterOk := filterAttrMap[conditionKey]
		if !filterOk {
			return false
		}
		// Assert the type of the judgment value interface {}
		switch conditionValue.(type) {
		case json.Number:      // !!!! This cannot be int64
			filterJson, ok1 := filterValue.(json.Number)
			conditionJson, ok2 := conditionValue.(json.Number)
			filter := filterJson.String()
			condition := conditionJson.String()
			if !ok1 || !ok2 || filter != condition {
				return false
			}
		case string:
			filter, ok1 := filterValue.(string)
			condition, ok2 := conditionValue.(string)
			if !ok1 || !ok2 || filter != condition {
				return false
			}
		case bool:
			filter, ok1 := filterValue.(bool)
			condition, ok2 := conditionValue.(bool)
			if !ok1 || !ok2 || filter != condition {
				return false
			}
		default:
			conditionValueMap, ok1 := conditionValue.(map[string]interface{})
			if !ok1 {
				logs.CtxFatal(ctx, "conf error, only support[int64, string, bool, map]")
				return false
			}
			filterValueMap, ok2 := filterValue.(map[string]interface{})
			if !ok1 || !ok2 {
				return false
			}
			// value type or map[string]interface {}, recursive parsing and comparison
			return CompareRecursiveV2(ctx, filterValueMap, conditionValueMap)
		}
	}
	return true
}

The first item of the above case cannot be int64, because the two comparison map s are transferred from json, and the number in the interface {} is only number type (actually stored in string). Therefore, it is necessary to type case number and convert it into string to judge whether the two value s are the same.

This is different from the following case. The following example: updateStudent is also a map[string]interface {} type
However, updateStudent is not converted by json, but is explicitly assigned by code

    var student_id int64 = 123456
    updateStudent["student_id"]= student_id
    id, ok := updateStudent["student_id"].(int64) // You can use int64
	if !ok {
		fmt.print("Assertion failed, key: stedent_id no int64 type")
	} else {
		fmt.printf(" key: stedent_id yes int64 type, And the value is:%d", id)
	}

Another comparison of two map[string]interface {} is added. Here is an additional regular, which is also of type map[string]interface {}
It is a json structure configured on the cloud. We can dynamically configure this regular on the cloud to define the fields and field types to be compared between the two map s (key and value are set to string type, which is convenient for switch case)
Every time you need to compare, pull the json from the cloud through the client and convert it into map[string]interface {} for comparison
For example:

{
    "student_id":"int64",
    "student_name":"string",
    "genre":"int64",
    "status":"int64",
    "extra_struct":{
        "father":"string",
        "risk_rate":"int64",
        "dynamic_struct":{
            "yuwen_score":"int64",
            "shuxue_score":"int64"
        }
    }
}
func CompareRecursive(ctx context.Context, regular, filterAttrMap, conditionMap map[string]interface{}) bool {
	for regularKey, regularValue := range regular {
		filterValue, filterOk := filterAttrMap[regularKey]
		conditionValue, conditionOk := conditionMap[regularKey]
		if filterOk && conditionOk {
			if regularValue == "int64" {
				filterJson, ok1 := filterValue.(json.Number)
				conditionJson, ok2 := conditionValue.(json.Number)
				filter := filterJson.String()
				condition := conditionJson.String()
				if !ok1 || !ok2 || filter != condition {
					return false
				}
			} else if regularValue == "string" {
				filter, ok1 := filterValue.(string)
				condition, ok2 := conditionValue.(string)
				if !ok1 || !ok2 || filter != condition {
					return false
				}
			} else if regularValue == "bool" {
				filter, ok1 := filterValue.(bool)
				condition, ok2 := conditionValue.(bool)
				if !ok1 || !ok2 || filter != condition {
					return false
				}
			} else {
				regularValueMap, ok := regularValue.(map[string]interface{})
				if !ok {
					logs.CtxFatal(ctx, "%s conf error, support[int64, string, bool, map]", regularKey)
					return false
				}
				filterValueMap, ok1 := filterValue.(map[string]interface{})
				conditionValueMap, ok2 := conditionValue.(map[string]interface{})
				if !ok1 || !ok2 {
					return false
				}
				return CompareRecursive(ctx, regularValueMap, filterValueMap, conditionValueMap)
			}
		} else if !filterOk && conditionOk {
			return false
		}
	}
	return true
}

Topics: Go JSON string