Go: Internationalization internationalization

Posted by johnnyk on Tue, 14 Dec 2021 07:15:06 +0100

How should we organize code when services need to deal with multilingual scenarios?

explain

Complex situations such as singular complex transformation are not discussed in this paper. If necessary, see here.

(too long to see the version)
  1. Read the language tag and the corresponding translation, and register the translation string
  2. When obtaining, analyze the label according to the language and obtain the Printer of the corresponding language, and search and generate the translation string according to the Key

The example code is as follows:

package main

import (
  "golang.org/x/text/language"
  "golang.org/x/text/message"
)

// 'und' here is undefined, which is used to display when there is no corresponding translation for the language
// Please prepare the corresponding key and translation string according to the translation file
var msg = map[string]string{"zh": "Hello,%s", "en": "Hello, %s", "und": "Hello, %s"}
var key = "HelloString" // Key is the key used by the dictionary to find the translation string; Of course, you can also use "Hello,% s" here

func register(k string, d map[string]string) {
  for lang, translation := range d {
    // Parsing language tags based on language strings
    tag, err := language.Parse(lang)
    if err != nil {
      // It is recommended to complete the language registration in the program initialization phase. An error may occur directly at this time
      panic(err)
    }
    // Set according to the language tag, key and translation string
    err = message.SetString(tag, key, translation)
    if err != nil {
      panic(err)
    }
  }
}

func getTranslation(k, lang string, content []interface{}) string {
  tag, err := language.Parse(lang)
  if err != nil {
    // Specifies the language you want to return if there is a parsing error
    tag = language.Und
  }
  // Get Printer by language
  p := message.NewPrinter(tag)
  return p.Sprintf(k, content...)
}

func main() {
  register(key, msg)
  println(getTranslation(key, "en", []interface{}{"en"}))
  println(getTranslation(key, "en-US", []interface{}{"en-US"}))
  println(getTranslation(key, "zh", []interface{}{"zh"}))
  // The parent s of ZH CN and zh SG are Zh, so they will be returned according to zh
  println(getTranslation(key, "zh-CN", []interface{}{"zh-CN"}))
  println(getTranslation(key, "zh-SG", []interface{}{"zh-SG"}))
  // The parent of zh TW is not zh, so the corresponding translation will be returned according to the und tag
  println(getTranslation(key, "zh-TW", []interface{}{"zh-TW"}))
  // Handling of errors in parsing language
  println(getTranslation(key, "???", []interface{}{"???"}))
}
/* Output:
Hello, en
Hello, en-US
 Hello, zh
 Hello, Zh CN
 Hello, Zh SG
Hello, zh-TW
Hello, ???
*/
(too long to see the end of the version)

Prerequisites: Principle

Language tags

We can usually see values like accept language: zh CN or accept language: en in the HTTP header. The corresponding value in the header here is the language tag. Similarly, Zh CMN Hans cn is also a language tag.

Syntax of language tags

We need to pay attention to the syntax of language tags:

Subject sub tag - extended language sub tag - text sub tag - region sub tag

zh-cmn-Hans-CN

  • Except that the subject sub tag is required, others are optional;
  • The extended language sub tag is 3 letters, and there can be at most three;
  • The text sub label is 4 letters (initial capital), with a maximum of one;
  • The region sub label is 2 letters (usually uppercase)

Therefore, Zh CMN Hans cn is interpreted as:

Chinese (zhongwen) - Putonghua (simplified mandarin) - Simplified Han Simplified Chinese mainland

In addition, the language tag is no longer recommended after 2009, because the extended language tag cmn implies that the language is zh (Chinese). Of course, it is recommended to use zh instead of cmn for compatibility.

Go stores the translation string

Go by calling message Setstring (tag language.tag, key string, msg string) to store the translation string; Where tag is the language tag, key is the key of the string, and msg is the value of the string in the language. for example

// tag: en
message.SetString(language.English, "HelloString", "Hello, %s")
// tag: zh-Hans
message.SetString(language.SimplifiedChinese, "HelloString", "Hello,%s") 
// tag: und
message.SetString(language.Und, "HelloString", "Hi, %s")

Go get translation string

Go by calling message Newprinter (tag language. Tag) to obtain the translation string dictionary by calling printer Sprint (key string, content... Interface {}) (or other methods similar to those in fmt) to generate (or print) translation strings. We use sprint as an example.

When printing, use the key and find the corresponding dictionary according to the language tag. If the key cannot be found in the language tag, continue to find it in its ancestor node in turn; If the root node (und) is found but not found, the effect is the same as calling fmt.Sprintf directly. Therefore, in a simple scenario, it is recommended to directly add translation statements under en, Zh and und; und is used to deal with unresolved language tags (for example,??) or unexpected language tags (for example, the ancestor nodes of ZH TW are zh hant and und in turn).

for example

message.NewPrinter(language.English, "en") // Hello, en
message.NewPrinter(language.AmericanEnglish, "en-US") //* Hello, en-US
message.NewPrinter(language.Chinese, "zh") //** Hi, zh
message.NewPrinter(language.SimplifiedChinese, "zh-Hans") // Hello, Zh Hans
/*
*   Here, because the parent node of en US is en, the corresponding translation string can be found
**  Here, because the parent node of ZH is und, the translation string in und is found (zh Hans is the child node of zh)
*/

practice

Step 1: prepare dictionary

It is recommended to prepare the translation string and language tag of each language in a separate text file. Call message by reading the file, parsing the tag and string Setstring setting.

Step 2: get the translation string according to the passed in language tag

Call language according to the language tag Parse gets the tag and uses it to get message Printer, use this printer to call printer according to the key Sprintf() generates a translation string.

gRPC Gateway gets the language tag in the request

gRPC Gateway stores HTTP headers in context and calls metadata.. Fromincomingcontext (CTX context. Context) can be obtained.

Note that the structure of map[string][]string is obtained, and the key of map is the names of HTTP headers, so it may be all lowercase (accept language) or initial uppercase (accept language). Therefore, it is necessary to use strings. Equalfold (k, "accept language") to ignore case comparison.