gopher's self-cultivation

Posted by 0riole on Mon, 20 Dec 2021 07:04:02 +0100

Write in front

go build, go run, go install, go get are the most frequently used commands by go language developers in their daily work or study, These commands can help us compile programs, run programs, install programs and obtain code packages. However, when executing these commands, have you thought about how to organize our source files in the go compilation process? What happened when go install? And do go get just download the package files we rely on? What other useful commands do go have? This is a basic article and can also be used as a quick reference manual for go commands to help us have a deeper understanding of go.

Several concepts

  • Command source file: it is declared as the source file of the main function that belongs to the main code package and contains no parameter declaration and result declaration. Generally, a project has only one command source file.
  • Library source code file: a common source code file that exists in a package without the characteristics of command source code
  • Test source file: to_ test.go is the source code file with suffix, and must contain functions with test or beachmark as the name prefix.

Gopath may be the most familiar thing for every go developer. Gopath is our workspace. You can view our gopath through go env, such as "GOPATH="/Users/code/go ". Gopath is our workspace. Under gopath, three folders of bin, src and pkg are generally established.

  • Bin: the executable file after go install will be placed in the bin directory. However, there is a GOBIN in the environment variable of go. By default, GOBIN is empty, but when we set GOBIN, GOPATH/bin will be meaningless, and the executable file of go install will be placed under GOBIN by default.
export GOBIN=/tmp //Declare GOBIN directory
go install $GOPATH/src/code //Install code program
/tmp/code The discovery program has been installed in GOBIN lower
  • src: the source files we write are all in this folder.
  • pkg: depends on the location where the library source code file after package installation is stored, that is, the archive file (. a end), that is, the static library file generated after program compilation.

Common auxiliary commands

Go has some auxiliary commands that can help us better understand some execution processes of go. The embodiment of auxiliary commands is go run/build  -x xx.go, - x is the auxiliary command- X can be of the following types:

  • -a: Force all involved code packages (including those in the standard library) to be rebuilt, even if they are already up-to-date.
  • -n: Print other commands used during the construction, but do not actually execute them. You can observe the construction process through - n
  • -race: used to detect data contention problems, such as map concurrent read and write
  • -v: It is used to print the code packages involved in the command execution process, including the code packages we depend on, and you can also see the code packages associated with the dependent code packages.
  • -work: used to print the name of the temporary working directory generated and used when the command is executed, and it will not be deleted after the command is executed. If this flag is not added, the temporary working directory will be deleted after the command is executed.
  • -x: Print all commands used during command execution and execute them at the same time.
  • -p n: number of parallels built (n). By default, the number of parallels is the same as the number of CPU s.
  • -o: Compiles the file specified for output to.

Minority auxiliary command

  • -asmflags: this tag can be followed by other tags, such as - D, - I, - S, etc. These tags followed by are used to control the behavior of the Go language compiler when compiling assembly language files.
  • -buildmode: Specifies the compilation mode, such as - buildmode=default. It mainly controls the compiler to generate static link library (that is. a file, that is, the archive file we mentioned earlier), dynamic link library (that is. so file) or / and executable file (that is. exe file under Windows).
  • -Compiler: Specifies the name of the compiler currently in use, which can be gc or gccgo. Among them, gc compiler is the editor of Go language, and gccgo compiler is the Go language compiler provided by GCC.
  • -Gccgo flags: Specifies the list of tags that need to be passed to the gccgo compiler or linker.
  • -gcflags: Specifies the list of tags that need to be passed to the go tool compile command.
  • -Installsuffix: this flag can be used to separate the current output directory from the default compiled output directory. The value of this tag is used as a suffix to the parent directory name of the result file. In fact, if the - race tag is used, this tag will be automatically appended and its value will be race. If we use both the - race tag and the - installsuffix tag, the value of the - installsuffix tag will be appended_ Race, and use it as the suffix actually used.
  • -ldflags: Specifies the list of tags to be passed to the go tool link command.
  • -linkshared: used with - buildmode=shared. The latter will cause the non main code packages that are the compilation target to be merged into a dynamic link library file, while the former will link on it.
  • pkgdir: you can specify a directory. The compiler will only load the archive files of code packages from this directory, and will place the archive files of code packages that may be generated by compilation in this directory.
  • -Tags: Specifies the list of compilation tags (also known as compilation constraints) that need to be accepted during actual compilation.
  • -toolexec: you can customize the way to use some Go language built-in tools (such as vet, asm, etc.) during compilation.
  • -Mod: the - mod [mode] mode is used. The [mode] supports readonly, release and vendor. When go build -mod=vendor is executed, the dependent packages of the project will be placed in the vendor directory of the main module while generating the executable file.
  • -modcacherw: keep the downloaded mod package in the module cache in read-write mode instead of making it read-only.
  • -trimpath: delete the source path information in the binary file. For example, when our program panic, it may print out the source path of relevant errors, which can be hidden.

compile

go run

We use go run - N main Go to see the construction process

mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=/Users/gopher/Library/Caches/go-build/f5/f58206d3722b7787f6341e5013f38f3887e44138ffbdafbb07b67da377632762-d
packagefile code/utils=/Users/gopher/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d
packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a
...
EOF
mkdir -p $WORK/b001/exe/
cd .
/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/main -importcfg $WORK/b001/importcfg.link -s -w -buildmode=exe -buildid=ttUV5epeZDG7q3yVlG2A/ek8iSJ8rjD-RiujnYKAd/hNomEMXuvdA3I1ks6XOE/ttUV5epeZDG7q3yVlG2A -extld=clang /Users/goper/Library/Caches/go-build/f5/f58206d3722b7787f6341e5013f38f3887e44138ffbdafbb07b67da377632762-d
$WORK/b001/exe/main
  1. Create temporary directory: $WORK/b001/
  2. Find dependent package information: $work / B001 / importcfg link
  3. Create exe Directory: $WORK/b001/exe/
  4. Link the library file (importcfg.link) through the link of go's tool tool to generate the executable file $WORK/b001/exe/main

Via go run - work main Go, let's look at the temporary folder:

go run -work main.go
WORK=/var/folders/s4/2cpbmp4s1_j4y3zv2s08m9q40000gn/T/go-build281107053

Switch to temporary directory:

└── b001
    ├── exe
    │   └── main
    └── importcfg.link
  • exe: is our executable.
  • importcfg.link: the package that our program depends on, and the dependent package is linked through the linker.

By default, the temporary folder is deleted after the go run command is run.

go build

go build is used to compile our program. By default, the compiled file is stored in the current folder. If - o is specified, it can be moved to the specified file.
We use go build - N main Go to see the build process:

#
# command-line-arguments
#

mkdir -p $WORK/b001/
cat >$WORK/b001/_gomod_.go << 'EOF' # internal
package main
import _ "unsafe"
//go:linkname __debug_modinfo__ runtime.modinfo
var __debug_modinfo__ = "0w\xaf\f\x92t\b\x02A\xe1\xc1\a\xe6\xd6\x18\xe6path\tcommand-line-arguments\nmod\tcode\t(devel)\t\n\xf92C1\x86\x18 r\x00\x82B\x10A\x16\xd8\xf2"
        EOF
cat >$WORK/b001/importcfg << 'EOF' # internal
# import config
packagefile code/utils=/Users/sunkang/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d
packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a
packagefile runtime=/Users/sunkang/Library/Caches/go-build/b4/b44856e241a6bb3baf596eb19e4566e956a490ef403c1ed31ba8f014542fcf81-d
EOF
cd /Users/gopher/go/src/code
/usr/local/go/pkg/tool/darwin_amd64/compile -o $WORK/b001/_pkg_.a -trimpath "$WORK/b001=>" -p main -lang=go1.15 -complete -buildid Fjkl2yr7MirhGqbO0lrl/Fjkl2yr7MirhGqbO0lrl -goversion go1.15.3 -D _/Users/sunkang/go/src/code -importcfg $WORK/b001/importcfg -pack -c=4 ./main.go $WORK/b001/_gomod_.go
/usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/_pkg_.a # internal
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=$WORK/b001/_pkg_.a
packagefile code/utils=/Users/sunkang/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d
packagefile fmt=/usr/local/go/pkg/darwin_amd64/fmt.a
...
EOF
mkdir -p $WORK/b001/exe/
cd .
/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=s0BcVGdGaAeuHGFL7teJ/Fjkl2yr7MirhGqbO0lrl/Fjkl2yr7MirhGqbO0lrl/s0BcVGdGaAeuHGFL7teJ -extld=clang $WORK/b001/_pkg_.a
/usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/exe/a.out # internal
mv $WORK/b001/exe/a.out main

The overall process is similar to go run. The only difference is that after compile and link, the generated executable file will be moved to the current folder and will not die with the temporary folder.

go install

Go install is used to compile and install the specified code package and its dependent packages. When the dependent packages of the specified code package have not been compiled and installed, the dependent packages will be installed first. Different from go build, go install will put the compiled installation package in the specified folder. The installed code package will be in the pkg directory of the current workspace, that is a's archive file. When we do not set GOBIN, the installed command source file will be stored in the bin directory of the current workspace. When we set GOBIN, it will be placed in GOBIN. Suppose the current project is as follows:

├── go.mod
├── main.go
└── utils
    └── utils.go

main.go is our entry file, that is, the command source file, and utils is our dependent package, that is, the library source file. When we only execute go install - N main in the current directory After go:

mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
...
EOF
mkdir -p $WORK/b001/exe/
cd .
/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=yfU8nbngCa6KbgUlJLKa/Fjkl2yr7MirhGqbO0lrl/khh18opCAdXA909bR95q/yfU8nbngCa6KbgUlJLKa -extld=clang /Users/sunkang/Library/Caches/go-build/ed/ed5868e69c99c66b8bf4b399e989ea410063b44143b546fd3f4a98f758d73a47-d
/usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/exe/a.out # internal
mkdir -p /Users/gopher/go/bin/
mv $WORK/b001/exe/a.out /Users/gopher/go/bin/main

You can see that the last line is different from go build in that it moves the executable to GOPATH/bin.
When we switch to the dependent package utils file and execute "go install -n":

mkdir -p $WORK/b001/
mkdir -p /Users/gopher/go/pkg/darwin_amd64/code/
mv /Users/gopher/Library/Caches/go-build/89/8931b87411d98f32ea6222ad605f3367700be896828286782db3e51de73217ff-d /Users/gopher/go/pkg/darwin_amd64/code/utils.a

It can be found that the dependent package will be installed under {GOPATH/pkg and named a end of the archive. darwin_amd64 is $goos_$ The assembly of goarch, namely the operating system and processor architecture, code is our project name.

/Users/gopher/go/pkg/darwin_amd64/code
└── utils.a

So the general process of go install is as follows:

go get

We can download and install a dependency package through the go get command. The source file downloaded by go get will be placed in the first workspace in GOPATH. Since go supports go mod from 1.11, when we enable go mod, the code obtained through go get will be downloaded in GOPAT/pkg/mod, and optional parameters are supported after go get

  • -d: Let the command program only perform the download action, not the installation action.
  • -u: Let the command use the network to update existing code packages and their dependent packages. By default, this command will only download code packages that do not exist locally from the network, and will not update existing code packages.
  • -t: Let the command program download and install the dependent code package in the test source file in the specified code package at the same time.
  • -Secure: allows the command program to use a non secure scheme (such as HTTP) to download the specified code package. If the code warehouse you use (such as Gitlab within the company) does not have HTTPS support, you can add this flag. Please use it when you are sure it is safe.
  • -v: Displays the command executed.

go mod

Go has supported go mod mode since 1.11. Now I believe everyone is basically using go mod. Compared with vendor s, your code can exist anywhere after using go mod.

  • GO111MODULE=on enable go mod mode through this environment variable
  • GOPROXY="https://goproxy.io ", it is recommended that you use this proxy to download the third-party dependency package of go.
  • GOPRIVE="git.xx.xx". prive is a private warehouse. For example, the company's own internal warehouse, you can configure the domain name of the private warehouse to obtain the internal code

go mod related commands

  1. go mod init module name} initializes a go project
  2. Go mod Download: Download modules to the local cache, that is, $GOPATH/pkg/mod/cache/download. Execute go mod download -json to output information in json format:
//go mod download -json
{
    "Path": "github.com/go-basic/uuid",
    "Version": "v1.0.0",
    "Info": "/Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.info",
    "GoMod": "/Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.mod",
    "Zip": "/Users/gopher/go/pkg/mod/cache/download/github.com/go-basic/uuid/@v/v1.0.0.zip",
    "Dir": "/Users/gopher/go/pkg/mod/github.com/go-basic/uuid@v1.0.0",
    "Sum": "h1:Faqtetcr8uwOzR2qp8RSpkahQiv4+BnJhrpuXPOo63M=",
    "GoModSum": "h1:yVtVnsXcmaLc9F4Zw7hTV7R0+vtuQw00mdXi+F6tqco="
}
  1. go mod edit

Edit go The mod file options are - json, - replace, You can use the help go help mod edit. For example, if you want to modify a package, you can directly use go mod edit -replace=old[@v]=new[@v]. Generally, you can edit go directly Mod file, this command is not used much.

  1. go mod graph

Print dependent packages in text mode, such as my go Mod yes

module code
go 1.15
require (
   github.com/gin-gonic/gin v1.7.4 // indirect
   github.com/go-basic/uuid v1.0.0 // indirect
)

At this time, execute go mod graph

//go mod graph
code github.com/gin-gonic/gin@v1.7.4
code github.com/go-basic/uuid@v1.0.0
github.com/gin-gonic/gin@v1.7.4 github.com/gin-contrib/sse@v0.1.0
github.com/gin-gonic/gin@v1.7.4 github.com/go-playground/validator/v10@v10.4.1
github.com/gin-gonic/gin@v1.7.4 github.com/golang/protobuf@v1.3.3
github.com/gin-gonic/gin@v1.7.4 github.com/json-iterator/go@v1.1.9
github.com/gin-gonic/gin@v1.7.4 github.com/mattn/go-isatty@v0.0.12
github.com/gin-gonic/gin@v1.7.4 
...
golang.org/x/text@v0.3.2 golang.org/x/tools@v0.0.0-20180917221912-90fa682c2a6e

We still can't see the dependency. Here we recommend to use {go get - u GitHub COM / paulxu CN / go mod graph chart / gmchart this package to view dependencies:

  1. go mod tidy

Add missing or remove unwanted modules.
My current go There is a uuid package in mod, but my code does not refer to it.

module code

go 1.15

require github.com/go-basic/uuid v1.0.0 // indirect

Execute go mod tidy:

module code

go 1.15

You'll find that you've removed unwanted bags for me.

6.go mod verify
Verify that the dependency is correct.
7. go mod why
Explain why packages and modules are needed, such as go mod why GitHub COM / go basic / UUID, and then output:

# github.com/go-basic/uuid
code/utils
github.com/go-basic/uuid

My understanding is that the "code/utils" package is useful to GitHub com/go-basic/uuid.

go.sum

go. The sum file serves two purposes:

  1. Checksum to prevent package tampering and checksum
  2. Update record of record package

When we go get a package, we will first download it to the local $GOPATH/pkg/mod/cache/download. After downloading, there will be a package named VX x. X.zip, and VX x. File of x.ziphash, VX x. The content of x.ziphash is VX x. The value of x.zip through hash, such as: h1:jwqTeEM3x6L9xDXrCxN0Hbg7vdGfPBOTIkr0+/LYZDA =% take uuid package as an example: when we go get GitHub COM / go basic / uuid, except in go After a require command is added to mod, it will be added in go Write two records in sum:

github.com/go-basic/uuid v1.0.0 h1:Faqtetcr8uwOzR2qp8RSpkahQiv4+BnJhrpuXPOo63M=
github.com/go-basic/uuid v1.0.0/go.mod h1:yVtVnsXcmaLc9F4Zw7hTV7R0+vtuQw00mdXi+F6tqco=

The first hash is the hash value of the zip package mentioned above. The second hash is if there is go in our dependent package Mod, then this is the go Hash value of mod. When preparing to update two hash value records to go In sum, in order to ensure the authenticity and reliability of the dependent package, go will check the hash value of the dependent package through the server pointed to by go's environment variable GOSUMDB="sum.golang.org" after downloading the dependent package. If the hash value queried is different from the local hash value, it will refuse to execute downward and will not update go sum.

go clean

  • -n: Print out the clear command to be executed, but do not execute it, so that you can easily know how the bottom layer works.
  • -i: Clear the associated installed packages and runnable files. We can combine - n to see how - I performs:
//go clean -i -n
cd /Users/gopher/go/job // Current project
rm -f job job.exe job.test job.test.exe main main.exe
rm -f /Users/gopher/go/bin/job

First switch to our project, and then try to delete some compiled files in the current directory, such as exe,. test, and finally try to delete $GOPATH/bin/job, the compiled file generated by go install.

  • -r: Circular cleanup package introduced in import:
// go clean -r -n 
cd /Users/gopher/go/job
rm -f job job.exe job.test job.test.exe main main.exe
cd /usr/local/go/src/fmt
rm -f fmt.test fmt.test.exe
cd /usr/local/go/src/errors
rm -f errors.test errors.test.exe
cd /usr/local/go/src/internal/reflectlite
rm -f reflectlite.test reflectlite.test.exe
....

The job project depends on many packages, and these dependent packages will also delete some compiled files in the current directory.

  • -x: The detailed commands printed out are actually -n printed versions.
  • -Cache: delete the cache of all go build commands.

The go build process generates some caches. These caches exist in the go environment variable. The - cache in GOCACHE deletes the relevant caches:

//go clean -n -cache
rm -r /Users/gopher/Library/Caches/go-build/00 
/Users/gopher/Library/Caches/go-build/01
/Users/gopher/Library/Caches/go-build/02 
/Users/gopher/Library/Caches/go-build/03 
/Users/gopher/Library/Caches/go-build/04 
...
rm -r /Users/gopher/Library/Caches/go-build/ff
  • -testcache: delete all test results of the current package.

When we use go test When running the test cases under a path, each test file under the path will be compiled and tested, and the test results will be cached to avoid unnecessary repeated tests. When the cache is successful, the second test run will find a cached ID.

for the first time
go test .
ok      job     0.431s
 The second time
go test .
ok      job     (cached)

In this case, go clean -testcache is used to delete the corresponding test cache.

go clean -testcache
go test .
ok      job     0.459s
  • -modcache: delete go Cache file in mod mode.

When we start go When we organize our go code in mod mode, the downloaded dependent packages will be placed in $GOPATH/pkg/mod. Through go clean -modcache, we delete all files under $GOPATH/pkg/mod.

//go clean -n -modcache
rm -rf /Users/gopher/go/pkg/mod

epilogue

The purpose of writing this article is mainly because of my lack of knowledge. I have used go for two years. However, there is still a sense of ambiguity about some commands of go and the compilation process of go, such as go, which has been used in the project mod，go. How does mod manage the code packages we rely on, go What is sum? Why go sum？ What happened during go build? On the other hand, this paper also lists the basic daily development commands, so that when you need to find commands, you don't have to go to online Baidu, which can be used as a quick reference manual.
Maybe our daily tasks don't need our understanding of the underlying knowledge, but curiosity is also a way for a programmer to make rapid progress. If you can chew down the boring things like principle and bottom layer, you will find that many problems have been figured out. Just like practicing martial arts, people with high martial arts will find that their internal mental skills are very strong. When their internal mental skills are strong, they will learn any martial arts quickly.

Topics: Go compiler

Hot Topics

©2024 Programmer Think