Go Scheduler Series Source Code Reading and Exploration

Posted by lkq on Thu, 09 May 2019 05:33:02 +0200

Dear friends, this time I would like to share with you the knowledge and experience related to the source code reading of Go Scheduler. There are many good articles on the network to analyze the source code. So this article is not another source code analysis article. It is not about the source code analysis and sharing, but to bring you some learning experience. I hope you can better read and master the implementation of Go Scheduler.

This article mainly divides into two parts:

Solve the problem of how to read the source code. Reading source code is essentially a scheduling design already in mind, to see whether it is so achieved, how it is achieved.
Give you a way to explore the implementation of Go Scheduler. Source code is ready, you can modify, snoop, through this way to solve the problems in the process of reading the source code, verify some ideas. For example: the task of scheduling is g0, how can schedule() be executed when it is currently g0?

How to read source code

Reading premise

Before reading the Go source code, you'd better have a good grasp of the design and principle of the Go scheduler. If you still can't answer the following questions:

Why do you need a Go scheduler?
What are the differences and relationships between Go scheduler and system scheduler?
What is G, P and M and what is their relationship?
How many defaults do P have?
How many P can M bind at the same time?
How does M get G?
What if M doesn't have G?
Why do you need a global G queue?
What are the two ways of load balancing in Go scheduler?
What is work stealing? What principle?
What is the impact of system calls on G, P, M?
What is Go scheduler preemption like? Are we sure to succeed?

It is recommended to read the Go Scheduler series and the references in this article:

Recommendation of Excellent Source Data

Now that you can answer the above questions, it shows that you have a certain grasp of the design of the Go Scheduler. There are many excellent data about the source code of the Go Scheduler. I recommend two here:

Rain Track Go Source Analysis Chapter 6 concurrent scheduling, not only the source code, is based on the source code for a detailed introduction of Go scheduler: ttps://github.com/qyuhen/book
GoNight Reading No. 12, goroutine scheduling in Goang, life states of M, P and G, and transformation relationships: https://reading.developerlear...

The source code of Go scheduler also involves GC and so on. When reading the source code, you can skip it temporarily and master the logic of scheduling.

In addition, the Go scheduler involves assembly. Maybe you don't understand assembly. Don't worry, the rainmark article explains the assembly part.

Finally, give you a flow chart, draw the main scheduling process, you can also read while drawing, increase understanding, HD version can be downloaded to the blog (original text jump).

How to Explore Scheduler

This section teaches you to explore the source code of Go scheduler and verify your ideas. The main idea is to download the source code of Go, add debugging and printing, compile and modify the source file, generate the modified go, and then run the test code with modified go to observe the results.

Download and compile Go

Github downloaded and switched to the go1.11.2 branch. All code modifications in this article are based on the go1.11.2 version.

$ GODIR=$GOPATH/src/github.com/golang/go
$ mkdir -p $GODIR
$ cd $GODIR/..
$ git clone https://github.com/golang/go.git
$ cd go
$ git fetch origin go1.11.2
$ git checkout origin/go1.11.2
$ git checkout -b go1.11.2
$ git checkout go1.11.2

First compilation, test run, take a little longer

$ cd $GODIR/src
$ ./all.bash

After each modification of go source code in the future, it can be compiled in about 4 minutes.

$ cd  $GODIR/src
$ time ./make.bash
Building Go cmd/dist using /usr/local/go.
Building Go toolchain1 using /usr/local/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for linux/amd64.
---
Installed Go for linux/amd64 in /home/xxx/go/src/github.com/golang/go
Installed commands in /home/xxx/go/src/github.com/golang/go/bin

real    1m11.675s
user    4m4.464s
sys    0m18.312s

The compiled go and gofmt are in the $GODIR/bin directory.

$ ll $GODIR/bin
total 16044
-rwxrwxr-x 1 vnt vnt 13049123 Apr 14 10:53 go
-rwxrwxr-x 1 vnt vnt  3377614 Apr 14 10:53 gofmt

In order to prevent the conflict between the modified go and the previously installed go, we create igo soft connection and point to the modified go.

$ mkdir -p ~/testgo/bin
$ cd ~/testgo/bin
$ ln -sf $GODIR/bin/go igo

Finally, by adding ~/testgo/bin to PATH, the code can be compiled using igo. Running igo, the version of go1.11.2 should be obtained:

$ igo version
go version go1.11.2 linux/amd64

At present, we have mastered the method of compiling and using modified go. Next, we will use a simple example to teach you how to verify ideas.

Verify that schedule() is executed by g0

Read the source code article, you already know that g0 is responsible for scheduling, and g0 is a global variable that can be used directly anywhere in the runtime package. You can see the schedule() code as follows (in the file: $GODIR/src/runtime/proc.go):

// One round of scheduler: find a runnable goroutine and execute it.
// Never returns.
func schedule() {
    // Get the current g, which should be g0 when dispatching
    _g_ := getg()

    if _g_.m.locks != 0 {
        throw("schedule: holding locks")
    }

    // M has been locked by some g. First stop the current m, wait for G to run, then execute g, and get the p where G is located.
    if _g_.m.lockedg != 0 {
        stoplockedm()
        execute(_g_.m.lockedg.ptr(), false) // Never returns.
    }

    // Omission...
}

Question: Since G0 is responsible for scheduling, why does schedule() execute _g_:= getg () every time? Can't we use G0 directly? Is schedule() really g0?

stay Go Scheduler Series (2) A Macro View of Scheduler In this article, I introduced the use of trace. When reading the code, I found that debug.schedtrace and print() functions can be used to print debugging information. Can we use this method to print the information we want to get? Certainly.

In addition, note that print() is not fmt.Print(), nor printf of C language, so it is not formatted output, it is assembled implementation, we do not go deep into its implementation, now we need to master its use:

// The print built-in function formats its arguments in an
// implementation-specific way and writes the result to standard error.
// Print is useful for bootstrapping and debugging; it is not guaranteed
// to stay in the language.
func print(args ...Type)

As you can see from the above, it accepts variable length parameters. When we use it, we only need to pass in, but we need to control the format manually.

We modify the schedule() function, use debug. schedtrace > 0 to control printing, add three lines of code, and print out the goid. If the goid is always printed as 0, then the scheduling is indeed executed by g0:

if debug.schedtrace > 0 {
    print("schedule(): goid = ", _g_.goid, "\n") // Will it be zero? Yes
}

schedule() is as follows:

// One round of scheduler: find a runnable goroutine and execute it.
// Never returns.
func schedule() {
    // Get the current g, which should be g0 when dispatching
    _g_ := getg()

    if debug.schedtrace > 0 {
        print("schedule(): goid = ", _g_.goid, "\n") // Will it be zero? Yes
    }

    if _g_.m.locks != 0 {
        throw("schedule: holding locks")
    }
    // ...
}

Compile igo:

$ cd  $GODIR/src
$ ./make.bash

Write a simple demo (not simpler):

package main

func main() {
}

As a result, you will find that all schedule() function calls print goid = 0, sufficiently proving that the scheduling of the Go scheduler is done by g0 (if you think it's still inconclusive, you can write a more complex demo):

$ GODEBUG=schedtrace=1000 igo run demo1.go
schedule(): goid = 0
schedule(): goid = 0
SCHED 0ms: gomaxprocs=8 idleprocs=6 threads=4 spinningthreads=1 idlethreads=0 runqueue=0 [0 0 0 0 0 0 0 0]
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
schedule(): goid = 0
// Eliminate hundreds of lines

Enlightenment is more important than conclusion. I hope that when you learn the Go Scheduler, you can explore and research more by yourself, not just by reading other people's articles.

Reference material

Installing Go from source

If this article is helpful to you, please give me a compliment / like it. Thank you.
The author of this paper: Tai bin
If you like this article, reproduce it at will, but please keep the link to the original text: http://lessisbetter.site/2019...

Topics: Go git github Linux network

Programmer Think