1.goroutine
1.1 process and thread introduction
● process is an execution process of a program in the operating system and the basic unit for resource allocation and scheduling of the system
● thread is an execution instance of a process and the smallest unit of program execution. It is a smaller basic unit that can run independently than a process
● a process can create and destroy multiple threads, and multiple threads in the same process can execute concurrently
● a program has at least one process, and a process has at least one thread
1.2 parallelism and concurrency
● multithreaded programs run on a single core, that is, concurrency
● multithreaded programs run on multiple cores, which is parallel
● Concurrency: because it is on a cpu, for example, there are 10 threads, and each thread performs 10 milliseconds (polling operation). From the perspective of people, it seems that these 10 threads are running, but from the micro point of view, at a certain point in time, there is actually only one thread executing, which is concurrency.
● parallelism: because it is on multiple CPUs (for example, there are 10 CPUs), for example, there are 10 threads, and each thread executes for 10 milliseconds (each executing on different CPUs). From the perspective of people, these 10 threads are running, but from the microscopic point of view, at a certain time point, there are also 10 threads executing at the same time, which is called parallelism
1.3 Go coroutine and Go main thread
● Go Main thread (directly called thread by programmers)/Process): a Go On a thread, there can be multiple coprocesses. It can be understood that a coprocess is a lightweight thread [optimized by the compiler] ● Go Characteristics of collaborative process ○ Independent stack space ○ Shared program heap space ○ Scheduling is controlled by users ○ A coroutine is a lightweight thread
1.4 goroutine quick start
(1) Case description
Write a program to complete the following functions:
In the main thread (which can be understood as a process), start a goroutine, which outputs "hello, world" every 1 second
After the main thread also outputs "hello, golang" every other second for 10 times, exit the program
The main thread and goroutine are required to execute simultaneously
(2) Code implementation
package main
import (
"fmt"
"strconv"
"time"
)
func test() {
for i := 1; i <= 10; i++ {
fmt.Println("test() hello,world" + strconv.Itoa(i))
time.Sleep(time.Second)
}
}
func main() {
go test() //Start a collaborative process
for i := 1; i <= 10; i++ {
fmt.Println("main() hello,golang" + strconv.Itoa(i))
time.Sleep(time.Second)
}
}
(3) Main thread and co process execution flow chart
(4) Quick start summary
● the main thread acts directly on the cpu as a physical thread. It is heavyweight and consumes cpu resources very much
● the coroutine starts from the main thread, which is a lightweight thread and a logical state. The consumption of resources is relatively small
● golang's collaborative process mechanism is an important feature, which can easily open tens of thousands of collaborative processes. The concurrency mechanism of other programming languages is generally based on threads. Opening too many threads leads to high resource consumption. Here, the advantage of golang in concurrency is highlighted
1.5 goroutine scheduling model
(1) Basic introduction of MPG mode
M: Main thread of operating system (physical thread)
P: Context required for collaborative process execution
G: Synergetic process
(2) MPG mode operation status 1
● there are three M's in the current program. If the three M's are running on one cpu, they are concurrent. If they are running on different CPUs, they are parallel
● M1, M2 and M3 are executing a G. there are three coordination queues of M1, three coordination queues of M2 and two coordination queues of M3
● as can be seen from the above figure: Go's coroutine is a lightweight thread in logical state. Go can easily start tens of thousands of coroutines.
● multithreading of other programs c/java is often in kernel mode, which is relatively heavyweight. Thousands of threads may consume cpu
(3) MPG mode operation status 2
● in two parts
● the original situation is that the MO main thread is executing the G0 process, and there are three other processes waiting in the queue
● if the GO process is blocked, such as reading files or databases, then the M1 main thread will be created (or M1 may be taken out from the existing thread pool), and the three waiting processes will be hung under M1 to start execution, and the GO under the MO main thread will still execute the reading and writing of files io.
● such MPG scheduling mode can not only allow GO to execute, but also prevent other processes in the queue from blocking all the time. It can still be executed concurrently / in parallel.
● when G0 is not blocked, MO will be put into the idle main thread to continue execution (from the existing thread pool), and G0 will be awakened at the same time.
1.6 set the number of CPUs that Golang runs
package main
import (
"fmt"
"runtime"
)
func main() {
//Getting the current cpu count returns the logical cpu count of the local machine
num := runtime.NumCPU()
//Set the number of CPUs to run the go program
runtime.GOMAXPROCS(num - 1)
fmt.Println("num=", num)
}
● go1. After 8, the program is allowed to run on multiple cores by default, which can be set without setting
● go1. Before 8, it needs to be set
2.channel
2.1 requirements
Requirements: calculate the factorial of each number from 1 to 200, put the factorial of each number into the map, and finally display it. It needs to be completed with goroutine
2.2 preliminary implementation, but there are problems
package main
import "fmt"
var myMap = make(map[int]int, 10)
//The test function is used to calculate the factorial and put the result into the map
func test(n int) {
res := 1
for i := 1; i <= n; i++ {
res *= i
}
myMap[n] = res
}
func main() {
//Open multiple collaborative processes to complete this task [200]
for i := 1; i <= 200; i++ {
go test(i)
}
//Sleep for 10 seconds
time.Sleep(time.Second * 10)
//Output results
for i, v := range myMap {
fmt.Printf("map[%d]=%d\n", i, v)
}
}
Run directly and report an error because of the concurrent write to the map
How to determine whether there is a resource competition problem in the program? When compiling the program, add a parameter - race
go build -race main.go
If there is a resource competition problem, it will be printed out
If you do not sleep, the program will not run completely
2.3 how to communicate between different goroutine s
● mutex of global variables
● use channel to solve the problem
2.4 improved program of locking synchronization using global variables
● because the global variable myMap is not locked, the problem of resource contention will occur, and the code will give an error, prompting fatal error: concurrent map writes error
● solution: add mutex lock
package main
import (
"fmt"
"sync"
)
var (
myMap = make(map[int]int, 10)
//Declare a global mutex
//lock is a global mutex
//sync is a package
//Mutex is mutually exclusive
lock sync.Mutex
)
//The test function is used to calculate the factorial and put the result into the map
func test(n int) {
res := 1
for i := 1; i <= n; i++ {
res *= i
}
//Lock
lock.Lock()
myMap[n] = res
//Unlock
lock.Unlock()
}
func main() {
//Open multiple collaborative processes to complete this task [200]
for i := 1; i <= 200; i++ {
go test(i)
}
//Sleep for 10 seconds
time.Sleep(time.Second * 10)
//Output results
//Lock
lock.Lock()
for i, v := range myMap {
fmt.Printf("map[%d]=%d\n", i, v)
}
//Unlock
lock.Unlock()
}
No error reported
Why should the output result be locked?
Because the main process does not know when the coordination process will be completed, resource contention may still occur at the bottom
The result of map[n] is 0 because the result is outside the int range
2.5 why channel is needed
● previously, global variable lock synchronization was used to solve goroutine communication, but it is not perfect
● it is difficult to determine the time when the main thread is waiting for all goroutine s to be loaded. We set 10 seconds here, which is just an estimate
● if the main thread sleeps for a long time, the waiting time will be increased. If the waiting time is short, there may be goroutine in working state, which will also be destroyed with the exit of the main thread
● communication is realized through locking and synchronization of global variables, which is not conducive to the reading and writing of global variables by multiple coprocessors
2.6 basic introduction to channel
● the essence of channel is a data structure - queue
● data is first in first out
● thread safe. When multiple goroutine s are accessed, they do not need to be locked. The channel itself is thread safe
● channels have types. A string channel can only store data of string type
2.7 definition / declaration of channel
var Variable name chan data structure give an example: var intChan chan int var mapChan chan map[int]string var perChan chan Person var perChan2 chan *Person explain: channel Is a reference type channel Initialization is required to write data, i.e make Before use There are types of pipes, intChan Only integers can be written int
2.8 pipeline initialization
package main
import "fmt"
func main() {
//1. Create a pipe that can store three int types
var intChan chan int
intChan = make(chan int, 3)
//2. What is intchan
fmt.Printf("intChan The value of is=%v\n", intChan) //The value of intChan is = 0xc00000d4080
//3. Write data to the pipeline
intChan <- 19
intChan <- 12
intChan <- 23
//Intchan < - 50 / / an error is reported. When we write data to the pipeline, we cannot exceed its capacity
//4. Pipe length and capacity
fmt.Printf("channel len=%v cap=%v\n", len(intChan), cap(intChan)) //channel len=3 cap=3
//5. Read data from the pipeline
var num int
num = <-intChan
fmt.Println("num=", num) //num= 19
fmt.Printf("channel len=%v cap=%v\n", len(intChan), cap(intChan)) //channel len=2 cap=3
//6. If we take out all the pipeline data without using the cooperation process, we will report an error deadlock if we take it out again
num2 := <-intChan
num3 := <-intChan
//Num4: = < - intchan / / error reported
fmt.Println(num2, num3) //12 23
}
matters needing attention:
● only the specified data type can be stored in the channel
● when the channel data is full, it cannot be put again
● if the data is taken out from the channel, it can be played again
● if the channel data is retrieved without using the cooperation process, it will report dead lock
2.9 reading and writing channel cases
be careful:
package main
import "fmt"
type Cat struct {
Name string
Age int
}
func main() {
var allChan chan interface{}
allChan = make(chan interface{}, 10)
cat1 := Cat{Name: "tom", Age: 10}
cat2 := Cat{Name: "tom~~", Age: 20}
allChan <- cat1
allChan <- cat2
allChan <- 121
allChan <- "abc"
//read
cat111 := <-allChan
cat222 := <-allChan
v1 := <-allChan
v2 := <-allChan
fmt.Println(cat111, cat222, v1, v2) //{tom 10} {tom~~ 20} 121 abc output is OK
//fmt.Println("Cat Name:",cat111.Name) / / an error is reported when compiling Name undefined (type interface{} has no field or method
//If you want to get name, you must use assertion
a := cat111.(Cat)
fmt.Println("Cat Name:", a.Name) //Cat Name: tom
}
2.10 closing of channel
Use the built-in function close to close the channel. When the channel is closed, you can no longer write data to the channel, but you can still read data from the channel
package main
import "fmt"
func main() {
intChan := make(chan int, 3)
intChan <- 23
intChan <- 34
close(intChan)
//After closing, an error will be reported if writing
//intChan <- 45 //panic: send on closed channel
// Data can be read normally after shutdown
n1 := <-intChan
fmt.Println(n1) //23
}
2.11 channel traversal
channel supports for range traversal. There are two points to note
● during traversal, if the channel is not closed, the deadlock error will appear
● during traversal, if the channel has been closed, the data will be traversed normally. After traversal is completed, the traversal will exit
package main
import "fmt"
func main() {
intChan1 := make(chan int, 100)
for i := 1; i <= 100; i++ {
intChan1 <- i * 2
}
//Traversing a pipeline cannot use a normal for loop because the length changes all the time
//During traversal, if the channel is not closed, a deadlock error will appear
// After printing the data, report fatal error: all goroutines are asleep - deadlock!
//During traversal, if the channel is closed, the data will be traversed normally. After traversal, the traversal will exit
close(intChan1)
for v := range intChan1 {
fmt.Println("v=", v)
}
}
2.12 application examples
Use goroutine and channel to work together to complete the case
requirement:
● start a writeData coroutine and write 50 integers into the pipeline intChan
● start a readData process to read the data written by writeData into the pipeline intChan
● note: writeData and readData operate on the same pipe
● the main thread needs to wait for both writeData and readData to complete before exiting
Train of thought analysis:
package main
import "fmt"
//writeData
func writeData(intChan chan int) {
for i := 1; i <= 50; i++ {
intChan <- i
fmt.Println("writeData:", i)
}
close(intChan)
}
//readData
func readData(intChan chan int, exitChan chan bool) {
for {
v, ok := <-intChan
if !ok {
break
}
fmt.Println("readData The data read is:", v)
}
//After the data reading is completed, the task is completed
exitChan <- true
close(exitChan)
}
func main() {
//Create 2 pipes
intChan := make(chan int, 10)
exitChan := make(chan bool, 1)
go writeData(intChan)
go readData(intChan, exitChan)
for {
_, ok := <-exitChan
if !ok {
break
}
}
}
2.13 blocking
If the read data of the above code is annotated with go ReadData (int Chan, exitchan), blocking will occur
Operation results:
writeData: 1
writeData: 2
writeData: 3
writeData: 4
writeData: 5
writeData: 6
writeData: 7
writeData: 8
writeData: 9
writeData: 10
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
C:/Project/GolangProject/src/go_code/Project01/chapter16/demo6/main.go:37 +0x92
goroutine 19 [chan send]:
main.writeData(0x0)
C:/Project/GolangProject/src/go_code/Project01/chapter16/demo6/main.go:8 +0x3b
created by main.main
C:/Project/GolangProject/src/go_code/Project01/chapter16/demo6/main.go:33 +0x86
exit status 2
If the program runs and finds that a pipeline has only write but no read, the pipeline will be blocked. It doesn't matter if the frequency of the write pipeline and the read pipeline are inconsistent
2.14 application goroutine and channel
Demand: in statistics 1-8000, which are prime numbers?
Idea:
● traditional: use a cycle, cycle judgment
● use concurrency / parallelism: assign the task of counting prime numbers to multiple goroutine s to complete, and the task completion time is short
package main
import (
"fmt"
)
//Write 1-8000 to pipe
func putNum(intChan chan int) {
for i := 1; i <= 8000; i++ {
intChan <- i
}
close(intChan)
}
//Find the prime number. If so, put it into the primeChan pipeline
func primeNum(intChan chan int, primeChan chan int, exitChan chan bool) {
var flag bool
for {
num, ok := <-intChan
if !ok { //I can't get it
break
}
flag = true
//Judge whether num is a prime number
for i := 2; i < num; i++ {
if num%i == 0 { //Description overview is not prime
flag = false
break
}
}
if flag {
primeChan <- num
}
}
fmt.Println("There is one primeNu The process exits because it cannot get data!!!")
exitChan <- true
}
func main() {
intChan := make(chan int, 1000)
primeChan := make(chan int, 2000)
exitChan := make(chan bool, 4)
start := time.Now().Unix()
//Start a collaborative process and put 1-80000 into intChan
go putNum(intChan)
//Open four co processes, take out data from intChan, and judge whether it is a prime number. If so, put it into primeChan
for i := 0; i < 4; i++ {
go primeNum(intChan, primeChan, exitChan)
}
//Main thread direct processing
go func() {
for i := 0; i < 4; i++ {
<-exitChan
}
end := time.Now().Unix()
fmt.Println("Time consuming to use collaborative process=", end - start)
//When we take four results from exitChan, we can safely close primeChan
close(primeChan)
}()
//Traverse primeChan and get the result
for {
res, ok := <-primeChan
if !ok {
break
}
fmt.Println("prime number=", res)
}
fmt.Println("main()Thread exit")
}
After using 4 co processes, the execution speed is at least 4 times higher than that of the ordinary method
2.15 channel read-only or write only
package main
import "fmt"
func main() {
//Pipes can be declared read-only or write only
//1. By default, the pipeline is bidirectional
//Var channel int / / readable and writable
//Declare write only
var chan2 chan<- int
chan2 = make(chan int, 3)
chan2 <- 20
//Num: = < - chan2 / / invalid operation: cannot receive from send only channel channel2 (variable of type Chan < - int)
fmt.Println("chan2=", chan2)
//Declared read-only
var chan3 <-chan int
num2 := <-chan3
// Chan3 < - 10 / / error
fmt.Println("num2=", num2)
}
2.16 using select can solve the blocking problem of fetching data from the pipeline
package main
import (
"fmt"
"strconv"
)
func main() {
//1. Define 10 data int s for a pipeline
intChan := make(chan int, 10)
for i := 0; i < 10; i++ {
intChan <- i
}
//2. Define 5 data string s for a pipeline
stringChan := make(chan string, 5)
for i := 0; i < 5; i++ {
stringChan <- "hello" + strconv.Itoa(i)
}
//label
//When traversing the pipeline, the traditional method will block and cause deadlock if it is not closed
//In the actual development, it is difficult for us to determine when to close the pipeline, which can be solved by select ing
for {
select {
//If intChan has not been closed, it will not be blocked all the time, and deadlock will automatically match to the next case
case v := <-intChan:
fmt.Println("from intChan Read data:", v)
case v := <-stringChan:
fmt.Println("from stringChan Read data:", v)
default:
fmt.Println("Can't get data:")
return
//Or use the label break label
}
}
}
2.17 using recover in goroutine to solve panic problems
If a panic occurs in a collaboration, if the panic is not captured, the whole program will crash
At this time, we need to use revolver in the collaboration to capture panic for processing, so that even if there is a problem in the collaboration, the main thread will not be affected and can continue to execute
package main
import (
"fmt"
"time"
)
func sayHello() {
for i := 0; i < 10; i++ {
fmt.Println("hello,world", i)
}
}
func test() {
//Define a map and assign values without make
var myMap map[int]string
myMap[0] = "golang" //report errors
}
func main() {
go sayHello()
go test()
for i := 0; i < 10; i++ {
fmt.Println("main()ok", i)
time.Sleep(time.Second)
}
}
Do not handle the error, the program reports an error and stops
solve
package main
import (
"fmt"
"time"
)
func sayHello() {
for i := 0; i < 10; i++ {
fmt.Println("hello,world", i)
}
}
func test() {
defer func() {
//Catch the panic thrown by test
if err := recover(); err != nil {
fmt.Println("test()An error occurred", err)
}
}()
//Define a map and assign values without make
var myMap map[int]string
myMap[0] = "golang" //report errors
}
func main() {
go sayHello()
go test()
for i := 0; i < 10; i++ {
fmt.Println("main()ok", i)
time.Sleep(time.Second)
}
}
