1. Goroutine synchronization [data synchronization]
-
Why goroutine synchronization is needed
-
The concept of gorotine synchronization and several ways of synchronization
1.1 why goroutine synchronization is needed
package main
import (
"fmt"
"sync"
)
var A = 10
var wg = sync.WaitGroup{}
func Add(){
defer wg.Done()
for i:=0;i<1000000;i++{
A += 1
}
}
func main() {
wg.Add(2)
go Add()
go Add()
wg.Wait()
fmt.Println(A)
}
# output: 1061865 # Every time we run it, the result is different, but it's not the result we expected 2000000.
Multi goroutine [multitask], with shared resources, and multi goroutine modifies shared resources, resulting in data insecurity [data error], ensuring data security and consistency, and requiring goroutine synchronization
1.2 goroutine synchronization
goroutine executes according to the agreed order to solve the problem of data insecurity.
1.3 goroutine synchronization mode
-
channel [csp model]
-
Mutex [traditional synchronization mechanism]
-
Read write lock [traditional synchronization mechanism]
-
Conditional variable [traditional synchronization mechanism]
2. Traditional synchronization mechanism
2.1 mutually exclusive lock
2.1.1 characteristics
If the lock is successful, the resource will be operated. If the lock is unsuccessful, the resource will wait until the lock is successful. All goroutine s are mutually exclusive. If one gets the lock, all the others will wait
It solves the problem of data security, reduces the performance of the program, and is suitable for scenarios where reading and writing are not frequent
2.1.2 lock particle size
Granularity refers to the range of locking, where to use resources and where to lock, and minimize the locking range
Basic use process of unit test
-
New unit test file
-
Write test cases
-
The gotest run generates the corresponding prof file
-
go tool to view the generated prof file
package main_test
import (
"fmt"
"sync"
"testing"
)
var A = 10
var wg = sync.WaitGroup{}
var mux sync.Mutex
func Add(){
defer wg.Done()
for i:=0;i<1000000;i++{
mux.Lock()
A += 1
mux.Unlock()
}
}
/*
// Increase lock particle size
func Add(){
defer wg.Done()
mux.Lock()
for i:=0;i<1000000;i++{
A += 1
}
mux.Unlock()
}*/
// Unit test format,
func TestMux(t *testing.T) {
wg.Add(2)
go Add()
go Add()
wg.Wait()
fmt.Println(A)
}
# To generate a profile, the - cpuprofile parameter specifies the type of profile to be generated cpu.profile specifies the name of the profile to be generated
go test mutex_test.go -cpuprofile cpu.prof
# View the generated prof file, pprof specifies the file type to view
go tool pprof cpu.prof
# Here is the output
Type: cpu
Time: Jul 10, 2019 at 2:38pm (CST)
Duration: 201.43ms, Total samples = 80ms (39.72%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top # Here, use the top command to view the cpu usage information in the test
Showing nodes accounting for 80ms, 100% of 80ms total
flat flat% sum% cum cum%
60ms 75.00% 75.00% 60ms 75.00% sync.(*Mutex).Unlock
20ms 25.00% 100% 20ms 25.00% sync.(*Mutex).Lock
0 0% 100% 80ms 100% command-line-arguments_test.Add
(pprof) svg #svg saves the visualization file, which can be visualized by browser
(pprof) list Add # View detailed time consumption information of corresponding function
Be careful:
The current test case is a programming error (this kind of fast computing, a goroutine is already competent, more often read-write separation, mutex lock is not suitable for this frequent read-write scenario), not a lock error
2.1.3 sync.once source reading
// Once is an object that will perform exactly one action.
type Once struct {
m Mutex
done uint32 // Identifies whether the task has been executed. If it is set to 1, the task has been executed
}
// DO calls the method executed by the user, only once
func (o *Once) Do(f func()) {
// Atomic operation judge done, has been set to 1. If done is 1, it means the method has been executed, and directly returns
if atomic.LoadUint32(&o.done) == 1 {
return
}
// Lock up
o.m.Lock()
defer o.m.Unlock()
// If done is 0, the user function method will be called
if o.done == 0 {
defer atomic.StoreUint32(&o.done, 1)
f()
}
}
2.2 read write lock
Reading and writing are mutually exclusive, and readers can lock repeatedly. Write lock needs to wait for all readers to unlock. During write lock, all readers wait
It is suitable for scenarios with less writing and more reading. Compared with mutex, it can improve program performance to a certain extent
-
Only readers
- Write lock update data
2.3 conditional variables
The function of condition variable does not guarantee that only one co program (thread) accesses a shared data resource at the same time, but notifies the co program (thread) blocking on a condition when the state of the corresponding shared data changes. The conditional variable is not a lock, and cannot achieve the goal of synchronization in concurrency. Therefore, the conditional variable is always used together with the lock. It can be considered that the conditional variable is a supplement to the lock, which to some extent improves the efficiency of the lock mechanism
2.3.1 introduction to conditional variable API
-
Create condition variable [cannot be copied after creation]
// Parameter to pass a lock, return pointer type
cond:=sync.NewCond(&sync.Mutex{})
Cond.Wait(), block and release the cup resource on the re condition variable
// Blocking on the condition variable will mount the current gorodine to the Cond queue cond.Wait() // 1. Release the lock, Mount yourself to the notification queue, block and wait for [atomic operation] // 2. Receive wake-up signal and try to acquire lock // 3. Return if the lock is acquired successfully
Cond.Signal() randomly wakes up a goroutine blocked on a conditional variable
// Wake up the goroutine blocked on the condition variable in the state of wait [calling cond.wait] // Randomly wakes a thread on the notification queue and removes it from the notification queue cond.Signal() // Send wake-up signal
Cond.Broadcast() broadcasts all goroutine s in the wait state
// Broadcast all goroutine s in wait state // Notify all gorotines on the notification queue and remove all gorotines from the notification queue cond.Broadcast()
2.3.2 use of conditional variables in the producer consumption model
-
Potential bug -- > deadlock
package main
import "fmt"
import "sync"
import "math/rand"
import "time"
var cond sync.Cond // Create global condition variable
// Producer
func producer(out chan<- int, idx int) {
for {
cond.L.Lock() // Condition variable corresponding mutex lock
for len(out) == 3 { // Product area full waiting for consumer consumption
cond.Wait() // Suspend the current process, wait for the condition variable to be met, and wake up by the consumer
}
num := rand.Intn(1000) // Generate a random number
out <- num // Write to channel (production)
fmt.Printf("%dth Producers, generating data %3d, Public area surplus%d Data\n", idx, num, len(out))
cond.L.Unlock() // Release the mutually exclusive lock after production
cond.Signal() // Wake up blocked consumers
time.Sleep(time.Second) // Rest for a while after production, give other cooperation execution opportunities, and solve the reduction of deadlock opportunities
}
}
//Consumer
func consumer(in <-chan int, idx int) {
for {
cond.L.Lock() // The condition variable corresponds to the mutex lock (the same as the producer)
for len(in) == 0 { // Product area is empty waiting for production by producers
cond.Wait() // Suspend the current process, wait for the condition variable to be met, and wake up by the producer
}
num := <-in // Read (consume) the data in the channel
fmt.Printf("---- %dth Consumer, Consumption data %3d,Public area surplus%d Data\n", idx, num, len(in))
cond.L.Unlock() // End of consumption, unlock mutually exclusive lock
cond.Signal() // Wake up blocked producers
time.Sleep(time.Millisecond * 500) //Take a rest after consumption, give other cooperation execution opportunities, and solve the reduction of deadlock opportunities
}
}
func main() {
rand.Seed(time.Now().UnixNano()) // Set random number seed
quit := make(chan bool) // Create a channel to end communication
product := make(chan int, 3) // channel simulation for product area (public area)
cond.L = new(sync.Mutex) // Creating mutexes and conditional variables
for i := 0; i < 5; i++ { // 5 consumers
go producer(product, i+1)
}
for i := 0; i < 3; i++ { // 3 Producers
go consumer(product, i+1)
}
<-quit // Main process blocking does not end
}
-
An analysis of the causes of deadlock
-
Extreme processing: 1 producer 2 consumes channle cache 1
-
In extreme cases, all producers and consumers will enter a wait state, and no one will wake up
-
-
Solve the bug ---- one way wake up, the producer wakes up the consumer
Wake up direction problem: wake up from the party with low speed and the party with high speed
package main
import (
"fmt"
"runtime"
)
import "sync"
import "math/rand"
import "time"
var cond sync.Cond // Create global condition variable
// Producer
func producer(out chan<- int, idx int) {
for {
num := rand.Intn(1000) // Generate a random number
cond.L.Lock() // Condition variable corresponding mutex lock
select {
// Attempt to write data to channel
case out <- num:
fmt.Printf("%dth Producers, generating data %3d, Public area surplus%d Data\n", idx, num, len(out))
default:
}
cond.L.Unlock() // Release the mutually exclusive lock after production
cond.Signal() // Wake up blocked consumers
runtime.Gosched() // Give no more chance to create locks
}
}
//Consumer
func consumer(in <-chan int, idx int) {
var num int
for {
cond.L.Lock() // The condition variable corresponds to the mutex lock (the same as the producer)
for len(in)==0{
cond.Wait()
}
num=<-in
fmt.Printf("%dth Consumer, consumer %d, Public area surplus%d Data\n", idx, num, len(in))
cond.L.Unlock() // End of consumption, unlock mutually exclusive lock
}
}
func main() {
rand.Seed(time.Now().UnixNano()) // Set random number seed
quit := make(chan bool) // Create a channel to end communication
product := make(chan int, 3) // channel simulation for product area (public area)
cond.L = new(sync.Mutex) // Creating mutexes and conditional variables
for i := 0; i < 3; i++ { // 3 Producers
go producer(product, i+1)
}
for i := 0; i < 5; i++ { // 5 consumers
go consumer(product, i+1)
}
<-quit // Main process blocking does not end
}
Question:
When we cancel the conditional variable and use the channel with cache, we can also complete the producer and consumer model well?
2.3.3 channel vs sync.Cond
Using channel to notify goroutine problems with multiple conditions of concern?
The function of off channle and broadcast is only used once
When the state is multiple, the channel fails. Use cond broadcast to update the state