channel overview
Hello, everyone! We meet again. In this article, we understand the channel and the underlying implementation of channel in Go language and some common interview questions.
channel is the first class type built in Go language, and it is also one of the distinctive features of Go language. Let's take a look at an application scenario. For example, during the execution of process a, sub processes A1 and A2 need to be created An, after creating the subprocess, collaboration a waits for the subprocess to exit. In this scenario, Go provides us with three solutions:
- Using channel to control subprocesses
- waitGroup semaphore mechanism control subprocess
- Context uses context to control subprocesses
The three solutions have their own advantages and disadvantages. For example, using channel to control sub processes has the advantages of simple implementation, but the disadvantage is that when a large number of sub processes need to be created, the same number of channels are required, so it is not convenient to control the sub processes that continue to derive.
First of all, think about why Go introduces channel and what kind of problems can channel provide us with? You can learn about the channel from the CSP model. The CSP model was published by Tony Hoare in 1978. CSP mainly speaks a concurrent programming language. CSP allows the use of process composition to describe the system. They run independently and communicate only through message passing. This paper has a great impact on the concurrent design of Go language by Go founder Rob Pike. Finally, the idea of CSP is realized by introducing a new type of channel.
When using channel type, you can use it without introducing a package. It is the built-in type of Go language. Unlike other libraries, you must introduce sync package and atomic package to use them.
Basic usage of channel
Channel many people often say the so-called channel, so the channel is also a similar channel in our life, which is used to transmit things. Computers can use channels to communicate. In Go language, it is common to allow data transmission between goroutines. In transmission, you need to clarify some rules. First, each channel is only allowed to exchange data of a specified type, also known as channel element type (similar to life, your own water pipe is only allowed to transport drinking water and gasoline, and you need to use another pipe). In the Go language, the chan keyword is used to declare a new channel, and the close() function is used to close the channel.
After defining the channel, you can send data to and receive data from the channel. You can also define three types: only accept, only send, or accept and send.
The format of channel type declaration is as follows:
var variable chan Element type
example
var ch1 chan int // Declare a channel that passes an integer var ch3 chan []int // Declare a channel that passes int slices
To create a channel:
var ch chan string fmt.Println(ch) //Output: < nil >
Note: the channel is a reference type, and the null value of the channel type is nil.
The declared channel can only be used after being initialized with the make function. The buffer size of the channel is also optional.
func main() {
//Initialize the channel with a buffer size of 2
ch := make(chan int,2)
ch <- 1
ch <- 2
ch <- 3 //An error will be reported because the buffer only allows a size of 2
x1 := <- ch
x2:= <- ch
fmt.Println(x1)
fmt.Println(x2)
}(1) Send data
Send a data to chan using CH < -, the format is as follows:
ch <- 1 //Send 1 to ch
(2) Receive data
To receive a piece of data from chan, use < - CH, and the received data is also a statement. The class hours are as follows:
x := <- ch //Receive only from ch and assign to variable x1. <- ch //Receive values from ch, ignoring results
Note: the thing to note about closing the channel is that the channel needs to be closed only when the receiver is notified that all data of goroutine has been sent. The channel can be recycled by the garbage collection mechanism. It is different from closing the file. Closing the file after the operation is necessary, but closing the channel is not necessary.
channel implementation principle
In this section, the interviewer will ask: what is the underlying implementation of channel?
Next, learn the data structure and initialization of channel together. There are three most important operation methods: send, recv and close. Carefully learn the implementation of the underlying principle of channel.
Source directory location: Runtime / chan Go, the data structure of chan type posted below is as follows:
type hchan struct {
qcount uint // total data in the queue
dataqsiz uint // size of the circular queue
buf unsafe.Pointer // points to an array of dataqsiz elements
elemsize uint16 //Element size in chan
closed uint32 //Has it been close d
elemtype *_type // Element type (chan element type)
sendx uint // send index (send is indexed in buf)
recvx uint // receive index (recv is indexed in buf)
recvq waitq // list of recv waiters
sendq waitq // list of send waiters
// lock protects all fields in hchan, as well as several
// fields in sudogs blocked on this channel.
//
// Do not change another G's status while holding this lock
// (in particular, do not ready a G), as this can deadlock
// with stack shrinking.
lock mutex //Mutexes protect all fields. The above comments have made it very clear
}(1) chan initialization
When compiling, Go will choose whether to call makechan64 or makechan according to the capacity. From the source code, we can know that makechan 64 only checks the size, and then the bottom layer finally calls makechan to implement it. (the goal of makechan is to generate hchan objects)
What did Makechan do? The source code is as follows:
func makechan(t *chantype, size int) *hchan {
elem := t.elem
//The compiler checks that the type is safe
// compiler checks this but be safe.
if elem.size >= 1<<16 {//Whether > = 2 ^ 16
throw("makechan: invalid channel element type")
}
if hchanSize%maxAlign != 0 || elem.align > maxAlign {
throw("makechan: bad alignment")
}
mem, overflow := math.MulUintptr(elem.size, uintptr(size))
if overflow || mem > maxAlloc-hchanSize || size < 0 {
panic(plainError("makechan: size out of range"))
}
var c *hchan
switch {
case mem == 0:
// If the size of chan or the size of the element is 0, you do not need to create a buf
c = (*hchan)(mallocgc(hchanSize, nil, true))
// The contention detector uses this location for synchronization
c.buf = c.raceaddr()
case elem.ptrdata == 0:
// Element is not a pointer. A contiguous block of memory is allocated to hchan data structure and buf
c = (*hchan)(mallocgc(hchanSize+mem, nil, true))
// The hchan data structure is followed by buf
c.buf = add(unsafe.Pointer(c), hchanSize)
default:
// Elements contain pointers.
c = new(hchan)
c.buf = mallocgc(mem, elem, true)
}
// Record the size, type and capacity of the element
c.elemsize = uint16(elem.size)
c.elemtype = elem
c.dataqsiz = uint(size)
lockInit(&c.lock, lockRankHchan)
if debugChan {
print("makechan: chan=", c, "; elemsize=", elem.size, "; dataqsiz=", size, "\n")
}
return c
}Summary: the bottom layer of channel allocates different objects according to different capacities and element types, initializes the fields of channel objects, and returns hchan objects.
(2) send method
send() is to send data to chan. The method is roughly divided into six parts. The source code is as follows:
Part I:
func chansend1(c *hchan, elem unsafe.Pointer) {
chansend(c, elem, true, getcallerpc())
}
func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
//Part I
if c == nil {
if !block {
return false
}
//Blocking sleep
gopark(nil, nil, waitReasonChanSendNilChan, traceEvGoStop, 2)
throw("unreachable")
}
// Part of the code is omitted...
}At the beginning of send, the first thing is to judge. If chan is nil, call gopark to block sleep. At this time, the caller will block forever. Then the code throw("unreachable") will not be executed.
Part II:
// In the second part, if chan is not close d and chan is full, it returns directly
if !block && c.closed == 0 && full(c) {
return false
}Part III:
// The third part is the scenario that chan has been close d
lock(&c.lock)
if c.closed != 0 {
unlock(&c.lock)
panic(plainError("send on closed channel"))
}In the third part, if chan has been close d and you send data to it, panic will appear. Panic will appear in the following code.
ch := make(chan int,1) close(ch) ch <- 1
Part IV:
//Part 4: if there is a recvq receiver, it means that there is no data in the buf, so it is directly sent from the sender to the receiver
if sg := c.recvq.dequeue(); sg != nil {
// Found a waiting receiver. We pass the value we want to send
// directly to the receiver, bypassing the channel buffer (if any).
send(c, sg, ep, func() { unlock(&c.lock) }, 3)
return true
}Part V:
// Part five, buf is not full
if c.qcount < c.dataqsiz {
// Space is available in the channel buffer. Enqueue the element to send.
qp := chanbuf(c, c.sendx)
if raceenabled {
racenotify(c, c.sendx, nil)
}
typedmemmove(c.elemtype, qp, ep)
c.sendx++
if c.sendx == c.dataqsiz {
c.sendx = 0
}
c.qcount++
unlock(&c.lock)
return true
}The fifth part explains that there is no receiver at present. You need to put the data into the buf. After putting it, it will be returned successfully.
Part VI:
// Part VI: buf full
//Chansend1 will not enter the if block because the block of chansend1 = true
if !block {
unlock(&c.lock)
return false
}The sixth part deals with the situation of buf full. If the buf is full, the sender's goroutine will be added to the sender's waiting queue until it is awakened. At this time, the data is either taken away or chan is close d.
(2)recv
When processing data received from chan, the source code is as follows:
Part I:
if c == nil { //Judge chan as nil
if !block {
return
}
gopark(nil, nil, waitReasonChanReceiveNilChan, traceEvGoStop, 2)
throw("unreachable")
}When getting data from chan, if chan is nil, the caller will be blocked forever.
Part II:
// In the second part, block=false and c is empty
if !block && empty(c) {
......
}Part III:
lock(&c.lock)//Lock and release the lock on return
//In the third part, c has been close d and chan is empty empty
if c.closed != 0 && c.qcount == 0 {
if raceenabled {
raceacquire(c.raceaddr())
}
unlock(&c.lock)
if ep != nil {
typedmemclr(c.elemtype, ep)
}
return true, false
}If chan has been close d and there are no cached elements in the queue, true and false are returned.
Part IV:
// In the fourth part, if there are sender s waiting to be sent in the sendq queue
if sg := c.sendq.dequeue(); sg != nil {
// Found a waiting sender. If buffer is size 0, receive value
// directly from sender. Otherwise, receive from head of queue
// and add sender's value to the tail of the queue (both map to
// the same buffer slot because the queue is full).
recv(c, sg, ep, func() { unlock(&c.lock) }, 3)
return true, true
}When the processing buf is full. At this time, if it is an unbuffer ed chan, directly copy the sender's data to the receiver. Otherwise, read a value from the queue head and add the sender's value to the end of the queue.
Part 5: deal with the sender without waiting. It shares a large lock with chansend, so there will be no concurrency problem. If buf has an element, it takes out an element to the receiver.
Part 6: deal with the case where there are no elements in buf. If there is no element, the current receiver will be blocked until it receives data from the sender or chan is close d.
(3)close
Through the close function, you can close chan, and the bottom layer calls the closechan method to execute. The specific source code is the same as the above two locations.
(4) Using channel to step on the pit
Common errors panic and goroutine leaks
Example 1:
ch := make(chan int,1) close(ch) ch <- 1
Add to chan, but when it is closed, Panic will appear. The solution is not to close.
Example 2:
ch := make(chan int,1)
ch <- 1
close(ch)
<- ch
close(ch)Take out the data from chan, but if it is close d, it will also Panic
(5) Introduce panic and recover
Panic and recover are also interview points. Simple attention
Panic: in Go language, panic represents a serious problem, which means that the program ends and exits. In Go, the panic keyword is used to throw an exception. Similar to throw in Java.
Recover: in Go language, it is used to recover the program state from serious errors to normal state. When Panic occurs, you need to use recover to capture. If you don't capture, the program will exit. Java like try catch catches exceptions.
Every time you like + collect + pay attention, it is my greatest motivation to create. Always on the road!