Defer is also a special key word in Go, which is mainly used to ensure that in the process of program execution, the function behind defer will be executed, generally used to close the connection, clean up resources and so on.
1. Overview of structure
1.1. defer
type _defer struct { siz int32 // Size of parameters started bool // Has it been implemented? sp uintptr // sp at time of defer pc uintptr fn *funcval _panic *_panic // panic in defer link *_defer // Defer list, defer in the function execution process, is concatenated by link }
1.2. panic
type _panic struct { argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink arg interface{} // argument to panic link *_panic // link to earlier panic recovered bool // whether this panic is over aborted bool // the panic was aborted }
1.3. g
Because defer panic is bound to running g, here are the attributes associated with defer panic in G
type g struct { _panic *_panic // Link List Composed of panic _defer *_defer // defer consists of a list of first-in-last-out links, on the same stack }
2. Source code analysis
2.1. main
At first, let's go tool to analyze what functions are used to implement the bottom level.
func main() { defer func() { recover() }() panic("error") }
go build -gcflags=all="-N -l" main.go
go tool objdump -s "main.main" main
▶ go tool objdump -s "main\.main" main | grep CALL main.go:4 0x4548d0 e81b00fdff CALL runtime.deferproc(SB) main.go:7 0x4548f2 e8b90cfdff CALL runtime.gopanic(SB) main.go:4 0x4548fa e88108fdff CALL runtime.deferreturn(SB) main.go:3 0x454909 e85282ffff CALL runtime.morestack_noctxt(SB) main.go:5 0x4549a6 e8d511fdff CALL runtime.gorecover(SB) main.go:4 0x4549b5 e8a681ffff CALL runtime.morestack_noctxt(SB)
From the decompilation results, we can see that the defer keyword first calls runtime.deferproc to define a delayed invocation object, and then calls runtime.deferreturn to complete the invocation of defer-defined functions before the end of the function.
The panic function calls runtime.gopanic to implement the relevant logic
Recovery calls runtime. gorecovery to implement recovery
2.2. deferproc
According to the function fn defined after the defer keyword and the size of the parameter, a delayed execution function is created, and the delayed function is hung on the linked list of the current g_defer.
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn sp := getcallersp() argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn) callerpc := getcallerpc() // Get a _defer object and put it in the head of the g._defer list d := newdefer(siz) // Set defer's fn pc sp, etc., and call later d.fn = fn d.pc = callerpc d.sp = sp switch siz { case 0: // Do nothing. case sys.PtrSize: // _ Memory behind defer stores the address information of argp *(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp)) default: // If it is not a pointer type parameter, copy the parameter to the memory space behind _defer memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz)) } return0() }
This function looks fairly straightforward. Get an object of _defer through newproc and add it to the head of the current g's _defer list. Then copy the pointer of the parameter or parameter to the memory space behind the acquired _defer object.
2.2.1. newdefer
The function of newdefer is to get a _defer object and push it into the head of the g._defer list.
func newdefer(siz int32) *_defer { var d *_defer // Judging the sizeclass to be allocated by deferclass is similar to determining several sizeclasses in advance according to memory allocation, and then determining sizeclass according to size to find the corresponding cached memory block. sc := deferclass(uintptr(siz)) gp := getg() // If the sizeclass is within the given sizeclass range, go to p bound by g. if sc < uintptr(len(p{}.deferpool)) { pp := gp.m.p.ptr() if len(pp.deferpool[sc]) == 0 && sched.deferpool[sc] != nil { // The number of caches in the current sizeclass = 0, not nil, gets a batch of caches from scheme systemstack(func() { lock(&sched.deferlock) for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil { d := sched.deferpool[sc] sched.deferpool[sc] = d.link d.link = nil pp.deferpool[sc] = append(pp.deferpool[sc], d) } unlock(&sched.deferlock) }) } // If the cache corresponding to sizeclass is not empty after being retrieved from scheme, allocation if n := len(pp.deferpool[sc]); n > 0 { d = pp.deferpool[sc][n-1] pp.deferpool[sc][n-1] = nil pp.deferpool[sc] = pp.deferpool[sc][:n-1] } } // p and sched do not find or have no corresponding sizeclass, allocate directly if d == nil { // Allocate new defer+args. systemstack(func() { total := roundupsize(totaldefersize(uintptr(siz))) d = (*_defer)(mallocgc(total, deferType, true)) }) } d.siz = siz // Insert into the list header of g._defer d.link = gp._defer gp._defer = d return d }
The idea of memory allocation is to acquire sizeclass according to size and classify and cache sizeclass according to size.
First allocate on p, then get local caches in batches from global scheme. This idea of secondary caching is really pervasive in all parts of go source code.
2.3. deferreturn
func deferreturn(arg0 uintptr) { gp := getg() // Get the first defer of the g defer list and the last declared defer d := gp._defer // Without defer, you don't need to do anything. if d == nil { return } sp := getcallersp() // If the sp of defer does not match callersp, it indicates that defer does not correspond. It may be a delay function that calls other stack frames. if d.sp != sp { return } // According to d.siz, the original stored parameter information is acquired and stored in arg0 switch d.siz { case 0: // Do nothing. case sys.PtrSize: *(*uintptr)(unsafe.Pointer(&arg0)) = *(*uintptr)(deferArgs(d)) default: memmove(unsafe.Pointer(&arg0), deferArgs(d), uintptr(d.siz)) } fn := d.fn d.fn = nil // defer is released when it's used. gp._defer = d.link freedefer(d) // Jump to execution defer jmpdefer(fn, uintptr(unsafe.Pointer(&arg0))) }
2.3.1.freedefer
The idea of releasing defer functions should be the same as that of scheduler and memory allocation.
func freedefer(d *_defer) { // sizeclass for judging defer sc := deferclass(uintptr(d.siz)) // Beyond the established sizeclass, it's directly allocated memory, and that's all. if sc >= uintptr(len(p{}.deferpool)) { return } pp := getg().m.p.ptr() // The buffer corresponding to p local sizeclass is full and half of the batch transfer to global sched if len(pp.deferpool[sc]) == cap(pp.deferpool[sc]) { // Use g0 to transfer systemstack(func() { var first, last *_defer for len(pp.deferpool[sc]) > cap(pp.deferpool[sc])/2 { n := len(pp.deferpool[sc]) d := pp.deferpool[sc][n-1] pp.deferpool[sc][n-1] = nil pp.deferpool[sc] = pp.deferpool[sc][:n-1] // First string the defer objects that need to be transferred into a linked list if first == nil { first = d } else { last.link = d } last = d } lock(&sched.deferlock) // Put this list in the header of sched.deferpool corresponding to sizeclass last.link = sched.deferpool[sc] sched.deferpool[sc] = first unlock(&sched.deferlock) }) } // Clear the attributes of defer currently being released d.siz = 0 d.started = false d.sp = 0 d.pc = 0 d.link = nil pp.deferpool[sc] = append(pp.deferpool[sc], d) }
The idea of secondary caching is Deeply Understanding the Implementation of Go-goroutine and Scheduler Analysis， Deeply Understanding the Principles of go-channel and select， Deep Understanding of Go-Garbage Recycling Mechanism Having analyzed it, we will not analyze it too much.
2.4. gopanic
func gopanic(e interface{}) { gp := getg() var p _panic p.arg = e p.link = gp._panic gp._panic = (*_panic)(noescape(unsafe.Pointer(&p))) atomic.Xadd(&runningPanicDefers, 1) // Defer objects that execute the g._defer linked list in turn for { d := gp._defer if d == nil { break } // If defer was started by earlier panic or Goexit (and, since we're back here, that triggered a new panic), // take defer off list. The earlier panic or Goexit will not continue running. // Normally, defer is removed after execution. Since the defer is not removed, there are only two reasons: 1. panic 2 is triggered in this defer. runtime.Goexit is triggered in this defer, but this defer has been executed and needs to be removed. If the defer is not removed, it is the first one. The reason is that the panic also needs to be removed, because the panic has been executed. Here, add a flag to the panic for subsequent removal. if d.started { if d._panic != nil { d._panic.aborted = true } d._panic = nil d.fn = nil gp._defer = d.link freedefer(d) continue } d.started = true // Record the panic that is running the defer. // If there is a new panic during the deferred call, that panic // will find d in the list and will mark d._panic (this panic) aborted. // Bind the current panic to the defer, and there may be panic in the defer. In this case, it will go into the logic of d.started above, and then terminate the current panic because it has already been executed. d._panic = (*_panic)(noescape(unsafe.Pointer(&p))) // Execute defer.fn p.argp = unsafe.Pointer(getargp(0)) reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz)) p.argp = nil // reflectcall did not panic. Remove d. if gp._defer != d { throw("bad defer entry in panic") } // Resolve the binding relationship between defer and panic, because the defer function has been executed, if there is panic or Goexit, it will not be executed here. d._panic = nil d.fn = nil gp._defer = d.link // trigger shrinkage to test stack copy. See stack_test.go:TestStackPanic //GC() pc := d.pc sp := unsafe.Pointer(d.sp) // must be pointer so it gets adjusted during stack copy freedefer(d) // When panic is recover ed, there is no need to continue panic and continue executing the remaining code if p.recovered { atomic.Xadd(&runningPanicDefers, -1) gp._panic = p.link // Aborted panics are marked but remain on the g.panic list. // Remove them from the list. // Remove aborted panic from the panic list, as explained below for gp._panic != nil && gp._panic.aborted { gp._panic = gp._panic.link } if gp._panic == nil { // must be done with signal gp.sig = 0 } // Pass information about recovering frame to recovery. gp.sigcode0 = uintptr(sp) gp.sigcode1 = pc // Call recovery to restore the scheduled execution of the current g mcall(recovery) throw("recovery failed") // mcall should not return } } // Print panic information preprintpanics(gp._panic) // panic fatalpanic(gp._panic) // should not return *(*int)(nil) = 0 // not reached }
Here's how gp._panic.aborted works. Here's an example
func main() { defer func() { // defer1 recover() }() panic1() } func panic1() { defer func() { // defer2 panic("error1") // panic2 }() panic("error") // panic1 }
- When executed to panic("error")
g._defer list: g._defer - > defer2 - > defer1
g._panic list: g._panic - > Panic1
- When executed to panic("error1")
g._defer list: g._defer - > defer2 - > defer1
g._panic list: g._panic - > panic 2 - > panic 1
-
Continue to the defer1 function and recover()
At this point, panic caused by panic2 will be restored, panic2.recovered = true. We should continue to process the next panic along the g._panic list, but we can find that panic1 has been executed, which is the logic of the following code, to remove the panic that has been executed.
for gp._panic != nil && gp._panic.aborted { gp._panic = gp._panic.link }
The logic of panic can be sorted out.
When the program encounters panic, it will no longer continue to execute. First, it mounts the current panic on the g._panic list, starts traversing the current g._defer list, then executes the functions defined by the _defer object, etc. If the panic happens again in the process of invoking the defer function, then it executes the gopanic function. Finally, it executes the gopanic function. The loop prints all panic information and exits the current G. However, if recovery is encountered in the process of invoking defer, the schedule (mcall(recovery) continues.
2.4.1. recovery
Restore a panic g, re-enter and continue scheduling
func recovery(gp *g) { // Info about defer passed in G struct. sp := gp.sigcode0 pc := gp.sigcode1 // Make the deferproc for this d return again, // this time returning 1. The calling function will // jump to the standard return epilogue. // Record the sp pc returned by defer gp.sched.sp = sp gp.sched.pc = pc gp.sched.lr = 0 gp.sched.ret = 1 // Restore execution scheduling gogo(&gp.sched) }
2.5. gorecover
gorecovery just sets the flag bit of g._panic.recovered
func gorecover(argp uintptr) interface{} { gp := getg() p := gp._panic // You need to determine whether it is called in defer function based on the address of argp if p != nil && !p.recovered && argp == uintptr(p.argp) { // Set a flag, which will be judged in the gopanic above p.recovered = true return p.arg } return nil }
2.6. goexit
We also overlooked the point that when we manually call runtime.Goexit() to exit, the defer function will also execute. Let's analyze this situation.
func Goexit() { // Run all deferred functions for the current goroutine. // This code is similar to gopanic, see that implementation // for detailed comments. gp := getg() // Traversing defer list for { d := gp._defer if d == nil { break } // If defer has been executed, panic bound to defer terminates if d.started { if d._panic != nil { d._panic.aborted = true d._panic = nil } d.fn = nil // Remove from defer list gp._defer = d.link // Release defer freedefer(d) continue } // Calling defer internal functions d.started = true reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz)) if gp._defer != d { throw("bad defer entry in Goexit") } d._panic = nil d.fn = nil gp._defer = d.link freedefer(d) // Note: we ignore recovers here because Goexit isn't a panic } // Call goexit0 to clear the attributes of the current g and re-enter the schedule goexit1() }
2.7. Graphic analysis
Source code is not very difficult to read, if there are any doubts, I hope the following picture can solve your doubts.
Excuse me for the slight inferiority of the drawing.
Step analysis:
- L3: Generate a defer1 and put it on the g._defer list
- L11: Generate a defer2 and mount it on the g._defer list
- L14: panic1 calls gopanic to place the current panic on the g._panic list
- L14: Because panic1, extract from the head of the g._defer list to defer2 and start executing
- L12: Execute defer2, another panic, mounted on the g._panic list
- L12: Because panic2, extracting defer2 from the head of the g._defer list, it is found that defer2 has executed the removal list, and defer2 is triggered by panic1, skipping defer2 and abort panic1.
- L12: Continue to extract the next part of the g._defer list and extract it to defer1
- L5: defer1 performs recovery, recovers panic2, removes the list, and judges that the next panic, panic1, has been removed by defer2 aborted, removes Panic1
- Defer1 is executed, defer1 is removed
3. Associated Documents
- Level 2 cache, sizeclass: Deep Understanding of Go-Garbage Recycling Mechanism
- Gogo exit0 scheduling: Deeply Understanding the Implementation of Go-goroutine and Scheduler Analysis
4. Reference Documents
- Go Language Learning Notes--Rain Traces