In the last article, we talked about the concurrency toolkit core/syncx in go zero.

From the overall analysis, concurrent components mainly communicate with each other in the control program through channel + mutex.

Do not communicate by sharing memory; instead, share memory by communicating.

Do not communicate through shared memory, but share memory through communication.

This article will talk about Go zero's concurrent components supported by goroutine in Go.

Let's review the tools that go natively supports for goroutine control?

1. go func() starts a coroutine
2. sync.WaitGroup controls the scheduling of multiple collaborative tasks
3. sync.Cond wake-up or wait

Then you may ask why go zero wants to talk about this? Back to go zero's design philosophy: tools are greater than conventions and documents.

So let's see what tools go zero provides?

threading

Although go func() is convenient, there are several problems:

• If the coroutine exits abnormally, the exception stack cannot be tracked
• If an abnormal request triggers panic, fault isolation should be done instead of the whole process exiting, which is easy to be attacked

Let's see what additional options the core/threading package offers:

func GoSafe(fn func()) {
    go RunSafe(fn)
}

func RunSafe(fn func()) {
    defer rescue.Recover()
    fn()
}

func Recover(cleanups ...func()) {
    for _, cleanup := range cleanups {
        cleanup()
    }

    if p := recover(); p != nil {
        logx.ErrorStack(p)
    }
}

GoSafe

threading.GoSafe() will help you solve this problem. Developers can pass in the logic they need to complete in the collaboration process in the form of closure, and go func() inside GoSafe();

When the developer's function exits abnormally, the exception stack will be printed in Recover(), so that the developer can quickly determine the exception point and call stack.

NewWorkerGroup

Let's look at the second one: WaitGroup. In fact, WaitGroup has nothing to say about daily development. You need N collaborative processes: WG Add (N), wait for all cooperation processes to complete the task: WG Wait(), which requires manual WG Done().

It can be seen that in the process of task start - > end - > wait, the developer needs to pay attention to the status of the task and then modify the status manually.

NewWorkerGroup helps developers reduce the burden. Developers only need to focus on:

1. Task logic [function]
2. Number of tasks [workers]

Then start workergroup Start(), the corresponding number of tasks will start:

func (wg WorkerGroup) Start() {
  // Packed sync WaitGroup
    group := NewRoutineGroup()
    for i := 0; i < wg.workers; i++ {
    // WG is maintained internally Add(1) wg. Done()
    // It is also carried out in goroutine security mode
        group.RunSafe(wg.job)
    }
    group.Wait()
}

The status of workers will be automatically managed and can be used to process the tasks of message queue with a fixed number of workers. The usage is as follows:

func main() {
  group := NewWorkerGroup(func() {
    // process tasks
    }, runtime.NumCPU())
    group.Start()
}

Pool

The pool here is not sync Pool. sync. One inconvenient thing about pool is that its pooled objects may be garbage collected, which makes developers wonder when the objects they create and store will disappear.

pool in go Zero:

1. The objects in the pool will be destroyed according to the usage time;
2. Use cond to notify and block object consumption and production;
3. Developers can customize their own production functions and destroy functions;

Let me see how production objects and consumption objects are implemented in the pool:

func (p *Pool) Get() interface{} {
  // Call cond You must hold the c.L lock when waiting
    p.lock.Lock()
    defer p.lock.Unlock()

    for {
    // 1. The object pool in the pool is a nodelist connected by a linked list
        if p.head != nil {
            head := p.head
            p.head = head.next
      // 1.1 if current node: current time > = last used time + maximum survival time of object
            if p.maxAge > 0 && head.lastUsed+p.maxAge < timex.Now() {
                p.created--
        // Indicates that the current node has expired - > destroy the object corresponding to the node, and then continue to find the next node
        // [ ⚠️:  Not destroy the node, but destroy the object corresponding to the node]
                p.destroy(head.item)
                continue
            } else {
                return head.item
            }
        }
        // 2. The object pool is lazily loaded, and the object linked list is created only when get ting
        if p.created < p.limit {
            p.created++
      // Input by the developer: production function
            return p.create()
        }

        p.cond.Wait()
    }
}

func (p *Pool) Put(x interface{}) {
    if x == nil {
        return
    }
    // Mutually exclusive access in pool
    p.lock.Lock()
    defer p.lock.Unlock()

    p.head = &node{
        item:     x,
        next:     p.head,
        lastUsed: timex.Now(),
    }
  // Put it into the head and inform other collaborative processes that are get ting [extremely critical]
    p.cond.Signal()
}

The above is the use of Cond by go zero. It can be compared with the producer consumer model, but instead of using channel for communication, Cond is used. Here are several features:

• Cond is associated with a Locker, which can be used to protect related dependency changes.
• Both Broadcast and Channel can support one method at the same time.

summary

Tools are bigger than conventions and documents, which has always been one of the main themes of go zero design; At the same time, it also precipitates the usual business into components, which is the meaning of framework and components.

For more articles on the design and implementation of go zero, you can continue to pay attention to us. Welcome to pay attention and use.

Project address

https://github.com/tal-tech/go-zero

Welcome to go zero and star support us!

Wechat communication group

Focus on the official account of micro service practice and return to the community to get the two-dimensional code of community groups.

go-zero series articles see official account of "micro service practice".