Chapter 8 Goroutines and Channels

Concurrent programs in Go language can be implemented by two means: goroutine and channel, which support sequential communication processes, or CSP for short. CSP is a concurrent programming model. In this concurrent programming model, values will be passed in different running instances. The second means is multi-threaded shared memory

8.5 concurrent loops

In this section, let's take a look at the common concurrency model of circular iteration in parallel. We generate some thumbnails from full-size images. The following function can help us stretch the images. We can use it directly

package thumbnail

func ImageFile(infile string) (string error)

Now let's build this function and call it makethumbnails

func makeThumbnails(filenames []string) (string, error) {
	for _, f := range filenames {
		if _, err := thumbnail.ImageFile(f); err != nil {
			log.Println(err)
		}
	}
}

Each of the above file s is independent. We use the go keyword to put each processing into goroutine. Note that we have no processing errors below

func makeThumbnails(filenames []string) (string, error) {
	for _, f := range filenames {
		go thumbnail.ImageFile(f)
	}
}

In this way, all the above file s are parallel, and when one goroutine completes first, makethumbnails will return. No matter whether other goroutines end or not, we now use the communication mechanism of channel to end the execution of all goroutines

func makeThumbnails(filenames []string) (string, error) {
	for _, f := range filenames {
		ch := make(chan struct{}) //Create a channel
		go func(f string) {       //Give the traversed function as the parameter of goroutine
			thumbnail.ImageFile(f)  //At the end of the goroutine function, a message is sent to the channel (here is the structure)
			ch <- struct{}{}
		}(f)
		for range filenames {
			<- ch                   //The receiver is at the end of the main function, which can ensure that all goroutine s end, because the program will not stop when the channel does not receive a value
		}
	}
}

In the above function, we call the function in goroutine. It is the case that the function will fail to call. Now we use channel technology to transfer possible errors from goroutine.

func makeThumbnails(filenames []string) (string, error) {
	for _, f := range filenames {
		errors := make(chan error)
		go func(f string) {
			_, err := thumbnail.ImageFile(f)    //Keep returned errors
			errors <- err                       //Send errors
		}(f)
	}
	for range filenames {
		if err := <- errors;err != nil {        //Receive error, return if any
			return err
		}
	}
	return nil                                //If not, it returns a null value
}

Now we can pass the errors that may occur when calling the function, but there is a problem with this program. Because we pass an error and the error of this channel is not received (the assignment here to the new variable err is not received), the channel will be blocked. In the future, even if there are errors in goroutine, the error cannot be successfully passed, Now let's create a new channel with cache, and let the function return appropriate information whether the processing is successful or not

func makeThumbnails(filenames []string) (thumbfiles []string, err error) {

	type item struct { //First define a structure. In order to integrate the return value into a variable, the internal members of the structure are two return values
		thumbfile string
		err       error
	}

	ch := make(chan item, len(filenames))    //Define a cached channel

	for _, f := range filenames {
		go func(f string) {
			var it item                     //Define a variable of structure type to call
			it.thumbfile, it.err = thumbnail.ImageFile(f)
			ch <- it
		}(f)
	}

	for range filenames {
		it := <- ch
		if it.err != nil {
			return nil,it.err
		}
		thumbfiles = append(thumbfiles,it.thumbfile)
	}
	return thumbfiles,nil
}

The above program is perfect, but we still want to change it. We want to get the number of files. Here we count each goroutine, and we use a new type sync Waitgroup, look at the code

func makeThumbnails(filenames <-chan string) int64 {
	
	sizes := make(chan int64)
	var wg sync.WaitGroup
	
	for f := range filenames {
		wg.Add(1)
		go func(f string) {
			defer wg.Done()
			thumb, err := thumbnail.ImageFile(f)
			if err != nil {
				log.Println(err)
				return
			}
			info, _ := os.Stat(err)
			sizes <- info.Size()
		}(f)
	}

	go func() {
		wg.Wait()
		close(sizes)
	}()

	var total int64
	
	for sise := range sizes {
		total += size
	}
	
	return total
}

The above program code structure is very common and authentic when we use concurrent loops but don't know the number of iterations