From: month https://lixueduan.com

Original text: https://www.lixueduan.com/post/grpc/12-client-side-loadbalance/
This paper mainly introduces the built-in load balancing strategy of gRPC and its configuration and use, including Name Resolver, ServiceConfig, etc.

For relevant codes of gRPC series, see Github

1. General

gRPC load balancing includes client load balancing and server load balancing. This paper mainly introduces the client load balancing.

gRPC client load balancing is mainly divided into two parts:

• 1)Name Resolver
• 2)Load Balancing Policy

1. NameResolver

For details, please refer to Official document - Name Resolver perhaps gRPC series tutorials (11) - NameResolver actual combat and principle analysis

The default name system in gRPC is DNS. At the same time, the client provides a mechanism to customize the name system in the form of a plug-in.

gRPC NameResolver will select the corresponding resolver according to the name system to resolve the server name provided by the user, and finally return the specific address list (IP + port number).

For example, DNS name system is used by default. We only need to provide the domain name of the server, that is, the port number. NameResolver will use DNS to resolve the IP list corresponding to the domain name and return it.

In this example, we will customize a NameResolver.

1.2 Load Balancing Policy

For details, please refer to Official document - Load Balancing Policy

Several load balancing algorithms are built into common gRPC libraries, such as gRPC-Go Built in pick_first and round_robin has two algorithms.

• pick_first: try to connect to the first address. If the connection is successful, use it for all RPC s. If the connection fails, try the next address (and continue to do so until a connection is successful).
• round_robin: connect to all the addresses it sees and send an RPC to each backend in turn. For example, the first RPC will be sent to backend-1, the second RPC will be sent to backend-2, and the third RPC will be sent to backend-1 again.

In this example, we will test the effects of the two load balancing strategies.

2. Demo

2.1 Server

package main

import (
	"context"
	"fmt"
	"log"
	"net"
	"sync"

	"google.golang.org/grpc"

	pb "github.com/lixd/grpc-go-example/features/proto/echo"
)

var (
	addrs = []string{":50051", ":50052"}
)

type ecServer struct {
	pb.UnimplementedEchoServer
	addr string
}

func (s *ecServer) UnaryEcho(ctx context.Context, req *pb.EchoRequest) (*pb.EchoResponse, error) {
	return &pb.EchoResponse{Message: fmt.Sprintf("%s (from %s)", req.Message, s.addr)}, nil
}

func startServer(addr string) {
	lis, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterEchoServer(s, &ecServer{addr: addr})
	log.Printf("serving on 0.0.0.0%s\n", addr)
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

func main() {
	var wg sync.WaitGroup
	for _, addr := range addrs {
		wg.Add(1)
		go func(addr string) {
			defer wg.Done()
			startServer(addr)
		}(addr)
	}
	wg.Wait()
}

Mainly through a for loop, the service is started on ports 50051 and 50052.

2.2 Client

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	pb "github.com/lixd/grpc-go-example/features/proto/echo"
	"google.golang.org/grpc"
	"google.golang.org/grpc/resolver"
)

const (
	exampleScheme      = "example"
	exampleServiceName = "lb.example.grpc.lixueduan.com"
)

var addrs = []string{"localhost:50051", "localhost:50052"}

func callUnaryEcho(c pb.EchoClient, message string) {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	r, err := c.UnaryEcho(ctx, &pb.EchoRequest{Message: message})
	if err != nil {
		log.Fatalf("could not greet: %v", err)
	}
	fmt.Println(r.Message)
}

func makeRPCs(cc *grpc.ClientConn, n int) {
	hwc := pb.NewEchoClient(cc)
	for i := 0; i < n; i++ {
		callUnaryEcho(hwc, "this is examples/load_balancing")
	}
}

func main() {
	// "pick_first" is the default, so there's no need to set the load balancer.
	pickfirstConn, err := grpc.Dial(
		fmt.Sprintf("%s:///%s", exampleScheme, exampleServiceName),
		grpc.WithInsecure(),
		grpc.WithBlock(),
	)
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer pickfirstConn.Close()

	fmt.Println("--- calling helloworld.Greeter/SayHello with pick_first ---")
	makeRPCs(pickfirstConn, 10)

	fmt.Println()

	// Make another ClientConn with round_robin policy.
	roundrobinConn, err := grpc.Dial(
		fmt.Sprintf("%s:///%s", exampleScheme, exampleServiceName),
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`), // This sets the initial balancing policy.
		grpc.WithInsecure(),
		grpc.WithBlock(),
	)
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer roundrobinConn.Close()

	fmt.Println("--- calling helloworld.Greeter/SayHello with round_robin ---")
	makeRPCs(roundrobinConn, 10)
}

You can see that on the client side, two connections are established using different load balancing policies. The first is the default policy pick_first, then round_robin, the core code is:

grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`)

At the same time, because it is a local test, it is inconvenient to use the built-in dns Resolver, so a Name Resolver is customized, and the relevant codes are as follows:

// Following is an example name resolver implementation. Read the name
// resolution example to learn more about it.

type exampleResolverBuilder struct{}

func (*exampleResolverBuilder) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOptions) (resolver.Resolver, error) {
	r := &exampleResolver{
		target: target,
		cc:     cc,
		addrsStore: map[string][]string{
			exampleServiceName: addrs,
		},
	}
	r.start()
	return r, nil
}
func (*exampleResolverBuilder) Scheme() string { return exampleScheme }

type exampleResolver struct {
	target     resolver.Target
	cc         resolver.ClientConn
	addrsStore map[string][]string
}

func (r *exampleResolver) start() {
	addrStrs := r.addrsStore[r.target.Endpoint]
	addrs := make([]resolver.Address, len(addrStrs))
	for i, s := range addrStrs {
		addrs[i] = resolver.Address{Addr: s}
	}
	r.cc.UpdateState(resolver.State{Addresses: addrs})
}
func (*exampleResolver) ResolveNow(o resolver.ResolveNowOptions) {}
func (*exampleResolver) Close()                                  {}

3. Test

Run the server and client respectively to view the results

lixd@17x:~/17x/projects/grpc-go-example/features/load_balancing/server$ go run main.go 
2021/05/23 09:47:59 serving on 0.0.0.0:50052
2021/05/23 09:47:59 serving on 0.0.0.0:50051

lixd@17x:~/17x/projects/grpc-go-example/features/load_balancing/client$ go run main.go 
--- calling helloworld.Greeter/SayHello with pick_first ---
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50051)

--- calling helloworld.Greeter/SayHello with round_robin ---
this is examples/load_balancing (from :50052)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50052)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50052)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50052)
this is examples/load_balancing (from :50051)
this is examples/load_balancing (from :50052)
this is examples/load_balancing (from :50051)

You can see pick_ During the first load balancing policy, the first service 50051, round is always requested_ Robin will alternate requests, which is also consistent with the load balancing strategy.

3. Summary

The load balancing described in this article belongs to client-side load balancing, which needs to be greatly changed on the client. Because the corresponding code has been implemented in grpc go, it is still very simple to use.

gRPC built-in load balancing:

• 1) According to the service name provided, use the corresponding name resolver to obtain the specific ip + port number list
• 2) Establish connections according to the specific service list
  • An internal gRPC connection pool is also maintained
• 3) Select a connection for rpc request according to the load balancing policy

For example, in the previous example, the service name is example:///lb.example.grpc.lixueduan.com , use the custom name resolver to parse the specific service list as localhost:50051,localhost:50052

Then the connection between the two services will be established when the dial is called. Finally, select a connection to initiate rpc request according to the load balancing policy. So pick_first will always request the 50051 service, while round_robin will alternately request 50051 and 50052.

4. Reference

https://github.com/grpc/grpc/blob/master/doc/naming.md

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md