Code address of this series: https://github.com/HashZhang/spring-cloud-scaffold/tree/master/spring-cloud-iiford

We use the Spring Cloud LoadBalancer officially recommended by Spring Cloud as our client load balancer. In the previous section, we learned about the structure of Spring Cloud LoadBalancer. Next, let's talk about the functions we want to achieve when using Spring Cloud LoadBalancer:

1. We need to realize that different clusters do not call each other. We can distinguish the instances of different clusters through the zone configuration in the instance metamap. Only instances with the same zone configuration in the metamap of the instance can call each other. This can be achieved by implementing a custom ServiceInstanceListSupplier
2. The polling algorithm of load balancing requires isolation between requests and cannot share the same position. The retry after a request fails is still the original failed instance. The default roundrobin loadbalancer seen in the previous section is that all threads share the same atomic variable position, adding 1 to each request atom. In this case, there will be a problem: suppose there are two instances of microservice A: instance 1 and instance 2. When request a arrives, the roundrobin loadbalancer returns instance 1. When request B arrives, the roundrobin loadbalancer returns instance 2. Then, if request a fails, try again, and the roundrobin loadbalancer returns instance 1. This is not what we expect.

For these two functions, we write our own implementations.

zone configuration in Spring Cloud LoadBalancer

Spring Cloud LoadBalancer defines LoadBalancerZoneConfig:

public class LoadBalancerZoneConfig {
    //Identifies which zone the current load balancer is in
	private String zone;
	public LoadBalancerZoneConfig(String zone) {
		this.zone = zone;
	}
	public String getZone() {
		return zone;
	}
	public void setZone(String zone) {
		this.zone = zone;
	}
}

If Eureka related dependencies are not introduced, the zone passes through spring cloud. loadbalancer. Zone configuration: LoadBalancerAutoConfiguration

@Bean
@ConditionalOnMissingBean
public LoadBalancerZoneConfig zoneConfig(Environment environment) {
	return new LoadBalancerZoneConfig(environment.getProperty("spring.cloud.loadbalancer.zone"));
}

If Eureka related dependencies are introduced, if a zone is configured in Eureka metadata, the zone will override the LoadBalancerZoneConfig in Spring Cloud LoadBalancer:

EurekaLoadBalancerClientConfiguration

@PostConstruct
public void postprocess() {
	if (!StringUtils.isEmpty(zoneConfig.getZone())) {
		return;
	}
	String zone = getZoneFromEureka();
	if (!StringUtils.isEmpty(zone)) {
		if (LOG.isDebugEnabled()) {
			LOG.debug("Setting the value of '" + LOADBALANCER_ZONE + "' to " + zone);
		}
		//Settings ` LoadBalancerZoneConfig`
		zoneConfig.setZone(zone);
	}
}

private String getZoneFromEureka() {
	String zone;
	//Is spring.com configured cloud. loadbalancer. eureka. Approximatezonefromhostname is true
	boolean approximateZoneFromHostname = eurekaLoadBalancerProperties.isApproximateZoneFromHostname();
	//If configured, try extracting from the host name configured by Eureka
    //The reality is that Split the host, and then the second is the zone
    //For example, www.zone1.com COM is Zone1
	if (approximateZoneFromHostname && eurekaConfig != null) {
		return ZoneUtils.extractApproximateZone(this.eurekaConfig.getHostName(false));
	}
	else {
	    //Otherwise, get the zone key from the metadata map
		zone = eurekaConfig == null ? null : eurekaConfig.getMetadataMap().get("zone");
		//If the key does not exist, take the first zone from the zone list with region as the current zone from the configuration
		if (StringUtils.isEmpty(zone) && clientConfig != null) {
			String[] zones = clientConfig.getAvailabilityZones(clientConfig.getRegion());
			// Pick the first one from the regions we want to connect to
			zone = zones != null && zones.length > 0 ? zones[0] : null;
		}
		return zone;
	}
}

Implement SameZoneOnlyServiceInstanceListSupplier

In order to filter instances in the same zone through the zone and never return instances in different zones, we write code:

SameZoneOnlyServiceInstanceListSupplier

/**
 * Only service instances in the same zone as the current instance are returned. Services in different zones do not call each other
 */
public class SameZoneOnlyServiceInstanceListSupplier extends DelegatingServiceInstanceListSupplier {
    /**
     * The instance metadata map represents the key of the zone configuration
     */
    private final String ZONE = "zone";
    /**
     * zone configuration of the current spring cloud loadbalancer
     */
    private final LoadBalancerZoneConfig zoneConfig;
    private String zone;

    public SameZoneOnlyServiceInstanceListSupplier(ServiceInstanceListSupplier delegate, LoadBalancerZoneConfig zoneConfig) {
        super(delegate);
        this.zoneConfig = zoneConfig;
    }

    @Override
    public Flux<List<ServiceInstance>> get() {
        return getDelegate().get().map(this::filteredByZone);
    }

    //Filter through zoneConfig
    private List<ServiceInstance> filteredByZone(List<ServiceInstance> serviceInstances) {
        if (zone == null) {
            zone = zoneConfig.getZone();
        }
        if (zone != null) {
            List<ServiceInstance> filteredInstances = new ArrayList<>();
            for (ServiceInstance serviceInstance : serviceInstances) {
                String instanceZone = getZone(serviceInstance);
                if (zone.equalsIgnoreCase(instanceZone)) {
                    filteredInstances.add(serviceInstance);
                }
            }
            if (filteredInstances.size() > 0) {
                return filteredInstances;
            }
        }
        /**
         * @see ZonePreferenceServiceInstanceListSupplier When there is no same zone instance, all instances are returned
         * We need to return an empty list in order to realize that different zone s do not call each other
         */
        return List.of();
    }

    //Read the zone of the instance. If it is not configured, it is null
    private String getZone(ServiceInstance serviceInstance) {
        Map<String, String> metadata = serviceInstance.getMetadata();
        if (metadata != null) {
            return metadata.get(ZONE);
        }
        return null;
    }
}

In the previous chapter, we mentioned that we use spring cloud sleuth as the link tracking library. We think we can distinguish whether it is the same request or not by the traceId.

RoundRobinWithRequestSeparatedPositionLoadBalancer

//It must implement the ReactorServiceInstanceLoadBalancer
//Instead of reactorloadbalancer < serviceinstance >
//Because the ReactorServiceInstanceLoadBalancer is registered
@Log4j2
public class RoundRobinWithRequestSeparatedPositionLoadBalancer implements ReactorServiceInstanceLoadBalancer {
    private final ServiceInstanceListSupplier serviceInstanceListSupplier;
    //Each request, including retry, will not exceed 1 minute
    //For more than 1 minute, the request must be heavy and should not be retried
    private final LoadingCache<Long, AtomicInteger> positionCache = Caffeine.newBuilder().expireAfterWrite(1, TimeUnit.MINUTES)
            //Random initial value to prevent calling from the first one every time
            .build(k -> new AtomicInteger(ThreadLocalRandom.current().nextInt(0, 1000)));
    private final String serviceId;
    private final Tracer tracer;


    public RoundRobinWithRequestSeparatedPositionLoadBalancer(ServiceInstanceListSupplier serviceInstanceListSupplier, String serviceId, Tracer tracer) {
        this.serviceInstanceListSupplier = serviceInstanceListSupplier;
        this.serviceId = serviceId;
        this.tracer = tracer;
    }

    @Override
    public Mono<Response<ServiceInstance>> choose(Request request) {
        return serviceInstanceListSupplier.get().next().map(serviceInstances -> getInstanceResponse(serviceInstances));
    }

    private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> serviceInstances) {
        if (serviceInstances.isEmpty()) {
            log.warn("No servers available for service: " + this.serviceId);
            return new EmptyResponse();
        }
        return getInstanceResponseByRoundRobin(serviceInstances);
    }

    private Response<ServiceInstance> getInstanceResponseByRoundRobin(List<ServiceInstance> serviceInstances) {
        if (serviceInstances.isEmpty()) {
            log.warn("No servers available for service: " + this.serviceId);
            return new EmptyResponse();
        }
        //In order to solve the problem of different original algorithms, call concurrency may cause a request to retry the same instance
        Span currentSpan = tracer.currentSpan();
        if (currentSpan == null) {
            currentSpan = tracer.newTrace();
        }
        long l = currentSpan.context().traceId();
        AtomicInteger seed = positionCache.get(l);
        int s = seed.getAndIncrement();
        int pos = s % serviceInstances.size();
        log.info("position {}, seed: {}, instances count: {}", pos, s, serviceInstances.size());
        return new DefaultResponse(serviceInstances.stream()
                //The order of the instance return list may be different. In order to maintain consistency, sort first and then retrieve
                .sorted(Comparator.comparing(ServiceInstance::getInstanceId))
                .collect(Collectors.toList()).get(pos));
    }
}

In the previous section, we mentioned that the default load balancer configuration can be configured through the @ LoadBalancerClients annotation. This is how we configure it here. First, in spring Add auto configuration class to factories:

spring.factories

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.github.hashjang.spring.cloud.iiford.service.common.auto.LoadBalancerAutoConfiguration

Then write this automatic configuration class. In fact, it is very simple to add an @ LoadBalancerClients annotation and set the default configuration class:

LoadBalancerAutoConfiguration

@Configuration(proxyBeanMethods = false)
@LoadBalancerClients(defaultConfiguration = DefaultLoadBalancerConfiguration.class)
public class LoadBalancerAutoConfiguration {
}

Write this default configuration class and assemble the above two classes:

DefaultLoadBalancerConfiguration

@Configuration(proxyBeanMethods = false)
public class DefaultLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier serviceInstanceListSupplier(
            DiscoveryClient discoveryClient,
            Environment env,
            ConfigurableApplicationContext context,
            LoadBalancerZoneConfig zoneConfig
    ) {
        ObjectProvider<LoadBalancerCacheManager> cacheManagerProvider = context
                .getBeanProvider(LoadBalancerCacheManager.class);
        return  //Enable service instance cache
                new CachingServiceInstanceListSupplier(
                        //Only service instances of the same zone can be returned
                        new SameZoneOnlyServiceInstanceListSupplier(
                                //Enable service discovery through discoveryClient
                                new DiscoveryClientServiceInstanceListSupplier(
                                        discoveryClient, env
                                ),
                                zoneConfig
                        )
                        , cacheManagerProvider.getIfAvailable()
                );
    }

    @Bean
    public ReactorLoadBalancer<ServiceInstance> reactorServiceInstanceLoadBalancer(
            Environment environment,
            ServiceInstanceListSupplier serviceInstanceListSupplier,
            Tracer tracer
    ) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RoundRobinWithRequestSeparatedPositionLoadBalancer(
                serviceInstanceListSupplier,
                name,
                tracer
        );
    }
}

In this way, we have implemented a custom load balancer. I also understand the use of Spring Cloud LoadBalancer.

In this section, we analyze in detail the functions to be realized by using Spring Cloud LoadBalancer in our project, implement a custom load balancer, and understand the use of Spring Cloud LoadBalancer. In the next section, we use unit tests to verify whether the functions we want to implement are effective.