Foreword: in the last blog, I introduced how zookeeper works as the registry of dubbo. There is a very important point. Our program is a distributed application. The service is deployed on several nodes (servers). When consumers call the service, zk returns a node list to dubbo, but dubbo only selects one server, So which one will it choose? This is dubbo's load balancing strategy. This blog will focus on dubbo's load balancing strategy.
Contents of this blog
1: Introduction to load balancing
1.1: introduction to load balancing
The following is wikipedia's definition of load balancing:
Load balancing improves performance across multiple computing resources (for example, workload distribution of computers, computer clusters, network links, central processing units or disk drives. Load balancing aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of any single resource. Using multiple components with load balancing instead of a single component can improve reliability and availability through redundancy. Negative Load balancing usually involves dedicated software or hardware
1.2: simple explanation
How to understand this concept? Generally speaking, if A request is initiated from the client, such as (querying the order list), the server should be selected for processing, but our cluster environment provides five servers A\B\C\D\E, and each server has the ability to process the request. At this time, the client must select A server for processing (there is no need to select A first, select C after processing, and then jump to D) To put it bluntly, it is A matter of choice. When there are too many requests, the load of each server should be considered. There are five servers in total. It is impossible to let one server handle it every time. For example, let other servers share the pressure. This is the advantage of load balancing: it can prevent A single server from responding to the same request, which is easy to cause server downtime, crash and other problems.
2: loadBalance interface of dubbo
1.1: loadBalance
Dubbo's load balancing strategy exposes the main body to the outside as an interface called loadplace, which is located on COM alibaba. dubbo. rpc. Under the cluster package, it is obvious that it is used to manage clusters according to the package name:
This interface is just a method. The function of the select method is to select a caller from the List of many calls. Invoker can be understood as the caller of the client. dubbo encapsulates a special class to represent it. URL is the URL request link initiated by the caller. From this URL, you can obtain the specific information of many requests. Invocation represents the specific process of the call, dubbo uses this class to simulate the detailed process of calling:
1.2: AbstractLoadBlance
This interface will be implemented in the following subclasses. Under the interface is an abstract class abstractloadlance
package com.alibaba.dubbo.rpc.cluster;
public interface LoadBalance {@Adaptive("loadbalance")
<T> Invoker<T> select(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException;
}Abstractloadplace abstract class inherits from LoadBalance. A static method indicates that it will run when the class is loaded. It means to calculate the warm-up loading weight. The parameter is uptime. It can be understood here as the service startup time, warm up is the warm-up time, and weight is the value of the weight. The comparison will be explained in detail below:
public abstract class AbstractLoadBalance implements LoadBalance{
static int calculateWarmupWeight(int uptime, int warmup, int weight){
//
}
@Override
public <T> Invoker<T> select(List<Invoker<T>> invokers, URL url,
Invocation invocation){
//
}
protected abstract <T> Invoker<T> doSelect(List<Invoker<T>>
invokers, URL url, Invocation invocation);
protected int getWeight(Invoker<?> invoker, Invocation invocation) {
}
}1.2.1: select method
There is a method select with method body in the abstract class method. First judge whether the List composed of callers is null. If it is null, it will return null. Then judge the size of the caller. If there is only one caller, return the unique caller (imagine that if the service calls another service, when there is only one service provider machine, you can return that one, because there is no choice!) If none of this holds, go on and follow the doSelect method:
@Override
public <T> Invoker<T> select(List<Invoker<T>> invokers, URL url, Invocation invocation) {
if (invokers == null || invokers.isEmpty())
return null;
if (invokers.size() == 1)
return invokers.get(0);
return doSelect(invokers, url, invocation);
}1.2.2:doSelect method
This method is abstract and is implemented by specific subclasses. Therefore, we can also think of a question: why does dubbo make an interface into an implementation abstract class first, and then implement it by different subclasses? The reason is that non abstract methods in abstract classes and subclasses must be implemented, and the difference of their subclasses is that they have different strategies to make specific choices. They extract the public logic and put it in the abstract class. Subclasses do not need to write redundant code, but only maintain and implement their own logic
protected abstract <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation);
1.2.3: getWeight method
As the name suggests, the meaning of this method is to obtain the weight. First, obtain the basic weight through the URL (an entity encapsulated by dubbo). If the weight is greater than 0, the service startup time will be obtained, and then the current time startup time is how long the service has been running so far. Therefore, this upTime can be understood as the service startup time, and then obtain the configured warm-up time, If the start-up time is less than the warm-up time, it will be called again to get the weight. This preheating method is actually a very suitable optimization made by dubbo for the JVM, because it takes a little time for the JVM to run to the best state from Startup to startup. This time is called warmup, and dubbo will set this time, and then calculate the weight when the service runtime is equal to warmup, so as to ensure the best running state of the service!
protected int getWeight(Invoker<?> invoker, Invocation invocation) {
int weight = invoker.getUrl().getMethodParameter(invocation.getMethodName(), Constants.WEIGHT_KEY, Constants.DEFAULT_WEIGHT);
if (weight > 0) {
long timestamp = invoker.getUrl().getParameter(Constants.REMOTE_TIMESTAMP_KEY, 0L);
if (timestamp > 0L) {
int uptime = (int) (System.currentTimeMillis() - timestamp);
int warmup = invoker.getUrl().getParameter(Constants.WARMUP_KEY, Constants.DEFAULT_WARMUP);
if (uptime > 0 && uptime < warmup) {
weight = calculateWarmupWeight(uptime, warmup, weight);
}
}
}
return weight;
}3: Several load balancing strategies of dubbo
3.1: overall structure diagram
It can be seen that there are four classes under the abstract load balancing. These four classes represent four load balancing strategies, namely consistent Hash balancing algorithm, random call method, polling method and least active call method
3.2:RandomLoadBalance
Load balancing is called randomly. This class implements the abstract AbstractLoadBalance interface and rewrites the doSelect method. The details of the method are to first traverse each service providing machine, obtain the weight of each service, and then accumulate the weight value to judge whether the weight of each service provider is the same. If the weight of each caller is different and each weight is greater than 0, Then a random number will be generated according to the total value of the weight, and then the random number will be used to subtract the weight of the caller each time according to the number of callers. Until the random number of the current service provider is calculated to be less than 0, select that provider! In addition, if the weight of each machine is the same, the weight will not participate in the calculation, and a selection generated by the random algorithm is directly selected, which is completely random. As you can see, random modulation,
public class RandomLoadBalance extends AbstractLoadBalance {
public static final String NAME = "random";
private final Random random = new Random();
@Override
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
int length = invokers.size(); // Number of invokers
int totalWeight = 0; // The sum of weights
boolean sameWeight = true; // Every invoker has the same weight?
for (int i = 0; i < length; i++) {
int weight = getWeight(invokers.get(i), invocation);
totalWeight += weight; // Sum
if (sameWeight && i > 0
&& weight != getWeight(invokers.get(i - 1), invocation)) {
sameWeight = false;
}
}
if (totalWeight > 0 && !sameWeight) {
// If (not every invoker has the same weight & at least one invoker's weight>0), select randomly based on totalWeight.
int offset = random.nextInt(totalWeight);
// Return a invoker based on the random value.
for (int i = 0; i < length; i++) {
offset -= getWeight(invokers.get(i), invocation);
if (offset < 0) {
return invokers.get(i);
}
}
}
// If all invokers have the same weight value or totalWeight=0, return evenly.
return invokers.get(random.nextInt(length));
}3.3: RoundRobinLoadBlance
Polling call. The process of polling call is mainly to maintain a LinkdesHashMap (sequential map) of local variables to store the corresponding relationship between the caller and the weight value, then traverse each caller, put the caller and the weight value greater than 0, and then accumulate the weight value. There is also a global variable map. To find the first service caller, first find the key value and method of each service. Here, it can be understood as the unique key identifying the first caller, and then guarantee the atomicity of + 1 (AtomicPositiveInteger is atomic) for its corresponding value. Then take the modular total weight of this value, and then set its weight value of - 1 each time, If you know that its modulus and total weight value are equal to 0, you can select the caller, which can be called "reduced weight modulus" (it is only a calculation level, not a real reduction of weight). Summary: polling calls are not simply called one after another. They are cycled according to the value of weight.
public class RoundRobinLoadBalance extends AbstractLoadBalance {
public static final String NAME = "roundrobin";
private final ConcurrentMap<String, AtomicPositiveInteger> sequences = new ConcurrentHashMap<String, AtomicPositiveInteger>();
@Override
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
int length = invokers.size(); // Number of invokers
int maxWeight = 0; // The maximum weight
int minWeight = Integer.MAX_VALUE; // The minimum weight
final LinkedHashMap<Invoker<T>, IntegerWrapper> invokerToWeightMap = new LinkedHashMap<Invoker<T>, IntegerWrapper>();
int weightSum = 0;
for (int i = 0; i < length; i++) {
int weight = getWeight(invokers.get(i), invocation);
maxWeight = Math.max(maxWeight, weight); // Choose the maximum weight
minWeight = Math.min(minWeight, weight); // Choose the minimum weight
if (weight > 0) {
invokerToWeightMap.put(invokers.get(i), new IntegerWrapper(weight));
weightSum += weight;
}
}
AtomicPositiveInteger sequence = sequences.get(key);
if (sequence == null) {
sequences.putIfAbsent(key, new AtomicPositiveInteger());
sequence = sequences.get(key);
}
int currentSequence = sequence.getAndIncrement();
if (maxWeight > 0 && minWeight < maxWeight) {
int mod = currentSequence % weightSum;
for (int i = 0; i < maxWeight; i++) {
for (Map.Entry<Invoker<T>, IntegerWrapper> each : invokerToWeightMap.entrySet()) {
final Invoker<T> k = each.getKey();
final IntegerWrapper v = each.getValue();
if (mod == 0 && v.getValue() > 0) {
return k;
}
if (v.getValue() > 0) {
v.decrement();
mod--;
}
}
}
}
// Round robin
return invokers.get(currentSequence % length);
}2.4: LeastActiveLoadBlance
Minimum active number calling usage: the main function of this method is to select the server according to the running state of the service provider. The main idea is to traverse each caller, and then obtain the running state of each server. If the current running state is less than the minimum state - 1, save it in the first position in leadindexes, It also determines that all callers have the same weight, and then directly returns the caller (the logic here is to find the least active number (the response at the code layer is the value of active)). If the calculated weight value is the same as the minimum weight value, save it in the leadindexes array and accumulate the weight values. If the current weight value is not equal to the initial value firstWeight, it is determined that not all callers have different weights. Then traverse lestIndexs, take the random number of the weighted cumulative value, generate the weight offset, subtract it, and return the caller when it is less than 0. If these do not match, randomly select an index from leastIndexs and return the caller!
public class LeastActiveLoadBalance extends AbstractLoadBalance {
public static final String NAME = "leastactive";
private final Random random = new Random();
@Override
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
int length = invokers.size(); // Number of invokers
int leastActive = -1; // The least active value of all invokers
int leastCount = 0; // The number of invokers having the same least active value (leastActive)
int[] leastIndexs = new int[length]; // The index of invokers having the same least active value (leastActive)
int totalWeight = 0; // The sum of weights
int firstWeight = 0; // Initial value, used for comparision
boolean sameWeight = true; // Every invoker has the same weight value?
for (int i = 0; i < length; i++) {
Invoker<T> invoker = invokers.get(i);
int active = RpcStatus.getStatus(invoker.getUrl(), invocation.getMethodName()).getActive(); // Active number
int weight = invoker.getUrl().getMethodParameter(invocation.getMethodName(), Constants.WEIGHT_KEY, Constants.DEFAULT_WEIGHT); // Weight
if (leastActive == -1 || active < leastActive) { // Restart, when find a invoker having smaller least active value.
leastActive = active; // Record the current least active value
leastCount = 1; // Reset leastCount, count again based on current leastCount
leastIndexs[0] = i; // Reset
totalWeight = weight; // Reset
firstWeight = weight; // Record the weight the first invoker
sameWeight = true; // Reset, every invoker has the same weight value?
} else if (active == leastActive) { // If current invoker's active value equals with leaseActive, then accumulating.
leastIndexs[leastCount++] = i; // Record index number of this invoker
totalWeight += weight; // Add this invoker's weight to totalWeight.
// If every invoker has the same weight?
if (sameWeight && i > 0
&& weight != firstWeight) {
sameWeight = false;
}
}
}
// assert(leastCount > 0)
if (leastCount == 1) {
// If we got exactly one invoker having the least active value, return this invoker directly.
return invokers.get(leastIndexs[0]);
}
if (!sameWeight && totalWeight > 0) {
// If (not every invoker has the same weight & at least one invoker's weight>0), select randomly based on totalWeight.
int offsetWeight = random.nextInt(totalWeight);
// Return a invoker based on the random value.
for (int i = 0; i < leastCount; i++) {
int leastIndex = leastIndexs[i];
offsetWeight -= getWeight(invokers.get(leastIndex), invocation);
if (offsetWeight <= 0)
return invokers.get(leastIndex);
}
}
// If all invokers have the same weight value or totalWeight=0, return evenly.
return invokers.get(leastIndexs[random.nextInt(leastCount)]);
}
}2.2.5: ConsistentHashLoadBalance
Consistency hash algorithm, doSelect method for selection. Consistent hash load balancing involves two main configuration parameters: hash Arguments and hash Nodes: when calling, the key is generated according to which parameters of the calling method, and the calling node is selected through the consistency hash algorithm according to the key. For example, call the method invoke(Strings1,Strings2); If hash If arguments is 1 (the default), only the parameter 1 (s1) of invoke is taken to generate hashCode.
hash.nodes: number of copies of the node.. The consistency hash of dubbo is implemented through the ConsistentHashLoadBalance class. ConsistentHashLoadBalance defines the ConsistentHashSelector class internally, and finally selects nodes through this class. The doSelect method implemented by ConsistentHashLoadBalance uses the created ConsistentHashSelector object to select nodes. The implementation of doSelect is as follows. When the method is called, it is created if the selector does not exist. Then select nodes through the select method of ConsistentHashSelector. ConsistentHashSelector will create replicaNumber virtual nodes inside the constructor and store these virtual nodes in TreeMap. Then generate the key according to the parameters of the calling method, and select a node in the TreeMap to call. In the above code, the hash(byte[]digest,intnumber) method is used to generate hashcode. This function converts the generated result into a long class because the generated result is a 32-bit number. If you save it with int, it may produce a negative number. The hashcode range of the logical ring generated by the consistency hash is 0-Max_ Between values. Therefore, it is a positive integer, so it should be cast to long type to avoid negative numbers. The node selection method is select. Finally, the node is selected through the sekectForKey method. During selection, if the hashcode is directly the same as the key of a virtual node, the node will be returned directly. If the hashcode falls on a node. If not, find the node corresponding to the smallest previous key.
public class ConsistentHashLoadBalance extends AbstractLoadBalance {
private final ConcurrentMap<String, ConsistentHashSelector<?>> selectors = new ConcurrentHashMap<String, ConsistentHashSelector<?>>();
@SuppressWarnings("unchecked")
@Override
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) {
String key = invokers.get(0).getUrl().getServiceKey() + "." + invocation.getMethodName();
int identityHashCode = System.identityHashCode(invokers);
ConsistentHashSelector<T> selector = (ConsistentHashSelector<T>) selectors.get(key);
//If the selector does not exist, create it
if (selector == null || selector.identityHashCode != identityHashCode) {
selectors.put(key, new ConsistentHashSelector<T>(invokers, invocation.getMethodName(), identityHashCode));
selector = (ConsistentHashSelector<T>) selectors.get(key);
}
return selector.select(invocation);
}
//Private inner class
private static final class ConsistentHashSelector<T> {
private final TreeMap<Long, Invoker<T>> virtualInvokers;
private final int replicaNumber;
private final int identityHashCode;
private final int[] argumentIndex;
ConsistentHashSelector(List<Invoker<T>> invokers, String methodName, int identityHashCode) {
this.virtualInvokers = new TreeMap<Long, Invoker<T>>();
this.identityHashCode = identityHashCode;
URL url = invokers.get(0).getUrl();
this.replicaNumber = url.getMethodParameter(methodName, "hash.nodes", 160);
String[] index = Constants.COMMA_SPLIT_PATTERN.split(url.getMethodParameter(methodName, "hash.arguments", "0"));
argumentIndex = new int[index.length];
for (int i = 0; i < index.length; i++) {
argumentIndex[i] = Integer.parseInt(index[i]);
}
for (Invoker<T> invoker : invokers) {
String address = invoker.getUrl().getAddress();
for (int i = 0; i < replicaNumber / 4; i++) {
byte[] digest = md5(address + i);
for (int h = 0; h < 4; h++) {
long m = hash(digest, h);
//Virtual caller
virtualInvokers.put(m, invoker);
}
}
}
}
//Select call
public Invoker<T> select(Invocation invocation) {
String key = toKey(invocation.getArguments());
byte[] digest = md5(key);
return selectForKey(hash(digest, 0));
}
//key value converted to service
private String toKey(Object[] args) {
StringBuilder buf = new StringBuilder();
for (int i : argumentIndex) {
if (i >= 0 && i < args.length) {
buf.append(args[i]);
}
}
return buf.toString();
}
//
private Invoker<T> selectForKey(long hash) {
//Search from TreeMap
Map.Entry<Long, Invoker<T>> entry = virtualInvokers.tailMap(hash, true).firstEntry();
if (entry == null) {
entry = virtualInvokers.firstEntry();
}
return entry.getValue();
}
//Calculate Hash value
private long hash(byte[] digest, int number) {
return (((long) (digest[3 + number * 4] & 0xFF) << 24)
| ((long) (digest[2 + number * 4] & 0xFF) << 16)
| ((long) (digest[1 + number * 4] & 0xFF) << 8)
| (digest[number * 4] & 0xFF))
& 0xFFFFFFFFL;
}
//md5 encryption
private byte[] md5(String value) {
MessageDigest md5;
try {
md5 = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException(e.getMessage(), e);
}
md5.reset();
byte[] bytes;
try {
bytes = value.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
throw new IllegalStateException(e.getMessage(), e);
}
md5.update(bytes);
return md5.digest();
}
}
}III. dubbo's default load balancing strategy
3.1: as can be seen from the @ SPI annotation, dubbo's default load balancing strategy is random call
3.2: how to change dubbo's load balancing strategy?
3.2.1: if it is a springboot project, directly Reference it in @ Reference and then indicate loadspace = "xx" Where xx is the value of name in each implementation class
3.2.2:xml configuration method
<dubbo:serviceinterface="..."loadbalance="roundrobin"/>
4: Summary
This blog describes dubbo's load balancing mechanism. It can be seen that in addition to the consistency Hash algorithm, others are calculated according to the weight. In practical distributed applications, understand how dubbo communicates with zookeeper, how to achieve load balancing, how to maintain the high availability of services, and the significance of load balancing for microservices, It will help us learn about distributed development.