The main functions of load balancing are as follows:
High concurrency: load balancing adjusts the load through the algorithm and tries to evenly distribute the workload of each node in the application cluster, so as to improve the concurrent processing capacity (throughput) of the application cluster.
Scalability: add or reduce the number of servers, and then the distribution is controlled by load balancing. This makes the application cluster scalable.
High availability: the load balancer can monitor candidate servers. When the server is unavailable, it will automatically skip and distribute the request to the available servers. This makes the application cluster highly available.
Security protection: some load balancing software or hardware provide security functions, such as black-and-white list processing, firewall, anti DDos attack, etc.
Load balancing algorithm
1，Random algorithm( Random)
private static final Random random = ThreadLocalRandom.current();
protected static MyNode doSelect(List<MyNode> nodes, String ip) {
int length = nodes.size();
AtomicInteger totalWeight = new AtomicInteger(0);
System.out.println("begin:"+totalWeight);
for (MyNode node : nodes) {
Integer weight = node.getWeight();
totalWeight.getAndAdd(weight);
}
System.out.println("totalWeight:"+totalWeight);
if (totalWeight.get() > 0) {
int offset = random.nextInt(totalWeight.get());
System.out.println("offset:"+offset);
for (MyNode node : nodes) {
System.out.println("node:"+node);
// Let the random value offset subtract the weight value
offset -= node.getWeight();
if (offset < 0) {
// Return the corresponding Node
return node;
}
}
}
// Direct random return
return nodes.get(random.nextInt(length));
}
2，Implementation of polling load balancing algorithm @@@The scheme does not do atomic operations
private final AtomicInteger position = new AtomicInteger(0);
@Override
protected N doSelect(List<N> nodes, String ip) {
int length = nodes.size();
// If the position value is already equal to the number of nodes, reset to 0
position.compareAndSet(length, 0);
N node = nodes.get(position.get());
position.getAndIncrement();
return node;
}
Thread t1 = new Thread(){
public void run(){
MyNode node = doSelect2(nodes,"");
System.out.println(node);
}
};
Thread t2 = new Thread(){
public void run(){
MyNode node = doSelect2(nodes,"");
System.out.println("t2:"+node);
}
};
t1.start();
t2.start();
System.out.println("start");
3. Other algorithms
Weighted Round Robin
Minimum active number
Weighted Source Hash
The source address hash (IP Hash) algorithm obtains a value through hash calculation according to the request source IP. The value is used for modular operation in the list of candidate servers, and the result is the selected server.
Consistent Hashing
The mainstream products of hardware load balancing are F5 and A10.
Mainstream software load balancing products include Nginx, HAProxy and LVS.
Nginx and HAProxy can be used as four or seven layer load balancers.
LVS can be used as a four layer load balancer. Its load balancing performance is better than Nginx.
Network communication classification
1) Seven layer load balancing: the request can be forwarded to a specific host according to the HTTP request header and URL information of the access user.
DNS redirection
HTTP redirection
Reverse proxy
2) Four layer load balancing: forwarding requests based on IP address and port.
Modify IP address
Modify MAC address
DNS is the domain name resolution service
DNS, the domain name resolution service, is the OSI layer 7 network protocol. DNS is designed as a distributed application with tree structure. From top to bottom, it is: root domain name server, primary domain name server, secondary domain name server, Local domain name server. Obviously, if all data is stored in the root domain name server, the load and overhead of DNS query will be very huge.
Therefore, compared with the DNS hierarchy, DNS query is a reverse recursive process. DNS clients request the local DNS server, the upper DNS server, the upper DNS server, The root DNS server (also known as authoritative DNS server) returns immediately once hit. In order to reduce the number of queries, DNS query cache will be set for each level of DNS server.
The working principle of DNS load balancing is to return the IP addresses of different servers according to the load based on DNS query cache.
Advantages of DNS redirection:
Simple to use: the load balancing work is handed over to the DNS server, which saves the trouble of maintaining the load balancing server
Improve performance: it can support address based domain name resolution and resolve it to the server address closest to the user (similar to the principle of CDN), which can speed up access speed and improve performance;
Disadvantages of DNS redirection:
Poor availability: DNS resolution is a multi-level resolution. After adding / modifying DNS, the resolution time is long; In the process of parsing, users will fail to visit the website;
Low scalability: the control of DNS load balancing is in the domain name provider, so it is impossible to improve and expand it;
Poor maintainability: it can not reflect the current running state of the server; Few supported algorithms; The difference between servers cannot be distinguished (the load cannot be judged according to the state of the system and service).
The mainstream product of reverse proxy service: Nginx Apache.
Reverse proxy: it occurs on the server side, and the user does not know the existence of the proxy.
Forward proxy: it occurs on the client and is initiated by the user
By actively accessing the proxy server, the client allows the proxy server to obtain the required external network data, and then forward it back to the client
Fortunately, the official version of redis is 2.6 12 the start SET command itself already contains the function of setting the expiration time.
If the timeout setting fails, release the lock to avoid deadlock
1. Cache set invalidation
Solution: expiration time = base time + random time
2. Cache penetration
Scheme 1: when querying the DB, if the data does not exist, preheat a special null value into the cache. In this way, subsequent queries will hit the cache, but special values need to be parsed.
Scheme 2: construct a BloomFilter to initialize the full amount of data. When receiving the request, judge whether the key exists in the BloomFilter. If it does not exist, just return it directly without querying the cache and DB
3. Cache large Key
Set a reasonable expiration time for large keys and try not to eliminate those large keys
The granularity of the cache should not be too large. If it is too large, it will easily lead to network congestion; It cannot be too small. If it is too small, the query frequency will be very high, and each request must be queried multiple times.
4. Cache data consistency
Scheme 1: when the cache update fails, retry. If the retry fails, write the failed key to the MQ message queue, and compensate the cache through asynchronous tasks to ensure data consistency.
Scheme 2: set a short expiration time. After the cache expires, the cache will reload the latest data through self-healing
5. Data concurrency contention preheating
Once the data in the cache expires or is deleted for some reason, resulting in empty data in the cache, a large number of concurrent thread requests (querying the same key) will query the database concurrently, and the pressure on the database will increase sharply
Scheme:
Introduce a global lock. When the cache misses, try to obtain the global lock first. If you get the lock, you are qualified to query the DB and preheat the data to the cache. Although the client side initiates a lot of requests, it can only wait because it can't get the lock. It sleeps for 500ms and tries three times. When the data in the cache is preheated successfully, it can be obtained from the cache