Overview of distributed locks
Before understanding distributed locks, we should first know why distributed locks are used. Let's first understand the following cache exceptions
Cache exception
Cache penetration
It refers to querying a certain nonexistent data. Because the cache does not hit, we will query the database, but the database does not have this record, and we are considering fault tolerance. We do not write the null of this query into the cache, which will cause the nonexistent data to be queried in the storage layer every time, losing the significance of the cache. When the traffic is heavy, the DB may hang up. If someone uses a nonexistent key to frequently attack our application, this is a vulnerability.
solve
Empty results are cached, but their expiration time will be short, up to five minutes.
Cache avalanche
Cache avalanche refers to an avalanche in which the same expiration time is used when we set the cache, resulting in the cache invalidation at the same time at a certain time, all requests are forwarded to the DB, and the DB is under excessive instantaneous pressure.
solve
Add a random value based on the original failure time, such as 1-5 minutes random, so that the repetition rate of the expiration time of each cache will be reduced, and it is difficult to cause collective failure events.
Buffer breakdown
For some keys with expiration time set, if these keys may be accessed at some time points, they are very "hot" data. At this time, we need to consider a problem: if the key fails just before a large number of requests come in at the same time, all data queries on the key will fall to db, which is called cache breakdown.
And cache avalanche:
Breakdown is a hot spot
Avalanches are a collective failure of many people
Install JMeter
To simulate the above cache exceptions, we need to install JMeter first. This is the software for stress testing, which can simulate the high concurrency state of data access
Download address: https://jmeter.apache.org/download_jmeter.cgi
First modify the configuration file to simplified Chinese
Click the jar package to execute the software
Analog buffer breakdown
First configure JMeter
Add thread group
Set concurrent requests: how many threads are opened in how much time, and how many requests each thread sends in a loop. The following is 100 * 100 = 10000 requests
Add HTTP Sampler: send HTTP request
Set request path
Configure view statistics
Service layer
@Service
public class SomeService {
@Autowired
RedisTemplate<String,Object> redisTemplate;
public String some(){
try {
//Amplification effect
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
String str = "";
ValueOperations<String, Object> opsForValue = redisTemplate.opsForValue();
if(!redisTemplate.hasKey("str")){
System.out.println("query data base");
str = "aaa";
opsForValue.set("str",str);
}else{
System.out.println("Query cache");
str = (String) opsForValue.get("str");
}
return str;
}
}
controller layer
@RestController
public class MyController {
@Autowired
SomeService someService;
@RequestMapping("/some")
public String some(){
return someService.some();
}
}
Test results:
Through console printing, we can see that we first query from the database many times, but we want to get information directly from the cache after only querying the database once. In this way, a large number of query requests will directly enter the database for query without caching. This is called cache breakdown
We can compare the following concepts of cache breakdown for understanding
For some keys with expiration time set, if these keys may be accessed at some time points, they are very "hot" data. At this time, we need to consider a problem: if the key fails just before a large number of requests come in at the same time, all data queries on the key will fall to db, which is called cache breakdown.
Review thread safety before introducing distributed locks
Review thread safety
Prerequisites for thread safety problems:
1. Multithreading
2. Shared data: member variables (local variables have no sharing problem, so there will be no problem. Each thread will open its own stack space)
3. Multiple statements modify shared variables
How to use synchronize
Method 1: synchronize code blocks
synchronized(Synchronization monitor){
Code to be synchronized (code to operate shared data)
}
explain:
- The code that operates shared data is the code that needs to be synchronized.
- Synchronization monitor: commonly known as lock, any object can become a lock.
- Requirement: multiple threads must share the same lock.
Add: in the way of implementing the Runnable interface to create multithreading, we can consider using this as the synchronization monitor. (it is not necessary to analyze specific problems, such as the following inheritance methods)
Note: in the way of inheriting Thread class to create multithreads, use this carefully as the synchronization monitor. You can consider the current class as the synchronization monitor.
Whoever uses it must ensure that it is unique. Multiple threads use unique objects
Note: when using synchronized to package code blocks, you cannot package more or less (if you package while(true) in the following code, it will become a single thread and will be unlocked only after printing in a window)
/**
* @author acoffee
* @create 2020-09-24 21:25
*/
public class WindowTest1 {
public static void main(String[] args) {
Window1 w = new Window1();
Thread t1 = new Thread(w);
Thread t2 = new Thread(w);
Thread t3 = new Thread(w);
t1.setName("Window 1");
t2.setName("Window 2");
t3.setName("Window 3");
t1.start();
t2.start();
t3.start();
}
}
class Window1 implements Runnable{
private int ticket = 100;//There is no need to add static here, because only one window object is created here
Object obj = new Object();
@Override
public void run() {
while (true){
//synchronized(this) {is also correct. It only represents w
synchronized(obj){
if(ticket > 0){
try{//We add blocking conditions here
Thread.sleep(100);
}catch(InterruptedException e){
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName()+":"+ticket);
ticket--;
}else{
break;
}
}
}
}
}
Execution results:
If we set Object obj = new Object(); If it is placed in the run method, synchronization cannot be realized, because the last thread has implemented the run method, and three locks are created, which is the same as not creating locks.
Use synchronous code blocks to solve Thread safety problems in the way of inheriting Thread class
/**
* Use synchronous code blocks to solve Thread safety problems in the way of inheriting Thread class
* @author acoffee
* @create 2020-09-25 18:35
*/
class window extends Thread{
private static int ticket = 100;
private static Object obj = new Object();
@Override
public void run() {
while (true){
//synchronized(this) {wrong way: This represents T1, T2 and T3 objects
//synchronized(window.class) {/ / is also correct. In fact, the class is also an object
//Equivalent to Class C1 = window class window. Class will be loaded only once
synchronized (obj) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (ticket > 0) {
System.out.println(getName() + ":Ticket number:" + ticket);
ticket--;
} else {
break;
}
}
}
}
}
public class WindowTest {
public static void main(String[] args) {
window w1 = new window();
window w2 = new window();
window w3 = new window();
w1.setName("Window 1");
w2.setName("Window 2");
w3.setName("Window 3");
w1.start();
w2.start();
w3.start();
}
}
Mode 2: synchronization method
If the code that operates on shared data is completely declared in a method, we might as well declare this method synchronous.
Using the synchronization method to solve the thread safety problem of time limited Runnable interface
/**
* Using the synchronization method to solve the thread safety problem of time limited Runnable interface
* @author acoffee
* @create 2020-09-25 19:14
*/
public class WindowTest2 {
public static void main(String[] args) {
Window2 w = new Window2();
Thread t1 = new Thread(w);
Thread t2 = new Thread(w);
Thread t3 = new Thread(w);
t1.setName("Window 1");
t2.setName("Window 2");
t3.setName("Window 3");
t1.start();
t2.start();
t3.start();
}
}
class Window2 implements Runnable{
private int ticket = 100;
@Override
public void run() {
while (true){
show();
}
}
public synchronized void show(){//Sync monitor: this
if(ticket > 0){
try{//We add blocking conditions here
Thread.sleep(100);
}catch(InterruptedException e){
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName()+":"+ticket);
ticket--;
}
}
}
Mode 3: Lock lock
import java.util.concurrent.locks.ReentrantLock;
/**
* Three ways to solve thread safety problems: Lock → jdk5 0 NEW
* @author acoffee
* @create 2020-09-26 10:40
*/
class Window implements Runnable{
private int ticket = 100;
//Instantiate ReentrantLock
private ReentrantLock lock = new ReentrantLock(true);
@Override
public void run() {
while (true){
try {
//2. Call lock()
lock.lock();
if(ticket > 0){
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName()+":Ticket No.:"+ticket);
ticket--;
}else{
break;
}
}finally {
//3. Call unlocking method: unlock()
lock.unlock();
}
}
}
}
public class LockTest {
public static void main(String[] args) {
Window w1 = new Window();
Thread t1 = new Thread(w1);
Thread t2 = new Thread(w1);
Thread t3 = new Thread(w1);
t1.setName("Window 1");
t2.setName("Window 2");
t3.setName("Window 3");
t1.start();
t2.start();
t3.start();
}
}
Use the synchronization method to solve the Thread safety problem of inheriting the Thread class
/**
* Use the synchronization method to solve the Thread safety problem of inheriting the Thread class
*
* @author acoffee
* @create 2020-09-25 19:27
*/
class window3 extends Thread {
private static int ticket = 100;
@Override
public void run() {
while (true) {
show();
}
}
public static synchronized void show() {//Synchronization monitor: windows 3 class
//public synchronized void show() {synchronization monitor: w1, w2,w3
if (ticket > 0) {
try {//We add blocking conditions here
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(Thread.currentThread().getName() + ":Ticket number:" + ticket);
ticket--;
}
}
}
public class WindowTest3 {
public static void main(String[] args) {
window3 w1 = new window3();
window3 w2 = new window3();
window3 w3 = new window3();
w1.setName("Window 1");
w2.setName("Window 2");
w3.setName("Window 3");
w1.start();
w2.start();
w3.start();
}
Summary:
- The synchronization method still involves the synchronization monitor, but we don't need to declare it explicitly
- Non static synchronization method. The synchronization monitor is this
- Synchronization method of static method, synchronization monitor: the current class itself
- To solve the problem of inheriting the Thread class, the Lock should also be declared with static to ensure that the same Lock is used
Interview question: what are the similarities and differences between synchronized and Lock?
Same: both can solve thread safety problems
Different:
- The synchronized mechanism automatically releases the synchronization monitor after executing the corresponding synchronization code
- Lock needs to manually start synchronization (lock()) and end synchronization (unlock())
Interview question: how to solve thread safety problems? How many ways? - Method 1: synchronize code blocks
- Mode 2: synchronization method
- Mode 3: Lock lock
Simulate thread safety issues (addition)
controller layer
@RequestMapping("/incr")
public String incr(){
someService.incr();
return "ok";
}
Service layer
@Service
public class SomeService {
@Autowired
RedisTemplate<String,Object> redisTemplate;
public void incr(){
redisTemplate.opsForValue().increment("num");
}
}
Execution results:
Let's now simulate the cluster
Configure nginx. For convenience, use Windows nginx
Copy a copy of the springboot program
Access port 80 of nginx
results of enforcement
Can we find or 1000
Redis instructions are executed in a single thread, which can ensure atomicity: the execution process of an instruction in redis will not be interrupted
The data here is still processed by redis, so there will be no thread problem. Next, I will process the data myself.
Change Service layer
public void incr(){
ValueOperations<String, Object> opsForValue = redisTemplate.opsForValue();
Object num = opsForValue.get("num");
if(num == null){
opsForValue.set("num","1");
}else{
int count = Integer.parseInt(num.toString());
count++;
opsForValue.set("num",count+"");
}
}
Execution results:
We can now see that the value is only 406, far less than 1000, which indicates that there is a thread safety problem.
Now we can add the synchronized lock we knew before to see if it can solve the thread safety problem
public synchronized void incr(){
ValueOperations<String, Object> opsForValue = redisTemplate.opsForValue();
Object num = opsForValue.get("num");
if(num == null){
opsForValue.set("num","1");
}else{
int count = Integer.parseInt(num.toString());
count++;
opsForValue.set("num",count+"");
}
}
Execution results:
We can still see that the result is less than 1000, but larger than the original (because each process is locked), so the synchronized synchronization lock can not solve the thread safety problem of cluster distributed. The reasons are as follows:
synchronized and lock locks:
The lock here is used for synchronization in the same process because multiple threads jointly access a shared resource. Its prerequisite is memory sharing in the same process. It is an inter process lock and cannot be used across processes, because here we use the two processes used for Nginx load balancing. A single entity can be locked, but cluster distributed synchronized does not work
Distributed lock principle
The underlying layer is implemented by Redis: whether the data can be operated needs to get the lock and obtain the lock in Redis
How to simulate locks in redis: store data into redis. The data already exists, saying someone is operating; You just wait until you can successfully store the data
setnx: save. Only one program can grab the lock
Set expiration time: the current application is locked. If it goes down, it can be unlocked normally
Unlocking can only unlock one's own lock, but not another's lock: thread id
Delete lock after execution: judge + delete. The execution of two instructions may be interrupted, and other people's locks may be deleted. Solution: Lua script: multiple Redis instructions can be written into a group, sent at one time and executed at the same time to ensure atomicity
Redisson
Redisson is the Java version of Redis client officially recommended by Redis. It provides many functions and is very powerful. Here we only use its distributed lock function.
Import dependency
<!--Redisson rely on-->
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson</artifactId>
<version>3.12.0</version>
</dependency>
Configuration class
@Configuration
public class RedisConfigruation {
@Bean
public RedissonClient redissonClient(){
Config config = new Config();
config.useSingleServer().setAddress("redis://192.168.195.157:6379");
config.useSingleServer().setPassword("123");
return Redisson.create(config);
}
}
The business layer uses distributed locks
public synchronized void incr(){
//Get the lock
RLock mylock = redissonClient.getLock("mylock");
//Start program lock
mylock.lock();
ValueOperations<String, Object> opsForValue = redisTemplate.opsForValue();
Object num = opsForValue.get("num");
if(num == null){
opsForValue.set("num","1");
}else{
int count = Integer.parseInt(num.toString());
count++;
opsForValue.set("num",count+"");
}
//Program end release lock
mylock.unlock();
}
Execution results:
We can see that the result of adding distributed lock is 1000, so the thread safety problem is solved.