Everyone has heard of the 10000 hour law, but is this really the case? After 10000 hours of CRUD, don't you only know CRUD? These days, you don't properly update your own technology stack. When you go out to chat and boast with others, you don't lose it (⊙ o ⊙)
Interview topics
What is distributed lock?
First, in order to ensure the availability of distributed locks, at least the following three conditions must be met
- Mutex. At any time, only one client can hold the lock
- No deadlock occurs. Even if one client runs away without actively unlocking while holding the lock, it can ensure that other subsequent clients can lock
- Whoever unties the bell must tie it. Locking and unlocking must be the same client. The client cannot unlock the lock added by others
How to implement distributed locking?
Both implementations are described below
- Using lua script to operate distributed locks
- The distributed lock is realized by using setnx and setex commands together
Distributed lock scenario
What is distributed lock?
- Distributed lock is a lock implementation that controls distributed systems or common access to shared resources between different systems
- If different systems or different hosts of the same system share a resource, mutual exclusion is often used to prevent mutual interference
Why distributed locks? [#](
)
It can ensure that in the distributed application cluster, the same method can only be executed by one thread on * * * * one machine.
Design requirements [#](
)
- Reentrant lock (avoid deadlock)
- Acquire and release locks are highly available
- Acquire and release locks high performance
Implementation scheme [#](
)
- To obtain the lock, use setnx(): SETNX key val: if and only if the key does not exist, set a string with key val and return 1
- If the key exists, nothing will be done, and [0] will be returned to lock. The value of the lock is the intranet IP number of the current lock server and the splicing task ID
- Judge when releasing the lock. And use the expire command to add a timeout time for the lock. If the timeout time is exceeded, the lock will be released automatically
- If 1 is returned, the lock is successfully obtained. It also sets a timeout time for acquiring. If it exceeds this time, the acquisition lock will be abandoned. The expiration of setex(key,value,expire) is in seconds
- When releasing the lock, judge whether it is the lock (that is, value is the intranet IP number of the current server and the splicing task ID). If it is the lock, execute delete to release the lock
Redis distributed lock implementation [#](
)
Create a SpringBoot project [#](
)
website: https://start.spring.io/
Step [#](
)
1. Annotate the startup class with @ enableshcheduling
2. Annotate the execution method @ Scheduled
Package and upload to Linux server to start [#](
)
Prepare three Linux servers, upload the jar packages to the three servers, and then start
Persistent startup mode of nohub [#](
)
nohup java -jar jar name &
Check whether all clusters in the cluster are started successfully
1,Install first lsof: yum install lsof 2,verification: lsof -i:8080
TCP triple handshake [#](
)
View local TCP connection status [#](
)
netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'Why three handshakes? [#](
)
It is mainly to prevent the invalid connection request message segment from being suddenly transmitted to B, so the message is disordered. It is assumed that the first connection request message segment sent by A is not lost, but stays in some network nodes for A long time, and does not arrive at B until A certain time after the connection is released. Originally, this is an already invalid message segment. However, after receiving the invalid connection request message segment, B mistakenly thinks that A sends A new connection request, so it sends A confirmation message segment to A and agrees to establish A connection.
Assuming that the triple handshake is not used, as long as B sends A confirmation, the new connection will be established. In this way, it has been waiting for A to send data, and many resources of B are wasted.
Illustration [#](
)
I shook hands three times. Why did I wave four times? [#](
)
Waving for the first time: the active closing party sends a fin to close the data transmission from the active party to the passive Closing Party, that is, the active Closing Party tells the passive closing party that I will no longer send you data (of course, if the data sent before the fin packet does not receive the corresponding ack confirmation message, the active closing party will still resend these data), but, At this time, the active shutdown party can also accept data.
Second wave: after receiving the FIN packet, the passive closing party sends an ACK to the other party, and the confirmation sequence number is the received sequence number + 1 (the same as SYN, one FIN occupies one sequence number).
The third wave: the passive closing party sends a FIN to close the data transmission from the passive closing party to the active Closing Party, that is, tell the active closing party that my data has been sent and will not send data to you again.
Fourth wave: after receiving FIN, the active closing party sends an ACK to the passive Closing Party, and confirms that the serial number is received serial number + 1. So far, four waves are completed.
Function [#](
)
Ensure complete data transmission
Redis distributed lock implementation source code explanation [#](
)
Graphic explanation [#](
)
Step [#](
)
- Distributed locks meet two conditions: one is the lock with effective time, and the other is high-performance unlocking
- redis commands setnx (set if not exist) and setex (set expire value) are used
- The unlocking process cannot be omitted, otherwise the task will never expire when it is executed once
- Put the locking code and task logic into the try catch code block, and the unlocking process into the finally code block
Project structure [#](
)
pom.xml[#](
)
<?xml version="1.0" encoding="UTF-8"?> 4.0.0 org.springframework.boot spring-boot-starter-parent 2.3.3.RELEASE
com.cyb yb-mobile-redis 0.0.1-SNAPSHOT yb-mobile-redis Demo project for Spring Boot application.properties[#](
)
spring.redis.database=0 spring.redis.host=192.168.199.142 spring.redis.port=6379 spring.redis.password=12345678 server.port=9001
RedisService.java[#](
)
package com.cyb.ybmobileredis.service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.redis.core.*; import org.springframework.stereotype.Service; import java.io.Serializable; import java.util.List; import java.util.Set; import java.util.concurrent.TimeUnit; /** * @ClassName: RedisService
* @Description: TODO
* @Author: chenyb
* @Date: 2020/8/16 5:39 afternoon
* @Versiion: 1.0 */ @Service public class RedisService { @Autowired private RedisTemplate redisTemplate; private static double size = Math.pow(2, 32); /** * Write cache
*
* @param key
* @param offset Bit 8Bit=1Byte
* @return */ public boolean setBit(String key, long offset, boolean isShow) { boolean result = false; try { ValueOperationsLockNxExJob.java[#](
)
package com.cyb.ybmobileredis.schedule; import com.cyb.ybmobileredis.service.RedisService; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Service; import java.net.Inet4Address; import java.net.InetAddress; import java.net.NetworkInterface; import java.util.Enumeration; /** * @ClassName: LockNxExJob
* @Description: Distributed lock acquisition and release
* @Author: chenyb
* @Date: 2020/8/16 5:44 afternoon
* @Versiion: 1.0 */ @Service public class LockNxExJob { private static final Logger logger = LoggerFactory.getLogger(LockNxExJob.class); @Autowired private RedisService redisService; @Autowired private RedisTemplate redisTemplate; private static String LOCK_PREFIX = "prefix_"; @Scheduled(fixedRate = 8000) public void lockJob() { String lock = LOCK_PREFIX + "LockNxExJob"; boolean nxRet=false; try{ //redistemplate setnx operation nxret = redistemplate opsForValue(). setIfAbsent(lock,getHostIp()); Object lockValue = redisService. get(lock); System. out. println(lockValue); // Failed to obtain lock if (! Nxret) {string value = (string) redisservice.get (lock); / / print the server IP logger.info (system. Currenttimemillis() + "get lock fail, lock long to: {}", value) currently occupying the lock; return; } Else {redistemplate. Opsforvalue(). Set (lock, gethostip(), 3600); / / successful lock acquisition logger.info (system. Currenttimemillis() + "start lock locknxexjob success"); thread.sleep (4000);}} catch (Exception e){ logger.error("lock error",e); } Finally {if (nxret) {system.out.println ("lock released successfully"); redisService.remove(lock);}}} / * ** Method for obtaining local intranet IP address
* @return */ private static String getHostIp(){ try{ Enumeration allNetInterfaces = NetworkInterface.getNetworkInterfaces(); while (allNetInterfaces.hasMoreElements()){ NetworkInterface netInterface = (NetworkInterface) allNetInterfaces.nextElement(); Enumeration addresses = netInterface.getInetAddresses(); while (addresses.hasMoreElements()){ InetAddress ip = (InetAddress) addresses.nextElement(); if (ip != null && ip instanceof Inet4Address && !ip.isLoopbackAddress() //Loopback address is the local address. The loopback range of IPv4 is 127.0 0.0 ~ 127.255. 255.255 && ip. getHostAddress(). indexOf(":")==-1){ return ip.getHostAddress(); } } } } catch(Exception e){ e.printStackTrace(); } return null; } }Verify [#](
)
Run one jar on Linux, one locally, one successfully obtains the lock, and the other fails to obtain the lock
Possible problems with Redis distributed locks [#](
)
We have implemented the distributed lock function with code above. Only one lock can be successfully obtained at the same time. It can be seen from the above figure that in extreme cases, after the first Server successfully obtains the lock, the service or Redis goes down, which will lead to the problem that the Redis lock cannot be released, and other servers always fail to obtain the lock.
Simulate server acquisition lock downtime [#](
)
First run the project and obtain the lock, immediately kill -9 process id and kill the current process. Then, when running the project, the console will always prompt that obtaining the lock failed.
Solutions (key) [#](
)
- If you execute one command at a time, this will not happen. Use Lua script
- Since 2.6, Redis supports the combination of setnx and setex
lua script [#](
)
- Add a suffix to the redource directory lua ending file
- Writing lua scripts
- key and arg passed in lua script
- Call redistemplate The execute method executes the script
Writing lua scripts
local lockKey = KEYS[1] local lockValue = KEYS[2] -- setnx info local result_1 = redis.call('SETNX',lockKey,lockValue) if result_1 == true then local result_2 = redis.call('SETEX',lockKey,3600,lockValue) return result_1 else return result_1 endEncapsulate and call lua script methods
last
Before the interview, you must brush the questions. In order to facilitate your review, I share a collection of personal interviews
- Java core knowledge sorting
Java core knowledge
- Spring family bucket (Practical Series)
- Other e-book materials
Step 3: Brush questions
Since it's an interview, I have to brush questions. In fact, I can't go anywhere after the Spring Festival. I brush a lot of interview questions myself, so I can know what knowledge points and high-frequency questions will be asked in the interview process. Therefore, brushing questions is a very important point in the early preparation of the interview.
The following is my private interview question bank:
Spring family bucket (Practical Series)
[external chain picture transferring... (IMG ovuoqrz8-1630306228601)]
- Other e-book materials
[external chain picture transferring... (img-FYm6fPfp-1630306228602)]
Step 3: Brush questions
Since it's an interview, I have to brush questions. In fact, I can't go anywhere after the Spring Festival. I brush a lot of interview questions myself, so I can know what knowledge points and high-frequency questions will be asked in the interview process. Therefore, brushing questions is a very important point in the early preparation of the interview.
The following is my private interview question bank:
[external chain picture transferring... (img-231cxDEZ-1630306228602)]