This article was originally published on my blog: https://ziyang.moe/article/mydb7.html
The code involved in this chapter is https://github.com/CN-GuoZiyang/MYDB/tree/master/src/main/java/top/guoziyang/mydb/backend/vm Yes.
preface
This section will conclude the VM layer, introduce the possible version jump caused by MVCC, and how MYDB can avoid the deadlock caused by 2PL and integrate it into Version Manager.
Version jump problem
Before the version jump, by the way, the implementation of MVCC makes it easy for MYDB to undo or rollback a transaction: just mark the transaction as aborted. According to the visibility mentioned in the previous chapter, each transaction can only see the data generated by other committed transactions. The data generated by an aborted transaction will not have any impact on other transactions, which is equivalent to that the transaction has never existed.
For version hopping, consider the following situations, assuming that X initially has only x0 version, and T1 and T2 are repeatable isolation levels:
T1 begin T2 begin R1(X) // T1 read x0 R2(X) // T2 read x0 U1(X) // T1 updates X to x1 T1 commit U2(X) // T2 update X to x2 T2 commit
This situation actually works well, but it is not logically correct. T1 updates x from x0 to x1, which is correct. However, T2 updates x from x0 to x2, skipping the X1 version.
Read submission allows version skipping, while repeatable reading does not allow version skipping. The idea to solve the version jump is also very simple: if Ti needs to modify x, and X has been modified by the transaction Tj invisible to Ti, Ti is required to roll back.
As summarized in the previous section, there are two situations for Tj where Ti is invisible:
- XID(Tj) > XID(Ti)
- Tj in SP(Ti)
Therefore, the version jump check is very simple. Take out the latest committed version of data X to be modified, and check whether the creator of the latest version is visible to the current transaction:
public static boolean isVersionSkip(TransactionManager tm, Transaction t, Entry e) {
long xmax = e.getXmax();
if(t.level == 0) {
return false;
} else {
return tm.isCommitted(xmax) && (xmax > t.xid || t.isInSnapshot(xmax));
}
}
Deadlock detection
The previous section mentioned that 2PL blocks transactions until the thread holding the lock releases the lock. This waiting relationship can be abstracted as a directed edge. For example, Tj is waiting for Ti, which can be expressed as Tj -- > ti. In this way, countless directed edges can form a graph (not necessarily a connected graph). It is simple to detect deadlock. You only need to check whether there are rings in the graph.
MYDB uses a LockTable object to maintain this graph in memory. The maintenance structure is as follows:
public class LockTable {
private Map<Long, List<Long>> x2u; // UID list of resources that a XID has obtained
private Map<Long, Long> u2x; // UID is held by a XID
private Map<Long, List<Long>> wait; // Waiting for XID list of UID s
private Map<Long, Lock> waitLock; // Waiting for lock on XID of resource
private Map<Long, Long> waitU; // XID UID waiting
private Lock lock;
...
}
Every time there is a waiting situation, try to add an edge to the graph and perform deadlock detection. If a deadlock is detected, the edge is revoked, adding is not allowed, and the transaction is revoked.
// If there is no need to wait, null is returned; otherwise, the lock object is returned
// An exception is thrown if it causes a deadlock
public Lock add(long xid, long uid) throws Exception {
lock.lock();
try {
if(isInList(x2u, xid, uid)) {
return null;
}
if(!u2x.containsKey(uid)) {
u2x.put(uid, xid);
putIntoList(x2u, xid, uid);
return null;
}
waitU.put(xid, uid);
putIntoList(wait, xid, uid);
if(hasDeadLock()) {
waitU.remove(xid);
removeFromList(wait, uid, xid);
throw Error.DeadlockException;
}
Lock l = new ReentrantLock();
l.lock();
waitLock.put(xid, l);
return l;
} finally {
lock.unlock();
}
}
Call add. If you need to wait, a locked Lock object will be returned. When the caller obtains the object, it needs to try to obtain the Lock of the object, so as to block the thread, for example:
Lock l = lt.add(xid, uid);
if(l != null) {
l.lock(); // Blocking at this step
l.unlock();
}
The algorithm for finding whether there is a ring in a graph is also very simple. It is a deep search. Just note that this graph is not necessarily a connected graph. The idea is to set an access stamp for each node, initialize it to - 1, then traverse all nodes, take each non-1 node as the root for deep search, and set all nodes encountered in the deep search connected graph to the same number, and the numbers of different connected graphs are different. In this way, if a node traversed before is encountered when traversing a graph, it indicates that a ring appears.
The implementation is simple:
private boolean hasDeadLock() {
xidStamp = new HashMap<>();
stamp = 1;
for(long xid : x2u.keySet()) {
Integer s = xidStamp.get(xid);
if(s != null && s > 0) {
continue;
}
stamp ++;
if(dfs(xid)) {
return true;
}
}
return false;
}
private boolean dfs(long xid) {
Integer stp = xidStamp.get(xid);
if(stp != null && stp == stamp) {
return true;
}
if(stp != null && stp < stamp) {
return false;
}
xidStamp.put(xid, stamp);
Long uid = waitU.get(xid);
if(uid == null) return false;
Long x = u2x.get(uid);
assert x != null;
return dfs(x);
}
When a transaction commit s or abort s, it can release all the locks it holds and delete itself from the wait graph.
public void remove(long xid) {
lock.lock();
try {
List<Long> l = x2u.get(xid);
if(l != null) {
while(l.size() > 0) {
Long uid = l.remove(0);
selectNewXID(uid);
}
}
waitU.remove(xid);
x2u.remove(xid);
waitLock.remove(xid);
} finally {
lock.unlock();
}
}
The while loop releases the locks of all resources held by this thread. These resources can be obtained by the waiting thread:
// Select a xid from the waiting queue to occupy the uid
private void selectNewXID(long uid) {
u2x.remove(uid);
List<Long> l = wait.get(uid);
if(l == null) return;
assert l.size() > 0;
while(l.size() > 0) {
long xid = l.remove(0);
if(!waitLock.containsKey(xid)) {
continue;
} else {
u2x.put(uid, xid);
Lock lo = waitLock.remove(xid);
waitU.remove(xid);
lo.unlock();
break;
}
}
if(l.size() == 0) wait.remove(uid);
}
Trying to unlock from the beginning of the List is still a fair Lock. When unlocking, unlock the Lock object, so that the business thread can obtain the Lock and continue to execute.
Implementation of VM
The VM layer provides functions to the upper layer through the VersionManager interface, as follows:
public interface VersionManager {
byte[] read(long xid, long uid) throws Exception;
long insert(long xid, byte[] data) throws Exception;
boolean delete(long xid, long uid) throws Exception;
long begin(int level);
void commit(long xid) throws Exception;
void abort(long xid);
}
At the same time, the implementation class of VM is also designed as the cache of Entry, which needs to inherit abstractcache < Entry >. The methods to get to and release from the cache are simple:
@Override
protected Entry getForCache(long uid) throws Exception {
Entry entry = Entry.loadEntry(this, uid);
if(entry == null) {
throw Error.NullEntryException;
}
return entry;
}
@Override
protected void releaseForCache(Entry entry) {
entry.remove();
}
begin() starts a transaction, initializes the structure of the transaction, and stores it in activeTransaction for checking and snapshot usage:
@Override
public long begin(int level) {
lock.lock();
try {
long xid = tm.begin();
Transaction t = Transaction.newTransaction(xid, level, activeTransaction);
activeTransaction.put(xid, t);
return xid;
} finally {
lock.unlock();
}
}
The commit() method commits a transaction, mainly free ing the relevant structure, releasing the held lock, and modifying the TM state:
@Override
public void commit(long xid) throws Exception {
lock.lock();
Transaction t = activeTransaction.get(xid);
lock.unlock();
try {
if(t.err != null) {
throw t.err;
}
} catch(NullPointerException n) {
System.out.println(xid);
System.out.println(activeTransaction.keySet());
Panic.panic(n);
}
lock.lock();
activeTransaction.remove(xid);
lock.unlock();
lt.remove(xid);
tm.commit(xid);
}
There are two methods for abort transactions, manual and automatic. Manual refers to calling abort() method, while automatic refers to automatically canceling and rolling back the transaction when a deadlock is detected in the transaction; Or when a version jump occurs, it will also be rolled back automatically:
private void internAbort(long xid, boolean autoAborted) {
lock.lock();
Transaction t = activeTransaction.get(xid);
if(!autoAborted) {
activeTransaction.remove(xid);
}
lock.unlock();
if(t.autoAborted) return;
lt.remove(xid);
tm.abort(xid);
}
The read() method reads an entry. Pay attention to the following visibility:
@Override
public byte[] read(long xid, long uid) throws Exception {
lock.lock();
Transaction t = activeTransaction.get(xid);
lock.unlock();
if(t.err != null) {
throw t.err;
}
Entry entry = super.get(uid);
try {
if(Visibility.isVisible(tm, t, entry)) {
return entry.data();
} else {
return null;
}
} finally {
entry.release();
}
}
insert() is to wrap the data into an Entry and hand it over to DM for insertion:
@Override
public long insert(long xid, byte[] data) throws Exception {
lock.lock();
Transaction t = activeTransaction.get(xid);
lock.unlock();
if(t.err != null) {
throw t.err;
}
byte[] raw = Entry.wrapEntryRaw(xid, data);
return dm.insert(xid, raw);
}
The delete() method looks slightly more complex:
@Override
public boolean delete(long xid, long uid) throws Exception {
lock.lock();
Transaction t = activeTransaction.get(xid);
lock.unlock();
if(t.err != null) {
throw t.err;
}
Entry entry = super.get(uid);
try {
if(!Visibility.isVisible(tm, t, entry)) {
return false;
}
Lock l = null;
try {
l = lt.add(xid, uid);
} catch(Exception e) {
t.err = Error.ConcurrentUpdateException;
internAbort(xid, true);
t.autoAborted = true;
throw t.err;
}
if(l != null) {
l.lock();
l.unlock();
}
if(entry.getXmax() == xid) {
return false;
}
if(Visibility.isVersionSkip(tm, t, entry)) {
t.err = Error.ConcurrentUpdateException;
internAbort(xid, true);
t.autoAborted = true;
throw t.err;
}
entry.setXmax(xid);
return true;
} finally {
entry.release();
}
}
In fact, there are three things ahead: first, visibility judgment, second, obtaining the lock of resources, and third, version jump judgment. The delete operation has only one setting, XMAX.
Today is December 24, 2021, Christmas Eve.
May you have a brilliant future
May your lovers get married
May you be happy on earth
I just want to face the sea and flowers bloom in spring