Concurrent programming - Condition implementation of blocking queue and source code analysis

Posted by mordeith on Mon, 01 Nov 2021 13:09:18 +0100

preface

In the last article, we used wait & notify Implement blocking queue Implemented a blocking queue. Today, let's see how to use the Condition object of ReentrantLock to implement the blocking queue

practice

public static void main(String[] args) {
        ConditionQueue conditionQueue = new ConditionQueue(5);
        for (int i = 0; i < 10; i++) {
            final int a = i;
            new Thread(() -> {
                try {
                    conditionQueue.put(a);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }).start();
        }
        for (int i = 0; i < 5; i++) {
            new Thread(() -> {
                try {
                    conditionQueue.get();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }).start();
        }

    }
    static class ConditionQueue{

        private LinkedList<Integer> queue = new LinkedList<>();

        private final Lock lock = new ReentrantLock();

        //producer
        private final Condition producer = lock.newCondition();

        //consumer
        private final Condition consumer = lock.newCondition();

        //Maximum capacity
        private int max;

        ConditionQueue(int max){
            this.max = max;
        }

        boolean isFull(){
            return queue.size() == max;
        }

        boolean isEmpty(){
            return queue.size() == 0;
        }

        void put(int i) throws InterruptedException {
            lock.lock();
            while (isFull()){
                producer.await();
            }
            queue.add(i);
            consumer.signalAll();
            System.out.println("insert data" + i);
            lock.unlock();
        }

        Integer get() throws InterruptedException {
            lock.lock();
            while (isEmpty()){
                consumer.await();
            }
            Integer s = queue.removeFirst();
            producer.signalAll();
            lock.unlock();
            System.out.println("Fetch data" + s);
            return s;
        }
    }

So how does condition implement blocking?
We used the await method and the signall method above
Let's see what these two methods do?

Source code analysis

After entering the source code, we can find that lock.newCondition() actually creates a new ConditionObject, which is an internal class of AbstractQueuedSynchronizer. ConditionObject itself maintains a waiting queue (this is different from the CLH queue of ReentrantLock)

await method

First, let's look at the await method

 public final void await() throws InterruptedException {
            //Determine whether the thread is interrupted
            if (Thread.interrupted())
                throw new InterruptedException();
            //Add the current node to the waiting queue of condition
            Node node = addConditionWaiter();
            //Release the lock occupied by the current thread
            int savedState = fullyRelease(node);
            int interruptMode = 0;
            //Judge whether the node is in the waiting queue. The waiting queue here refers to the CLH queue
            //Note that the thread will not be in the waiting queue at first. It may be in the CLH queue only after the thread is awakened
            while (!isOnSyncQueue(node)) {
                //If you are not in the queue, suspend yourself directly
                LockSupport.park(this);
                //Here is to see if the thread has been interrupted. There are three cases
                if ((interruptMode = checkInterruptWhileWaiting(node)) != 0)
                    break;
            }
            //Then start competing for locks
            if (acquireQueued(node, savedState) && interruptMode != THROW_IE)
                interruptMode = REINTERRUPT;
            if (node.nextWaiter != null) // clean up if cancelled
                //Clear all nodes whose status is not equal to condition
                unlinkCancelledWaiters();
            if (interruptMode != 0)
                //Here is to see if the thread has been interrupted, compensate the current thread or throw an exception
                reportInterruptAfterWait(interruptMode);
        }

Look at the addConditionWaiter method added to the waiting queue. This method actually puts a NODE in CONDITION status into the waiting queue. Note that the NODE here is the same object as the NODE in ReentrantLock

 private Node addConditionWaiter() {
            //The last node in the queue
            //This node object is the same as that in the previous CLH queue
            Node t = lastWaiter;
            if (t != null && t.waitStatus != Node.CONDITION) {
                //Clear all nodes whose status is not equal to condition
                unlinkCancelledWaiters();
                //Reassign the t node
                t = lastWaiter;
            }
            //Create node
            Node node = new Node(Thread.currentThread(), Node.CONDITION);
            //Put the node at the end of the queue
            if (t == null)
                firstWaiter = node;
            else
                t.nextWaiter = node;
            lastWaiter = node;
            return node;
        }

Then look at the isOnSyncQueue method, which determines whether the current node is in the CLH queue

final boolean isOnSyncQueue(Node node) {
        //If the node status is condition or node is the first node
        if (node.waitStatus == Node.CONDITION || node.prev == null)
            return false;
        //next is not equal to null, indicating that it must be in the waiting queue
        if (node.next != null) // If has successor, it must be on queue
            return true;
        /*
         * node.prev can be non-null, but not yet on queue because
         * the CAS to place it on queue can fail. So we have to
         * traverse from tail to make sure it actually made it.  It
         * will always be near the tail in calls to this method, and
         * unless the CAS failed (which is unlikely), it will be
         * there, so we hardly ever traverse much.
         */
        //Here is a thorough. Traverse the CLH queue from the back to the front to see if you can find the current node
        return findNodeFromTail(node);
    }

The checkInterruptWhileWaiting method determines whether a thread has been interrupted when it is awakened

private int checkInterruptWhileWaiting(Node node) {
            //Note that if the signal is received, the status of waitStatus is 0

            //1. It has not been interrupted and directly returns 0
            //2. The transferaftercanceledwait is interrupted before being signal ed and returns true
            //3. Transferaftercanceledwait interrupted after signal returns false
            return Thread.interrupted() ?
                (transferAfterCancelledWait(node) ? THROW_IE : REINTERRUPT) :
                0;
        }

Then the acquirequeueueueued method, which we should be familiar with, is the competitive lock method of ReentranLock. It will not be carried out here. Interested students can take a look at the previous issue reentrantLcok lock unlock source code

await method process summary

When the thread executes the await method, it will add itself to the waiting queue of the conditionObject, and then release its lock resources,
To judge whether you are in the CLH queue, there are two cases:
1. It is not in the CLH queue. Note that threads that are not signal ed will not be in the CLH queue. At this time, they will directly park themselves and wait to be unpark ed
2. In the CLH queue, it indicates that it has been signal ed at this time, and then enter the next step to enter the process of competing for lock lock

Then let's take a look at how signal puts nodes into CLH

Analysis of signal method

signal method

public final void signal() {
            //Determine whether the thread with the lock is operating
            if (!isHeldExclusively())
                throw new IllegalMonitorStateException();
            //First node of the queue
            Node first = firstWaiter;
            if (first != null)
                doSignal(first);
        }

Then look at the doSignal method. If it is the signalAll method, it is actually a while loop through the entire queue and call the transferForSignal method for all nodes

private void doSignal(Node first) {
            do {
                //Set first node
                if ( (firstWaiter = first.nextWaiter) == null)
                    lastWaiter = null;
                first.nextWaiter = null;
                //If the wake-up is unsuccessful or the next node is not empty, wake up the next node until a node is successfully awakened
            } while (!transferForSignal(first) &&
                     (first = firstWaiter) != null);
        }

Take another look at the transferForSignal method

    final boolean transferForSignal(Node node) {
        /*
         * If cannot change waitStatus, the node has been cancelled.
         */
        //Set the status of the current node to 0 (the node status in the waiting queue itself is CONDITION, which is set in the await method)
        if (!compareAndSetWaitStatus(node, Node.CONDITION, 0))
            return false;

        /*
         * Splice onto queue and try to set waitStatus of predecessor to
         * indicate that thread is (probably) waiting. If cancelled or
         * attempt to set waitStatus fails, wake up to resync (in which
         * case the waitStatus can be transiently and harmlessly wrong).
         */
        //Put the current node into the CLH queue because there may be other nodes waiting to wake up
        //Note that the p node returned here is the front node of the current node
        Node p = enq(node);
        int ws = p.waitStatus;
        //If the front node is CANCELLED, or the state of the front node has been changed, wake up the thread
        //In fact, this is an insurance function. The purpose is to ensure that the state of the front node must be SIGNAL. In this way, it will be unpark when it is unlock ed
        if (ws > 0 || !compareAndSetWaitStatus(p, ws, Node.SIGNAL))
            //Wake up a thread
            LockSupport.unpark(node.thread);
        return true;
    }

signal method summary

When the thread executes the signal method, it will first judge whether it has a lock, and then wake up the first node in the queue. When waking up, it will first set the waiting state of the node to 0, and then put the current node into the CLH queue to wait for the lock to be obtained successfully

summary

The blocking queue of Condition is actually implemented based on the internal class ConditionObject of AbstractQueuedSynchronizer,
ConditionObject maintains a queue internally, which is independent of the CLH queue of Lock

For example, producer and consumer in the above example can be understood as two queues when calling
When using the producer.await method: put a task node into the producer queue and block itself
When using the producer.singal method: put the first task node of the producer queue into the CLH queue, and then wait for the lock to be unpark ed successfully
It can be understood that there are not only producer nodes but also consumer nodes in the CLH queue

in other words
The Condition queue controls whether the thread is qualified to rob the lock
The CLH queue controls the order in which threads acquire locks

I hope you don't confuse the two queues

ending

I hope you can point out any improper understanding and encourage each other

Topics: Java Concurrent Programming