Series articles
Source code analysis of Reactor3 SpscLinkedArrayQueue ðĨ
Source code analysis of Reactor3 MpscLinkedQueue ðĨðĨ
Reactor3 Flux.create and flux Correct opening mode of push ðĨðĨðĨ
Reactor3 Flux.create and flux Push difference and source code analysis (I) ðĨðĨðĨðĨðĨ
edition
Reactor 3.4.9
Write in front
In order to introduce the source code of Flux's create and push, I wrote the above three blogs to pave the way. It's not easy ðŧ.
If you don't understand flux To use create and push, please read: Reactor3 Flux.create and flux Correct opening mode of push ð.
If you don't understand SpscLinkedArrayQueue, click: Source code analysis of Reactor3 SpscLinkedArrayQueue ð
If you don't understand MpscLinkedQueue, please enter: Source code analysis of Reactor3 MpscLinkedQueue ð
stay Last In, we analyzed flux Create and flux Push some external source code, and the internal core source code design is left in this article.
Let's first review reactor core. publisher. Steps of fluxcreate#subscribe method:
- First, a BaseSink is created. Both push and create use the default BufferAsyncSink. There is no code here. The logic is not complex.
- Pass the baselink to the Subscriber as a Subscription, as the default BufferAsyncSink used by FluxCreatepush and create. Bridge with Subscriber.
- Then there is the black magic. Through the different CreateMode modes, we use SerializedFluxSink to encapsulate the sink of create.
You can see that BaseSink is used first, then BufferAsyncSink and SerializedFluxSink. Let's analyze them one by one in this order.
BaseSink source code analysis
In the previous article, we saw that BaseSink implements two interfaces, FluxSink and Subscription, and acts as a direct bridge between FluxCreate and Subscriber.
BaseSink is an abstract class, and other Sink inherit it. It implements some general methods, such as request, complete, error, cancel, etc. next, an important method of issuing elements, has the following subclasses to implement according to their own characteristics.
BaseSink construction method
BaseSink(CoreSubscriber<? super T> actual) {
this.actual = actual;
this.ctx = actual.currentContext();
}
General operation, save the subscriber to its own member variable for later communication with the subscriber.
BaseSink complete,error,cancel
public void complete() {
if (isTerminated()) {
return;
}
try {
actual.onComplete();
}
finally {
disposeResource(false);
}
}
- Determine whether it has been destroyed
- If it is not destroyed, the subscriber's onComplete method will be called, and finally the destruction method will be called.
Other methods are similar. There is no black magic, so I won't describe them one by one.
BufferAsyncSink source code analysis
The BaseSink above is only an appetizer and implements some general methods. From the name of BufferAsyncSink, you can guess that it has two characteristics: buffering and asynchrony. It's flux Push the queue directly, of course flux Create is also used, but it is encapsulated. In the last article, we said flux Push only allows a single thread to submit tasks. How is it implemented?
Introduction to BufferAsyncSink member variable
//Buffer queue for issuing elements final Queue<T> queue; //Record error Throwable error; //Determine whether to end volatile boolean done; //It can be understood as how many times the drain method has been called, //Only when the read wip is 0, the thread is allowed to queue //The operation of distributing elements in to subscribers. volatile int wip;
The Buffer and Async of BufferAsyncSink are ready to come out. Its member variable queue is used as a Buffer. First submit the task to the queue, and then issue it asynchronously to subscribers.
BufferAsyncSink construction method
BufferAsyncSink(CoreSubscriber<? super T> actual, int capacityHint) {
super(actual);
this.queue = Queues.<T>unbounded(capacityHint).get();
}
The focus is on the creation of the queue. Finally, a SpscLinkedArrayQueue is returned. Yes Source code analysis of Reactor3 SpscLinkedArrayQueue All students know the characteristics of this queue: single producer single consumer. If you don't know the specific implementation, you can click Source code analysis of Reactor3 SpscLinkedArrayQueue ð.
BufferAsyncSink request method
public FluxSink<T> next(T t) {
queue.offer(t);
drain();
return this;
}
- Put elements directly into the queue
- Call the drain method
When we analyze the SpscLinkedArrayQueue, although it is said to be a single producer, it does not control the whole feature by itself, but it is left to the caller to control itself. In BufferAsyncSink, it also has no control. Only one thread is allowed to insert elements into the queue. So, if you use multiple threads concurrently, you can use flux When pushing to distribute elements, there will be concurrency security problems.
Flux. An example of pushing multithreaded elements
public void testFluxPush() throws InterruptedException {
//Multi thread calling push method
Flux<String> f = Flux.push(sink -> {
outSink = sink;
});
f.subscribe(e -> System.out.println(e));
//do something
//Issue element
for (int i = 0; i < 10; i++) {
int finalI = i;
new Thread(() -> outSink.next("I'm coming." + finalI)).start();
}
Thread.sleep(1000);
}
results of enforcement
We can see that several tasks are missing.
We're using flux You need to control when pushing. Only a single thread can call flux The element of push is distributed. Readers can replace push with create to see how the results are different.
BufferAsyncSink drain method
In the next method, this method will be called after the element is distributed. In fact, not only next, BufferAsyncSink, but all other operations will call drain.
void drain() {
//If the wip is not 0, it will be returned directly. Only one thread can enter the control
//Any thread that does not enter will increase the wip by 1,
if (WIP.getAndIncrement(this) != 0) {
return;
}
final Flow.Subscriber<? super T> a = actual;
final Queue<T> q = queue;
for (; ; ) {
long r = requested;
//Record the number of current circular consumption elements
long e = 0L;
while (e != r) {
if (isCancelled()) {
// Operators.onDiscardQueueWithClear(q, ctx, null);
if (WIP.decrementAndGet(this) != 0) {
continue;
} else {
return;
}
}
boolean d = done;
//Get element from queue
T o = q.poll();
boolean empty = o == null;
//The queue is empty and has been cancelled. Return directly
if (d && empty) {
Throwable ex = error;
if (ex != null) {
super.error(ex);
} else {
super.complete();
}
return;
}
//Queue is empty, jump out of internal loop
if (empty) {
break;
}
//Distribute elements to subscribers
a.onNext(o);
e++;
}
if (e == r) {
if (isCancelled()) {
// Operators.onDiscardQueueWithClear(q, ctx, null);
if (WIP.decrementAndGet(this) != 0) {
continue;
} else {
return;
}
}
boolean d = done;
boolean empty = q.isEmpty();
if (d && empty) {
Throwable ex = error;
if (ex != null) {
super.error(ex);
} else {
super.complete();
}
return;
}
}
if (e != 0) {
//request minus the number of these purchases
Operators.produced(REQUESTED, this, e);
}
//The loop continues until the value of the wip becomes 0 or is cancelled
if (WIP.decrementAndGet(this) == 0) {
break;
}
}
}
- After obtaining the value of the wip, add the value of the wip + 1;
- If wip is not 0, it returns directly
- The following is the logic to be executed when wip is 0
- Then, the subscriber and queue are assigned to local variables, and then an infinite for loop is used to continuously extract elements from the queue and distribute them to the subscriber. The condition for infinite loop exit is that the wip is 0 or terminated.
The code is very long. In fact, it is such a thing to do. Many are exception handling such as judging whether they have been destroyed.
from
From controlling the wip to 0 to enter, it limits that only one thread can consume elements from the queue. Therefore, there will be no multi-threaded access during consumption, and the single consumer qualification rule of SpscLinkedArrayQueue will not be broken.
BufferAsyncSink summary
It works as flux The Subscription used by push uses SpscLinkedArrayQueue as the buffer queue. Therefore, it is limited that it can only submit elements by a single thread and consume elements by a single thread. In BufferAsyncSink, only the number of threads consumed is limited, while the number of threads submitted is not limited. So we're using flux When pushing, you need to know whether the distribution element is a single thread.
When writing code, be sure to know what you're doing ðĪŠ
SerializedFluxSink source code analysis
Flux. The FluxSink used by cretate is a BufferAsyncSink wrapped in serializedfluxink, and then supports multi-threaded access. How does it do that? As the name suggests, we can guess that it does a serialization operation to turn multiple threads into linear through some kind of black magic.
SerializedFluxSink member variable
//Store BufferAsyncSink
final BaseSink<T> sink;
//Record exception
volatile Throwable error;
//Different from the wip of BufferAsyncSink, it can also be understood as the total number of operations called.
volatile int wip;
//It is different from the queue of BufferAsyncSink. It is a self-contained queue
final Queue<T> mpscQueue;
volatile boolean done;
Although there is a BufferAsyncSink in SerializedFluxSink memory, it does not share a wip and queue with it, but generates new wip and queue by itself. The queue it uses is no longer SpscLinkedArrayQueue, but MpscLinkedQueue. It is a multi producer single consumer thread. If you don't understand MpscLinkedQueue, please enter: Source code analysis of Reactor3 MpscLinkedQueue ð
SerializedFluxSink netx method
public FluxSink<T> next(T t) {
Objects.requireNonNull(t, "t is null in sink.next(t)");
if (sink.isTerminated() || done) {
Operators.onNextDropped(t);
return this;
}
//If a thread has entered the execution process to send a message, the message will be pushed to the local concurrent queue first
if (WIP.get(this) == 0 && WIP.compareAndSet(this, 0, 1)) {
try {
sink.next(t);
} catch (Throwable ex) {
Operators.onOperatorError(sink, ex, t);
}
//If wip is equal to 0, it means that only this thread has operation elements to send, and there are no redundant messages, which can be returned directly.
//wip can also be regarded as the largest number of circular consumption
if (WIP.decrementAndGet(this) == 0) {
return this;
}
} else {
//Push local concurrent queue
this.mpscQueue.offer(t);
//If the wip is greater than 0, it will return. If it is equal to 0, it will enter the logic of sending messages from the local queue
if (WIP.getAndIncrement(this) != 0) {
return this;
}
//The condition that the branch does not return directly from here is that the wip value is 0, that is, the task of the above branch has been executed,
// After the thread of the above branch directly operates sink, if another thread submits a task, it will continue to call drawloop and continue to operate sink,
//Therefore, the production thread of sink is not switched.
//If no one submits, it means that you are not exiting directly in the case of high concurrency.
//In the upper branch, only when the last value of wip is 0 will it exit directly, otherwise it will enter the lower drawloop
//That is, the thread executing the upper branch is just finished, and the thread of the lower branch performs the get operation of wip, successfully taking over the upper thread to operate sink,
//Control that only one thread can operate sink at a time
}
//The value of the wip entered here will not be 0,
drainLoop();
return this;
}
- If wip is 0, it is allowed to directly manipulate the BufferAsyncSink push element. After that, if it is found that the wip is still 0 after subtracting one, it means that there are no other messages, it will be returned directly.
- If wip is not 0, it indicates that other threads are operating BufferAsyncSink. It indicates that BufferAsyncSink cannot be operated any more and can only be stored in the local mpscQueue first. mpscQueue, which supports multi-threaded push messages.
- Finally, lucky people will call drawloop and fall into an infinite loop. What drawloop does is push the message of mpscQueue into BufferAsyncSink. Of course, the lucky one needs control, which can only be operated by one thread.
Using this method, we can see the dark magic of SerializedFluxSink turning multithreading into single thread. When there is only single thread access, you can directly operate the BufferAsyncSink that allows only single thread access. According to the wip judgment, when multiple threads access concurrently, first push these elements into a queue supporting multiple production, MpscLinkedQueue, cache these elements locally, and then select one of these threads to push MpscLinkedQueue to BufferAsyncSink, so as to convert multiple threads into a single thread.
Serializedfluxsink drawloop method
void drainLoop() {
BaseSink<T> e = sink;
Queue<T> q = mpscQueue;
for (; ; ) {
for (; ; ) {
if (e.isCancelled()) {
Operators.onDiscardQueueWithClear(q);
if (WIP.decrementAndGet(this) == 0) {
return;
} else {
continue;
}
}
if (ERROR.get(this) != null) {
Operators.onDiscardQueueWithClear(q);
//noinspection ConstantConditions
e.error(Exceptions.terminate(ERROR, this));
return;
}
boolean d = done;
T v = q.poll();
boolean empty = v == null;
if (d && empty) {
e.complete();
return;
}
if (empty) {
break;
}
try {
e.next(v);
} catch (Throwable ex) {
Operators.onOperatorError(sink, ex, v);
}
}
//Reduce the wip by one after each consumption. If it is equal to 0, it means there is no message and exit the cycle
if (WIP.decrementAndGet(this) == 0) {
break;
}
}
}
In fact, the logic is similar to the draw method of BufferAsyncSink, so I won't repeat the description.
summary
This chapter describes in detail the causes of flux Push and flux The essential reason of create and the source code analysis of their respective implementations. I wonder if readers will have some doubts about concurrency when reading it. What skills can we learn from their concurrency processing? The next article continues ð