The previous articles have analyzed the server startup and client connection of Netty in detail, so that you should have a more comprehensive and macro understanding of Netty. In the following article, I will analyze the source code of each main component of Netty. Personally, I think that the order from macro to detail, from surface to point, is more scientific. In this article, I will first introduce EventLoop and EventLoopGroup.
I EventLoopGroup and EventExecutorGroup
The concept of loopgroup is similar to that of the parent method of loopgroup, which is equivalent to one method of loopgroup.
- EventLoop next();
This method returns the next EventLoop to be used, which is equivalent to the thread pool returning the next thread used to process the task.
- ChannelFuture register(Channel channel);
Through this method, a Channel is registered (or bound) to an EventLoop, and a Channel will only be bound to one EventLoop in its life cycle (but one EventLoop can bind multiple channels), so that subsequent IO events of the Channel will be handled by the EventLoop. Therefore, Netty can ensure that the Channel must be thread safe.
- Future<?> shutdownGracefully();
- boolean isShuttingDown();
- void shutdown();
These three methods are inherited from the parent interface EventExecutorGroup. The shutdown gracefully () method can "gracefully" close the EventLoopGroup. The so-called elegance is to ensure that the tasks submitted to the EventLoopGroup before calling the method can be executed, that is, it will not be closed until the remaining tasks are executed. The method is asynchronous, so it returns a Future. Corresponding to this method is the shutdown() method, which cannot guarantee that the remaining tasks can be executed (this method is outdated in version 4.1.47). isShuttingDown() returns true only in two cases, 1 All eventloops of the EventLoopGroup have been closed; 2. The EventLoopGroup called the shutdown gracefully() method and is in the process of shutdown.
- Future<?> submit(Runnable task);
- ScheduledFuture<?> schedule(Runnable command, long delay, TimeUnit unit);
- ScheduledFuture<?> scheduleAtFixedRate(Runnable command, long initialDelay, long period, TimeUnit unit);
- ScheduledFuture<?> scheduleWithFixedDelay(Runnable command, long initialDelay, long delay, TimeUnit unit);
These methods also inherit from the parent interface EventExecutorGroup. Their functions can also be seen from the method name. They submit a common task, scheduled task or delayed task to EventLoopGroup. For a detailed explanation of these methods, see Java util. concurrent. The notes in scheduledexecutorservice will not be repeated here.
We can understand EventLoopGroup and EventExecutorGroup in this way: EventExecutorGroup is equivalent to a general thread pool that can execute various types of tasks and manage the thread pool in its life cycle. On this basis, EventLoopGroup is a thread pool specially used to handle IO tasks of the Channel.
After finishing the interface methods, let's see how some main subclasses of EventLoopGroup implement these interface methods.
MultithreadEventExecutorGroup
MultithreadEventExecutorGroup has two important member variables, children and chooser. Children is an array of EventExecutor, which is equivalent to the thread array in the thread pool. Chooser is a selector used to select the next EventExecutor to be used.
private final EventExecutor[] children;
private final EventExecutorChooserFactory.EventExecutorChooser chooser;
The construction method of MultithreadEventExecutorGroup is as follows. The logic of the method is very simple. First, a threadpertaskeexecutor will be instantiated, and then each child will be obtained by calling the newChild() method, and then chooser will be obtained through chooserFactory. In this way, the two most important member variables of MultithreadEventExecutorGroup will be obtained.
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
EventExecutorChooserFactory chooserFactory, Object... args) {
if (nThreads <= 0) {
throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
}
if (executor == null) {
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
}
children = new EventExecutor[nThreads];
for (int i = 0; i < nThreads; i ++) {
boolean success = false;
try {
children[i] = newChild(executor, args);
success = true;
} catch (Exception e) {
// TODO: Think about if this is a good exception type
throw new IllegalStateException("failed to create a child event loop", e);
} finally {
if (!success) {
for (int j = 0; j < i; j ++) {
children[j].shutdownGracefully();
}
for (int j = 0; j < i; j ++) {
EventExecutor e = children[j];
try {
while (!e.isTerminated()) {
e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
}
} catch (InterruptedException interrupted) {
// Let the caller handle the interruption.
Thread.currentThread().interrupt();
break;
}
}
}
}
}
chooser = chooserFactory.newChooser(children);
final FutureListener<Object> terminationListener = new FutureListener<Object>() {
@Override
public void operationComplete(Future<Object> future) throws Exception {
if (terminatedChildren.incrementAndGet() == children.length) {
terminationFuture.setSuccess(null);
}
}
};
for (EventExecutor e: children) {
e.terminationFuture().addListener(terminationListener);
}
Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);
Collections.addAll(childrenSet, children);
readonlyChildren = Collections.unmodifiableSet(childrenSet);
}The newthild () method of MultithreadEventExecutorGroup is an abstract method that needs to be implemented by subclasses. Let's take a look at NioEventLoopGroup. Its newChild() method is to instantiate a NioEventLoop. The parameters will be described in detail later when we talk about NioEventLoop. NioEventLoopGroup's execute() method inherits from AbstractEventExecutorGroup. The logic is to hand over the task to the next EventLoop to be used. NioEventLoopGroup's register() method inherits from MultithreadEventLoopGroup. The logic is similar. It also hands over the register operation of the channel to EventLoop.
//NioEventLoopGroup.java
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
EventLoopTaskQueueFactory queueFactory = args.length == 4 ? (EventLoopTaskQueueFactory) args[3] : null;
return new NioEventLoop(this, executor, (SelectorProvider) args[0],
((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2], queueFactory);
}
//AbstractEventExecutorGroup.java
public void execute(Runnable command) {
next().execute(command);
}
//MultithreadEventExecutorGroup.java
public EventExecutor next() {
return chooser.next();
}
//MultithreadEventLoopGroup.java
public ChannelFuture register(Channel channel) {
return next().register(channel);
}So far, we can summarize the overall logic of EventLoopGroup. It is equivalent to a thread pool dedicated to processing IO operations of the Channel. It has an array of children and a selector chooser inside. When the EventLoopGroup is instantiated, it will call the newChild() method to create each EventLoop, The Channel can bind itself to an EventLoop through the register() method. EventLoopGroup selects an EventLoop by calling the next() method, and the next() method returns the EventLoop (or EventExecutor) selected by the policy of selector chooser.
II EventLoop and EventExecutor
The previous section talked about the relationship and difference between EventLoopGroup and EventExecutorGroup, so it is the same for EventLoop and EventExecutor. In fact, in most semantics, there is no need to distinguish between EventLoop and EventExecutor. They can be regarded as the same thing.
- EventExecutor next();
Yes, EventExecutor also has a next() method, but its next() method only returns a reference of itself
- EventExecutorGroup parent();
Returns the EventExecutorGroup to which the EventExecutor belongs
- boolean inEventLoop();
Determine whether the current thread is the EventLoop thread
The interface method is relatively simple. We are mainly concerned about how the implementation class realizes these functions, which is also the focus of this article.
NioEventLoop
NioEventLoop is the most commonly seen EventLoop by users using Netty. Let's mainly talk about its logic.
- public ChannelFuture register(Channel channel)
NioEventLoop itself does not handle the registration operation of the Channel. The specific operation is still left to the Channel itself. This was mentioned in detail in the article about the startup of Netty server before
Netty source code analysis (II) server startup source code_ Blog of Benjamin 1n77 - CSDN blog
What NioEventLoop does is encapsulate the result of channel registration into a ChannelFuture.
@Override
public ChannelFuture register(Channel channel) {
return register(new DefaultChannelPromise(channel, this));
}
@Override
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
promise.channel().unsafe().register(this, promise);
return promise;
}- private void execute(Runnable task, boolean immediate)
The logic of the execute() method is the core part of NioEventLoop, which inherits from SingleThreadEventExecutor. When we call the execute() method to submit a Runnable task to NioEventLoop, it will be added to the queue. SingleThreadEventExecutor has a task queue to store the tasks submitted to him.
private final Queue<Runnable> taskQueue;
Then, if the thread calling the execute() method is not NioEventLoop's own thread (the inEventLoop() method returns false), the startThread() method will be called. It is worth noting that if you submit a task to a NioEventLoop for the first time, the startThread() method must be called, because the first call to the execute() method must be called by an external thread, It will not be called by NioEventLoop's own thread.
private void execute(Runnable task, boolean immediate) {
boolean inEventLoop = inEventLoop();
addTask(task);
if (!inEventLoop) {
startThread();
if (isShutdown()) {
boolean reject = false;
try {
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
if (!addTaskWakesUp && immediate) {
wakeup(inEventLoop);
}
}- private void startThread()
The logic of the startThread() method is very simple. If the status of EventLoop is ST_NOT_STARTED, then set the status to ST_STARTED, and then invoke the doStartThread() method to open the thread. In other words, the doStartThread() method will only be called once, that is, when the task is submitted to EventLoop for the first time, it is equivalent to an initialization method.
The life cycle of EventLoop has the following states:
- ST_NOT_STARTED: EventLoop has not started working yet
- ST_STARTED: EventLoop has started working normally and can submit tasks to it normally
- ST_SHUTTING_DOWN: the state changes to st after calling shutdown gracefully()_ SHUTTING_ Down, you can still submit tasks to EventLoop at this time.
- ST_SHUTDOWN: after the remaining tasks in the queue are executed and the runShutdownHooks() method is executed, the status changes to ST_SHUTDOWN, you can no longer submit tasks to EventLoop (except those submitted in the hook method), but it does not mean that all tasks in the task queue have been executed, because new tasks may be submitted in the hook method.
- ST_TERMINATED: all tasks in the EventLoop have been completed, and the selector has been closed (if it is NioEventLoop).
Note that from ST_SHUTTING_DOWN to ST_SHUTDOWN and slave ST_SHUTDOWN to St_ The two calls of confirmShutDown() by terminated are different. When the former calls the confirmShutDown() method, tasks can be added to the hook method, while when the latter calls the confirmShutDown() method, only the remaining tasks in the queue can be executed, and no new tasks can be added. The confirmShutDown() method will be analyzed in more detail later in the article. Let's put it down for the time being.
private void startThread() {
if (state == ST_NOT_STARTED) {
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
boolean success = false;
try {
doStartThread();
success = true;
} finally {
if (!success) {
STATE_UPDATER.compareAndSet(this, ST_STARTED, ST_NOT_STARTED);
}
}
}
}
}- private void doStartThread()
NioEventLoop calls the doStartThread() method to start executing the task. This method calls its own internal executor to execute an anonymous Runnable. This executor is the executor that really executes the task. EventLoop is actually a wrapper for this executor. The main logic of the anonymous Runnable's run() method is as follows:
- First, save the threads used by the underlying layer of the EventLoop.
- Call SingleThreadEventExecutor Run () method, which is an abstract method of SingleThreadEventExecutor and needs subclasses to implement. In general, this method will continuously take a task from the queue. Next, we will analyze nioeventloop run().
- The execution of the run() method ends, which means that the EventLoop should stop working, and its life cycle has at least reached St_ SHUTTING_ In the step of down, all subsequent operations are to handle the shutdown operation of EventLoop. The logic of shutdown has been described earlier.
private void doStartThread() {
assert thread == null;
executor.execute(new Runnable() {
@Override
public void run() {
thread = Thread.currentThread();
if (interrupted) {
thread.interrupt();
}
boolean success = false;
updateLastExecutionTime();
try {
SingleThreadEventExecutor.this.run();
success = true;
} catch (Throwable t) {
logger.warn("Unexpected exception from an event executor: ", t);
} finally {
for (;;) {
int oldState = state;
if (oldState >= ST_SHUTTING_DOWN || STATE_UPDATER.compareAndSet(
SingleThreadEventExecutor.this, oldState, ST_SHUTTING_DOWN)) {
break;
}
}
// Check if confirmShutdown() was called at the end of the loop.
if (success && gracefulShutdownStartTime == 0) {
if (logger.isErrorEnabled()) {
logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " +
SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must " +
"be called before run() implementation terminates.");
}
}
try {
// Run all remaining tasks and shutdown hooks. At this point the event loop
// is in ST_SHUTTING_DOWN state still accepting tasks which is needed for
// graceful shutdown with quietPeriod.
for (;;) {
if (confirmShutdown()) {
break;
}
}
// Now we want to make sure no more tasks can be added from this point. This is
// achieved by switching the state. Any new tasks beyond this point will be rejected.
for (;;) {
int oldState = state;
if (oldState >= ST_SHUTDOWN || STATE_UPDATER.compareAndSet(
SingleThreadEventExecutor.this, oldState, ST_SHUTDOWN)) {
break;
}
}
// We have the final set of tasks in the queue now, no more can be added, run all remaining.
// No need to loop here, this is the final pass.
confirmShutdown();
} finally {
try {
cleanup();
} finally {
// Lets remove all FastThreadLocals for the Thread as we are about to terminate and notify
// the future. The user may block on the future and once it unblocks the JVM may terminate
// and start unloading classes.
// See https://github.com/netty/netty/issues/6596.
FastThreadLocal.removeAll();
STATE_UPDATER.set(SingleThreadEventExecutor.this, ST_TERMINATED);
threadLock.countDown();
int numUserTasks = drainTasks();
if (numUserTasks > 0 && logger.isWarnEnabled()) {
logger.warn("An event executor terminated with " +
"non-empty task queue (" + numUserTasks + ')');
}
terminationFuture.setSuccess(null);
}
}
}
}
});
}- protected void run()
run() method is an abstract method defined by SingleThreadEventExecutor. Here we are talking about the implementation of NioEventLoop. run() method is the core method of NioEventLoop. In an endless loop, the run() method will do the following things over and over again:
- Call selector Select() or selectNow() to judge whether there is an IO event task
- If so, processSelectedKeys() is called to process the IO event task
- Then call runAllTasks() to handle other tasks
protected void run() {
int selectCnt = 0;
for (;;) {
try {
int strategy;
try {
strategy = selectStrategy.calculateStrategy(selectNowSupplier, hasTasks());
switch (strategy) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.BUSY_WAIT:
// fall-through to SELECT since the busy-wait is not supported with NIO
case SelectStrategy.SELECT:
long curDeadlineNanos = nextScheduledTaskDeadlineNanos();
if (curDeadlineNanos == -1L) {
curDeadlineNanos = NONE; // nothing on the calendar
}
nextWakeupNanos.set(curDeadlineNanos);
try {
if (!hasTasks()) {
strategy = select(curDeadlineNanos);
}
} finally {
// This update is just to help block unnecessary selector wakeups
// so use of lazySet is ok (no race condition)
nextWakeupNanos.lazySet(AWAKE);
}
// fall through
default:
}
} catch (IOException e) {
// If we receive an IOException here its because the Selector is messed up. Let's rebuild
// the selector and retry. https://github.com/netty/netty/issues/8566
rebuildSelector0();
selectCnt = 0;
handleLoopException(e);
continue;
}
selectCnt++;
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
boolean ranTasks;
if (ioRatio == 100) {
try {
if (strategy > 0) {
processSelectedKeys();
}
} finally {
// Ensure we always run tasks.
ranTasks = runAllTasks();
}
} else if (strategy > 0) {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
ranTasks = runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
} else {
ranTasks = runAllTasks(0); // This will run the minimum number of tasks
}
if (ranTasks || strategy > 0) {
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS && logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
selectCnt = 0;
} else if (unexpectedSelectorWakeup(selectCnt)) { // Unexpected wakeup (unusual case)
selectCnt = 0;
}
} catch (CancelledKeyException e) {
// Harmless exception - log anyway
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
selector, e);
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}Let's explain it in detail step by step
1. Get selectStrategy
selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())
This step is to get selectStrategy. If there are no tasks in the queue, selectStrategy will be returned Select. If there is no task, call selectNow() and return the result.
public int calculateStrategy(IntSupplier selectSupplier, boolean hasTasks) throws Exception {
return hasTasks ? selectSupplier.get() : SelectStrategy.SELECT;
}
private final IntSupplier selectNowSupplier = new IntSupplier() {
@Override
public int get() throws Exception {
return selectNow();
}
};2. select or selectNow
try {
switch (strategy) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.BUSY_WAIT:
// fall-through to SELECT since the busy-wait is not supported with NIO
case SelectStrategy.SELECT:
long curDeadlineNanos = nextScheduledTaskDeadlineNanos();
if (curDeadlineNanos == -1L) {
curDeadlineNanos = NONE; // nothing on the calendar
}
nextWakeupNanos.set(curDeadlineNanos);
try {
if (!hasTasks()) {
strategy = select(curDeadlineNanos);
}
} finally {
// This update is just to help block unnecessary selector wakeups
// so use of lazySet is ok (no race condition)
nextWakeupNanos.lazySet(AWAKE);
}
// fall through
default:
}
}SelectStrategy has three values:
- SelectStrategy.CONTINUE (-2): skip this loop
- SelectStrategy.BUSY_WAIT (-3): busy, non blocking access to IO events, not supported in NioEventLoop
- SelectStrategy.SELECT (-1): select once
When there are still tasks in the task queue, selectStrategy is the value returned by selectNow(), which is not equal to any of the above cases; When there is no task, selectStrategy is selectStrategy Select, and then determine which select method to call according to curDeadlineNanos.
- If there is no scheduled task, call selector Select(), because this method is blocking, so if no IO event occurs all the time, the thread will block here.
- There is a scheduled task, and the deadline of the scheduled task is less than 5 microseconds, indicating that there is a scheduled task to be executed immediately, and the thread does not need to be blocked, so the selector is called selectNow().
- If there is a scheduled task and the deadline of the scheduled task is greater than 5 minutes, call selector Select (time out millis), just ensure that the thread can wake up in time before the deadline.
private int select(long deadlineNanos) throws IOException {
if (deadlineNanos == NONE) {
return selector.select();
}
// Timeout will only be 0 if deadline is within 5 microsecs
long timeoutMillis = deadlineToDelayNanos(deadlineNanos + 995000L) / 1000000L;
return timeoutMillis <= 0 ? selector.selectNow() : selector.select(timeoutMillis);
}In addition, we need to pay attention to one thing, that is, if there are no ordinary tasks in the task queue of NioEventLoop, and the deadline of scheduled tasks is very long or there are no scheduled tasks, then call selector Select () blocks the thread until an IO event occurs. If a new task is submitted during thread blocking, will the task not be executed because of thread blocking? In fact, it won't, because when you call the addTask(Runnable task) method to submit a task, you can wake up the thread. The code is as follows.
public void execute(Runnable task) {
ObjectUtil.checkNotNull(task, "task");
//If the task is not a task of LazyRunnable type, the second parameter passed is true
execute(task, !(task instanceof LazyRunnable) && wakesUpForTask(task));
}
//This method always returns true. Of course, subclasses can override this method, but NioEventLoop does not
protected boolean wakesUpForTask(Runnable task) {
return true;
}
private void execute(Runnable task, boolean immediate) {
boolean inEventLoop = inEventLoop();
addTask(task);
if (!inEventLoop) {
startThread();
if (isShutdown()) {
boolean reject = false;
try {
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
//For NioEventLoop, the addTaskWakesUp parameter is false by default, so if immediate is true, the wakeup method will be called after the task is added to the queue.
if (!addTaskWakesUp && immediate) {
wakeup(inEventLoop);
}
}
protected void wakeup(boolean inEventLoop) {
if (!inEventLoop && nextWakeupNanos.getAndSet(AWAKE) != AWAKE) {
selector.wakeup();
}
}3. If an exception occurs, the selector needs to be rebuilt
Nothing to say in this section is to call the rebuildSelector0() method to rebuild a selector. The logic of the rebuildSelector0() method is not complicated, that is, re open a new selector and close the old selector. I won't repeat it here.
catch(IOException e){
// If we receive an IOException here its because the Selector is messed up. Let's rebuild
// the selector and retry. https://github.com/netty/netty/issues/8566
rebuildSelector0();
selectCnt = 0;
handleLoopException(e);
continue;
}4. Perform IO tasks and general tasks
Judge the value of ioRatio. This value represents the proportion of the time NioEventLoop takes to execute IO tasks and ordinary tasks. If this value is smaller, the more time NioEventLoop takes to execute ordinary tasks. For example, if ioRatio=20, the time ratio between executing IO tasks and executing ordinary tasks is 20:80, that is, 1:4. By default, ioRatio=50. In addition, if ioRatio=100, all ordinary tasks will be executed after the IO task is executed, regardless of how much time it takes.
selectCnt++;
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
boolean ranTasks;
if (ioRatio == 100) {
try {
if (strategy > 0) {
processSelectedKeys();
}
} finally {
// Ensure we always run tasks.
//ioRatio is 100, and the runAllTasks() method without parameters is called to execute all tasks in the queue, no matter how long they take
ranTasks = runAllTasks();
}
} else if (strategy > 0) {
final long ioStartTime = System.nanoTime();
try {
//Perform IO tasks
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
//To execute ordinary tasks, call the runAllTasks method with parameters here. This method can ensure to return within the specified time. Even if there are tasks in the queue that have not been executed, they will not continue to be executed
ranTasks = runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
} else {
//If there are no IO tasks, only the minimum number of ordinary tasks will be executed. (minimum 64)
ranTasks = runAllTasks(0); // This will run the minimum number of tasks
}
if (ranTasks || strategy > 0) {
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS && logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
selectCnt = 0;
} else if (unexpectedSelectorWakeup(selectCnt)) { // Unexpected wakeup (unusual case)
selectCnt = 0;
}5. Close NioEventLoop
At the end of each cycle, the state of NioEventLoop will be judged. If isShuttingDown() returns true, it means that a thread has called shutdown gracefully(), which means that the NioEventLoop will be closed. Then call closeAll() to close all selectionkeys and channels. Finally, call confirmShutdown() again to confirm whether it really needs to be closed. If so, the run() method returns and the loop ends.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}So far, the logic of NioEventLoop's run() method has been finished, and there are two more important methods left. processSelectedKeys(): process IO tasks; runAllTasks(): handle common tasks.
- private void processSelectedKeys()
This method will call processSelectedKeysOptimized() or processSelectedKeysPlain() according to whether selectedKeys is null. selectedKeys is the optimization of JDK NIO native SelectionKey by NioEventLoop, selector selectedKeys() returns a Set < SelectionKey >, and here selectedKeys is a SelectionKey array. Compared with Set, the traversal speed of the array must be faster.
private void processSelectedKeys() {
if (selectedKeys != null) {
processSelectedKeysOptimized();
} else {
processSelectedKeysPlain(selector.selectedKeys());
}
}
- private void processSelectedKeysOptimized()
Call processSelectedKey() in the loop to handle all selectedKey in turn, and check needsToSelectAgain at the end of each cycle. If true, reset selectedKeys is needed and the selectNow() is called again. Needstoselectagain will only be set to true if the selectedkey is cancelled multiple times.
private void processSelectedKeysOptimized() {
for (int i = 0; i < selectedKeys.size; ++i) {
final SelectionKey k = selectedKeys.keys[i];
// null out entry in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.keys[i] = null;
final Object a = k.attachment();
if (a instanceof AbstractNioChannel) {
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
if (needsToSelectAgain) {
// null out entries in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.reset(i + 1);
selectAgain();
i = -1;
}
}
}
private void selectAgain() {
needsToSelectAgain = false;
try {
selector.selectNow();
} catch (Throwable t) {
logger.warn("Failed to update SelectionKeys.", t);
}
}- private void processSelectedKey(SelectionKey k, AbstractNioChannel ch)
Here, first judge whether the selectionKey is valid. If the selectionKey is not valid and the corresponding channel is still registered on the EventLoop, the channel should be closed. If the channel is not registered on the EventLoop, it is obvious that there is no right to close the channel here.
Next, determine the type of IO event and then call the response of channel's unsafe. If OP_CONNETC, call unsafe Finishconnect() and unregister OP_CONNETC event, otherwise it may lead to empty polling BUG of JDK; If OP_WIRITE, call unsafe() forceFlush()； If OP_ACCEPT or OP_READ, call unsafe read().
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
// If the channel implementation throws an exception because there is no event loop, we ignore this
// because we are only trying to determine if ch is registered to this event loop and thus has authority
// to close ch.
return;
}
// Only close ch if ch is still registered to this EventLoop. ch could have deregistered from the event loop
// and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
// still healthy and should not be closed.
// See https://github.com/netty/netty/issues/5125
if (eventLoop == this) {
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
}
return;
}
try {
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}- protected boolean runAllTasks() and {protected boolean runAllTasks(long timeoutNanos)
The nonparametric runAllTasks method will complete all tasks in the task queue. The runAllTasks method with timeoutNanos parameter will only execute for a certain time. If it exceeds this time, the method return s and the remaining tasks in the queue will not be executed.
In the nonparametric runAllTasks() method, the scheduled tasks will be taken from the scheduled task queue and added to the normal task queue, and then all tasks in the normal queue will be executed until the fetchFromScheduledTaskQueue() method returns true. Finally, call the afterRunningAllTasks() method again.
protected boolean runAllTasks() {
assert inEventLoop();
boolean fetchedAll;
boolean ranAtLeastOne = false;
do {
fetchedAll = fetchFromScheduledTaskQueue();
if (runAllTasksFrom(taskQueue)) {
ranAtLeastOne = true;
}
} while (!fetchedAll); // keep on processing until we fetched all scheduled tasks.
if (ranAtLeastOne) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
}
afterRunningAllTasks();
return ranAtLeastOne;
}The logic of the fetchfromscheduledtasqueue() method is to take the tasks that meet the execution conditions from the scheduled task queue to the normal queue until the normal queue is full or there are no tasks to be executed in the scheduled task queue. If the former is false, the latter returns true.
private boolean fetchFromScheduledTaskQueue() {
//If the scheduled task queue is empty and there is no scheduled task, return true directly
if (scheduledTaskQueue == null || scheduledTaskQueue.isEmpty()) {
return true;
}
long nanoTime = AbstractScheduledEventExecutor.nanoTime();
for (;;) {
//Get the scheduled task that has met the execution time
Runnable scheduledTask = pollScheduledTask(nanoTime);
if (scheduledTask == null) {
//If not, return true directly
return true;
}
if (!taskQueue.offer(scheduledTask)) {
//Add the scheduled task to be executed to the normal task queue. If the normal task queue is full, false is returned, which means that the method needs to be called again
// No space left in the task queue add it back to the scheduledTaskQueue so we pick it up again.
scheduledTaskQueue.add((ScheduledFutureTask<?>) scheduledTask);
return false;
}
}
}The only difference between the runAllTasks() method with parameters is that it will check whether it times out every 64 tasks. If it times out, it will not continue to execute the task and return directly.
protected boolean runAllTasks(long timeoutNanos) {
fetchFromScheduledTaskQueue();
Runnable task = pollTask();
if (task == null) {
afterRunningAllTasks();
return false;
}
final long deadline = timeoutNanos > 0 ? ScheduledFutureTask.nanoTime() + timeoutNanos : 0;
long runTasks = 0;
long lastExecutionTime;
for (;;) {
safeExecute(task);
runTasks ++;
// Check timeout every 64 tasks because nanoTime() is relatively expensive.
// XXX: Hard-coded value - will make it configurable if it is really a problem.
if ((runTasks & 0x3F) == 0) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
if (lastExecutionTime >= deadline) {
break;
}
}
task = pollTask();
if (task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
afterRunningAllTasks();
this.lastExecutionTime = lastExecutionTime;
return true;
}(end)