preface
The notes are based on black horse's Netty teaching handout and some of my own understanding. I feel that this is a very good video I've seen. There's basically no nonsense. Video address: Black horse Netty . From here on, we try to look at the source code to understand the underlying operating mechanism of Netty.
1. nio startup process review
Let's review what nio needs to do to start the server
//1. NioEventLoopGroup (nio boss thread for short) is used in netty to encapsulate threads and selector s Selector selector = Selector.open(); //2 create NioServerSocketChannel, initialize its associated handler and store config for native ssc NioServerSocketChannel attachment = new NioServerSocketChannel(); //3 when creating NioServerSocketChannel, a java Native serversocketchannel is created. The function of serversocketchannel can be understood as a registrar. The ssc in netty is registered on the native ssc as an attachment ServerSocketChannel serverSocketChannel = ServerSocketChannel.open(); serverSocketChannel.configureBlocking(false); //4 start the nio boss thread to perform the following operations //5 Registration (only the selector and NioServerSocketChannel are associated), and events are not followed SelectionKey selectionKey = serverSocketChannel.register(selector, 0, attachment); //6 head - > initializer - > serverbootstrap acceptor - > tail. The initializer is one-time and only for adding acceptors //7 binding port serverSocketChannel.bind(new InetSocketAddress(8080)); //8 trigger the channel active event and focus on the op in the head_ Accept event //Indicates that this event is triggered when the client accesses selectionKey.interestOps(SelectionKey.OP_ACCEPT);
The main steps are as follows:
1. Selector selector = Selector.open();
2. ServerSocketChannel ssc = ServerSocketChannel.open();
3. SelectionKey key = ssc.register(selector, 0, nettySsc);
4. ssc.bind(new InetSocketAddress("localhost", 8080));
5. key.interestOps(SelectionKey.OP_ACCRPT);
2. Source code method description
Use the following code to illustrate:
public class TestSourceServer {
public static void main(String[] args) {
new ServerBootstrap()
//EventLoop has a thread and actuator selector, which is used to focus on events and solve some tasks
.group(new NioEventLoopGroup())
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<NioSocketChannel>(){
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler());
}
}).bind(8080);
}
}
| 1. Selector selector = Selector.open(); |
This part is created in new NioEventLoopGroup(). Because EventLoop maintains a selector and a thread, the selector is created here
| 2. The following four methods |
bind() -> return bind(new InetSocketAddress(inetPort)) -> return doBind(ObjectUtil.checkNotNull(localAddress, "localAddress"));
Important method: doBind(ObjectUtil.checkNotNull(localAddress, "localAddress"))
private ChannelFuture doBind(final SocketAddress localAddress) {
//1. Initialize NioServerSocketChannel and register with ssc
// regFuture is actually a promise object. When the result is generated, the following doBind0 will be called
final ChannelFuture regFuture = initAndRegister();
final Channel channel = regFuture.channel();
if (regFuture.cause() != null) {
return regFuture;
}
if (regFuture.isDone()) {
// At this point we know that the registration was complete and successful.
ChannelPromise promise = channel.newPromise();
//Binding port
doBind0(regFuture, channel, localAddress, promise);
return promise;
} else {
final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
regFuture.addListener(new ChannelFutureListener() {
//Nio thread call
@Override
public void operationComplete(ChannelFuture future) throws Exception {
Throwable cause = future.cause();
if (cause != null) {
promise.setFailure(cause);
} else {
promise.registered();
//2. Binding port
doBind0(regFuture, channel, localAddress, promise);
}
}
});
return promise;
}
}
When looking at the code below, note that the dobind method is executed in the main thread, but thread switching occurs when the initAndRegister method is executed.
The following is a brief introduction to the two methods (initAndRegister and doBind0):
-
init and register regFuture processing
- init (main thread execution)
a. Create NioServerSocketChannel (main thread execution)
b. Add NioServerSocketChannel initialization handler (main thread execution)
Initialize handler and wait for call (main not called)
Add accept handler to nio ssc (establish connection after accept event) - register (switch thread)
a. Start nio boss thread (executed by main thread)
b. The native ssc(ServerSocketChannel) registers with the selector and does not pay attention to events (NiO thread execution)
c. Execute NioServerSocketChannel initialization handler (NiO thread execution)
- init (main thread execution)
-
regFuture waiting for callback doBind0
- Native ServerSocketChannel binding (NiO thread execution)
- Trigger NioServerSocketChannel active event (NiO thread execution)
3. initAndRegister
The form of method annotation is based on a general method and assisted by internal methods
final ChannelFuture initAndRegister() {
//Initialize channel
Channel channel = null;
try {
//Create a channel through the channelFactory factory
//Created reflective channelfactory (nioserversocketchannel. Class)
//In fact, the interior is created by using the reflection method to get the constructor
channel = channelFactory.newChannel(); // 1
init(channel); // 2
} catch (Throwable t) {
if (channel != null) {
channel.unsafe().closeForcibly();
return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
}
return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
}
ChannelFuture regFuture = config().group().register(channel);//3
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
return regFuture;
}
1. channel = channelFactory.newChannel()
| 1. channel = channelFactory.newChannel(); |
This method uses the channel factory to create it internally. newChannel uses the reflection mechanism to call the constructor of NioServerSocketChannel class. We go directly to NioServerSocketChannel class to see the source code
public NioServerSocketChannel() {
this(newSocket(DEFAULT_SELECTOR_PROVIDER));
}
Enter newSocket(DEFAULT_SELECTOR_PROVIDER): in fact, the provider is called Openserversocketchannel() method
In fact, after careful observation, this line of code is the execution process of the open method of ServerSocketChannel. The following is the open method of ServerSocketChannel:
Summary: this line of code actually calls serversocketchannel Open creates a NioServerSocketChannel, and internally creates the ServerSockerChannel of JDK
2. init(channel)
| 2. init(channel) |
@Override
void init(Channel channel) {
setChannelOptions(channel, newOptionsArray(), logger);
setAttributes(channel, attrs0().entrySet().toArray(EMPTY_ATTRIBUTE_ARRAY));
//p here is the pipeline of NioServerSocketChannel
ChannelPipeline p = channel.pipeline();
final EventLoopGroup currentChildGroup = childGroup;
final ChannelHandler currentChildHandler = childHandler;
final Entry<ChannelOption<?>, Object>[] currentChildOptions;
synchronized (childOptions) {
currentChildOptions = childOptions.entrySet().toArray(EMPTY_OPTION_ARRAY);
}
final Entry<AttributeKey<?>, Object>[] currentChildAttrs = childAttrs.entrySet().toArray(EMPTY_ATTRIBUTE_ARRAY);
//Starting from here is the point
//An initialized handler is added to NioServerSocketChannel
//The method inside will only be executed once
p.addLast(new ChannelInitializer<Channel>( ) {
@Override
public void initChannel(final Channel ch) {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
pipeline.addLast(handler);
}
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
}
Summary: in this line of code, an initialization processor is created for NioServerSocketChannel, waiting to be called. This processor is called only once
3. ChannelFuture regFuture = config().group().register(channel)
| 3. ChannelFuture regFuture = config().group().register(channel) |
@Override
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
ObjectUtil.checkNotNull(eventLoop, "eventLoop");
if (isRegistered()) {
promise.setFailure(new IllegalStateException("registered to an event loop already"));
return;
}
if (!isCompatible(eventLoop)) {
promise.setFailure(
new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
return;
}
AbstractChannel.this.eventLoop = eventLoop;
//This method is to check whether the current thread is equal to Nio thread. The current thread is definitely not equal to Nio thread, because the main thread is walking
if (eventLoop.inEventLoop()) {
register0(promise);
} else {
try {
//Create a task object and give it to the Nio thread in eventLoop to execute
//Then, the thread is switched from the main thread to the Nio thread
//register0 is lazy loading, and the thread is created only when it is called here
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
logger.warn(
"Force-closing a channel whose registration task was not accepted by an event loop: {}",
AbstractChannel.this, t);
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}
}
Enter the register0(promise) method:
private void register0(ChannelPromise promise) {
try {
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
boolean firstRegistration = neverRegistered;
//The core code is here
doRegister();
neverRegistered = false;
registered = true;
pipeline.invokeHandlerAddedIfNeeded();
safeSetSuccess(promise);
pipeline.fireChannelRegistered();
if (isActive()) {
if (firstRegistration) {
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
beginRead();
}
}
} catch (Throwable t) {
// Close channel
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}
Enter the doRegister() method of register0, which completes the registration, that is, selectionkey = SSc Register (selector, 0, nettyssc)
@Override
protected void doRegister() throws Exception {
boolean selected = false;
for (;;) {
try {
//Get the java Native ServerSocketChannel and register it with the selector
//0: indicates that there is no event of interest
//this: the current NioServerSocketChannel object
selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
return;
} catch (CancelledKeyException e) {
if (!selected) {
eventLoop().selectNow();
selected = true;
} else {
throw e;
}
}
}
}
Enter the pipeline of register0 In the invokehandleraddedifneeded () method, this method is actually called internally The ChannelInitializer set in init (channel) initializes the initChannel method of the processor, so the function of this method is to call the initialization method not called in the second step
So this method is actually called in initialization.
p.addLast(new ChannelInitializer<Channel>() {
@Override
public void initChannel(final Channel ch) {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
pipeline.addLast(handler);
}
//A serverbootstrap acceptor processor is added to NioServerSocketChanel to establish a connection after the accept event
//The task of adding a processor is submitted to the channel and guaranteed to run in the thread in the channel
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
Enter the safeSetSuccess(promise) method of register0. In fact, this method is used to set the result of promise. Note: the promise in doBind of the source method comes from the same promise, so it will enter doBind0 method after this step is completed.
Summary: config () in this line of code group(). Register (channel) completes the registration of ssc, that is, the register method, and sets a handler in it. This handler is executed after the accept event, and sets the result for promise in the source doBind, resulting in entering doBind0 of the doBind method
4. doBind0
doBind0(regFuture, channel, localAddress, promise) is still a familiar routine inside. The task is handed over to the nio ssc thread for execution
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}
The following is the execution chain of the method: layer by layer
Finally came to the working method, which is too deep in the call chain: similarly, we only focus on two places
@Override
public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
assertEventLoop();
if (!promise.setUncancellable() || !ensureOpen(promise)) {
return;
}
// See: https://github.com/netty/netty/issues/576
if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
localAddress instanceof InetSocketAddress &&
!((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
!PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
// Warn a user about the fact that a non-root user can't receive a
// broadcast packet on *nix if the socket is bound on non-wildcard address.
logger.warn(
"A non-root user can't receive a broadcast packet if the socket " +
"is not bound to a wildcard address; binding to a non-wildcard " +
"address (" + localAddress + ") anyway as requested.");
}
boolean wasActive = isActive();
try {
//1. Core calling method
doBind(localAddress); // 1
} catch (Throwable t) {
safeSetFailure(promise, t);
closeIfClosed();
return;
}
//Determine if it is isActive
if (!wasActive && isActive()) { // 2
invokeLater(new Runnable() {
@Override
public void run() {
pipeline.fireChannelActive();
}
});
}
safeSetSuccess(promise);
}
1. doBind(localAddress)
@SuppressJava6Requirement(reason = "Usage guarded by java version check")
@Override
protected void doBind(SocketAddress localAddress) throws Exception {
//Judge whether the java version is > 7
if (PlatformDependent.javaVersion() >= 7) {
//javaChannel is actually ServerSocketChannel
//register
javaChannel().bind(localAddress, config.getBacklog());
} else {
javaChannel().socket().bind(localAddress, config.getBacklog());
}
}
Summary: this method is used to bind according to different java versions
2. pipeline.fireChannelActive()
To enter this method, first judge whether it is isActive, that is, whether the current ServerSocketChannel is available after a series of operations.
Up to now, there are three processors on the pipeline:
head --> acceptor --> tail
Call pipeline After firechannelactive(), the methods of all processors will be executed in sequence. Of course, acceptor and tail have little impact at this time, because acceptor has been handled in initAndRegister. So now navigate directly to the HeadContext, in the defaultchannelpipeline Java class. Navigate to the ChannelActice method:
@Override
public void channelActive(ChannelHandlerContext ctx) {
ctx.fireChannelActive();
//Focus on events
readIfIsAutoRead();
}
We trace the calling process of readIfIsAutoRead():
Find the final calling method:
@Override
protected void doBeginRead() throws Exception {
// Get selectionKey
final SelectionKey selectionKey = this.selectionKey;
//Judge whether it is illegal
if (!selectionKey.isValid()) {
return;
}
readPending = true;
//Get events of interest
final int interestOps = selectionKey.interestOps();
//If the event of interest is not
if ((interestOps & readInterestOp) == 0) {
// Paying attention to reading events is actually equivalent to adding, for example, 0000 | 0010 = 0010
//Of course, readInterestOp = 16 can be added only when it is 0
selectionKey.interestOps(interestOps | readInterestOp);
}
}
Conclusion: this method is actually binding the event of interest
If there is any error, please point it out!!!