netty - start process

Posted by superdude on Fri, 17 Jan 2020 12:33:42 +0100

netty start process

  • Start instance:
public final class EchoServer {
    static final boolean SSL = System.getProperty("ssl") != null;
    static final int PORT = Integer.parseInt(System.getProperty("port", "8007"));

    public static void main(String[] args) throws Exception {
        // Configure SSL.
        final SslContext sslCtx;
        if (SSL) {
            SelfSignedCertificate ssc = new SelfSignedCertificate();
            sslCtx = SslContextBuilder.forServer(ssc.certificate(), ssc.privateKey()).build();
        } else {
            sslCtx = null;
        }
        // Configure the server.
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        final EchoServerHandler serverHandler = new EchoServerHandler();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
             .channel(NioServerSocketChannel.class)
             .option(ChannelOption.SO_BACKLOG, 100)
             .handler(new LoggingHandler(LogLevel.INFO))
             .childHandler(new ChannelInitializer<SocketChannel>() {
                 @Override
                 public void initChannel(SocketChannel ch) throws Exception {
                     ChannelPipeline p = ch.pipeline();
                     if (sslCtx != null) {
                         p.addLast(sslCtx.newHandler(ch.alloc()));
                     }
                     //p.addLast(new LoggingHandler(LogLevel.INFO));
                     p.addLast(serverHandler);
                 }
             });
            // Start the server.
            ChannelFuture f = b.bind(PORT).sync();
            // Wait until the server socket is closed.
            f.channel().closeFuture().sync();
        } finally {
            // Shut down all event loops to terminate all threads.
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
}

Structural mode

Start step

  • . create bossGroup
EventLoopGroup bossGroup=new NioEventLoopGroup();//Equivalent to mainReactor
  • Create workerGroup:
EventLoopGroup workerGroup=new NioEventGroup();//Equivalent to subReactor
  • Create startup class
ServerBootstrap sb=new ServerBootstrap()
  • Set properties and business processing Handler for startup class:
 b.group(bossGroup, workerGroup)
             .channel(NioServerSocketChannel.class)
             .option(ChannelOption.SO_BACKLOG, 100)
             .handler(new LoggingHandler(LogLevel.INFO))
             .childHandler(new ChannelInitializer<SocketChannel>() {
                 @Override
                 public void initChannel(SocketChannel ch) throws Exception {
                     ChannelPipeline p = ch.pipeline();
                     if (sslCtx != null) {
                         p.addLast(sslCtx.newHandler(ch.alloc()));
                     }
                     //p.addLast(new LoggingHandler(LogLevel.INFO));
                     p.addLast(serverHandler);
                 }
             });
  • Binding port:
	 ChannelFuture f = b.bind(PORT).sync();

Source code process analysis

  1. Start bossGroup, each EventLoopGroup has a group of nioeventloops, and each NioEventLoop will start a Selector
1.Establish
EventLoopGroup bossGroup = new NioEventLoopGroup();
2.Get into NioEventLoopGroup in
  public NioEventLoopGroup() {
        this(0);
    }
    //The number of threads will be created based on the number of threads provided
    public NioEventLoopGroup(int nThreads) {
        this(nThreads, (Executor) null);
    }
    .....
 3.   //Finally, the parent class MultithreadEventLoopGroup is called. If the number of threads initialized is 0, the initialization value is 2*cpu core number.
    protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
        super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
    }
    //Create EventNioLoop
    protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
                                            EventExecutorChooserFactory chooserFactory, Object... args) {
        if (nThreads <= 0) {
            throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
        }
//If the executor is null, create a default threadpertask executor, using Netty's default thread factory.
        if (executor == null) {
            executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
        }
    //Create an array of thread pools (singleton thread pools) based on the number of incoming threads (CPU*2).
        children = new EventExecutor[nThreads];
	//Loop through the elements in the array. If not, close all singleton thread pools.
        for (int i = 0; i < nThreads; i ++) {
            boolean success = false;
            try {
            	//Call the implementation of the subclass NioEventLoopGroup to create a NioEventLoop
                children[i] = newChild(executor, args);
                success = true;
            } catch (Exception e) {
                // TODO: Think about if this is a good exception type
                throw new IllegalStateException("failed to create a child event loop", e);
            } finally {
                if (!success) {
                    for (int j = 0; j < i; j ++) {
                        children[j].shutdownGracefully();
                    }

                    for (int j = 0; j < i; j ++) {
                        EventExecutor e = children[j];
                        try {
                            while (!e.isTerminated()) {
                                e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
                            }
                        } catch (InterruptedException interrupted) {
                            // Let the caller handle the interruption.
                            Thread.currentThread().interrupt();
                            break;
                        }
                    }
                }
            }
        }
//Create a thread selector according to the thread selection factory. The default is to take the remainder (bit operation) of 2, which can also be obtained in sequence.
        chooser = chooserFactory.newChooser(children);

        final FutureListener<Object> terminationListener = new FutureListener<Object>() {
            @Override
            public void operationComplete(Future<Object> future) throws Exception {
                if (terminatedChildren.incrementAndGet() == children.length) {
                    terminationFuture.setSuccess(null);
                }
            }
        };
		//Add a shutdown listener for each singleton thread pool.
        for (EventExecutor e: children) {
            e.terminationFuture().addListener(terminationListener);
        }
	//Add all singleton thread pools to a HashSet
        Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);
        Collections.addAll(childrenSet, children);
        readonlyChildren = Collections.unmodifiableSet(childrenSet);
    }
    4.Get into NioEventLoop
    //Create NioEventLoop and initialize a series of parameters
      NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
                 SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler,
                 EventLoopTaskQueueFactory queueFactory) {
        super(parent, executor, false, newTaskQueue(queueFactory), newTaskQueue(queueFactory),
                rejectedExecutionHandler);
        if (selectorProvider == null) {
            throw new NullPointerException("selectorProvider");
        }
        if (strategy == null) {
            throw new NullPointerException("selectStrategy");
        }
        provider = selectorProvider;
     	//Initialize create Selector
        final SelectorTuple selectorTuple = openSelector();
        selector = selectorTuple.selector;
        unwrappedSelector = selectorTuple.unwrappedSelector;
        selectStrategy = strategy;
    }

2. Bind listening port
Create a service startup auxiliary class ServerBootstrap, call the Bind method of this class, and start the service

//Create service startup class
  ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
             .channel(NioServerSocketChannel.class)
             .option(ChannelOption.SO_BACKLOG, 100)
             .handler(new LoggingHandler(LogLevel.INFO))
             .childHandler(new ChannelInitializer<SocketChannel>() {
                 @Override
                 public void initChannel(SocketChannel ch) throws Exception {
                     ChannelPipeline p = ch.pipeline();
                     if (sslCtx != null) {
                         p.addLast(sslCtx.newHandler(ch.alloc()));
                     }
                     //p.addLast(new LoggingHandler(LogLevel.INFO));
                     p.addLast(serverHandler);
                 }
             });
            // Start the server.
            ChannelFuture f = b.bind(PORT).sync();
  • Call the bind() method of ServerBootstrap to bind the port
	//1. Call the parent class abstractbootstrap × bind
	public ChannelFuture bind(int inetPort) {
        return bind(new InetSocketAddress(inetPort));
    }
    ....
     public ChannelFuture bind(SocketAddress localAddress) {
        //2. Verify whether EventLoopGroup parentGroup is empty. parentGroup is set through serverbootstrap ා group
        //3. Judge whether channelFactory is empty. channelFactory is the reflective channelFactory instance created by calling the channel method
        validate();
        return doBind(ObjectUtil.checkNotNull(localAddress, "localAddress"));
    }
  • Enter the doBind operation and execute the binding port. The doBind method mainly includes the following operations: 1. Perform initialization and registration operation; 2. Perform binding operation
private ChannelFuture doBind(final SocketAddress localAddress) {
		//1. Perform initialization and registration
        final ChannelFuture regFuture = initAndRegister();
        final Channel channel = regFuture.channel();
        if (regFuture.cause() != null) {
            return regFuture;
        }
        if (regFuture.isDone()) {
            // At this point we know that the registration was complete and successful.
            ChannelPromise promise = channel.newPromise();
           	//
            doBind0(regFuture, channel, localAddress, promise);
            return promise;
        } else {
            // Registration future is almost always fulfilled already, but just in case it's not.
            final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
            regFuture.addListener(new ChannelFutureListener() {
                @Override
                public void operationComplete(ChannelFuture future) throws Exception {
                    Throwable cause = future.cause();
                    if (cause != null) {
                        // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
                        // IllegalStateException once we try to access the EventLoop of the Channel.
                        promise.setFailure(cause);
                    } else {
                        // Registration was successful, so set the correct executor to use.
                        // See https://github.com/netty/netty/issues/2586
                        promise.registered();

                        doBind0(regFuture, channel, localAddress, promise);
                    }
                }
            });
            return promise;
        }
    }
  • Channel initialization and registration operations:
final ChannelFuture initAndRegister() {
        Channel channel = null;
        try {
        	//Call the factory method to create the channel. The channelFactory is created through the serverbootstrap ා Channel(NioServerSocketChannel.class). The channelFactory class is ReflectiveChannelFactory. Through the ReflectiveChannelFactory construction method, the NioServerSocketChannel channel channel class passed from the Channel(NioServerSocketChannel.class) is created by reflection
            channel = channelFactory.newChannel();
            //Initialize channel
            init(channel);
        } catch (Throwable t) {
            if (channel != null) {
                // channel can be null if newChannel crashed (eg SocketException("too many open files"))
                channel.unsafe().closeForcibly();
                // as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
                return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
            }
            // as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
            return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
        }
        //Start registering service to selector
        ChannelFuture regFuture = config().group().register(channel);
        if (regFuture.cause() != null) {
            if (channel.isRegistered()) {
                channel.close();
            } else {
                channel.unsafe().closeForcibly();
            }
        }
        return regFuture;
    }
 @Override
    void init(Channel channel) {
 		//1. Set channelOption 
        setChannelOptions(channel, options0().entrySet().toArray(newOptionArray(0)), logger);
        //2. set attr
        setAttributes(channel, attrs0().entrySet().toArray(newAttrArray(0)));
        ChannelPipeline p = channel.pipeline();
        final EventLoopGroup currentChildGroup = childGroup;//workerBoss
        final ChannelHandler currentChildHandler = childHandler;
        final Entry<ChannelOption<?>, Object>[] currentChildOptions =            childOptions.entrySet().toArray(newOptionArray(0));
        final Entry<AttributeKey<?>, Object>[] currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(0));
         //3. add a custom ChannelInitializer to the pipeline of channel. The initChannel method of ChannelInitializer will be called after the channel registration is completed, and initChannel has done a more important thing to add a ServerBootstrapAcceptor to channel (NioServerSocketChannel) pipeline and rewrite it in the NioServerSocketChannel. D method (for the Server side, this method is called when a new connection is established), in which new connection (for the ServerBootStrap, it can be called childChannel) option and attr are set, and at the same time, the childGroup.register method is called to register in the eventLoop thread selected in the workerGroup 
        p.addLast(new ChannelInitializer<Channel>() {
            @Override
            public void initChannel(final Channel ch) {
                final ChannelPipeline pipeline = ch.pipeline();
                ChannelHandler handler = config.handler();
                if (handler != null) {
                    pipeline.addLast(handler);
                }
                //The serverbootstrappaccepter handle is responsible for accepting the client connection and initializing the connection after creating the connection
                ch.eventLoop().execute(new Runnable() {
                    @Override
                    public void run() {
                        pipeline.addLast(new ServerBootstrapAcceptor(
                                ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
                    }
                });
            }
        });
    }
  • After initializing the channel properties, register the selector, return to the InitAndRegister method, and call config().group().register(channel); that is, call multithreadeventloopgroup ා register (channel channel channel) method
 @Override
    public ChannelFuture register(Channel channel) {
    	//1. next() will select NioEventLoop; create eventLoop selectors PowerOfTwoEventExecutorChooser and GenericEventExecutorChooser through defaulteventexecutorchooserfactory ා newchooser().
    	//2. Select a NioEventLoop, and each NioEventLoop will start a Selector, and then start to register the ServerSocketChannel
        return next().register(channel);
    }
  • Call nioeventloop ා register() to register, enter the parent class SingleThreadEventLoop method
 @Override
    public ChannelFuture register(final ChannelPromise promise) {
        ObjectUtil.checkNotNull(promise, "promise");
        //Call the internal class Unsafe of channel to register. The channel here is the NioServerSocketChannel above
        promise.channel().unsafe().register(this, promise);
        return promise;
    }
  • Enter NioServerSocketChannel. Entering the unsafe method, calling the procedure to call AbstractChannel (newUnsafe) for constructing the method call for the newUnsafe, and finally implementing the NioMessageUnsafe() instance for the subclass AbstractNioMessageChannel#newUnsafe() method, finally calling its parent class AbstractUnsafe#register, then the flow goes to the calling AbstractNioChannel#doRegister() method.
  @Override
    protected void doRegister() throws Exception {
        boolean selected = false;
        for (;;) {
            try {
                //If ops is 0, note that there is no channel registered here, but the selectionkey is obtained. There are two ways to register a channel. One is to call the channel's register method, and the other is to set the value of the selectionkey's interestOps. Netty uses the second method to register the events concerned by the channel by setting the interestOps of the selectionkey, which delays the actual registration
                selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
                return;
            } catch (CancelledKeyException e) {
                if (!selected) {
                    // Force the Selector to select now as the "canceled" SelectionKey may still be
                    // cached and not removed because no Select.select(..) operation was called yet.
                    eventLoop().selectNow();
                    selected = true;
                } else {
                    // We forced a select operation on the selector before but the SelectionKey is still cached
                    // for whatever reason. JDK bug ?
                    throw e;
                }
            }
        }
    }
  • At this point, the channel initialization and delay registration are completed, and then the binding port is completed: continue to execute abstractbootstrap ා dobind0() to start binding data
   private static void doBind0(
            final ChannelFuture regFuture, final Channel channel,
            final SocketAddress localAddress, final ChannelPromise promise) {

        // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up
        // the pipeline in its channelRegistered() implementation.
        //Start asynchronous execution of registration binding listening
        channel.eventLoop().execute(new Runnable() {
            @Override
            public void run() {
                if (regFuture.isSuccess()) {
                    channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
                } else {
                    promise.setFailure(regFuture.cause());
                }
            }
        });
    }
  • Call the bind method of channel to bind, and finally enter the bind() method of AbstractChannel class
    @Override
    public ChannelFuture bind(SocketAddress localAddress) {
    	//Call the bind method of pipLine to bind the address. The pipeLine is DefaultChannelPipeline
        return pipeline.bind(localAddress);
    }
  • Finally, the abstractchannel ා bind() method will be called
 @Override
        public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
            assertEventLoop();

            if (!promise.setUncancellable() || !ensureOpen(promise)) {
                return;
            }

            // See: https://github.com/netty/netty/issues/576
            if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
                localAddress instanceof InetSocketAddress &&
                !((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
                !PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
                // Warn a user about the fact that a non-root user can't receive a
                // broadcast packet on *nix if the socket is bound on non-wildcard address.
                logger.warn(
                        "A non-root user can't receive a broadcast packet if the socket " +
                        "is not bound to a wildcard address; binding to a non-wildcard " +
                        "address (" + localAddress + ") anyway as requested.");
            }

            boolean wasActive = isActive();
            try {
                doBind(localAddress);
            } catch (Throwable t) {
                safeSetFailure(promise, t);
                closeIfClosed();
                return;
            }
			//Determine whether it has been bound. After binding, call pipline ා firechannelactive() method
            if (!wasActive && isActive()) {
                invokeLater(new Runnable() {
                    @Override
                    public void run() {
                        pipeline.fireChannelActive();
                    }
                });
            }

            safeSetSuccess(promise);
        }
  • Call the subclass nioserversocketchannel ා dobind() method to bind the port, but the listening event has not been registered. Next, go on
    protected void doBind(SocketAddress localAddress) throws Exception {
        if (PlatformDependent.javaVersion() >= 7) {
            javaChannel().bind(localAddress, config.getBacklog());
        } else {
            javaChannel().socket().bind(localAddress, config.getBacklog());
        }
    }
  • Determines whether the binding has been bound, and calls pipLine#fireChannelActive() after binding.
 @Override
    public final ChannelPipeline fireChannelActive() {
    	//To execute the channelActive method of Channelhandler, we all know that there is a call chain handler in ChannelPipeline, which is divided into chain head and chain tail. Here, the incoming chain head is processed. The head is an instance of HeadContext
        AbstractChannelHandlerContext.invokeChannelActive(head);
        return this;
    }
  • Enter AbstractChannelHandlerContext, execute channelActive method of ChannelHandler. It is said above that head is HeadContext instance, and finally execute and call HeadContext ා channelActive() method
    @Override
        public void channelActive(ChannelHandlerContext ctx) {
        	//Call back to fireChannelActive, and finally the tailcontext ා channelactive method will be executed
            ctx.fireChannelActive();
            //Register read events here to create connection / read events
            readIfIsAutoRead();
        }
  • Enter the readIfIsAutoRead() method
 private void readIfIsAutoRead() {
 			//netty set auto read
            if (channel.config().isAutoRead()) {
            	//Execute the read method of channel
                channel.read();
            }
        }
  • Enter NioServerSocketChannel and trace back to abstractchannel
 @Override
    public Channel read() {
        pipeline.read();
        return this;
    }
  • It's back to calling the pipeline class for execution
@Override
    public final ChannelPipeline read() {
        tail.read();
        return this;
    }
  • The tail.read method is called, and tail is an instance of TailContext class
@Override
    public ChannelHandlerContext read() {
        final AbstractChannelHandlerContext next = findContextOutbound(MASK_READ);
        EventExecutor executor = next.executor();
        if (executor.inEventLoop()) {
            next.invokeRead();
        } else {
            Tasks tasks = next.invokeTasks;
            if (tasks == null) {
                next.invokeTasks = tasks = new Tasks(next);
            }
            executor.execute(tasks.invokeReadTask);
        }

        return this;
    }
  • Finally, execute to headcontext "read" (channelhandlercontext CTX)
 @Override
        public void read(ChannelHandlerContext ctx) {
            unsafe.beginRead();
        }
  • Finally, go to the method of unsafe.beginRead(). Unsafe is the instance of NioMessageUnsafe. Finally, call its parent class abstractniochannel ා dobeginread()
  @Override
    protected void doBeginRead() throws Exception {
        // Channel.read() or ChannelHandlerContext.read() was called
        final SelectionKey selectionKey = this.selectionKey;
        if (!selectionKey.isValid()) {
            return;
        }
        readPending = true;
        //The ops used at the beginning of registration above is 0. Here, it is determined whether the selectionkey.op'accept and op'read events have been registered. If not, the registration starts
        final int interestOps = selectionKey.interestOps();
        if ((interestOps & readInterestOp) == 0) {
            selectionKey.interestOps(interestOps | readInterestOp);
        }
    }
  • Set intersops on
  public NioServerSocketChannel(ServerSocketChannel channel) {
        super(null, channel, SelectionKey.OP_ACCEPT);
        config = new NioServerSocketChannelConfig(this, javaChannel().socket());
    }

So far, the whole process has been registered

48 original articles published, 10 praised, 50000 visitors+
Private letter follow

Topics: Netty socket SSL github