Source Code Analysis of Bootstrap Initialization Process--Start of netty Client

Posted by gonsman on Sat, 22 Jun 2019 21:04:18 +0200

Bootstrap initialization process

netty's client boot class is Bootstrap. Let's take a look at the initialization process of Bootstrap in the client part of spark's rpc.

TransportClientFactory.createClient(InetSocketAddress address)

Simply post out the Bootstrap initialization code

// Client boot object
Bootstrap bootstrap = new Bootstrap();
// Setting various parameters
bootstrap.group(workerGroup)
  .channel(socketChannelClass)
  // Disable Nagle's Algorithm since we don't want packets to wait
  // Turn off Nagle algorithm
  .option(ChannelOption.TCP_NODELAY, true)
  .option(ChannelOption.SO_KEEPALIVE, true)
  .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, conf.connectionTimeoutMs())
  .option(ChannelOption.ALLOCATOR, pooledAllocator);

// socket receiving buffer
if (conf.receiveBuf() > 0) {
  bootstrap.option(ChannelOption.SO_RCVBUF, conf.receiveBuf());
}

// socket send buffer
// The settings of receiving and sending buffers should be calculated using the following formulas:
// Delay*Bandwidth
// For example, if the delay is 1ms and the bandwidth is 10Gbps, the buffer size should be set to 1.25MB.
if (conf.sendBuf() > 0) {
  bootstrap.option(ChannelOption.SO_SNDBUF, conf.sendBuf());
}

final AtomicReference<TransportClient> clientRef = new AtomicReference<>();
final AtomicReference<Channel> channelRef = new AtomicReference<>();

// Setting handler (processor object)
bootstrap.handler(new ChannelInitializer<SocketChannel>() {
  @Override
  public void initChannel(SocketChannel ch) {
    TransportChannelHandler clientHandler = context.initializePipeline(ch);
    clientRef.set(clientHandler.getClient());
    channelRef.set(ch);
  }
});

// Connect to the remote server
long preConnect = System.nanoTime();
// Establish a connection with the server and start the method
ChannelFuture cf = bootstrap.connect(address);

There are several main steps:

  • First, create a Bootstrap object that calls a parametric constructor
  • Setting various parameters, such as channel type, closing Nagle algorithm, receiving and sending buffer size, setting processor
  • Call connect to establish a connection with the server

Next, we analyze Bootstrap's start-up process mainly through two clues: constructor and connect. For the process of setting parameters, we only assign values to some internal member variables, so we don't need to expand in detail.

Bootstrap.Bootstrap()

Bootstrap inherits AbstractBootstrap and looks at their parametric construction methods. They are all empty methods... So this step, we will save, instantly feel like flying up or not.^^

Bootstrap.connect(SocketAddress remoteAddress)

public ChannelFuture connect(SocketAddress remoteAddress) {
    // Check non-empty
    ObjectUtil.checkNotNull(remoteAddress, "remoteAddress");
    // Also check for non-null member variables, mainly EventLoopGroup, ChannelFactory, handler objects
    validate();
    return doResolveAndConnect(remoteAddress, config.localAddress());
}

Mainly do some non-null checks, it should be noted that the ChannelFactory object settings, the previous spark in the Bootstrap initialization settings call. channel(socketChannelClass) method, this method is as follows:

public B channel(Class<? extends C> channelClass) {
    return channelFactory(new ReflectiveChannelFactory<C>(
            ObjectUtil.checkNotNull(channelClass, "channelClass")
    ));
}

A Reflective ChannelFactory object is created and assigned to the internal channelFactory member. This factory class creates a Channel instance by reflection based on the incoming Class object.

doResolveAndConnect

As can be seen from the logic of this method, the process of creating a connection is divided into two main steps.

  • Initialize a Channel object and register it in EventLoop
  • Call the doResolveAndConnect0 method to complete the establishment of tcp connection

It is noteworthy that the initAndRegister method returns a Future object, which is typically used for the implementation of asynchronous mechanisms. Here, if registration is not immediately successful, a listener is added to the returned futrue object to establish a tcp connection after registration is successful.

private ChannelFuture doResolveAndConnect(final SocketAddress remoteAddress, final SocketAddress localAddress) {
    // Initialize a Channel object and register it in EventLoop
    final ChannelFuture regFuture = initAndRegister();
    final Channel channel = regFuture.channel();

    if (regFuture.isDone()) {
        // If registration fails, the world returns the failed future object
        if (!regFuture.isSuccess()) {
            return regFuture;
        }
        return doResolveAndConnect0(channel, remoteAddress, localAddress, channel.newPromise());
    } else {// If registration is still in progress, you need to add a listener to the future object to do some work when registration is successful, and the listener is actually a callback object.
        // Registration future is almost always fulfilled already, but just in case it's not.
        final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
        regFuture.addListener(new ChannelFutureListener() {
            @Override
            public void operationComplete(ChannelFuture future) throws Exception {
                // Directly obtain the cause and do a null check so we only need one volatile read in case of a
                // failure.
                Throwable cause = future.cause();
                if (cause != null) {
                    // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
                    // IllegalStateException once we try to access the EventLoop of the Channel.
                    promise.setFailure(cause);
                } else {
                    // Registration was successful, so set the correct executor to use.
                    // See https://github.com/netty/netty/issues/2586
                    promise.registered();
                    // After successful registration, the doResolveAndConnect0 method is still called to complete the connection establishment process.
                    doResolveAndConnect0(channel, remoteAddress, localAddress, promise);
                }
            }
        });
        return promise;
    }

initAndRegister

There are still two steps:

  • Create a channel object through the channel factory class, retrieve the specified channel type's parametric constructor by reflection, and call the constructor to create the object
  • Initialize the channel object by calling init method, which is an abstract method. Bootstrap and Server Bootstrap are implemented differently.
  • Register channel into EventLoopGroup

Note a comment in the source code that is very helpful for understanding netty's thread model.

  • If the current code is executed in the EventLoopEvent thread, then it runs here to show that channel has successfully registered with EventLoopEvent, at which point it is no problem to call the bind() or connect() method again.
  • If the current code is not executed in the EventLoopEvent thread, that is to say, the current thread is another thread, it is still safe to continue calling the bind() or connect() method here, and there is no confusion in the execution order of the method due to concurrency, because only one channel in netty is bound to one thread. All operations related to the channel include registration, bind() and connect(). Or connect will be executed serially in a thread in the form of queuing tasks, which also avoids many thread security problems for netty, thus reducing a lot of lock and synchronization code, reducing thread switching caused by competing resources between threads, and improving thread execution efficiency on the side.

final ChannelFuture initAndRegister() {
Channel channel = null;
try {
// Create a channel object through the channel factory class
channel = channelFactory.newChannel();
// Initialize channel by calling init method
init(channel);
} catch (Throwable t) {
if (channel != null) {
// channel can be null if newChannel crashed (eg SocketException("too many open files"))
channel.unsafe().closeForcibly();
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(channel, GlobalEventExecutor.INSTANCE).setFailure(t);
}
// as the Channel is not registered yet we need to force the usage of the GlobalEventExecutor
return new DefaultChannelPromise(new FailedChannel(), GlobalEventExecutor.INSTANCE).setFailure(t);
}

    // Register in EventLoopGroup
    ChannelFuture regFuture = config().group().register(channel);
    // If an exception occurs, the established connection needs to be closed
    if (regFuture.cause() != null) {
        if (channel.isRegistered()) {
            channel.close();
        } else {
            channel.unsafe().closeForcibly();
        }
    }

    // If we are here and the promise is not failed, it's one of the following cases:
    // 1) If we attempted registration from the event loop, the registration has been completed at this point.
    //    i.e. It's safe to attempt bind() or connect() now because the channel has been registered.
    // 2) If we attempted registration from the other thread, the registration request has been successfully
    //    added to the event loop's task queue for later execution.
    //    i.e. It's safe to attempt bind() or connect() now:
    //         because bind() or connect() will be executed *after* the scheduled registration task is executed
    //         because register(), bind(), and connect() are all bound to the same thread.
    
    return regFuture;
}

NioSocket Channel initialization

DEFAULT_SELECTOR_PROVIDER is the default Selector Provider object, when a class defined in jdk is used to generate selector objects and channel channel objects.

public NioSocketChannel() {
    this(DEFAULT_SELECTOR_PROVIDER);
}

In newSocket, a SocketChannel object is created by calling the provider.openSocketChannel() method. Its default implementation is SocketChannelImpl.
public NioSocketChannel(SelectorProvider provider) {
this(newSocket(provider));
}

After several calls, the following constructor is finally called. First, the constructor of the parent class AbstractNioByteChannel is called.
Then a SocketChannelConfig object is created, which is somewhat similar to the facade pattern, encapsulating some parameters setting and acquisition interfaces of NioSocketChannel object and Socket object.
public NioSocketChannel(Channel parent, SocketChannel socket) {
super(parent, socket);
config = new NioSocketChannelConfig(this, socket.socket());
}

Let's move on to the construction of the parent class AbstractNioByteChannel

AbstractNioByteChannel(Channel parent, SelectableChannel ch)

Notice that there is an additional parameter, SelectionKey.OP_READ, which represents the event of interest at the beginning of the channel. Channel is interested in read event just after it is created.
protected AbstractNioByteChannel(Channel parent, SelectableChannel ch) {
super(parent, ch, SelectionKey.OP_READ);
}

AbstractNioChannel(Channel parent, SelectableChannel ch, int readInterestOp)

The main thing is to call the construction method of the parent class.

protected AbstractNioChannel(Channel parent, SelectableChannel ch, int readInterestOp) {
    // Parent Class Construction Method
    super(parent);
    this.ch = ch;
    this.readInterestOp = readInterestOp;
    try {
        // Setting up non-blocking
        ch.configureBlocking(false);
    } catch (IOException e) {
        try {
            // If an exception occurs, close the channel
            ch.close();
        } catch (IOException e2) {
            if (logger.isWarnEnabled()) {
                logger.warn(
                        "Failed to close a partially initialized socket.", e2);
            }
        }

        throw new ChannelException("Failed to enter non-blocking mode.", e);
    }
}

AbstractChannel(Channel parent)

The most critical initialization logic is in this top-level base class, which contains two heavy objects, Unsafe object and ChannelPipeline object. The former encapsulates the invocation of jdk underlying api, and the latter is the core class to realize netty's chain processing of events.

protected AbstractChannel(Channel parent) {
    this.parent = parent;
    // Create a ChannelId object that uniquely identifies the channel
    id = newId();
    // Unsafe object, encapsulating jdk underlying api calls
    unsafe = newUnsafe();
    // Create a DefaultChannelPipeline object
    pipeline = newChannelPipeline();
}

Summary

In the previous section, we briefly analyzed the initialization process of NioSocket Channel. We can see that the most important logic is the construction method of AbstractChannel. Here we see two important class creation processes.

Bootstrap.init

Back to the AbstractBootstrap.initAndRegister method, after invoking the NioSocket Channel construction method through reflection and creating an instance, the newly created Channel instance will be initialized immediately. Let's take a look at the initialization process of Bootstrap for the newly created Channel:

  • Add a processor to channel's Pipeline. channel Pipeline can be understood as a pipeline. There are various processors on this pipeline. When a channel event is generated, it will propagate on this pipeline, passing through all processors in turn.
  • Setting parameters, that is, some parameters with ChannelOption as the key, can be seen through the DefaultChannelConfig.setOption method which parameters can be set.
  • set a property

    void init(Channel channel) throws Exception {
    ChannelPipeline p = channel.pipeline();
    // Add a processor to Channel Pipeline, which is the processor we set up earlier.
    p.addLast(config.handler());

      final Map<ChannelOption<?>, Object> options = options0();
      // Setting parameters, finally setting interface parameters by calling some parameters of SocketChannelConfig
      synchronized (options) {
          setChannelOptions(channel, options, logger);
      }
    
      final Map<AttributeKey<?>, Object> attrs = attrs0();
      // set a property
      synchronized (attrs) {
          for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) {
              channel.attr((AttributeKey<Object>) e.getKey()).set(e.getValue());
          }
      }

    }

MultithreadEventLoopGroup.register

After the creation and initialization of channel is completed, we will register the channel into an EventLoop. NioNioEventLoop inherits from the MultithreadEventLoop Group and completes the registration by calling the register method of SingleThreadEventLoop.

public ChannelFuture register(Channel channel) {
    return next().register(channel);
}

As you can see, one of the EventLoop s is selected by the next() method for registration. Multithread Event Loop Group is the encapsulation of multiple real Event Loop Groups. Each real Event Loop Group that implements the actual functionality runs in one thread.
So next we should look at a single EventLoopGroup registration method.

SingleThreadEventLoop.register

A DefaultChannelPromise object is created here for use as a return value.

public ChannelFuture register(Channel channel) {
    return register(new DefaultChannelPromise(channel, this));
}

Finally, the Unsafe register method is called to bind channel to the current EventLoopGroup object.
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
promise.channel().unsafe().register(this, promise);
return promise;
}

AbstractChannel.AbstractUnsafe.register

  • First, do some pre-checks, including variable non-null checks, repeated registration checks, check whether the channel type and EventLoopGroup type match.
  • Bind this channel to the specified eventLoop object.
  • Call register0 to complete registration

      public final void register(EventLoop eventLoop, final ChannelPromise promise) {
          // Do some non-empty checks
          if (eventLoop == null) {
              throw new NullPointerException("eventLoop");
          }
          // If the registration is repeated, an exception is thrown through the future object
          // A channel can only be registered on an EventLoopGroup object
          if (isRegistered()) {
              promise.setFailure(new IllegalStateException("registered to an event loop already"));
              return;
          }
          // Check whether the channel type matches the EventLoopGroup type
          if (!isCompatible(eventLoop)) {
              promise.setFailure(
                      new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
              return;
          }
    
          // Set EvetLoop members inside channel as corresponding objects
          // That is, bind the channel to the specified top eventLoop
          AbstractChannel.this.eventLoop = eventLoop;
    
          // Here's a judgment. If you're currently in the thread corresponding to EvetLoop, execute the code directly.
          // If the currently running thread is not the same as EvetLoop, add the registered task to the EvetLoop task queue
          if (eventLoop.inEventLoop()) {
              register0(promise);
          } else {
              try {
                  eventLoop.execute(new Runnable() {
                      @Override
                      public void run() {
                          register0(promise);
                      }
                  });
              } catch (Throwable t) {
                  logger.warn(
                          "Force-closing a channel whose registration task was not accepted by an event loop: {}",
                          AbstractChannel.this, t);
                  closeForcibly();
                  closeFuture.setClosed();
                  safeSetFailure(promise, t);
              }
          }
      }

AbstractChannel.AbstractUnsafe.register0

This method implements the actual registration logic.

  • There are still some pre-settings and checks to be done, including checking whether channel is alive or not, which can not be cancelled during the registration process.
  • Call the api of jdk to complete registration. For example, the registration of jdk Nio channels is to call Selectable Channel. register (Selector sel, int ops, Object att)
  • Call the ChannelHandler.handlerAdded method of all the added processor nodes, which in fact also calls the handler.handlerRemoved method, if handler has been removed before that.
  • Notify that the future object has been registered successfully
  • Trigger an event that registers successfully with channel, which will be propagated in the pipeline, and all registered handler s will receive the event in turn and process it accordingly
  • If it's the first registration, it also needs to trigger a channel survival event, so that all handler s do the appropriate processing.

      private void register0(ChannelPromise promise) {
          try {
              // check if the channel is still open as it could be closed in the mean time when the register
              // call was outside of the eventLoop
              // Set ChannelPromise to be irrevocable and check if channel is still alive. Check if channel is alive by channeling the internal jdk
              if (!promise.setUncancellable() || !ensureOpen(promise)) {
                  return;
              }
              // Is it the first registration?
              // How many times will the TODO be registered?
              boolean firstRegistration = neverRegistered;
              // Complete the actual registration, that is, the invocation of the underlying api
              // If the channel registration for jdk Nio is called Selectable Channel. register (Selector sel, int ops, Object att)
              doRegister();
              // Update flag variables
              neverRegistered = false;
              registered = true;
    
              // Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
              // user may already fire events through the pipeline in the ChannelFutureListener.
              // Call the ChannelHandler.handlerAdded method of all added processor nodes
              pipeline.invokeHandlerAddedIfNeeded();
    
              // Registered successfully through the future object
              safeSetSuccess(promise);
              // Trigger an event that registers successfully with channel, which will propagate in pipeline.
              // All registered handler s receive the event in turn and process it accordingly
              pipeline.fireChannelRegistered();
              // Only fire a channelActive if the channel has never been registered. This prevents firing
              // multiple channel actives if the channel is deregistered and re-registered.
              if (isActive()) {
                  if (firstRegistration) {
                      // If it's the first registration, it also needs to trigger a channel survival event, so that all handler s do the appropriate processing.
                      pipeline.fireChannelActive();
                  } else if (config().isAutoRead()) {
                      // This channel was registered before and autoRead() is set. This means we need to begin read
                      // again so that we process inbound data.
                      //
                      // See https://github.com/netty/netty/issues/4805
                      // Start receiving read events
                      // For Nio-type channel s, the event of interest is registered by calling the relevant api of jdk
                      beginRead();
                  }
              }
          } catch (Throwable t) {
              // Close the channel directly to avoid FD leak.
              closeForcibly();
              closeFuture.setClosed();
              safeSetFailure(promise, t);
          }
      }

Summary

So far, we have completed the process of creating, initializing and registering channel to EventLoop. The whole process is not complicated, but the nesting of code is deep and the inheritance structure is complex. Some simple functions may need several layers to find the real place, so we need patience and familiarity. Here, I will refine the trunk logic, remove all the details of the logic, once again can have a holistic understanding:

  • First, a NioSocket Channel is created by reflection (invoking the parametric constructor by reflection)
  • Then the channel object is initialized, mainly to add the handler set by the user in Channel Pipeline of the channel.
  • Finally, the channel is registered on an EventLoop. The registration process designs a call to the selector registration api at the bottom of jdk, calls the handler's callback method, and triggers a channel registered event in the channel Pipeline, which finally calls back the channelRegistered method of each handler object.

Next, we go back to the Bootstrap.doResolveAndConnect method and continue to analyze the process of establishing the connection.

Bootstrap.doResolveAndConnect0

Connection is established in the method doResolveAndConnect0:

The main work of this method is to parse remote addresses, such as domain names through dns servers.
Then use the parsed address to establish the connection, which calls the doConnect method.

private ChannelFuture doResolveAndConnect0(final Channel channel, SocketAddress remoteAddress,
                                           final SocketAddress localAddress, final ChannelPromise promise) {
    try {
        final EventLoop eventLoop = channel.eventLoop();
        // Get an address resolver
        final AddressResolver<SocketAddress> resolver = this.resolver.getResolver(eventLoop);

        // If the parser does not support the address or if the address has been resolved, then start creating the connection directly.
        if (!resolver.isSupported(remoteAddress) || resolver.isResolved(remoteAddress)) {
            // Resolver has no idea about what to do with the specified remote address or it's resolved already.
            doConnect(remoteAddress, localAddress, promise);
            return promise;
        }

        // Resolution of remote addresses
        final Future<SocketAddress> resolveFuture = resolver.resolve(remoteAddress);

        if (resolveFuture.isDone()) {
            final Throwable resolveFailureCause = resolveFuture.cause();

            if (resolveFailureCause != null) {
                // Failed to resolve immediately
                channel.close();
                promise.setFailure(resolveFailureCause);
            } else {
                // Succeeded to resolve immediately; cached? (or did a blocking lookup)
                // Connect after successful parsing
                doConnect(resolveFuture.getNow(), localAddress, promise);
            }
            return promise;
        }

        // Wait until the name resolution is finished.
        // Add a callback to the future object and connect it asynchronously.
        resolveFuture.addListener(new FutureListener<SocketAddress>() {
            @Override
            public void operationComplete(Future<SocketAddress> future) throws Exception {
                if (future.cause() != null) {
                    channel.close();
                    promise.setFailure(future.cause());
                } else {
                    doConnect(future.getNow(), localAddress, promise);
                }
            }
        });
    } catch (Throwable cause) {
        promise.tryFailure(cause);
    }
    return promise;
}

Bootstrap.doConnect

Call the connect method of channel to complete the connection process.
Maybe it's the habit of looking at scala code before. Looking back at Java code, it feels redundant. A lot of code expresses that logic. It feels that the information density is too low. Nowadays, many people think that Java will gradually decline. In the language that is most likely to replace java, scala is absolutely one of the strong competitors. Without comparison, scala is harmless. Compared with java, scala is really a language. It is too concise to express the logic precisely and directly in a few sentences. It seems a little closer to declarative programming.

private static void doConnect(
        final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise connectPromise) {

    // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up
    // the pipeline in its channelRegistered() implementation.
    final Channel channel = connectPromise.channel();
    channel.eventLoop().execute(new Runnable() {
        @Override
        public void run() {
            if (localAddress == null) {
                // Call the channel.connect method to complete the connection
                channel.connect(remoteAddress, connectPromise);
            } else {
                channel.connect(remoteAddress, localAddress, connectPromise);
            }
            connectPromise.addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
        }
    });
}

AbstractChannel.connect

public ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
    return pipeline.connect(remoteAddress, promise);
}

DefaultChannelPipeline.connect

If you are familiar with netty, you should know that netty uses the responsibility chain model for handling io events. That is, users can set up multiple processors. These processors form a chain. The io events propagate on the chain and are processed by specific processors. Among them, two special processors are h. Ead and tail, which are the head and tail of the chain, exist mainly to realize some special logic.

public final ChannelFuture connect(SocketAddress remoteAddress, ChannelPromise promise) {
    return tail.connect(remoteAddress, promise);
}

AbstractChannelHandlerContext.connect

After several calls, the method is finally called. Here is a key code, findContextOutbound(MASK_CONNECT). I will not post the code of this method. I will talk about its function, more specific mechanism and so on. This method traverses the processor chain backwards and forwards until it finds a processor capable of handling connection events. Whether or not a certain type of event can be handled is determined by bits. Each AbstractChannelHandlerContext object has an int variable that stores bits that mark various types of events. Generally, the connection event will be handled by the head er node, the DefaultChannelPipeline.HeadContext class, so let's look directly at the DefaultChannelPipeline.HeadContext.connect method.

public ChannelFuture connect(
        final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise promise) {

    if (remoteAddress == null) {
        throw new NullPointerException("remoteAddress");
    }
    if (isNotValidPromise(promise, false)) {
        // cancelled
        return promise;
    }

    // Find the next one that can connect, where bits are used to mark different types of operations.
    final AbstractChannelHandlerContext next = findContextOutbound(MASK_CONNECT);
    EventExecutor executor = next.executor();
    if (executor.inEventLoop()) {
        // Call AbstractChannelHandlerContext.invokeConnect
        next.invokeConnect(remoteAddress, localAddress, promise);
    } else {
        safeExecute(executor, new Runnable() {
            @Override
            public void run() {
                next.invokeConnect(remoteAddress, localAddress, promise);
            }
        }, promise, null);
    }
    return promise;
}

DefaultChannelPipeline.HeadContext.connect

public void connect(
            ChannelHandlerContext ctx,
            SocketAddress remoteAddress, SocketAddress localAddress,
            ChannelPromise promise) {
        unsafe.connect(remoteAddress, localAddress, promise);
    }

Assignment of unsafe objects:

    HeadContext(DefaultChannelPipeline pipeline) {
        super(pipeline, null, HEAD_NAME, HeadContext.class);
        unsafe = pipeline.channel().unsafe();
        setAddComplete();
    }

So let's look directly at unsafe.connect.

AbstractNioChannel.connect

Main logic:

  • State check, non-empty check
  • Call the doConnect method to connect
  • If the connection succeeds immediately, set the future object to succeed
  • If the timeout is greater than 0, a delayed scheduling task is submitted, which is performed after the timeout arrives to check whether the connection is successful. If the connection timeout is indicated for a successful connection, the channel needs to be closed.
  • Add a callback to the future object to close the channel when the future is cancelled by an external caller

So the core method of establishing connection is doConnect, which is an abstract method. Let's look at NioSocket Channel, that is, the process of establishing tcp connection. Look at the implementation class of AbstractNioChannel and find UDP,SCTP and other protocols.

public final void connect(
            final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise promise) {
        // Check promise status, channel survival status
        if (!promise.setUncancellable() || !ensureOpen(promise)) {
            return;
        }

        try {
            // Preventing duplicate connections
            if (connectPromise != null) {
                // Already a connect in process.
                throw new ConnectionPendingException();
            }

            boolean wasActive = isActive();
            // Call the doConnect method to connect
            if (doConnect(remoteAddress, localAddress)) {
                // If the connection succeeds immediately, set the future object to succeed
                fulfillConnectPromise(promise, wasActive);
            } else {
                connectPromise = promise;
                requestedRemoteAddress = remoteAddress;

                // Schedule connect timeout.
                int connectTimeoutMillis = config().getConnectTimeoutMillis();
                // If the timeout is greater than 0, the connection will be checked for success after the timeout arrives.
                if (connectTimeoutMillis > 0) {
                    connectTimeoutFuture = eventLoop().schedule(new Runnable() {
                        @Override
                        public void run() {
                            ChannelPromise connectPromise = AbstractNioChannel.this.connectPromise;
                            ConnectTimeoutException cause =
                                    new ConnectTimeoutException("connection timed out: " + remoteAddress);
                            // If connectPromise can be marked as a failure, it means that the connection has not succeeded at this time, that is, the connection has timed out.
                            // The channel needs to be closed at this time.
                            if (connectPromise != null && connectPromise.tryFailure(cause)) {
                                close(voidPromise());
                            }
                        }
                    }, connectTimeoutMillis, TimeUnit.MILLISECONDS);
                }

                // Add a callback to the future object to close the channel when the future is cancelled by an external caller
                promise.addListener(new ChannelFutureListener() {
                    @Override
                    public void operationComplete(ChannelFuture future) throws Exception {
                        if (future.isCancelled()) {
                            if (connectTimeoutFuture != null) {
                                connectTimeoutFuture.cancel(false);
                            }
                            connectPromise = null;
                            close(voidPromise());
                        }
                    }
                });
            }
        } catch (Throwable t) {
            promise.tryFailure(annotateConnectException(t, remoteAddress));
            closeIfClosed();
        }
    }

NioSocketChannel.doConnect

  • First bind the specified local address
  • Call SocketUtils.connect to establish a connection

    protected boolean doConnect(SocketAddress remoteAddress, SocketAddress localAddress) throws Exception {
    // Bind the specified local address
    if (localAddress != null) {
    doBind0(localAddress);
    }

      // This variable marks whether the connection initiation was successful
      // Successful initiation of connection establishment does not mean that the connection has been successfully established
      boolean success = false;
      try {
          // Statements that actually establish connections
          boolean connected = SocketUtils.connect(javaChannel(), remoteAddress);
          if (!connected) {
              selectionKey().interestOps(SelectionKey.OP_CONNECT);
          }
          success = true;
          // Whether the return connection has been successfully established
          return connected;
      } finally {
          if (!success) {
              doClose();
          }
      }

    }

SocketUtils.connect

As you can see, the connection is finally established by calling jdk's api, the SocketChannel.connect method

public static boolean connect(final SocketChannel socketChannel, final SocketAddress remoteAddress)
        throws IOException {
    try {
        return AccessController.doPrivileged(new PrivilegedExceptionAction<Boolean>() {
            @Override
            public Boolean run() throws IOException {
                // Call jdk api to establish a connection, SocketChannel.connect
                return socketChannel.connect(remoteAddress);
            }
        });
    } catch (PrivilegedActionException e) {
        throw (IOException) e.getCause();
    }
}

summary

In a word, the code is really deep! Very direct, if you look at it for the first time, it is easy to get lost in the hierarchical inheritance structure without a code framework diagram beside it. Many code layer calls, and the really useful logic is hidden deeply. So you must have patience, perseverance, and the determination to break the casserole to the end. However, the advantage of such a complex code structure is also obvious, that is, good scalability, you can extend at any level.

Summarizing the process of establishing connections, I think it can be summed up in three main aspects:

  • First, the code that actually establishes the logic must still be the jdk api
  • Second, the main function of so many method calls is to cater to the requirements of the framework, essentially for code extensibility, such as the processor chain of Channel Pipeline.
  • Thirdly, another main task is to process the future object, which is an important means to achieve asynchrony. The future object is also a link between the external caller and the internal state of the object. The caller completes some functions through the future object, such as checking the state, sending out cancel actions, and realizing blocking waiting.

Topics: PHP JDK Netty socket Scala