Non-blocking io uses a single thread or only a small number of multi-threads. Each connection shares a single thread. Thread resources can be released to handle other requests while waiting (without events). The main thread allocates resources to handle related events by notifying (waking up) the main thread through event-driven model when events such as accept/read/write occur. java.nio.channels.Selector is the observer of events in this model. It can register multiple events of SocketChannel on a Selector. When no event occurs, the Selector is blocked and wakes up the Selector when events such as accept/read/write occur in SocketChannel.
This selector uses a single thread model, mainly to describe event-driven model. To optimize performance, a good thread model is needed. At present, the better nio frameworks are Netty, mina of apache and so on. Thread model is shared later. Here we focus on the blocking and wake-up principle of Selector.
Let's start with a simple code for Selector
selector = Selector.open();
ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.configureBlocking(false);
ssc.socket().bind(new InetSocketAddress(port));
ssc.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
// select() block, waiting for an event to wake up
int selected = selector.select();
if (selected > 0) {
Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext()) {
SelectionKey key = selectedKeys.next();
if ((key.readyOps() & SelectionKey.OP_ACCEPT) == SelectionKey.OP_ACCEPT) {
// Handling accept events
} else if ((key.readyOps() & SelectionKey.OP_READ) == SelectionKey.OP_READ) {
// Handling read events
} else if ((key.readyOps() & SelectionKey.OP_WRITE) == SelectionKey.OP_WRITE) {
// Handling write events
}
selectedKeys.remove();
}
}
} The key points in the code are:
Selector.open();
selector.select();
The post-blocking wake-up can be initiated by registering an event with the socket on the selector or or by timeOut timeout or selector.wakeup().
The whole process of blocking and waking up involves a lot of points. It is easy to understand that the whole picture is sorted out first and then the source code is entered.
The source code in openjdk now parses each link in the figure above:
1.
Selector.open()
Selector.java
-----
public static Selector open() throws IOException {
return SelectorProvider.provider().openSelector();
} SelectorProvider.java
-----
public static SelectorProvider provider() {
synchronized (lock) {
if (provider != null)
return provider;
return (SelectorProvider)AccessController
.doPrivileged(new PrivilegedAction() {
public Object run() {
if (loadProviderFromProperty())
return provider;
if (loadProviderAsService())
return provider;
provider = sun.nio.ch.DefaultSelectorProvider.create();
return provider;
}
});
}
} Provider = sun. nio. ch. DefaultSelector Provider. create (); will return different implementation classes according to the operating system, and windows platform will return to windows Selector Provider;
Here we mainly comb the whole process with the implementation of windows, get the provider and see the implementation in openSelector().
WindowsSelectorProvider.java
----
public AbstractSelector openSelector() throws IOException {
return new WindowsSelectorImpl(this);
}
WindowsSelectorImpl.java
----
WindowsSelectorImpl(SelectorProvider sp) throws IOException {
super(sp);
pollWrapper = new PollArrayWrapper(INIT_CAP);
wakeupPipe = Pipe.open();
wakeupSourceFd = ((SelChImpl)wakeupPipe.source()).getFDVal();
// Disable the Nagle algorithm so that the wakeup is more immediate
SinkChannelImpl sink = (SinkChannelImpl)wakeupPipe.sink();
(sink.sc).socket().setTcpNoDelay(true);
wakeupSinkFd = ((SelChImpl)sink).getFDVal();
pollWrapper.addWakeupSocket(wakeupSourceFd, 0);
}
The following things are done in this code
Pipe.open() Open a pipe (see after the implementation of opening the pipe); get wakeup Source Fd and wakeup SinkFd file descriptors; put wakeup Source Fd in pollWrapper;
So why do we need a pipeline and how does it work? Next, look at what Pipe.open() did.
Pipe.java
----
public static Pipe open() throws IOException {
return SelectorProvider.provider().openPipe();
} Similarly, SelectorProvider.provider() is also an implementation related to acquiring an operating system
SelectorProvider.java
----
public Pipe openPipe() throws IOException {
return new PipeImpl(this);
} PipeImpl.java
----
PipeImpl(final SelectorProvider sp) throws IOException {
try {
AccessController.doPrivileged(new Initializer(sp));
} catch (PrivilegedActionException x) {
throw (IOException)x.getCause();
}
} Create a PipeImpl object, AccessController.doPrivileged calls and then execute the initializer's run method
PipeImpl.Initializer
-----
public Object run() throws IOException {
ServerSocketChannel ssc = null;
SocketChannel sc1 = null;
SocketChannel sc2 = null;
try {
// loopback address
InetAddress lb = InetAddress.getByName("127.0.0.1");
assert(lb.isLoopbackAddress());
// bind ServerSocketChannel to a port on the loopback address
ssc = ServerSocketChannel.open();
ssc.socket().bind(new InetSocketAddress(lb, 0));
// Establish connection (assumes connections are eagerly
// accepted)
InetSocketAddress sa
= new InetSocketAddress(lb, ssc.socket().getLocalPort());
sc1 = SocketChannel.open(sa);
ByteBuffer bb = ByteBuffer.allocate(8);
long secret = rnd.nextLong();
bb.putLong(secret).flip();
sc1.write(bb);
// Get a connection and verify it is legitimate
for (;;) {
sc2 = ssc.accept();
bb.clear();
sc2.read(bb);
bb.rewind();
if (bb.getLong() == secret)
break;
sc2.close();
}
// Create source and sink channels
source = new SourceChannelImpl(sp, sc1);
sink = new SinkChannelImpl(sp, sc2);
} catch (IOException e) {
try {
if (sc1 != null)
sc1.close();
if (sc2 != null)
sc2.close();
} catch (IOException e2) { }
IOException x = new IOException("Unable to establish"
+ " loopback connection");
x.initCause(e);
throw x;
} finally {
try {
if (ssc != null)
ssc.close();
} catch (IOException e2) { }
}
return null;
} This is the process of creating pipe for the bottom part of the figure above. The implementation under windows is to create two local socket Channels and then connect them (the process of linking is to write a random long to check the links of two sockets). The two socket Channels realize the source and sink ends of the pipeline respectively.
The source side is put into pollWrapper (pollWrapper.addWakeupSocket(wakeupSourceFd, 0)) by the Windows Selector Impl mentioned earlier.
PollArrayWrapper.java
----
private AllocatedNativeObject pollArray; // The fd array
// Adds Windows wakeup socket at a given index.
void addWakeupSocket(int fdVal, int index) {
putDescriptor(index, fdVal);
putEventOps(index, POLLIN);
}
// Access methods for fd structures
void putDescriptor(int i, int fd) {
pollArray.putInt(SIZE_POLLFD * i + FD_OFFSET, fd);
}
void putEventOps(int i, int event) {
pollArray.putShort(SIZE_POLLFD * i + EVENT_OFFSET, (short)event);
}
//Here, the POOLLIN event of source is identified as interesting. When data is written to sink, the file descriptor wakeupSourceFd corresponding to source is ready.
Java Code Collection Code
AllocatedNativeObject.java
----
class AllocatedNativeObject extends NativeObject
AllocatedNativeObject(int size, boolean pageAligned) {
super(size, pageAligned);
}
NativeObject.java
----
protected NativeObject(int size, boolean pageAligned) {
if (!pageAligned) {
this.allocationAddress = unsafe.allocateMemory(size);
this.address = this.allocationAddress;
} else {
int ps = pageSize();
long a = unsafe.allocateMemory(size + ps);
this.allocationAddress = a;
this.address = a + ps - (a & (ps - 1));
}
}
As you can see from the above, pollArray is a block of system memory allocated by unsafe.allocateMemory(size + ps)
Selector.open() has been completed here, mainly completing the establishment of Pipe, and putting the wakeup Source Fd of Pipe into pollArray, which is the hub of Selector. Here is the implementation of Windows. Under Windows, Pipe is implemented through two linked socket Channels, while under linux, pipes directly use the system.
2. serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
AbstractSelectableChannel.java --> register() --> SelectorImpl.java
----
protected final SelectionKey register(AbstractSelectableChannel ch,int ops,Object attachment)
{
if (!(ch instanceof SelChImpl))
throw new IllegalSelectorException();
SelectionKeyImpl k = new SelectionKeyImpl((SelChImpl)ch, this);
k.attach(attachment);
synchronized (publicKeys) {
implRegister(k);
}
k.interestOps(ops);
return k;
} WindowsSelectorImpl.java
----
protected void implRegister(SelectionKeyImpl ski) {
growIfNeeded();
channelArray[totalChannels] = ski;
ski.setIndex(totalChannels);
fdMap.put(ski);
keys.add(ski);
pollWrapper.addEntry(totalChannels, ski);
totalChannels++;
}
PollArrayWrapper.java
----
void addEntry(int index, SelectionKeyImpl ski) {
putDescriptor(index, ski.channel.getFDVal());
} Here we put the socketChannel file descriptor in pollArray.
3. selector.select();
SelectorImpl.java
----
public int select(long timeout) throws IOException
{
if (timeout < 0)
throw new IllegalArgumentException("Negative timeout");
return lockAndDoSelect((timeout == 0) ? -1 : timeout);
}
private int lockAndDoSelect(long timeout) throws IOException {
synchronized (this) {
if (!isOpen())
throw new ClosedSelectorException();
synchronized (publicKeys) {
synchronized (publicSelectedKeys) {
return doSelect(timeout);
}
}
}
}
WindowsSelectorImpl.java
----
protected int doSelect(long timeout) throws IOException {
if (channelArray == null)
throw new ClosedSelectorException();
this.timeout = timeout; // set selector timeout
processDeregisterQueue();
if (interruptTriggered) {
resetWakeupSocket();
return 0;
}
// Calculate number of helper threads needed for poll. If necessary
// threads are created here and start waiting on startLock
adjustThreadsCount();
finishLock.reset(); // reset finishLock
// Wakeup helper threads, waiting on startLock, so they start polling.
// Redundant threads will exit here after wakeup.
startLock.startThreads();
// do polling in the main thread. Main thread is responsible for
// first MAX_SELECTABLE_FDS entries in pollArray.
try {
begin();
try {
subSelector.poll();
} catch (IOException e) {
finishLock.setException(e); // Save this exception
}
// Main thread is out of poll(). Wakeup others and wait for them
if (threads.size() > 0)
finishLock.waitForHelperThreads();
} finally {
end();
}
// Done with poll(). Set wakeupSocket to nonsignaled for the next run.
finishLock.checkForException();
processDeregisterQueue();
int updated = updateSelectedKeys();
// Done with poll(). Set wakeupSocket to nonsignaled for the next run.
resetWakeupSocket();
return updated;
}
private int poll() throws IOException{ // poll for the main thread
return poll0(pollWrapper.pollArrayAddress,
Math.min(totalChannels, MAX_SELECTABLE_FDS),
readFds, writeFds, exceptFds, timeout);
}
private native int poll0(long pollAddress, int numfds, int[] readFds, int[] writeFds, int[] exceptFds, long timeout);
Others are some preparations. The key is subSelector.poll(), which finally calls poll0 of native and passes pollWrapper. pollArray Address as a parameter to poll0. What does poll0 do to pollArray?
WindowsSelectorImpl.c
----
Java_sun_nio_ch_WindowsSelectorImpl_00024SubSelector_poll0(JNIEnv *env, jobject this,
jlong pollAddress, jint numfds,
jintArray returnReadFds, jintArray returnWriteFds,
jintArray returnExceptFds, jlong timeout)
{
// Code... 10,000 words are omitted here
/* Call select */
if ((result = select(0 , &readfds, &writefds, &exceptfds, tv)) == SOCKET_ERROR) {
// Code... 10,000 words are omitted here
for (i = 0; i < numfds; i++) {
// Code... 10,000 words are omitted here
}
}
}The code is almost forgotten, but here you can see that the idea is to call the select method of c, where the select corresponds to the sys_select call in the kernel. Sys_select first copies the fd_set pointed by the second and third four parameters to the kernel, then polls each descriptor call by SET, and records it in the temporary result (fdset), if an event occurs, the select will occur. Write temporary results to user space and return; when no event occurs after polling, if a timeout time is specified, select will sleep to timeout, poll again after sleep, and write temporary results to user space, and then return.
Here select is to poll the FD in pollArray to see if there is an event. If there is an event, collect all the FD that happened and exit the blocking.
References on select system calls "select, poll, epoll comparison" This article, at the same time, we can see that the implementation of nio select on different platforms is different. Through epoll on linux, we can avoid polling. After the first call, event information will be associated with the corresponding epoll descriptor. Callback functions are registered on the descriptor to be used. When an event occurs, callback functions are responsible for storing the event in the ready event list and writing it to the user finally. Space.
Now it's clear that the way to exit the blockage is to regist er the socket Channel on the selector in a ready state (FD ready for the socket Channel in poll Array) or wakeup Source Fd in poll Array in Section 1. The former (socket Channel) is ready to wake up, which corresponds to the blocking - > event-driven - > wake-up process at the beginning of the article. The latter (wakeup Source Fd) is the active wakeup to be seen below.
4.
selector.wakeup()
WindowsSelectorImpl.java
----
public Selector wakeup() {
synchronized (interruptLock) {
if (!interruptTriggered) {
setWakeupSocket();
interruptTriggered = true;
}
}
return this;
}
// Sets Windows wakeup socket to a signaled state.
private void setWakeupSocket() {
setWakeupSocket0(wakeupSinkFd);
}
private native void setWakeupSocket0(int wakeupSinkFd);native Implementation Summary:
WindowsSelectorImpl.c
----
Java_sun_nio_ch_WindowsSelectorImpl_setWakeupSocket0(JNIEnv *env, jclass this,
jint scoutFd)
{
/* Write one byte into the pipe */
send(scoutFd, (char*)&POLLIN, 1, 0);
}This completes writing a byte to the sink end of the pipe that was originally created, and the source file descriptor will be ready, and the poll method will return, resulting in the return of the select method. (It turned out that you built your own socket chain with your other socket to do this.)