1, Basic introduction to Java NIO

The full name of Java NIO is java non blocking IO, which refers to the new API provided by JDK. From jdk1 Since the beginning of 4, Java has provided a series of improved input / output new features, which are collectively referred to as NiO (i.e. New IO), which are synchronous and non blocking

NIO related classes are placed in Java NIO package and sub package, and for the original Java Many classes in the IO package are rewritten.

NIO is buffer oriented or fast programming oriented. The data is read into a buffer that will be processed later. It can move back and forth in the buffer when needed, which increases the flexibility in the processing process. It can provide a non blocking high scalability network

   the non blocking mode of Java NIO enables a thread to send requests or read data from a channel, but it can only get the currently available data. If there is no data available, nothing will be obtained instead of keeping the thread blocked. Therefore, the thread can continue to do other things until the data can be read. The same is true for non blocking write. A thread requests to write some data to a channel, but does not need to wait for it to write completely. This thread can do other things at the same time.

  popular understanding: NIO can handle multiple operations with one thread. Assuming 10000 requests come, 50 or 100 threads can be allocated to process them according to the actual situation. Unlike the previous blocking IO, 10000 must be allocated.

  HTTP2.0 uses multiplexing technology to process multiple requests simultaneously in the same connection, and the number of concurrent requests is higher than that of HTTP 1 1 is several orders of magnitude larger.

2, Comparison of NIO and BIO

1. BIO processes data in stream mode, while NIO processes data in block mode. The efficiency of block I/O is much higher than that of I/O
2. BIO is blocking and NIO is non blocking
3. BIO operates based on byte stream and character stream, while NIO operates based on channel and buffer. Data is always read from the channel to the buffer or written from the buffer to the channel. The selector is used to listen to events of multiple channels (such as connection request, data arrival, etc.), so a single thread can listen to multiple client channels

3, NIO three core

  the following figure describes the relationship among Buffer, Channel and Selector

• Each channel corresponds to a Buffer
• The Selector corresponds to one thread, and one thread corresponds to multiple channels (connections)
• The figure shows that three channel s are registered with the selector / / program
• The channel to which the program switches is determined by events. Event is an important concept
• The Selector will switch on each channel according to different events
• Buffer is a memory block, and there is an array at the bottom
• Data is read and written through the Buffer, which is different from bio. Bio is either an input stream or an output stream, which cannot be bidirectional, but NIO's Buffer can be read or written. It needs to switch the flip method. The channel is bidirectional and can return to the underlying operating system, such as Linux and the underlying operating system
• The passage is bidirectional

3.1 Buffer

Basic introduction

   Buffer: a Buffer is essentially a memory block that can read and write data. It can be understood as a container object (including an array). This object provides a set of methods to use the memory block more easily. The Buffer object has built-in mechanisms to track and record the state changes of the Buffer. The Channel provides a Channel for reading data from files and networks, but the read or written data must pass through the Buffer,

Buffer class and its subclasses

1. In NIO, Buffer is a top-level parent class. It is an abstract class. The hierarchical relationship diagram of the class:

The function of each specific Buffer can also be seen intuitively through the name.

2. The Buffer class defines four attributes that all buffers have to provide information about the data elements they contain:

private int mark = -1;
private int position = 0;
private int limit;
    private int capacity;    

3. List of Buffer class related methods

public abstract class Buffer {
//JDK1.4, the api is introduced
public final int capacity( )//Returns the capacity of this buffer
public final int position( )//Returns the location of this buffer
public final Buffer position (int newPositio)//Set the location of this buffer
public final int limit( )//Returns the limit for this buffer
public final Buffer limit (int newLimit)//Set the limit for this buffer
public final Buffer mark( )//Set a flag at the location of this buffer
public final Buffer reset( )//Resets the location of this buffer to the previously marked location
public final Buffer clear( )//Clear this buffer, that is, each mark is restored to the initial state, but the data is not really erased, and subsequent operations will be overwritten
public final Buffer flip( )//Invert this buffer
public final Buffer rewind( )//Rewind this buffer
public final int remaining( )//Returns the number of elements between the current position and the limit
public final boolean hasRemaining( )//Tells whether there is an element between the current position and the limit
public abstract boolean isReadOnly( );//Tells whether this buffer is read-only
//JDK1. api introduced in 6
public abstract boolean hasArray();//Tells whether the buffer has an accessible array of underlying implementations
public abstract Object array();//Returns the underlying implementation array of this buffer
public abstract int arrayOffset();//Returns the offset of the first buffer element in the underlying implementation array of this buffer
public abstract boolean isDirect();//Tells whether this buffer is a direct buffer
}

4,ByteBuffer

As can be seen from the above, for the basic data types in Java (except boolean), there is a Buffer type corresponding to it. The most commonly used is naturally the ByteBuffer class (binary data). The main methods of this class are as follows

public abstract class ByteBuffer {
//Buffer creation related APIs
public static ByteBuffer allocateDirect(int capacity)//Create direct buffer
public static ByteBuffer allocate(int capacity)//Sets the initial capacity of the buffer
public static ByteBuffer wrap(byte[] array)//Put an array into the buffer for use
//Construct a buffer with initialization position offset and upper bound length
public static ByteBuffer wrap(byte[] array,int offset, int length)
//Cache access related API s
public abstract byte get( );//Get from the current position. After get, position will automatically + 1
public abstract byte get (int index);//get from absolute position
public abstract ByteBuffer put (byte b);//After adding put from the current position, position will automatically + 1
public abstract ByteBuffer put (int index, byte b);//From absolute position
 }

3.2 Channel

Basic introduction

1. NIO channels are similar to streams, but some differences are as follows:

1. Channels can read and write at the same time, while streams can only read or write
2. The channel can read and write data asynchronously
3. The channel can read data from the buffer or write data to the buffer

2. The stream in BIO is unidirectional. For example, the FileInputStream object can only read data, while the channel in NIO is bidirectional and can be read or written.

3. Channel is an interface in NIO. Public interface channel extensions are closed {}

4. Common Channel classes include FileChannel, datagram Channel, ServerSocketChannel and SocketChannel. [ServerSocketChannel is similar to ServerSocket, and SocketChannel is similar to Socket]

5. FileChannel is used for file data reading and writing, datagram channel is used for UDP data reading and writing, and ServerSocketChannel and SocketChannel are used for TCP data reading and writing.

FileChannel class

FileChannel is mainly used to perform IO operations on local files. Common methods include

public int read(ByteBuffer dst) ，Read data from the channel and put it in the buffer
public int write(ByteBuffer src) ，Write buffer data to the channel
public long transferFrom(ReadableByteChannel src, long position, long count)，Copy data from the target channel to the current channel
public long transferTo(long position, long count, WritableByteChannel target)，Copy data from the current channel to the target channel

Example 1 - local file write data

Complete some simple NIO file operations through the contents described above. The code is as follows:

package com.dpb.netty.nio;

import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

/**
 * @program: netty4demo
 * @description:
 * @author: Bobo roast duck
 * @create: 2019-12-28 11:37
 */
public class NioChannel01 {

public static void main(String[] args) throws Exception{

        String str = "hello,bobokaoya";
//Create an output stream - > Channel
        FileOutputStream fileOutputStream = new FileOutputStream("c:\\tools\\netty.txt");

//Obtain the corresponding FileChannel through fileOutputStream
//The real type of this fileChannel is FileChannelImpl
        FileChannel fileChannel = fileOutputStream.getChannel();
//Create a buffer ByteBuffer
        ByteBuffer byteBuffer = ByteBuffer.allocate(1024)
//Put str into byteBuffer
        byteBuffer.put(str.getBytes());
//flip byteBuffer
        byteBuffer.flip();
//Write byteBuffer data to fileChannel
        fileChannel.write(byteBuffer);
        fileOutputStream.close();
    }
}

Example 2 - local file read data

package com.dpb.netty.nio;

import java.io.File;
import java.io.FileInputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

/**
 * @program: netty4demo
 * @description:
 * @author: Bobo roast duck
 * @create: 2019-12-28 11:41
 */
public class NioChannel02 {
public static void main(String[] args) throws Exception {

//Create an input stream for the file
        File file = new File("c:\\tools\\netty.txt");
        FileInputStream fileInputStream = new FileInputStream(file);

//Get the corresponding filechannel - > actual type FileChannelImpl through fileInputStream
        FileChannel fileChannel = fileInputStream.getChannel();

//Create buffer
        ByteBuffer byteBuffer = ByteBuffer.allocate((int) file.length());

//Read the data of the channel into the Buffer
        fileChannel.read(byteBuffer);

//Convert byte data of byteBuffer to String
        System.out.println(new String(byteBuffer.array()));
        fileInputStream.close();
    }
}

Example 3 - use a Buffer to complete file reading

package com.dpb.netty.nio;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
/**
 * @program: netty4demo
 * @description:
 * @author: Bobo roast duck
 * @create: 2019-12-28 11:44
 */
public class NioChannel03 {
public static void main(String[] args) throws Exception {
        FileInputStream fileInputStream = new FileInputStream("1.txt");
        FileChannel fileChannel01 = fileInputStream.getChannel();
        FileOutputStream fileOutputStream = new FileOutputStream("2.txt");
        FileChannel fileChannel02 = fileOutputStream.getChannel();
        ByteBuffer byteBuffer = ByteBuffer.allocate(512);
while (true) { //Cyclic reading
//Here is an important operation. Don't forget
/*
             public final Buffer clear() {
                position = 0;
                limit = capacity;
                mark = -1;
                return this;
            }
             */
            byteBuffer.clear(); //Empty buffer
int read = fileChannel01.read(byteBuffer);
            System.out.println("read =" + read);
if(read == -1) { //Indicates that you have finished reading
break;
            }
//Write the data in the buffer to filechannel02 -- 2 txt
            byteBuffer.flip();
            fileChannel02.write(byteBuffer);
        }

//Close the associated flow
        fileInputStream.close();
        fileOutputStream.close();
    }
}

Example 4 - copy file transferFrom method

Next, we use the transferFrom method to copy a file.

package com.dpb.netty.nio;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.channels.FileChannel;

/**
 * @program: netty4demo
 * @description:
 * @author: Bobo roast duck
 * @create: 2019-12-28 11:49
 */
public class NioChannel04 {
public static void main(String[] args)  throws Exception {

//Create correlation flow
        FileInputStream fileInputStream = new FileInputStream("c:\\tools\\a1.jpg");
        FileOutputStream fileOutputStream = new FileOutputStream("c:\\tools\\a2.jpg");
//Get the filechannel corresponding to each stream
        FileChannel sourceCh = fileInputStream.getChannel();
        FileChannel destCh = fileOutputStream.getChannel();
//Use transferForm to complete the copy
        destCh.transferFrom(sourceCh,0,sourceCh.size());
//Close relevant channels and streams
        sourceCh.close();
        destCh.close();
        fileInputStream.close();
        fileOutputStream.close();
    }
}

Precautions and details about Buffer and Channel

ByteBuffer supports typed put and get. What data type is put in, get should use the corresponding data type to get it, otherwise there may be BufferUnderflowException.

You can convert a normal Buffer to a read-only Buffer [example]

NIO also provides MappedByteBuffer, which allows files to be modified directly in memory (memory outside the heap), and NIO completes how to synchronize to files

The read and write operations mentioned above are all completed through one Buffer. NIO also supports reading and write operations through multiple buffers (i.e. Buffer arrays), i.e. Scattering and Gathering

3.3 Selector

Basic introduction

NIO of Java uses non blocking IO mode. You can use one thread to process multiple client connections, and you will use the selector

   the Selector can detect whether events occur on multiple registered channels (Note: multiple channels can be registered to the same Selector in the form of events). If an event occurs, it will obtain the event and then process each event accordingly. In this way, only one single thread can manage multiple channels, that is, manage multiple connections and requests.

Only when the connection / channel really has a read-write event, it can read and write, which greatly reduces the system overhead, and there is no need to create a thread for each connection and maintain multiple threads

It avoids the overhead caused by context switching between multiple threads

Description of features:

1. Netty's IO thread NioEventLoop aggregates selectors (selectors, also known as multiplexers), which can handle hundreds of client connections simultaneously.
2. When a thread reads and writes data from a client Socket channel, if no data is available, the thread can perform other tasks.
3. Threads usually use the idle time of non blocking IO to perform IO operations on other channels, so a single thread can manage multiple input and output channels.
4. Since read and write operations are non blocking, this can fully improve the operation efficiency of IO threads and avoid thread suspension caused by frequent I/O blocking.
5. One I/O thread can handle N client connections and read-write operations concurrently, which fundamentally solves the traditional synchronous blocking I/O connection thread model, and greatly improves the performance, scalability and reliability of the architecture.

Selector class related methods

Selector class is an abstract class. Common methods and descriptions are as follows:

public abstract class Selector implements Closeable { 
public static Selector open();//Get a selector object
//Monitor all registered channels. When there are IO operations available, add the corresponding SelectionKey to the internal collection and return it. The parameter is used to set the timeout time
public int select(long timeout);
public Set<SelectionKey> selectedKeys();//Get all selectionkeys from the internal collection  
}

matters needing attention

The ServerSocketChannel function in NIO is similar to ServerSocket, and the SocketChannel function is similar to Socket

Description of selector related methods

selector.select()//block
selector.select(1000);//Block for 1000 ms and return after 1000 ms
selector.wakeup();//Wake up selector
selector.selectNow();//No blocking, return immediately