Netty framework learning -- HTTP/HTTPS application based on netty

Posted by tmed on Tue, 25 Jan 2022 03:10:09 +0100


Secure applications via SSL/TLS

SSL and TLS security protocols are superimposed on other protocols to achieve data security. To support SSL/TLS, Java provides javax net. SSL package, its SSLContext and SSLEngine classes make the implementation of decryption and encryption quite simple. Netty implements this API through a ChannelHandler called sslhandler, where sslhandler uses ssengine internally to do the actual work

Netty also provides an implementation of SSLEngine based on OpenSSL toolkit, which has better performance than SSLEngine provided by JDK. If OpenSSL is available, you can configure the netty application to use OpenSSL engine by default. If not available, netty will go back to the JDK implementation

The following code shows how to use ChannelInitializer to add SslHandler to ChannelPipeline

public class SslChannelInitializer extends ChannelInitializer<Channel> {

    private final SslContext context;
    private final boolean startTls;

    public SslChannelInitializer(SslContext context, boolean startTls) {
        this.context = context;
        this.startTls = startTls;
    }

    @Override
    protected void initChannel(Channel ch) throws Exception {
        SSLEngine engine = context.newEngine(ch.alloc());
        ch.pipeline().addFirst("ssl", new SslHandler(engine, startTls));
    }
}

In most cases, Sslhandler will be the first ChannelHandler in ChannelPipeline, which ensures that encryption will not occur until all other channelhandlers apply their logic to the data

SslHandler has some useful methods, as shown in the table. For example, in the handshake phase, the two nodes will verify each other and agree on an encryption method. You can modify its behavior by configuring SslHandler, or provide a notification once the SSL/TLS handshake is completed. After the handshake phase, all data will be encrypted

Method namedescribe
setHandshakeTimeout(long, TimeUnit)
setHandshakeTimeoutMillis(long)
getHandshakeTimeoutMillis()
Set and obtain the timeout. After the timeout, handshake ChannelFuture will be notified of failure
setCloseNotifyTimeout(long, TimeUnit)
setCloseNotifyTimeoutMillis(long)
getCloseNotifyTimeoutMillis()
Set and obtain the timeout. After the timeout, a close notification will be triggered and the connection will be closed, which will also lead to the failure of notifying the ChannelFuture
handshakeFuture()Return a ChannelFuture that will be notified after the handshake is completed. If the handshake has been executed previously, return a ChannelFuture that contains the results of the previous handshake
close()
close(ChannelPipeline)
close(ChannelHandlerContext, ChannelPromise)
Send close_notify to request the shutdown and destruction of the underlying SslEngine

HTTP codec

HTTP is based on the request / response mode. The client sends an HTTP request to the server, and then the server will return an HTTP response. Netty provides a variety of encoders and decoders to simplify the use of this protocol

The following figure shows the methods of producing and consuming HTTP requests and HTTP responses respectively


As shown in the figure, an HTTP request / response may consist of multiple data parts and always ends with a LastHttpContent part

The following table outlines the HTTP decoders and encoders that process and generate these messages

namedescribe
HttpRequestEncoderEncode HTTPRequest, HttpContent, and LastHttpContent messages into bytes
HttpResponseEncoderEncode HTTPResponse, HttpContent, and LastHttpContent messages into bytes
HttpRequestDecoderEncode bytes into HTTPRequest, HttpContent, and LastHttpContent messages
HttpResponseDecoderEncode bytes into HTTPResponse, HttpContent, and LastHttpContent messages

The HttpPipelineInitializer class in the following code shows how easy it is to add HTTP support to your application - just add the correct ChannelHandler to ChannelPipeline

public class HttpPipelineInitializer extends ChannelInitializer<Channel> {

    private final boolean client;

    public HttpPipelineInitializer(boolean client) {
        this.client = client;
    }

    @Override
    protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        if (client) {
            // If it is a client, add HttpResponseDecoder to process the response from the server
            pipeline.addLast("decoder", new HttpResponseDecoder());
            // If it is a client, add httprequesteencoder to send a request to the server
            pipeline.addLast("encoder", new HttpRequestEncoder());
        } else {
            // If it is a server, add an HttpRequestDecoder to process requests from the client
            pipeline.addLast("decoder", new HttpRequestDecoder());
            // If it is a client, add HttpResponseEncoder to send a response to the client
            pipeline.addLast("encoder", new HttpResponseEncoder());
        }
    }
}

Aggregate HTTP messages

After the ChannelInitializer installs the ChannelHandler into the ChannelPipeline, you can handle different types of HTTP object messages. However, since HTTP requests and responses may consist of many parts, you need to aggregate them to form a complete message. Netty provides an aggregator that can combine multiple message parts into FullHttpRequest or FullHttpResponse messages

Since message segments need to be buffered until the next complete message can be forwarded to the next ChannelInboundHandler, this operation has a slight overhead. The advantage is that you don't have to care about message fragments

The introduction of this automatic aggregation mechanism is just to add another ChannelHandler to the ChannelPipeline. The following code shows how to do this:

public class HttpAggregatorInitializer extends ChannelInitializer<Channel> {

    private final boolean isClient;

    public HttpAggregatorInitializer(boolean isClient) {
        this.isClient = isClient;
    }

    @Override
    protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        if (isClient) {
            // If it is a client, add HttpClientCodec
            pipeline.addLast("codec", new HttpClientCodec());
        } else {
            // If it is a server, add HttpServerCodec
            pipeline.addLast("codec", new HttpServerCodec());
        }
        // Add an HTTPObjectAggregator with a maximum message size of 512KB to the ChannelPipeline
        pipeline.addLast("aggregator", new HttpObjectAggregator(512 * 1024));
    }
}

HTTP compression

When using HTTP, it is recommended to turn on the compression function to reduce the size of transmitted data as much as possible. Although compression can be costly, it is generally a good idea, especially for text data

Netty provides a ChannelHandler implementation for both compression and decompression, which supports both gzip and deflate coding

The client can indicate the compression format supported by the server by providing the following header information

GET /encrypted-area HTTP/1.1

Host: www.example.com

Accept-Encoding: gzip, deflate

However, it should be noted that the server has no obligation to compress the data it sends

public class HttpCompressionInitializer extends ChannelInitializer<Channel> {

    private final boolean isClient;

    public HttpCompressionInitializer(boolean isClient) {
        this.isClient = isClient;
    }

    @Override
    protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        if (isClient) {
            // If it is a client, add http clientcodec
            pipeline.addLast("codec", new HttpClientCodec());
            // If it is a client, add HttpContentDecompressor to process compressed content from the server
            pipeline.addLast("decompressor", new HttpContentDecompressor());
        } else {
            // If it is a server, add HttpServerCodec
            pipeline.addLast("codec", new HttpServerCodec());
            // If it is a server, add HttpContentDecompressor to compress the data
            pipeline.addLast("decompressor", new HttpContentDecompressor());
        }
    }
}

HTTPS

To enable HTTPS, you only need to add SslHandler to the ChannelHandler combination of ChannelPipeline

public class HttpsCodecInitializer extends ChannelInitializer<Channel> {

    private final SslContext context;
    private final boolean isClient;

    public HttpsCodecInitializer(SslContext context, boolean isClient) {
        this.context = context;
        this.isClient = isClient;
    }

    @Override
    protected void initChannel(Channel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();
        SSLEngine engine = context.newEngine(ch.alloc());
        pipeline.addLast("ssl", new SslHandler(engine));
        if (isClient) {
            pipeline.addLast("codec", new HttpClientCodec());
        } else {
            pipeline.addLast("codec", new HttpServerCodec());
        } 
    }
}

WebSocket

WebSocket solves a long-standing problem: since the underlying protocol (HTTP) is an interactive sequence of request / response mode, how to publish information in real time? AJAX solves this problem to some extent, but the data flow is still driven by the request sent by the client

WebSocket provides two-way communication over a single TCP connection. It provides an alternative to HTTP polling for two-way communication between web pages and remote servers

To add WebSocket support to your application, you need to add the appropriate client or server WebSocketChannelHandler to the ChannelPipeline. This class will handle the special message type called frame defined by WebSocket. As shown in the table, WebSocketFrame can be classified as data frame or control frame

namedescribe
BinaryWebSocketFrameData frames: binary data
TextWebSocketFrameData frames: text data
ContinuationWebSocketFrameData frame: text or binary data belonging to the previous BinaryWebSocketFrame or TextWebSocketFrame
CloseWebSocketFrameControl frame: a CLOSE request, closing status code and closing reason
PingWebSocketFrameControl frame: request a PongWebSocketFrame
PongWebSocketFrameControl frame: response to PingWebSocketFrame request

Because Netty is mainly a server-side technology, we focus on creating WebSocket server. The following code shows a simple example of using WebSocketChannelHandler. This class will handle protocol upgrade handshake and three control frames - Close, Ping and Pong. Text and Binary data frames will be passed to the next ChannelHandler for processing

public class WebSocketServerInitializer extends ChannelInitializer<Channel> {

    @Override
    protected void initChannel(Channel ch) throws Exception {
        ch.pipeline().addLast(
                new HttpServerCodec(),
                new HttpObjectAggregator(65536),
                // If the requested endpoint is / websocket, the upgrade handshake is processed
                new WebSocketServerProtocolHandler("/websocket"),
                // TextFrameHandler handles TextWebSocketFrame
                new TextFrameHandler(),
                // BinaryFrameHandler handles BinaryWebSocketFrame
                new BinaryFrameHandler(),
                // ContinuationFrameHandler handles continuationwebsocketframe
                new ContinuationFrameHandler());
    }

    public static final class TextFrameHandler extends SimpleChannelInboundHandler<TextWebSocketFrame> {

        @Override
        protected void messageReceived(ChannelHandlerContext ctx, TextWebSocketFrame msg) throws Exception {
            // do something
        }
    }

    public static final class BinaryFrameHandler extends SimpleChannelInboundHandler<BinaryWebSocketFrame> {

        @Override
        protected void messageReceived(ChannelHandlerContext ctx, BinaryWebSocketFrame msg) throws Exception {
            // do something
        }
    }

    public static final class ContinuationFrameHandler extends SimpleChannelInboundHandler<ContinuationWebSocketFrame> {

        @Override
        protected void messageReceived(ChannelHandlerContext ctx, ContinuationWebSocketFrame msg) throws Exception {
            // do something
        }
    }
}

Topics: Java Netty http