Introduction: This paper focuses on the design and implementation of proxy gateway itself, rather than the management and maintenance of proxy resources.

Author Xin Ran
Source: Ali technical official account

I. problem background

• The platform side purchases a number of naked agents to advertise, show and audit in different places. Agents purchased from outside are used as follows:
• Extract the proxy IP:PORT through the given HTTP API, and the returned result will give the valid duration of the proxy for 3 ~ 5 minutes and the region of the proxy;

Select the designated region from the extracted agents, add authentication information, and request to obtain the results;

This paper designs and implements a proxy gateway through:

• Manage and maintain agent resources and do agent authentication;
• Expose a unified proxy entry, rather than a dynamically changing proxy IP:PORT;
• Flow filtering and flow restriction, such as: static resources do not go through agents;

This paper focuses on the design and implementation of proxy gateway itself, rather than the management and maintenance of proxy resources.

Note: This article contains a lot of executable JAVA code to explain the principles of proxy

II. Technical route

The technical route of this paper. Before implementing the proxy gateway, first introduce the principle of proxy and how to implement it

• Transparent agent;
• Non transparent agent;
• Transparent upstream agent;
• Non transparent upstream agent;

Finally, this paper constructs a proxy gateway, which is essentially a non transparent upstream proxy, and gives a detailed design and implementation.

1 transparent proxy

Transparent proxy is the basis of proxy gateway. This paper introduces it in detail with JAVA Native NIO. When implementing the proxy gateway, the NETTY framework is actually used. The implementation of native NIO is helpful to understand the implementation of NETTY.

Transparent proxy designs three interacting parties, client, proxy service and server. Its principle is as follows:

• When the proxy service receives the connection request, it determines that if it is a CONNECT request, it needs to respond to the proxy connection success message to the client;
• After the connection request response is completed, the proxy service needs to CONNECT to the remote server specified by CONNECT, and then directly forward the communication between the client and the remote service;
• When the proxy service receives a non CONNECT request, it needs to parse the remote server of the request, and then directly forward the communication between the client and the remote service;

The points to note are:

• Usually, an HTTPS request will send a CONNECT request before passing through the agent; Handshake protocol for encrypted communication on the channel after successful connection; Therefore, the time to CONNECT to the remote is when the CONNECT request is received, because after that, the data is encrypted;
• When the transparent agent receives the CONNECT request, it does not need to pass it to the remote service (the remote service does not recognize the request);
• When receiving a non CONNECT request, the transparent agent shall forward it unconditionally;

The implementation of a complete transparent proxy is less than 300 lines of code. The complete excerpt is as follows:

@Slf4j
public class SimpleTransProxy {

    public static void main(String[] args) throws IOException {
        int port = 8006;
        ServerSocketChannel localServer = ServerSocketChannel.open();
        localServer.bind(new InetSocketAddress(port));
        Reactor reactor = new Reactor();
        // REACTOR thread
        GlobalThreadPool.REACTOR_EXECUTOR.submit(reactor::run);

        // WORKER single thread debugging
        while (localServer.isOpen()) {
            // Wait connection blocked here
            SocketChannel remoteClient = localServer.accept();

            // Worker thread
            GlobalThreadPool.WORK_EXECUTOR.submit(new Runnable() {
                @SneakyThrows
                @Override
                public void run() {
                    // Proxy to remote
                    SocketChannel remoteServer = new ProxyHandler().proxy(remoteClient);

                    // Transparent transmission
                    reactor.pipe(remoteClient, remoteServer)
                            .pipe(remoteServer, remoteClient);
                }
            });
        }
    }
}

@Data
class ProxyHandler {
    private String method;
    private String host;
    private int port;
    private SocketChannel remoteServer;
    private SocketChannel remoteClient;

    /**
     * Original information
     */
    private List<ByteBuffer> buffers = new ArrayList<>();
    private StringBuilder stringBuilder = new StringBuilder();

    /**
     * Connect to remote
     * @param remoteClient
     * @return
     * @throws IOException
     */
    public SocketChannel proxy(SocketChannel remoteClient) throws IOException {
        this.remoteClient = remoteClient;
        connect();
        return this.remoteServer;
    }

    public void connect() throws IOException {
        // Resolve METHOD, HOST and PORT
        beforeConnected();

        // Link REMOTE SERVER
        createRemoteServer();

        // CONNECT request response, other requests WRITE THROUGH
        afterConnected();
    }

    protected void beforeConnected() throws IOException {
        // Read HEADER
        readAllHeader();

        // Resolve HOST and PORT
        parseRemoteHostAndPort();
    }

    /**
     * Create remote connection
     * @throws IOException
     */
    protected void createRemoteServer() throws IOException {
        remoteServer = SocketChannel.open(new InetSocketAddress(host, port));
    }

    /**
     * Preprocessing after connection establishment
     * @throws IOException
     */
    protected void afterConnected() throws IOException {
        // When a CONNECT request is made, 200 is written to the CLIENT by default
        if ("CONNECT".equalsIgnoreCase(method)) {
            // CONNECT defaults to port 443, which is re parsed according to HOST
            remoteClient.write(ByteBuffer.wrap("HTTP/1.0 200 Connection Established\r\nProxy-agent: nginx\r\n\r\n".getBytes()));
        } else {
            writeThrouth();
        }
    }

    protected void writeThrouth() {
        buffers.forEach(byteBuffer -> {
            try {
                remoteServer.write(byteBuffer);
            } catch (IOException e) {
                e.printStackTrace();
            }
        });
    }

    /**
     * Read request content
     * @throws IOException
     */
    protected void readAllHeader() throws IOException {
        while (true) {
            ByteBuffer clientBuffer = newByteBuffer();
            int read = remoteClient.read(clientBuffer);
            clientBuffer.flip();
            appendClientBuffer(clientBuffer);
            if (read < clientBuffer.capacity()) {
                break;
            }
        }
    }

    /**
     * Resolve HOST and PROT
     * @throws IOException
     */
    protected void parseRemoteHostAndPort() throws IOException {
        // Read the first batch and get METHOD
        method = parseRequestMethod(stringBuilder.toString());

        // The default is port 80, which is re parsed according to HOST
        port = 80;
        if ("CONNECT".equalsIgnoreCase(method)) {
            port = 443;
        }

        this.host = parseHost(stringBuilder.toString());

        URI remoteServerURI = URI.create(host);
        host = remoteServerURI.getHost();

        if (remoteServerURI.getPort() > 0) {
            port = remoteServerURI.getPort();
        }
    }

    protected void appendClientBuffer(ByteBuffer clientBuffer) {
        buffers.add(clientBuffer);
        stringBuilder.append(new String(clientBuffer.array(), clientBuffer.position(), clientBuffer.limit()));
    }

    protected static ByteBuffer newByteBuffer() {
        // The buffer must be greater than 7 to ensure that the method can be read
        return ByteBuffer.allocate(128);
    }

    private static String parseRequestMethod(String rawContent) {
        // create uri
        return rawContent.split("\r\n")[0].split(" ")[0];
    }

    private static String parseHost(String rawContent) {
        String[] headers = rawContent.split("\r\n");
        String host = "host:";
        for (String header : headers) {
            if (header.length() > host.length()) {
                String key = header.substring(0, host.length());
                String value = header.substring(host.length()).trim();
                if (host.equalsIgnoreCase(key)) {
                    if (!value.startsWith("http://") && !value.startsWith("https://")) {
                        value = "http://" + value;
                    }
                    return value;
                }
            }
        }
        return "";
    }

}

@Slf4j
@Data
class Reactor {

    private Selector selector;

    private volatile boolean finish = false;

    @SneakyThrows
    public Reactor() {
        selector = Selector.open();
    }

    @SneakyThrows
    public Reactor pipe(SocketChannel from, SocketChannel to) {
        from.configureBlocking(false);
        from.register(selector, SelectionKey.OP_READ, new SocketPipe(this, from, to));
        return this;
    }

    @SneakyThrows
    public void run() {
        try {
            while (!finish) {
                if (selector.selectNow() > 0) {
                    Iterator<SelectionKey> it = selector.selectedKeys().iterator();
                    while (it.hasNext()) {
                        SelectionKey selectionKey = it.next();
                        if (selectionKey.isValid() && selectionKey.isReadable()) {
                            ((SocketPipe) selectionKey.attachment()).pipe();
                        }
                        it.remove();
                    }
                }
            }
        } finally {
            close();
        }
    }

    @SneakyThrows
    public synchronized void close() {
        if (finish) {
            return;
        }
        finish = true;
        if (!selector.isOpen()) {
            return;
        }
        for (SelectionKey key : selector.keys()) {
            closeChannel(key.channel());
            key.cancel();
        }
        if (selector != null) {
            selector.close();
        }
    }

    public void cancel(SelectableChannel channel) {
        SelectionKey key = channel.keyFor(selector);
        if (Objects.isNull(key)) {
            return;
        }
        key.cancel();
    }

    @SneakyThrows
    public void closeChannel(Channel channel) {
        SocketChannel socketChannel = (SocketChannel)channel;
        if (socketChannel.isConnected() && socketChannel.isOpen()) {
            socketChannel.shutdownOutput();
            socketChannel.shutdownInput();
        }
        socketChannel.close();
    }
}

@Data
@AllArgsConstructor
class SocketPipe {

    private Reactor reactor;

    private SocketChannel from;

    private SocketChannel to;

    @SneakyThrows
    public void pipe() {
        // Cancel listening
        clearInterestOps();

        GlobalThreadPool.PIPE_EXECUTOR.submit(new Runnable() {
            @SneakyThrows
            @Override
            public void run() {
                int totalBytesRead = 0;
                ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
                while (valid(from) && valid(to)) {
                    byteBuffer.clear();
                    int bytesRead = from.read(byteBuffer);
                    totalBytesRead = totalBytesRead + bytesRead;
                    byteBuffer.flip();
                    to.write(byteBuffer);
                    if (bytesRead < byteBuffer.capacity()) {
                        break;
                    }
                }
                if (totalBytesRead < 0) {
                    reactor.closeChannel(from);
                    reactor.cancel(from);
                } else {
                    // Reset listening
                    resetInterestOps();
                }
            }
        });
    }

    protected void clearInterestOps() {
        from.keyFor(reactor.getSelector()).interestOps(0);
        to.keyFor(reactor.getSelector()).interestOps(0);
    }

    protected void resetInterestOps() {
        from.keyFor(reactor.getSelector()).interestOps(SelectionKey.OP_READ);
        to.keyFor(reactor.getSelector()).interestOps(SelectionKey.OP_READ);
    }

    private boolean valid(SocketChannel channel) {
        return channel.isConnected() && channel.isRegistered() && channel.isOpen();
    }
}

Above, learn from NETTY:

• First initialize the REACTOR thread, and then turn on the agent listening. When the agent request is received, it will be processed.
• When the proxy service receives the proxy request, it first preprocesses the proxy, and then the SocketPipe does two-way forwarding between the client and the remote server.
• Proxy preprocessing: first read the first HTTP request and parse method, host and port.
• If it is a CONNECT request, send a response Connection Established, then CONNECT to the remote server and return to SocketChannel
• If it is a non CONNECT request, CONNECT to the remote server, write the original request, and return the SocketChannel
• SocketPipe does two-way forwarding at the client and remote server; It registers the SocketChannel of the client and server with the REACTOR
• Upon detecting the READABLE CHANNEL, the REACTOR sends it to SocketPipe for two-way forwarding.

test

The agent test is relatively simple. After pointing to the code, the agent service listens to port 8006. At this time:

curl -x 'localhost:8006' http://httpbin.org/get Test HTTP request

curl -x 'localhost:8006' https://httpbin.org/get Test HTTPS requests

Note that the proxy service proxies HTTPS requests at this time, but the - k option is not required to indicate non secure proxies. Because the proxy service itself does not act as an intermediary and does not parse the communication content between the client and the remote server. In the case of non transparent proxy, this problem needs to be solved.

2 non transparent proxy

The non transparent proxy needs to parse the content transmitted by the client and the remote server and handle it accordingly.

When the transmission is HTTP protocol, the data transmitted by SocketPipe is plaintext data, which can be intercepted and processed directly.

When the transmission is HTTPS protocol, the valid data transmitted by SocketPipe is encrypted data and cannot be processed transparently.

In addition, no matter the HTTP protocol or HTTPS protocol, SocketPipe reads incomplete data, which needs to be processed in batch.

• The SocketPipe batching problem can be implemented in a mode similar to BufferedInputStream decorating InputStream, which is relatively simple; For details, please refer to NETTY's HttpObjectAggregator;
• The processing of encryption and decryption of HTTPS original request and result data requires NIO SOCKET CHANNEL;

Packaging principle of SslSocketChannel

Considering that the SocketChannel of NIO provided with JDK does not support SSL; The existing SSLSocket is a blocked OIO. As shown in the figure:

It can be seen that

• SSL SESSION handshake is required for each inbound data and outbound data;
• Decrypt inbound data and encrypt outbound data;
• Handshake, data encryption and data decryption are a unified set of state machines;

The following code implements SslSocketChannel

public class SslSocketChannel {

    /**
     * Four storage required for handshake encryption and decryption
     */
    protected ByteBuffer myAppData; // Plaintext
    protected ByteBuffer myNetData; // ciphertext
    protected ByteBuffer peerAppData; // Plaintext
    protected ByteBuffer peerNetData; // ciphertext

    /**
     * Asynchronous actuator used in handshake encryption and decryption
     */
    protected ExecutorService executor = Executors.newSingleThreadExecutor();

    /**
     * CHANNEL of original NIO
     */
    protected SocketChannel socketChannel;

    /**
     * SSL engine
     */
    protected SSLEngine engine;

    public SslSocketChannel(SSLContext context, SocketChannel socketChannel, boolean clientMode) throws Exception {
        // Original NIO SOCKET
        this.socketChannel = socketChannel;

        // Initialize BUFFER
        SSLSession dummySession = context.createSSLEngine().getSession();
        myAppData = ByteBuffer.allocate(dummySession.getApplicationBufferSize());
        myNetData = ByteBuffer.allocate(dummySession.getPacketBufferSize());
        peerAppData = ByteBuffer.allocate(dummySession.getApplicationBufferSize());
        peerNetData = ByteBuffer.allocate(dummySession.getPacketBufferSize());
        dummySession.invalidate();

        engine = context.createSSLEngine();
        engine.setUseClientMode(clientMode);
        engine.beginHandshake();
    }

    /**
     * reference resources https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
     * SSL handshake protocol
     * @return
     * @throws IOException
     */
    protected boolean doHandshake() throws IOException {
        SSLEngineResult result;
        HandshakeStatus handshakeStatus;

        int appBufferSize = engine.getSession().getApplicationBufferSize();
        ByteBuffer myAppData = ByteBuffer.allocate(appBufferSize);
        ByteBuffer peerAppData = ByteBuffer.allocate(appBufferSize);
        myNetData.clear();
        peerNetData.clear();

        handshakeStatus = engine.getHandshakeStatus();
        while (handshakeStatus != HandshakeStatus.FINISHED && handshakeStatus != HandshakeStatus.NOT_HANDSHAKING) {
            switch (handshakeStatus) {
                case NEED_UNWRAP:
                    if (socketChannel.read(peerNetData) < 0) {
                        if (engine.isInboundDone() && engine.isOutboundDone()) {
                            return false;
                        }
                        try {
                            engine.closeInbound();
                        } catch (SSLException e) {
                            log.debug("received END OF STREAM，Close connection.", e);
                        }
                        engine.closeOutbound();
                        handshakeStatus = engine.getHandshakeStatus();
                        break;
                    }
                    peerNetData.flip();
                    try {
                        result = engine.unwrap(peerNetData, peerAppData);
                        peerNetData.compact();
                        handshakeStatus = result.getHandshakeStatus();
                    } catch (SSLException sslException) {
                        engine.closeOutbound();
                        handshakeStatus = engine.getHandshakeStatus();
                        break;
                    }
                    switch (result.getStatus()) {
                        case OK:
                            break;
                        case BUFFER_OVERFLOW:
                            peerAppData = enlargeApplicationBuffer(engine, peerAppData);
                            break;
                        case BUFFER_UNDERFLOW:
                            peerNetData = handleBufferUnderflow(engine, peerNetData);
                            break;
                        case CLOSED:
                            if (engine.isOutboundDone()) {
                                return false;
                            } else {
                                engine.closeOutbound();
                                handshakeStatus = engine.getHandshakeStatus();
                                break;
                            }
                        default:
                            throw new IllegalStateException("Invalid handshake state: " + result.getStatus());
                    }
                    break;
                case NEED_WRAP:
                    myNetData.clear();
                    try {
                        result = engine.wrap(myAppData, myNetData);
                        handshakeStatus = result.getHandshakeStatus();
                    } catch (SSLException sslException) {
                        engine.closeOutbound();
                        handshakeStatus = engine.getHandshakeStatus();
                        break;
                    }
                    switch (result.getStatus()) {
                        case OK :
                            myNetData.flip();
                            while (myNetData.hasRemaining()) {
                                socketChannel.write(myNetData);
                            }
                            break;
                        case BUFFER_OVERFLOW:
                            myNetData = enlargePacketBuffer(engine, myNetData);
                            break;
                        case BUFFER_UNDERFLOW:
                            throw new SSLException("After encryption, the message content is empty and an error is reported");
                        case CLOSED:
                            try {
                                myNetData.flip();
                                while (myNetData.hasRemaining()) {
                                    socketChannel.write(myNetData);
                                }
                                peerNetData.clear();
                            } catch (Exception e) {
                                handshakeStatus = engine.getHandshakeStatus();
                            }
                            break;
                        default:
                            throw new IllegalStateException("Invalid handshake state: " + result.getStatus());
                    }
                    break;
                case NEED_TASK:
                    Runnable task;
                    while ((task = engine.getDelegatedTask()) != null) {
                        executor.execute(task);
                    }
                    handshakeStatus = engine.getHandshakeStatus();
                    break;
                case FINISHED:
                    break;
                case NOT_HANDSHAKING:
                    break;
                default:
                    throw new IllegalStateException("Invalid handshake state: " + handshakeStatus);
            }
        }

        return true;
    }

    /**
     * reference resources https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
     * SSL transport read protocol
     * @param consumer
     * @throws IOException
     */
    public void read(Consumer<ByteBuffer> consumer) throws IOException {
        // BUFFER initialization
        peerNetData.clear();
        int bytesRead = socketChannel.read(peerNetData);
        if (bytesRead > 0) {
            peerNetData.flip();
            while (peerNetData.hasRemaining()) {
                peerAppData.clear();
                SSLEngineResult result = engine.unwrap(peerNetData, peerAppData);
                switch (result.getStatus()) {
                    case OK:
                        log.debug("The returned result message received from the remote is:" + new String(peerAppData.array(), 0, peerAppData.position()));
                        consumer.accept(peerAppData);
                        peerAppData.flip();
                        break;
                    case BUFFER_OVERFLOW:
                        peerAppData = enlargeApplicationBuffer(engine, peerAppData);
                        break;
                    case BUFFER_UNDERFLOW:
                        peerNetData = handleBufferUnderflow(engine, peerNetData);
                        break;
                    case CLOSED:
                        log.debug("Remote connection close message received.");
                        closeConnection();
                        return;
                    default:
                        throw new IllegalStateException("Invalid handshake state: " + result.getStatus());
                }
            }
        } else if (bytesRead < 0) {
            log.debug("received END OF STREAM，Close connection.");
            handleEndOfStream();
        }
    }

    public void write(String message) throws IOException {
        write(ByteBuffer.wrap(message.getBytes()));
    }

    /**
     * reference resources https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html
     * SSL transport write protocol
     * @param message
     * @throws IOException
     */
    public void write(ByteBuffer message) throws IOException {
        myAppData.clear();
        myAppData.put(message);
        myAppData.flip();
        while (myAppData.hasRemaining()) {
            myNetData.clear();
            SSLEngineResult result = engine.wrap(myAppData, myNetData);
            switch (result.getStatus()) {
                case OK:
                    myNetData.flip();
                    while (myNetData.hasRemaining()) {
                        socketChannel.write(myNetData);
                    }
                    log.debug("The message written to the remote is: {}", message);
                    break;
                case BUFFER_OVERFLOW:
                    myNetData = enlargePacketBuffer(engine, myNetData);
                    break;
                case BUFFER_UNDERFLOW:
                    throw new SSLException("The message content is empty after encryption.");
                case CLOSED:
                    closeConnection();
                    return;
                default:
                    throw new IllegalStateException("Invalid handshake state: " + result.getStatus());
            }
        }
    }

    /**
     * Close connection
     * @throws IOException
     */
    public void closeConnection() throws IOException  {
        engine.closeOutbound();
        doHandshake();
        socketChannel.close();
        executor.shutdown();
    }

    /**
     * END OF STREAM(-1)The default is to close the connection
     * @throws IOException
     */
    protected void handleEndOfStream() throws IOException  {
        try {
            engine.closeInbound();
        } catch (Exception e) {
            log.error("END OF STREAM Closing failed.", e);
        }
        closeConnection();
    }

}

above:

• Realize unified handshake based on SSL protocol;
• The decryption of reading and the encryption method of writing are realized respectively;
• Implement SslSocketChannel as the Decorator of SocketChannel;

SslSocketChannel test server

Based on the above encapsulation, the simple test server is as follows

@Slf4j
public class NioSslServer {

    public static void main(String[] args) throws Exception {
        NioSslServer sslServer = new NioSslServer("127.0.0.1", 8006);
        sslServer.start();
        // Use curl -vv -k 'https://localhost:8006 'Connect
    }

    private SSLContext context;

    private Selector selector;

    public NioSslServer(String hostAddress, int port) throws Exception {
        // Initialize SSL Context
        context = serverSSLContext();

        // Register listener
        selector = SelectorProvider.provider().openSelector();
        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.configureBlocking(false);
        serverSocketChannel.socket().bind(new InetSocketAddress(hostAddress, port));
        serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
    }

    public void start() throws Exception {

        log.debug("Waiting for connection.");

        while (true) {
            selector.select();
            Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
            while (selectedKeys.hasNext()) {
                SelectionKey key = selectedKeys.next();
                selectedKeys.remove();
                if (!key.isValid()) {
                    continue;
                }
                if (key.isAcceptable()) {
                    accept(key);
                } else if (key.isReadable()) {
                    ((SslSocketChannel)key.attachment()).read(buf->{});
                    // Respond directly to an OK
                    ((SslSocketChannel)key.attachment()).write("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nOK\r\n\r\n");
                    ((SslSocketChannel)key.attachment()).closeConnection();
                }
            }
        }
    }

    private void accept(SelectionKey key) throws Exception {
        log.debug("Receive new requests.");

        SocketChannel socketChannel = ((ServerSocketChannel)key.channel()).accept();
        socketChannel.configureBlocking(false);

        SslSocketChannel sslSocketChannel = new SslSocketChannel(context, socketChannel, false);
        if (sslSocketChannel.doHandshake()) {
            socketChannel.register(selector, SelectionKey.OP_READ, sslSocketChannel);
        } else {
            socketChannel.close();
            log.debug("Handshake failed. Close the connection.");
        }
    }
}

above:

• Because it is NIO, simple testing needs to use NIO's basic component Selector for testing;
• First initialize ServerSocketChannel and listen to port 8006;
• After receiving the request, encapsulate the SocketChannel as SslSocketChannel and register it with the Selector
• After receiving the data, read and write through SslSocketChannel;

SslSocketChannel test client

Based on the above server encapsulation, the simple test client is as follows

@Slf4j
public class NioSslClient {

    public static void main(String[] args) throws Exception {
        NioSslClient sslClient = new NioSslClient("httpbin.org", 443);
        sslClient.connect();
        // 'request ' https://httpbin.org/get '
    }

    private String remoteAddress;

    private int port;

    private SSLEngine engine;

    private SocketChannel socketChannel;

    private SSLContext context;

    /**
     * Remote HOST and PORT are required
     * @param remoteAddress
     * @param port
     * @throws Exception
     */
    public NioSslClient(String remoteAddress, int port) throws Exception {
        this.remoteAddress = remoteAddress;
        this.port = port;

        context = clientSSLContext();
        engine = context.createSSLEngine(remoteAddress, port);
        engine.setUseClientMode(true);
    }

    public boolean connect() throws Exception {
        socketChannel = SocketChannel.open();
        socketChannel.configureBlocking(false);
        socketChannel.connect(new InetSocketAddress(remoteAddress, port));
        while (!socketChannel.finishConnect()) {
            // Through the REACTOR, there will be no waiting
            //log.debug("connecting..);
        }

        SslSocketChannel sslSocketChannel = new SslSocketChannel(context, socketChannel, true);
        sslSocketChannel.doHandshake();

        // After the handshake is completed, turn on the SELECTOR
        Selector selector = SelectorProvider.provider().openSelector();
        socketChannel.register(selector, SelectionKey.OP_READ, sslSocketChannel);

        // Write request
        sslSocketChannel.write("GET /get HTTP/1.1\r\n"
            + "Host: httpbin.org:443\r\n"
            + "User-Agent: curl/7.62.0\r\n"
            + "Accept: */*\r\n"
            + "\r\n");

        // Read results
        while (true) {
            selector.select();
            Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
            while (selectedKeys.hasNext()) {
                SelectionKey key = selectedKeys.next();
                selectedKeys.remove();
                if (key.isValid() && key.isReadable()) {
                    ((SslSocketChannel)key.attachment()).read(buf->{
                        log.info("{}", new String(buf.array(), 0, buf.position()));
                    });
                    ((SslSocketChannel)key.attachment()).closeConnection();
                    return true;
                }
            }
        }
    }
}

above:

The encapsulation test of the client is to verify that both directions of the encapsulated SSL protocol are OK,
In the following non transparent upstream proxy, SslSocketChannel will be used as the server and client at the same time
The above encapsulation is similar to the server encapsulation, except that the SocketChannel is initialized and connect ed instead of bound

summary

above:

• The non transparent agent needs to get complete request data, which can be implemented in batch through Decorator mode;
• The non transparent agent needs to get the decrypted HTTPS request data, which can be realized by encapsulating the original SocketChannel through SslSocketChannel;
• Finally, after receiving the request, do the corresponding processing, and finally realize the non transparent agent.

3 transparent upstream agent

Transparent upstream agent is simpler than transparent agent. The difference is

• The transparent agent needs to respond to the CONNECT request. The transparent upstream agent does not need to forward it directly;
• The transparent agent needs to resolve the HOST and PORT in the CONNECT request and CONNECT to the server; The transparent upstream agent only needs to CONNECT to the IP:PORT of the downstream agent and forward the request directly;
• The transparent upstream agent is just a simple SocketChannel pipeline; Determine the downstream proxy server, connect and forward the request;

You only need to make the above simple modifications to the transparent agent to realize the transparent upstream agent.

4 non transparent upstream agent

Non transparent upstream agents are more complex than non transparent agents

The above is divided into four components: client, server handler, client handler and server

• If it is an HTTP request, the data directly passes through the client < - > serverhandler < - > clienthandler < - > server. The proxy gateway only needs to do a simple request aggregation batch, and the corresponding management strategy can be applied;
• If it is an HTTPS request, the agent, as the intermediary between the client and the server, can only get the encrypted data; Therefore, the proxy gateway needs to communicate with the client as the service party of HTTPS; Then, it communicates with the server as the client of HTTPS;
• When an agent is an HTTPS server, it needs to consider that it is a non transparent agent and implement the protocols related to the non transparent agent;
• When an agent is an HTTPS client, it needs to consider that its downstream is a transparent agent, and the real service party is the service Party requested by the client;

III. design and Implementation

What this paper needs to build is a non transparent upstream agent. The following gives a detailed design and implementation using NETTY framework. The unified proxy gateway is divided into two parts, ServerHandler and ClientHandler, as follows:

• This paper introduces the implementation of proxy gateway server;
• This paper introduces the implementation of proxy gateway client;

1 proxy gateway server

Mainly including

• Initialize proxy gateway server
• Initialize server processor
• Server protocol upgrade and processing

Initialize proxy gateway service

public void start() {
    HookedExecutors.newSingleThreadExecutor().submit(() ->{
        log.info("Start the proxy server and listen on the port:{}", auditProxyConfig.getProxyServerPort());
        EventLoopGroup bossGroup = new NioEventLoopGroup(auditProxyConfig.getBossThreadCount());
        EventLoopGroup workerGroup = new NioEventLoopGroup(auditProxyConfig.getWorkThreadCount());
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                .channel(NioServerSocketChannel.class)
                .handler(new LoggingHandler(LogLevel.DEBUG))
                .childHandler(new ServerChannelInitializer(auditProxyConfig))
                .bind(auditProxyConfig.getProxyServerPort()).sync().channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("The proxy server was interrupted.", e);
            Thread.currentThread().interrupt();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    });
}

Proxy gateway initialization is relatively simple,

• The bossGroup thread group is responsible for receiving requests
• The worker group thread group is responsible for processing the received request data. The specific processing logic is encapsulated in the ServerChannelInitializer.

The request processor of the proxy gateway service is defined in ServerChannelInitializer as

@Override
  protected void initChannel(SocketChannel ch) throws Exception {
      ch.pipeline()
          .addLast(new HttpRequestDecoder())
          .addLast(new HttpObjectAggregator(auditProxyConfig.getMaxRequestSize()))
          .addLast(new ServerChannelHandler(auditProxyConfig));
  }

Firstly, the HTTP request is parsed, then the batch processing is done, and finally the ServerChannelHandler implements the proxy gateway protocol;

Proxy gateway protocol:

• Determine whether it is a CONNECT request. If so, the CONNECT request will be stored; Pause reading, send generation
• Manage the successful response, and upgrade the agreement after the response is successful;
• The upgrade engine essentially uses SslSocketChannel to transparently encapsulate the original SocketChannel;
• Finally, CONNECT the remote server according to the CONNECT request;

The detailed implementation is as follows:

 @Override
 public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
     FullHttpRequest request = (FullHttpRequest)msg;

     try {
         if (isConnectRequest(request)) {
             // CONNECT request, store pending
             saveConnectRequest(ctx, request);

             // Prohibit reading
             ctx.channel().config().setAutoRead(false);

             // Send response
             connectionEstablished(ctx, ctx.newPromise().addListener(future -> {
                 if (future.isSuccess()) {
                     // upgrade
                     if (isSslRequest(request) && !isUpgraded(ctx)) {
                         upgrade(ctx);
                     }

                     // Open message reading
                     ctx.channel().config().setAutoRead(true);
                     ctx.read();
                 }
             }));

         } else {
             // Other requests to determine whether it has been upgraded
             if (!isUpgraded(ctx)) {

                 // Upgrade engine
                 upgrade(ctx);
             }

             // Connect remote
             connectRemote(ctx, request);
         }
     } finally {
         ctx.fireChannelRead(msg);
     }
 }

2 proxy gateway client

The proxy gateway server needs to connect to the remote service and enter the proxy gateway client part.

Proxy gateway client initialization:

 /**
  * Initialize remote connection
  * @param ctx
  * @param httpRequest
  */
 protected void connectRemote(ChannelHandlerContext ctx, FullHttpRequest httpRequest) {
     Bootstrap b = new Bootstrap();
     b.group(ctx.channel().eventLoop()) // use the same EventLoop
         .channel(ctx.channel().getClass())
         .handler(new ClientChannelInitializer(auditProxyConfig, ctx, safeCopy(httpRequest)));

     // Dynamic connection agent
     FullHttpRequest originRequest = ctx.channel().attr(CONNECT_REQUEST).get();
     if (originRequest == null) {
         originRequest = httpRequest;
     }
     ChannelFuture cf = b.connect(new InetSocketAddress(calculateHost(originRequest), calculatePort(originRequest)));
     Channel cch = cf.channel();
     ctx.channel().attr(CLIENT_CHANNEL).set(cch);    
 }

above:

• Reuse the workerGroup thread group of the proxy gateway server;
• The processing of requests and results is encapsulated in ClientChannelInitializer;
• The HOST and PORT of the connected remote server can be resolved in the request received by the server.

Initialization logic of the processor of the proxy gateway client:

@Override
  protected void initChannel(SocketChannel ch) throws Exception {
      SocketAddress socketAddress = calculateProxy();
      if (!Objects.isNull(socketAddress)) {
          ch.pipeline().addLast(new HttpProxyHandler(calculateProxy(), auditProxyConfig.getUserName(), auditProxyConfig
              .getPassword()));
      }
      if (isSslRequest()) {
          String host = host();
          int port = port();
          if (StringUtils.isNoneBlank(host) && port > 0) {
              ch.pipeline().addLast(new SslHandler(sslEngine(host, port)));
          }
      }
      ch.pipeline().addLast(new ClientChannelHandler(clientContext, httpRequest));
  }

above:

• If the downstream is an agent, HttpProxyHandler will be used to communicate with the remote server through the downstream agent;
• If the SSL protocol needs to be upgraded, the SocketChannel will be transparently encapsulated to realize SSL communication.
• Finally, the ClientChannelHandler is just the forwarding of simple messages; The only difference is that because the proxy gateway intercepts the first request, it needs to forward the intercepted request to the server.

IV. other issues

Possible problems of proxy gateway implementation:

1 memory problem

The common problem faced by agents is OOM. When implementing the proxy gateway, this paper ensures that the HTTP/HTTPS request body currently being processed is cached in memory. Theoretically, the upper limit of memory usage is the number of requests processed in real time * the average size of the request body. The request results of HTTP/HTTPS directly use off heap memory and zero copy forwarding.

2 performance issues

Performance issues should not be considered in advance. In this paper, the proxy gateway implemented by NETTY framework uses a lot of out of heap memory and zero copy forwarding to avoid performance problems.

After the first phase of the proxy gateway went online, it faced a performance problem caused by a long connection,

• After the CLIENT and SERVER establish a long TCP connection (for example, TCP heartbeat detection), usually either the CLIENT closes the TCP connection or the SERVER closes;
• If both parties occupy TCP connection resources for a long time without closing, SOCKET resources will be leaked; The phenomenon is: CPU resources are full and idle connections are processed; The new connection cannot be established;

Use IdleStateHandler to regularly monitor idle TCP connections and force them to close; This problem is solved.

V. summary

This paper focuses on the core of the unified proxy gateway and introduces the technical principles related to the proxy in detail.

The management part of the proxy gateway can be maintained in the ServerHandler part or the ClientHandler part;

• ServerHandler can intercept conversion requests
• ClientHanlder controls the exit of the request

Note: This article uses Netty's zero copy; Storing the request for parsing processing; However, the processing of RESPONSE is not realized; That is, RESPONSE is directly through the gateway, which avoids the common problems related to proxy implementation, memory leakage and OOM;

Finally, after implementing the proxy gateway, this paper makes corresponding control over the proxy resources and requests flowing through the proxy gateway, mainly including:

When a request for static resources is encountered, the proxy gateway will directly request the remote server, not through the downstream proxy
When the request HEADER contains a region ID, the proxy gateway will try its best to ensure that the request enters the specified region agent and accesses the remote server through the region agent.

Original link
This article is the original content of Alibaba cloud and cannot be reproduced without permission.