Basic introduction of encoding and decoding

1. When writing network applications, because the data transmitted in the network is binary bytecode data, it needs to be encoded when sending data and decoded when receiving data
2. Codec has two components: decoder and encoder The encoder is responsible for converting business data into bytecode data, and the decoder is responsible for converting bytecode data into business data

Analysis of the encoding and decoding mechanism and problems of Netty

1. Netty itself provides some codecs
2. Encoder provided by Netty
   1. StringEncoder, which encodes string data
   2. ObjectEncoder, which encodes Java objects
   3. ......
3. Decoder provided by Netty
   1. StringDecoder: decodes string data
   2. ObjectDecoder: decodes Java objects
   3. ......
4. The ObjectDecoder and ObjectEncoder provided by Netty can be used to encode and decode Pojo objects or various business objects. Java serialization technology is still used at the bottom, but the efficiency of Java serialization technology is not high. There are the following problems
   1. Cannot cross language
   2. The volume after serialization is too large, which is more than 5 times that of binary coding
   3. Serialization performance is too low
5. =>Lead to a new solution [Google's ProtoBuf]

Protobuf

1. Protobuf basic introduction and use diagram
2. Protobuf fer is an open source project released by Google. Its full name is Google Protocol Buffers. It is a lightweight and efficient structured data storage format. It can be used for structured data serialization, or serialization. It is very suitable for data storage or RPC [remote procedure call] data exchange format
   1. At present, many companies use HTTP + JSON - > TCP + protobuf
3. Reference documents: https://developers.google.com/protocol-buffers/docs/proto Language Guide
4. Protobuf manages data in the form of message
5. Support cross platform and cross language, i.e. [client and server can be written in different languages] (support most current languages, such as C++, C#, Java, Python, etc.)
6. High performance and high reliability
7. Using protobuf compiler can automatically generate code. Protobuf uses the definition of class Proto file description, description, written in IDEA When the proto file is, it will automatically prompt whether to download it Proto write plug-ins, which can highlight the syntax
8. Then through protocol Exe compiler according to proto is automatically generated java file
9. Schematic diagram of Protobuf

Protobuf quick start case

Write a program and use Protobuf to complete the following functions

1. The client can send a Student POJO object to the server (encoded by Protobuf)
2. The server can accept Student POJO objects and display information (decoded by Protobuf)

Introducing Protobuf dependency

<!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java -->
<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.6.1</version>
</dependency>

data type

Special field

Define other complex type References: https://blog.csdn.net/lijingjingchn/article/details/89466437

Software

New student proto

syntax = "proto3"; // edition
option java_outer_classname = "StudentPOJO"; // External class name of Java
// protobuf uses message to manage data
message Student { // An internal class Student will be generated in the external class of StudentPOJO, which is the real sent POJO object

  /**
    Indicates that there is an attribute in the Student class with the name of id and the type of int32(Protobuf type). 1 indicates the sequence number, not the value
   */
  int32 id = 1;

  string name = 2;
}

According to Generate Java files from proto files

Put the written file into the bin folder

Start cmd at current location

Execute compilation

protoc.exe --java_out=. Student.proto

Copy the generated file to the project

Generated file

Don't paste it [it's too big]. Have a look for yourself

New NettyServer

package com.dance.netty.netty.protobuf;

import com.dance.netty.netty.protobuf.pojo.StudentPOJO;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.protobuf.ProtobufDecoder;

public class NettyServer {

    public static void main(String[] args) throws InterruptedException {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.group(bossGroup, workerGroup) //Set up two thread groups
                    .channel(NioServerSocketChannel.class) // NioServerSocketChannel is used as the channel implementation of the server
                    .option(ChannelOption.SO_BACKLOG, 128) // Set the number of thread queues waiting for connections
                    .childOption(ChannelOption.SO_KEEPALIVE, true) // Set keep active connection status
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            ChannelPipeline pipeline = socketChannel.pipeline();
                            // The decoder added to protobuf specifies the default instance of decoding
                            pipeline.addLast(new ProtobufDecoder(StudentPOJO.Student.getDefaultInstance()));
                            pipeline.addLast(new NettyServerHandler());
                        }
                    });

            System.out.println("server is ready......");
            ChannelFuture channelFuture = serverBootstrap.bind(6668).sync();
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }

}

New NettyServerhandler

package com.dance.netty.netty.protobuf;

import com.dance.netty.netty.protobuf.pojo.StudentPOJO;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import java.nio.charset.StandardCharsets;
public class NettyServerHandler extends ChannelInboundHandlerAdapter {
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        // Read the amount of studentpojo sent from the client student
        StudentPOJO.Student student = (StudentPOJO.Student) msg;
        System.out.println("Data sent by client:" + student.getId() + " " + student.getName());
    }
    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.writeAndFlush(Unpooled.copiedBuffer("Hello client", StandardCharsets.UTF_8));
    }
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        ctx.close();
        cause.printStackTrace();
    }
}

New NettyClient

package com.dance.netty.netty.protobuf;

import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.protobuf.ProtobufDecoder;
import io.netty.handler.codec.protobuf.ProtobufEncoder;

public class NettyClient {

    public static void main(String[] args) throws InterruptedException {

        EventLoopGroup eventExecutors = new NioEventLoopGroup();

        try {
            Bootstrap bootstrap = new Bootstrap();

            bootstrap.group(eventExecutors)
                    .channel(NioSocketChannel.class)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            ChannelPipeline pipeline = socketChannel.pipeline();
                            // Add encoder for processing Protobuf
                            pipeline.addLast(new ProtobufEncoder());
                            pipeline.addLast(new NettyClientHandler());
                        }
                    });
            System.out.println("client ok !");
            ChannelFuture sync = bootstrap.connect("127.0.0.1", 6668).sync();
            sync.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            eventExecutors.shutdownGracefully();
        }
    }

}

New NettyClientHandler

package com.dance.netty.netty.protobuf;

import com.dance.netty.netty.protobuf.pojo.StudentPOJO;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;

import java.nio.charset.StandardCharsets;

public class NettyClientHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        // Send a Student object to the server
        StudentPOJO.Student dance = StudentPOJO.Student.newBuilder().setId(4).setName("dance").build();
        ctx.writeAndFlush(dance);
    }
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        ByteBuf byteBuf = (ByteBuf) msg;
        System.out.println("Messages replied by the server: " + byteBuf.toString(StandardCharsets.UTF_8));
        System.out.println("server address: " + ctx.channel().remoteAddress());
    }
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        ctx.close();
        cause.printStackTrace();
    }
}

test

Start Server

Start Client

server is ready......
Data sent by client:4 dance
 client ok !
Messages replied by the server: Hello client
 server address: /127.0.0.1:6668

Defining generics using SimpleChannelInBoundHandler

package com.dance.netty.netty.protobuf;

import com.dance.netty.netty.protobuf.pojo.StudentPOJO;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.SimpleChannelInboundHandler;

import java.nio.charset.StandardCharsets;
//public class NettyServerHandler extends ChannelInboundHandlerAdapter {
public class NettyServerHandler extends SimpleChannelInboundHandler<StudentPOJO.Student> {
    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.writeAndFlush(Unpooled.copiedBuffer("Hello client", StandardCharsets.UTF_8));
    }
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        ctx.close();
        cause.printStackTrace();
    }

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, StudentPOJO.Student msg) throws Exception {
        System.out.println("Data sent by client:" + msg.getId() + " " + msg.getName());
    }
}

Protobuf quick start instance 2

demand

1. Write a program and use Protobuf to complete the following functions
2. The client can randomly send studentPOJO/workerPOJO objects to the server (encoded by Protobuf)
3. The server receives the StudentPOJO/WorkerPOJO object (it needs to determine the type) and displays the information (decoded by Protobuf)
4. See the teacher's show time for details

Write proto file

syntax = "proto3"; // edition

option optimize_for = SPEED; // Speed up analysis

option java_package = "com.dance.netty.netty.protobuf2.pojo"; // Specify package

option java_outer_classname = "MyDataInfo"; // External class name of Java

message MyMessage{
  // Define an enumeration type
  enum DataType {
    StudentType = 0;
    WorkerType = 1;
  }

  // Use DataType to identify which enumeration type is passed
  DataType dataType = 1;

  // Indicates that at most one of the enumeration types can appear each time, saving space
  oneof dataBody{
    Student student = 2;
    Worker worker = 3;
  }
}

message Student {
  int32 id = 1;
  string name = 2;
}

message Worker {
  int32 id = 1;
  string name = 2;
}

Generate java types and copy them to the project

So big~

Modified according to previous case 1

Modify NettyClientHandler

@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
  // Send students or workers randomly
  int i = new Random().nextInt(3);
  MyDataInfo.MyMessage myMessage = null;
  if(0 == i){
    myMessage = MyDataInfo.MyMessage.newBuilder()
      .setDataType(MyDataInfo.MyMessage.DataType.StudentType)
      .setStudent(MyDataInfo.Student.newBuilder().setId(1).setName("flower").build()).build();
  }else{
    myMessage = MyDataInfo.MyMessage.newBuilder()
      .setDataType(MyDataInfo.MyMessage.DataType.StudentType)
      .setWorker(MyDataInfo.Worker.newBuilder().setId(2).setName("dance").build()).build();
  }
  ctx.writeAndFlush(myMessage);

}

Modify NettyServer

pipeline.addLast(new ProtobufDecoder(MyDataInfo.MyMessage.getDefaultInstance()));

Modify NettyServerHandler

public class NettyServerHandler extends SimpleChannelInboundHandler<MyDataInfo.MyMessage> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, MyDataInfo.MyMessage msg) throws Exception {
    if(msg.getDataType() == MyDataInfo.MyMessage.DataType.StudentType){
        System.out.println("Data sent by client:" + msg.getStudent().getId() + " " + msg.getStudent().getName());
    }else if(msg.getDataType() == MyDataInfo.MyMessage.DataType.WorkerType){
        System.out.println("Data sent by client:" + msg.getWorker().getId() + " " + msg.getWorker().getName());
    }
}

test

client

client ok !
Messages replied by the server: Hello client
 server address: /127.0.0.1:6668

Server

server is ready......
Data sent by client:2 dance
 Data sent by client:2 dance
 Data sent by client:1 flower