Raise a chestnut gorpc - Encoding and decoding of messages

Posted by webbnino on Tue, 01 Feb 2022 08:28:52 +0100

The first rpc in 2022, earlier than ever...

Stay in Hangzhou for the New Year... Write something

Initialize project gorpc

With go module we can easily create a new project

mkdir gorpc
go mod init github.com/taadis/gorpc

// output:
go: creating new go.mod: module github.com/taadis/gorpc

Message conventions

The communication between the client and the server of rpc requires the transmission of data messages. A typical message structure generally consists of two parts

  • Header header - Used to host Convention content, generally relatively fixed.
  • body Message body - Used to hold user data, usually of variable length, that is, dynamic

Let's start with a more general definition of the message body, because it is dynamic, so we can define it directly with interface {} in go without specifically declaring a structure.

Then let's define a header structure

// codec.go

type Header struct {
    Sequence      uint64 // sequence number chosen by client
    ServiceMethod string // format "Service.Method"
    Error         error

The Sequence Sequence Number is brought over by the client. Each request must be somewhat different so that the server can distinguish between different calls based on the Sequence Number, which is understood to be a unique ID.

ServiceMethod is a method term under a service that you want to call remotely, as opposed to the method name of a structure in the go language, such as the user-created method "User.Create".

Errors are Error messages used to place errors that occur at one end so that another receives a message and handles it based on the Error, rather than losing the response directly.

Coding and decoding of messages

Messages that communicate between the client and the server of rpc have their own unique format, so encoding and decoding are the key steps involved, which are known as serialization and deserialization.

For Abstract understanding, we define a uniform Codec interface

// codec.go

type Codec interface {
    ReadHeader(*Header) error
    ReadBody(interface{}) error
    Write(*Header, interface{}) error

ReadHeader reads the header and returns an error if there is an error.

ReadBody reads the message body and the data is dynamic, so use interface {} as a parameter and return an error if there is an error.

After the Write message is received and processed, we need to inform the client of the result and a write operation is required.

Here we use encoding/gob built into the standard library to increase productivity.

Of course, you can also use encoding/json, encoding/xml or other codec packages. Select encoding/gob here just because it is unique to go. JUST GO.

Next we implement a gobCodec based on encoding/gob

rpc requests are network requests, essentially I/O, so we can use io.ReadWriteCloser to define the network link conn.

Through gob.Decoder decodes the data stream from the request to the corresponding structure parameter.

After the service-side call is completed, use gob again for the returned results. Encoder is encoded into the data stream,

Finally through bufio.Writer writes data to complete the response.

// codec.go

type gobCodec struct {
    conn     io.ReadWriteCloser
    decoder  *gob.Decoder
    encoder  *gob.Encoder
    writeBuf *bufio.Writer

Encapsulates a newGobCodec function for subsequent calls.

func newGobCodec(conn io.ReadWriteCloser) Codec {
    writeBuf := bufio.NewWriter(conn)
    return &gobCodec{
        conn:     conn,
        decoder:  gob.NewDecoder(conn),
        encoder:  gob.NewEncoder(writeBuf),
	writeBuf: writeBuf,

Implement ReadHeader method in Codec interface

func (c *gobCodec) ReadHeader(header *Header) error {
    return c.decoder.Decode(header)

Implement ReadBody method in Codec interface

func (c *gobCodec) ReadBody(body interface{}) error {
    return c.decoder.Decode(body)

Implementing Write methods in Codec interfaces

func (c *gobCodec) Write(header *Header, body interface{}) error {
    defer func() {
        if c.writeBuf.Flush() != nil {

    if err := c.encoder.Encode(header); err != nil {
    	return err

    if err := c.encoder.Encode(body); err != nil {
	return err

    return nil

So far, we have abstracted the Codec interface from the lower level data encoding and decoding in rpc and implemented gobCodec with encoding/gob.

Topics: Go rpc