Since the rpc used by the project team is based on the google protobuf rpc Protocol, I spent some time learning about protobuf rpc. rpc is certainly no stranger to distributed systems. For children's shoes that rpc doesn't know, you can google by yourself. Here's just a brief introduction. The main function of rpc is to make the implementation of distributed system simpler and provide powerful remote calls without losing the simplicity of local call semantics. In order to achieve this goal, the rpc framework needs to provide a transparent invocation mechanism so that users do not have to show whether to distinguish local calls or remote calls. The components involved in rpc architecture are as follows:
The client calls the remote interface method like calling the local method. The RPC framework provides the proxy implementation of the interface, and the actual call will be entrusted to the proxy RpcProxy. The proxy encapsulates the call information and forwards the call to RpcInvoker # for actual execution. The RpcInvoker of the client maintains the channel RpcChannel with the server through the connector RpcConnector, uses RpcProtocol to encode the protocol, and sends the encoded request message to the server through the channel. The RPC server receiver rpcceptor receives the call request from the client, and also uses RpcProtocol to perform protocol decoding. The decode d call information is passed to RpcProcessor ， to control the processing call process, and finally delegate the call to RpcInvoker ， to actually execute and return the call result.
Protobuf rpc mainly plays the role of RpcProtocol in the above components, which eliminates the design of protocol, and protobuf protocol is very efficient in coding and space efficiency, which is also the reason why many companies use protobuf as data serialization and communication protocol. Meanwhile, protobuf rpc defines an abstract rpc framework, as shown in the following figure:
Rpcservicesub and RpcService classes are generated by the protobuf compiler according to the proto definition. RpcService defines the function interface exposed by the server to the client. The specific implementation requires the user to inherit this class. Rpcservicesub defines the description of the exposed function of the server, and uniformly converts the client's calls to the functions in rpcservicesub to calling the CallMethod method in RpcChannel. The CallMethod encodes the rpc call through the function descriptor and function parameters passed by rpcservicesub, and finally sends it to the service party through RpcConnecor. The other party finally calls the function defined in RpcService in the opposite process of the client. In fact, the protobuf rpc framework only defines an empty CallMethod in RpcChannel, so how to encode and call RpcConnector should be implemented by yourself. RpcConnector is not defined in protobuf, so it is implemented by the user. Its function is to send and receive rpc message packets. On the server side, RpcChannel specifically calls the functions exposed to the client in RpcService by calling the CallMethod in RpcService.
After introducing so much, how to use protobuf rpc to implement an rpc must be a fog. Let's use protobuf rpc to implement a simple python rpc demo.
The demo is directly given below to describe the proto file of PRC. For the preparation rules of proto file, please refer to the official website of protobuf.
common.proto file:
1 package game;
2
3 message RequestMessage
4 {
5 required string message = 1;
6 }
7
8 message ResponseMessage
9 {
10 required string message = 1;
11 }
game_service.proto file:
1 package game;
2
3 import "common.proto";
4 option py_generic_services = true;
5
6 service GameService
7 {
8 rpc connect_server(RequestMessage) returns(RequestMessage);
9 }
common. The proto file describes the messages sent and received in RPC; game_service.proto describes the connect exported by the server_ Server function, which accepts the RequestMessage object as a parameter and returns the RequestMessage object. Option py must be added when using PRC protocol_ generic_ services = true; Optional, otherwise the compiler will not generate a file containing connect_ GameService description of the server function.
1 class MyRpcChannel(service.RpcChannel):
2 def __init__(self, rpc_service, conn):
3 super(MyRpcChannel, self).__init__()
4 self.logger = LogManager.get_logger("MyRpcChannel")
5
6 def CallMethod(self, method_descriptor, rpc_controller, request, response_class, done):
7 """"protol buffer rpc Required function to send rpc call"""
8 self.logger.info('CallMethod')
9 cmd_index = method_descriptor.index
10 assert(cmd_index < 65535)
11 data = request.SerializeToString()
12 total_len = len(data) + 2
13 self.conn.send_data(''.join([pack('<I', total_len), pack('<H', cmd_index), data]))
14
15 def from_request(self):
16 """"A function that is adjusted after a complete request is resolved from the network."""
17 index_data = self.rpc_request.data[0:2]
18 cmd_index = unpack('<H', index_data)[0]
19 rpc_service = self.rpc_service
20 s_descriptor = rpc_service.GetDescriptor()
21 method = s_descriptor.methods[cmd_index]
22 try:
23 request = rpc_service.GetRequestClass(method)()
24 serialized = self.rpc_request.data[2:]
25 request.ParseFromString(serialized)
26 rpc_service.CallMethod(method, self.controller, request, None)
27 except:
28 self.logger.error("Call rpc method failed!")
29 self.logger.log_last_except()
30 return True
Finally, inherit GameService and implement connect_ The server function.
1 class GameService(game_service_pb2.GameService):
2 def __init__(self):
3 self.logger = LogManager.get_logger("GameService")
4
5 def connect_server(self, rpc_controller, request, callback):
6 self.logger.info('%s', request.message)
As for the RpcConnector for sending and receiving messages on the network, it can be implemented using python's asyncore library. The specific implementation is not discussed here.
From the above implementation, the implementation of protobuf rpc mainly includes writing proto files and compiling to generate corresponding services_ PB2 file, inherit RpcChannel and implement CallMethod and CallMethod calling Service, inherit Service to implement the functions exposed to the client.