muduo learning notes: implementation of net part TCP network programming library TcpClient

Posted by Dathremar on Thu, 16 Dec 2021 21:07:48 +0100

With the foregoing [muduo learning notes: implementation of net part TCP network programming library Connector] The implementation of tcpclient is not difficult. muduo uses tcpclient to initiate a connection. Tcpclient has a Connector connector. Tcpclient uses Connector to initiate a connection. After the connection is established successfully, tcpclient uses socket to create TcpConnection to manage the connection. Each TcpClient class only manages one TcpConnection. After the connection is established successfully, set the corresponding callback function. Obviously, tcpclient is used to manage client connections, and the real connections are handed over to the Connector.

Its code is even somewhat similar to TcpServer, except that TcpClient only manages one TcpConnection. Let's talk about some key points first:

  • TcpClient has the function of reconnecting after TcpConnection is disconnected, and the Connector has the function of repeatedly trying to connect, so the startup sequence of the client and server is irrelevant. You can start the client first. Once the server starts, the connection can be restored within half a minute (controlled by the Connector::kMaxRetryDelayMs constant); When the client is running, the server can be restarted and the client will be automatically reconnected.
  • The delay time of the first retry after the connection is disconnected is random. For example, when the server crashes, all its client connections are disconnected at the same time, and then the connection is initiated again after 0.5s. This may not only cause SYN packet loss, but also bring short-term heavy load to the server and affect its service quality. Therefore, each TcpClient should wait for a random period of time (0.5~2s) and retry to avoid congestion.
  • If TCP SYN packet loss occurs during connection initiation, the default retry interval of the system is 3s, during which no error code will be returned, and this interval seems to be difficult to modify. If you need to shorten the interval, you can use another timer to initiate another link after 0.5s or 1s. If required, this function can be implemented in the Connector.

1. TcpClient definition

TcpClient uses connecor to initiate a connection. After the connection is established successfully, TcpConnection is created with socket to manage the connection. Each TcpClient class manages only one TcpConnection.

class TcpClient : noncopyable
{
 public:
  // TcpClient(EventLoop* loop);
  // TcpClient(EventLoop* loop, const string& host, uint16_t port);
  TcpClient(EventLoop* loop,
            const InetAddress& serverAddr,
            const string& nameArg);
  ~TcpClient();  // force out-line dtor, for std::unique_ptr members.

  void connect();
  void disconnect();
  void stop();

  TcpConnectionPtr connection() const
  {
    MutexLockGuard lock(mutex_);
    return connection_;
  }

  EventLoop* getLoop() const { return loop_; }
  bool retry() const { return retry_; }
  void enableRetry() { retry_ = true; }

  const string& name() const { return name_; }

  /// Set connection callback.
  /// Not thread safe.
  void setConnectionCallback(ConnectionCallback cb) { connectionCallback_ = std::move(cb); }

  /// Set message callback.
  /// Not thread safe.
  void setMessageCallback(MessageCallback cb) { messageCallback_ = std::move(cb); }

  /// Set write complete callback.
  /// Not thread safe.
  void setWriteCompleteCallback(WriteCompleteCallback cb) { writeCompleteCallback_ = std::move(cb); }

 private:
  /// Not thread safe, but in loop
  void newConnection(int sockfd);
  /// Not thread safe, but in loop
  void removeConnection(const TcpConnectionPtr& conn);

  EventLoop* loop_;			// The EvenetLoop to which it belongs
  ConnectorPtr connector_; 	// Use the Connector smart pointer to avoid the introduction of header files
  const string name_;		// Connection name
  
  ConnectionCallback connectionCallback_;		// Callback function to establish connection
  MessageCallback messageCallback_;				// Callback function for message arrival
  WriteCompleteCallback writeCompleteCallback_; // Callback function after sending data
  bool retry_;   // atomic 		//  Is the connection reconnected after disconnection
  bool connect_; // atomic
  // always in loop thread
  int nextConnId_;				// name_+nextConnId_  Used to identify a connection
  mutable MutexLock mutex_;
  TcpConnectionPtr connection_ GUARDED_BY(mutex_);
};

2. TcpClient implementation

When constructing TcpClient, initialize the Connector and register the event callback for communication with the server, and set the callback function for successful connection.

2.1 constructor

The initialization list creates a Connector to connect to the server. If the connection is successful, call the TcpClient::newConnection() callback function.

TcpClient::TcpClient(EventLoop* loop,
                     const InetAddress& serverAddr,
                     const string& nameArg)
  : loop_(CHECK_NOTNULL(loop)),
    connector_(new Connector(loop, serverAddr)),	// Create a Connector
    name_(nameArg),
    connectionCallback_(defaultConnectionCallback),
    messageCallback_(defaultMessageCallback),
    retry_(false), 		// Default no reconnection
    connect_(true), 	// Start connection
    nextConnId_(1) 		// Serial number of the current connection
{
  // Set the callback function to establish the connection
  connector_->setNewConnectionCallback(std::bind(&TcpClient::newConnection, this, _1));
  // FIXME setConnectFailedCallback
  LOG_INFO << "TcpClient::TcpClient[" << name_ << "] - connector " << get_pointer(connector_);
}

2.2. Establish connection and disconnect

The connect() function calls Connector::start() to initiate a connection.

void TcpClient::connect()
{
  // FIXME: check state
  LOG_INFO << "TcpClient::connect[" << name_ << "] - connecting to " << connector_->serverAddress().toIpPort();
  connect_ = true;
  connector_->start();  // Call Connector::start to initiate the connection
}

2.3. The connection is established successfully

If the connection is successful, call back the TcpClient::newConnection() function. The parameter is the sockfd of the local established connection, which creates a TcpConnection for subsequent message sending.

void TcpClient::newConnection(int sockfd)
{
  loop_->assertInLoopThread();
  InetAddress peerAddr(sockets::getPeerAddr(sockfd));    // Get the address of the opposite end
  char buf[32];
  snprintf(buf, sizeof buf, ":%s#%d", peerAddr.toIpPort().c_str(), nextConnId_);
  ++nextConnId_;
  string connName = name_ + buf;  // Connection name

  InetAddress localAddr(sockets::getLocalAddr(sockfd));  // Get the address of this section
  // FIXME poll with zero timeout to double confirm the new connection
  // FIXME use make_shared if necessary

  // Construct a TcpConnection object and set the corresponding callback function
  TcpConnectionPtr conn(new TcpConnection(loop_,
                                          connName,
                                          sockfd,
                                          localAddr,
                                          peerAddr));
  conn->setConnectionCallback(connectionCallback_);
  conn->setMessageCallback(messageCallback_);
  conn->setWriteCompleteCallback(writeCompleteCallback_);
  conn->setCloseCallback(std::bind(&TcpClient::removeConnection, this, _1)); // FIXME: unsafe
  {
    MutexLockGuard lock(mutex_);
    connection_ = conn;  // Save to member variable
  }
  conn->connectEstablished(); // Register with Poller and listen for IO events
}

2.4. disconnect(), close the connection stop()

Disconnect, only turn off the write function, and still receive peer messages.

void TcpClient::disconnect()
{
  connect_ = false;
  {
    MutexLockGuard lock(mutex_);
    if (connection_) {
      connection_->shutdown();  // Semi closed, can continue to receive complete messages from the opposite end
    }
  }
}

If the connection is closed, the client will be completely closed and data receiving and sending can no longer be performed.

void TcpClient::stop()
{
  connect_ = false;
  connector_->stop();
}

2.5. Deconstruction

TcpClient::~TcpClient()
{
  LOG_INFO << "TcpClient::~TcpClient[" << name_ << "] - connector " << get_pointer(connector_);
  TcpConnectionPtr conn;
  bool unique = false;
  {
    MutexLockGuard lock(mutex_);
    unique = connection_.unique();  // Is there only one holder
    conn = connection_;
  }
  if (conn)  // The connection has been established successfully. TcpConnectionPtr is not empty
  {
    assert(loop_ == conn->getLoop());
    // FIXME: not 100% safe, if we are in different thread
    CloseCallback cb = std::bind(&detail::removeConnection, loop_, _1);
    loop_->runInLoop(std::bind(&TcpConnection::setCloseCallback, conn, cb));
    if (unique){
      conn->forceClose();
    }
  }
  else{  // TcpConnectionPtr is null
    connector_->stop();   // Close Connector connection
    // FIXME: HACK
    loop_->runAfter(1, std::bind(&detail::removeConnector, connector_)); 
  }
}

3. Testing

EchoClient

#include <muduo/net/TcpClient.h>

#include <muduo/base/Logging.h>
#include <muduo/base/Thread.h>
#include <muduo/net/EventLoop.h>
#include <muduo/net/InetAddress.h>

#include <utility>

#include <stdio.h>
#include <unistd.h>

using namespace muduo;
using namespace muduo::net;

int numThreads = 0;
class EchoClient;
std::vector<std::unique_ptr<EchoClient>> clients;
int current = 0;

class EchoClient : noncopyable
{
 public:
  EchoClient(EventLoop* loop, const InetAddress& listenAddr, const string& id)
    : loop_(loop),
      client_(loop, listenAddr, "EchoClient"+id)
  {
    client_.setConnectionCallback(std::bind(&EchoClient::onConnection, this, _1));
    client_.setMessageCallback(std::bind(&EchoClient::onMessage, this, _1, _2, _3));
    //client_.enableRetry();
  }

  void connect()
  {
    client_.connect();
  }
  // void stop();

 private:
  void onConnection(const TcpConnectionPtr& conn)
  {
    LOG_TRACE << conn->localAddress().toIpPort() << " -> "
        << conn->peerAddress().toIpPort() << " is "
        << (conn->connected() ? "UP" : "DOWN");

    if (conn->connected()) {
      ++current;
      if (implicit_cast<size_t>(current) < clients.size()) {
        clients[current]->connect();
      }
      LOG_INFO << "*** connected " << current;
    }
    conn->send("world\n");
  }

  void onMessage(const TcpConnectionPtr& conn, Buffer* buf, Timestamp time)
  {
    string msg(buf->retrieveAllAsString());
    LOG_TRACE << conn->name() << " recv " << msg.size() << " bytes at " << time.toString();
    if (msg == "quit\n") {
      conn->send("bye\n");
      conn->shutdown();
    }
    else if (msg == "shutdown\n") {
      loop_->quit();
    }
    else {
      conn->send(msg);
    }
  }

  EventLoop* loop_;
  TcpClient client_;
};

int main(int argc, char* argv[])
{
  argc = 2;
  argv[1] = const_cast<char*>("localhost");

  LOG_INFO << "pid = " << getpid() << ", tid = " << CurrentThread::tid();
  if (argc > 1)
  {
    EventLoop loop;
    bool ipv6 = argc > 3;
    InetAddress serverAddr(argv[1], 2000, ipv6);

    int n = 1;
    if (argc > 2){
      n = atoi(argv[2]);
    }

    clients.reserve(n);
    for (int i = 0; i < n; ++i){
      char buf[32];
      snprintf(buf, sizeof buf, "%d", i+1);
      clients.emplace_back(new EchoClient(&loop, serverAddr, buf));
    }

    clients[current]->connect();
    loop.loop();
  }
  else{
    printf("Usage: %s host_ip [current#]\n", argv[0]);
  }
}

Topics: net muduo