Source code analysis of Sylar server framework log module

Posted by vent on Thu, 17 Feb 2022 16:05:32 +0100

Source code analysis of Sylar server framework log module

Overall design of log library module

It is used to format the output program log, which is convenient to locate the problems in the running process of the program from the log. In addition to the log content itself, the log here should also include additional information such as file name / line number, timestamp, thread / collaboration number, module name and log level. Even when printing a fatal log, the stack backtracking information of the program should be attached to facilitate analysis and troubleshooting.

In terms of design, a complete log module should have the following functions:

  1. Distinguish different levels, such as DEBUG/INFO/WARN/ERROR. The log module can only output logs above a certain level by specifying the level, which can flexibly switch some unimportant log output. For example, the program can set a lower level in the debugging stage to see more scheduling log information. After the program is released, a higher level can be set to reduce log output and improve performance.

  2. Distinguish between different output targets. Different logs can be output to different locations, such as standard output, file, syslog, log server on the network, and even the same log can be output to multiple output locations at the same time.

  3. Distinguish between different categories. Logs can be classified and named. Each module of a program can output logs with different names, which makes it easy to judge which program module outputs the current log.

  4. The log format can be configured flexibly. You can specify whether each log contains file name / line number, timestamp, thread / collaboration number, log level, startup time, etc. as needed.

  5. The above functions can be configured through configuration files

First, the figure above

Key classes

sylar log library has 6 log levels: unknown, DEBUG, INFO, WARN, ERROR and FATAL

// log level
class LogLevel;
// Log format
class LogFormatter;
// Log output target
class LogAppender;
// Appender output to console
class StdoutLogAppender : public LogAppender;
// Appender output to file
class FileLogAppender : public LogAppender;
// Logger
class Logger;
// Log events
class LogEvent;
// Log event wrapper
class LogEventWrap;
// Logger manager class
class LogManager;

Detailed design of the above categories:

  • LogFormatter: log formatter, which can be used to format log events. The format method is used to format log events into strings;
  • LogAppender: log output target, which is used to output log events to the corresponding output destination. This class includes LogFormatter formatting method and log method, which can be output to different output destinations by inheriting this class;
  • Logger: logger, which is responsible for log output and provides log methods. If the log needs to be output, call LogAppender to output the log, otherwise discard it;
  • LogEvent: log time, used to record log information, such as the level of the log, file name / line number, log message, thread / collaboration number, name of the logger, etc.;
  • LogEventWrap: a log event wrapper that wraps the log event and the logger together, simplifies the use of the log module through macro definition, and calls the log method of the logger for output during the current deconstruction;
  • LogManager: logger management class, single instance mode. It is used to manage all loggers in a unified way. It provides the creation and acquisition methods of loggers. It internally maintains a map of the name to the logger. When the obtained logger exists, it directly returns the corresponding logger pointer. Otherwise, it creates the corresponding logger and returns.

Description of the overall process and important functions of the log Library (the LogFormatter of the LogAppender is responsible for calling the LogFormatter to complete the final log output)

1. Initialize Logger, LogFormatter and LogAppender;
2. Generate LogEventWrap object through macro definition;
3. When the LogEventWrap object is destructed, call the log method to output the log.
Summary: manage loggers to write logs through Loggermanager: trigger logevents on different loggers through LogEventWrap destructor. Different loggers write logs to different loggerappenders.

The red line is the transfer process of formatter class in different classes

sylar::Logger::ptr logger(new sylar::Logger);
// console output 
logger->addAppender(sylar::LogAppender::ptr(new sylar::StdoutLogAppender));
// File output, output to ". / log.txt"
sylar::FileLogAppender::ptr file_appender(new sylar::FileLogAppender("./log.txt"));
// Standardized format (custom)
sylar::LogFormatter::ptr fmt(new sylar::LogFormatter("%d%T%p%T%m%n"));
// Sets the standardized format for output to a file
// Sets the log level for output to a file
// Add file output destination
// Set log events
sylar::LogEvent::ptr event2(new sylar::LogEvent(logger,sylar::LogLevel::ERROR1,__FILE__, __LINE__, 0, 1, 2, time(0),"root"));
// Bind log events
sylar::LogEventWrap wrap(event2);
// Output hello
wrap.getEvent()->getSS() << "hello";
// output
logger->log(sylar::LogLevel::ERROR, wrap.getEvent());
wrap.getEvent()->format("test macro fmt error %c", 'c');

Line by line analysis of important codes

LogFormatter::init -- format parsing function (about 100 lines)


Log formatter is used to format a log event and convert it into a string. Because a log event includes a lot of content (refer to the previous section, where each content is called an item), in fact, users do not want to output all these items every time they output the log, but want to be free to choose the items to be output. In addition, users may need to add some specified characters to each log, such as adding a colon between the file name and the line number. In order to realize this function, LogFormatter uses a template string to specify the formatting method. The template string is a string of characters like the following:

"%d{%Y-%m-%d %H:%M:%S} [%rms]%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n"

The template string consists of ordinary characters and escape characters. The escape characters start with%, such as% m,% p, etc. Except for escape characters, all the rest are ordinary characters, including spaces.

LogFormatter formats log events according to the template string. First, a string of template characters will be specified when constructing the LogFormatter object. LogFormatter will first parse the template string and parse the escape characters and ordinary characters. Then, when formatting the log event, replace the escape character with the specific content of the log event according to the template string, and the ordinary character remains unchanged.

For details, see

     * @brief Constructor
     * @param[in] pattern format template
     * @details 
     *  %m news
     *  %p log level
     *  %r Cumulative milliseconds
     *  %c Log name
     *  %t Thread id
     *  %n Line feed
     *  %d time
     *  %f file name
     *  %l Line number
     *  %T Tab
     *  %F Synergetic id
     *  %N Thread name
     *  Default format '% d {% Y -% m -% d% H:% m:% s}% T% T% T% n% T% F% t [% P]% t [% C]% T% F:% l% T% m% n'
2022-02-17 18:34:34     1       test    2       [DEBUG] [root]  /home/knight/framework/tests/ 
  • %d{%Y-%m-%d %H:%M:%S}%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n

  • The first traversal result is str = "d" -- fmt = "% Y -% m -% d% H:% m:% s"

  • "T" "t"

  • nstr is used to parse [] and other similar characters -- corresponding to stringformattitem,

  • After being added respectively, the logger - > log function is responsible for calling one by one in order, and finally realizing log printing

General process

Traverse from the first letter

Whether the i-th is%
yesIs the i+1 letter
No, join nstr
yesIs the i+1 yes{
No, (and not {) exit the loop
Yes, enter the parsing state until}
No, exit the loop
       // Lines 12 ~ 19 of the source code are useless
		/*if((i + 1) < m_pattern.size()) {
            if(m_pattern[i + 1] == '%') { // Two%% cases will not be encountered and can be ignored
                nstr.append(1, '%'); // Add a% at the end of the string
// %d{%Y-%m-%d %H:%M:%S}%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n
// Initialize and parse log template
void LogFormatter::init() {
    //str, format, type
    std::vector<std::tuple<std::string, std::string, int> > vec;
    std::string nstr;
    for(size_t i = 0; i < m_pattern.size(); ++i) { // Ergodic m_pattern string
        if(m_pattern[i] != '%') {
            nstr.append(1, m_pattern[i]); // Add an m at the end of the string_ pattern[i]

        size_t n = i + 1; // Traverse from i + 1
        int fmt_status = 0; // Is it in resolution state
        size_t fmt_begin = 0; // Parsing start position

        std::string str;
        std::string fmt;
        while(n < m_pattern.size()) { // Parse {}
            if(!fmt_status && (!isalpha(m_pattern[n]) && m_pattern[n] != '{'
                    && m_pattern[n] != '}')) {
                str = m_pattern.substr(i + 1, n - i - 1);
            if(fmt_status == 0) {
                if(m_pattern[n] == '{') {
                    str = m_pattern.substr(i + 1, n - i - 1); // Take one character
                    //std::cout << "*" << str << std::endl;
                    fmt_status = 1; //Analytical format
                    fmt_begin = n;
            } else if(fmt_status == 1) {
                if(m_pattern[n] == '}') {
                    fmt = m_pattern.substr(fmt_begin + 1, n - fmt_begin - 1); //Intercept {}
                    //std::cout << "#" << fmt << std::endl;
                    fmt_status = 0;
            if(n == m_pattern.size()) {
                if(str.empty()) {
                    str = m_pattern.substr(i + 1);

        if(fmt_status == 0) {
            if(!nstr.empty()) { // Parse []
                vec.push_back(std::make_tuple(nstr, std::string(), 0));
            vec.push_back(std::make_tuple(str, fmt, 1));
            i = n - 1; // i start from} and traverse again
        } else if(fmt_status == 1) {
            std::cout << "pattern parse error: " << m_pattern << " - " << m_pattern.substr(i) << std::endl;
            m_error = true;
            vec.push_back(std::make_tuple("<<pattern_error>>", fmt, 0));

    if(!nstr.empty()) {
        vec.push_back(std::make_tuple(nstr, "", 0));
    static std::map<std::string, std::function<FormatItem::ptr(const std::string& str)> > s_format_items = {
#define XX(str, C) \
        {#str, [](const std::string& fmt) { return FormatItem::ptr(new C(fmt));}}

        XX(m, MessageFormatItem),           //m: News
        XX(p, LevelFormatItem),             //p: Log level
        XX(r, ElapseFormatItem),            //r: Cumulative milliseconds
        XX(c, NameFormatItem),              //c: Log name
        XX(t, ThreadIdFormatItem),          //t: Thread id
        XX(n, NewLineFormatItem),           //n: Line feed
        XX(d, DateTimeFormatItem),          //d: Time
        XX(f, FilenameFormatItem),          //f: File name
        XX(l, LineFormatItem),              //l: Line number
        XX(T, TabFormatItem),               //T:Tab
        XX(F, FiberIdFormatItem),           //F: Synergetic id
        XX(N, ThreadNameFormatItem),        //N: Thread name
#undef XX

    for(auto& i : vec) {
        if(std::get<2>(i) == 0) {
            m_items.push_back(FormatItem::ptr(new StringFormatItem(std::get<0>(i)))); // Put []
        } else {
            auto it = s_format_items.find(std::get<0>(i));
            if(it == s_format_items.end()) {
                m_items.push_back(FormatItem::ptr(new StringFormatItem("<<error_format %" + std::get<0>(i) + ">>")));
                m_error = true;
                std::cout << "error" << std::endl;
            } else {

        // std::cout << "(" << std::get<0>(i) << ") - (" << std::get<1>(i) << ") - (" << std::get<2>(i) << ")" << std::endl;
    // std::cout << m_items.size() << std::endl;

Topics: Operation & Maintenance server