Source code analysis of Sylar server framework log module
Overall design of log library module
It is used to format the output program log, which is convenient to locate the problems in the running process of the program from the log. In addition to the log content itself, the log here should also include additional information such as file name / line number, timestamp, thread / collaboration number, module name and log level. Even when printing a fatal log, the stack backtracking information of the program should be attached to facilitate analysis and troubleshooting.
In terms of design, a complete log module should have the following functions:
-
Distinguish different levels, such as DEBUG/INFO/WARN/ERROR. The log module can only output logs above a certain level by specifying the level, which can flexibly switch some unimportant log output. For example, the program can set a lower level in the debugging stage to see more scheduling log information. After the program is released, a higher level can be set to reduce log output and improve performance.
-
Distinguish between different output targets. Different logs can be output to different locations, such as standard output, file, syslog, log server on the network, and even the same log can be output to multiple output locations at the same time.
-
Distinguish between different categories. Logs can be classified and named. Each module of a program can output logs with different names, which makes it easy to judge which program module outputs the current log.
-
The log format can be configured flexibly. You can specify whether each log contains file name / line number, timestamp, thread / collaboration number, log level, startup time, etc. as needed.
-
The above functions can be configured through configuration files
First, the figure above
Key classes
sylar log library has 6 log levels: unknown, DEBUG, INFO, WARN, ERROR and FATAL
// log level class LogLevel; // Log format class LogFormatter; // Log output target class LogAppender; // Appender output to console class StdoutLogAppender : public LogAppender; // Appender output to file class FileLogAppender : public LogAppender; // Logger class Logger; // Log events class LogEvent; // Log event wrapper class LogEventWrap; // Logger manager class class LogManager;
Detailed design of the above categories:
- LogFormatter: log formatter, which can be used to format log events. The format method is used to format log events into strings;
- LogAppender: log output target, which is used to output log events to the corresponding output destination. This class includes LogFormatter formatting method and log method, which can be output to different output destinations by inheriting this class;
- Logger: logger, which is responsible for log output and provides log methods. If the log needs to be output, call LogAppender to output the log, otherwise discard it;
- LogEvent: log time, used to record log information, such as the level of the log, file name / line number, log message, thread / collaboration number, name of the logger, etc.;
- LogEventWrap: a log event wrapper that wraps the log event and the logger together, simplifies the use of the log module through macro definition, and calls the log method of the logger for output during the current deconstruction;
- LogManager: logger management class, single instance mode. It is used to manage all loggers in a unified way. It provides the creation and acquisition methods of loggers. It internally maintains a map of the name to the logger. When the obtained logger exists, it directly returns the corresponding logger pointer. Otherwise, it creates the corresponding logger and returns.
Description of the overall process and important functions of the log Library (the LogFormatter of the LogAppender is responsible for calling the LogFormatter to complete the final log output)
1. Initialize Logger, LogFormatter and LogAppender;
2. Generate LogEventWrap object through macro definition;
3. When the LogEventWrap object is destructed, call the log method to output the log.
Summary: manage loggers to write logs through Loggermanager: trigger logevents on different loggers through LogEventWrap destructor. Different loggers write logs to different loggerappenders.
The red line is the transfer process of formatter class in different classes
sylar::Logger::ptr logger(new sylar::Logger);
// console output
logger->addAppender(sylar::LogAppender::ptr(new sylar::StdoutLogAppender));
// File output, output to ". / log.txt"
sylar::FileLogAppender::ptr file_appender(new sylar::FileLogAppender("./log.txt"));
// Standardized format (custom)
sylar::LogFormatter::ptr fmt(new sylar::LogFormatter("%d%T%p%T%m%n"));
// Sets the standardized format for output to a file
file_appender->setFormatter(fmt);
// Sets the log level for output to a file
file_appender->setLevel(sylar::LogLevel::ERROR);
// Add file output destination
logger->addAppender(file_appender);
// Set log events
sylar::LogEvent::ptr event2(new sylar::LogEvent(logger,sylar::LogLevel::ERROR1,__FILE__, __LINE__, 0, 1, 2, time(0),"root"));
// Bind log events
sylar::LogEventWrap wrap(event2);
// Output hello
wrap.getEvent()->getSS() << "hello";
// output
logger->log(sylar::LogLevel::ERROR, wrap.getEvent());
wrap.getEvent()->format("test macro fmt error %c", 'c');
Line by line analysis of important codes
LogFormatter::init -- format parsing function (about 100 lines)
LogFormatter
Log formatter is used to format a log event and convert it into a string. Because a log event includes a lot of content (refer to the previous section, where each content is called an item), in fact, users do not want to output all these items every time they output the log, but want to be free to choose the items to be output. In addition, users may need to add some specified characters to each log, such as adding a colon between the file name and the line number. In order to realize this function, LogFormatter uses a template string to specify the formatting method. The template string is a string of characters like the following:
"%d{%Y-%m-%d %H:%M:%S} [%rms]%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n"
The template string consists of ordinary characters and escape characters. The escape characters start with%, such as% m,% p, etc. Except for escape characters, all the rest are ordinary characters, including spaces.
LogFormatter formats log events according to the template string. First, a string of template characters will be specified when constructing the LogFormatter object. LogFormatter will first parse the template string and parse the escape characters and ordinary characters. Then, when formatting the log event, replace the escape character with the specific content of the log event according to the template string, and the ordinary character remains unchanged.
For details, see https://github.com/zhongluqiang/sylar-from-scratch/blob/main/sylar/log.cpp
/**
* @brief Constructor
* @param[in] pattern format template
* @details
* %m news
* %p log level
* %r Cumulative milliseconds
* %c Log name
* %t Thread id
* %n Line feed
* %d time
* %f file name
* %l Line number
* %T Tab
* %F Synergetic id
* %N Thread name
*
* Default format '% d {% Y -% m -% d% H:% m:% s}% T% T% T% n% T% F% t [% P]% t [% C]% T% F:% l% T% m% n'
*/
result
2022-02-17 18:34:34 1 test 2 [DEBUG] [root] /home/knight/framework/tests/test.cc:7
-
%d{%Y-%m-%d %H:%M:%S}%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n
-
The first traversal result is str = "d" -- fmt = "% Y -% m -% d% H:% m:% s"
-
"T" "t"
-
nstr is used to parse [] and other similar characters -- corresponding to stringformattitem,
-
After being added respectively, the logger - > log function is responsible for calling one by one in order, and finally realizing log printing
General process
Traverse from the first letter
| Whether the i-th is% | ||
|---|---|---|
| yes | Is the i+1 letter | |
| No, join nstr | ||
| yes | Is the i+1 yes{ | |
| No, (and not {) exit the loop | ||
| Yes, enter the parsing state until} | ||
| No, exit the loop |
// Lines 12 ~ 19 of the source code are useless
/*if((i + 1) < m_pattern.size()) {
if(m_pattern[i + 1] == '%') { // Two%% cases will not be encountered and can be ignored
nstr.append(1, '%'); // Add a% at the end of the string
continue;
}
}*/
// %d{%Y-%m-%d %H:%M:%S}%T%t%T%N%T%F%T[%p]%T[%c]%T%f:%l%T%m%n
// Initialize and parse log template
void LogFormatter::init() {
//str, format, type
std::vector<std::tuple<std::string, std::string, int> > vec;
std::string nstr;
for(size_t i = 0; i < m_pattern.size(); ++i) { // Ergodic m_pattern string
if(m_pattern[i] != '%') {
nstr.append(1, m_pattern[i]); // Add an m at the end of the string_ pattern[i]
continue;
}
size_t n = i + 1; // Traverse from i + 1
int fmt_status = 0; // Is it in resolution state
size_t fmt_begin = 0; // Parsing start position
std::string str;
std::string fmt;
while(n < m_pattern.size()) { // Parse {}
if(!fmt_status && (!isalpha(m_pattern[n]) && m_pattern[n] != '{'
&& m_pattern[n] != '}')) {
str = m_pattern.substr(i + 1, n - i - 1);
break;
}
if(fmt_status == 0) {
if(m_pattern[n] == '{') {
str = m_pattern.substr(i + 1, n - i - 1); // Take one character
//std::cout << "*" << str << std::endl;
fmt_status = 1; //Analytical format
fmt_begin = n;
++n;
continue;
}
} else if(fmt_status == 1) {
if(m_pattern[n] == '}') {
fmt = m_pattern.substr(fmt_begin + 1, n - fmt_begin - 1); //Intercept {}
//std::cout << "#" << fmt << std::endl;
fmt_status = 0;
++n;
break;
}
}
++n;
if(n == m_pattern.size()) {
if(str.empty()) {
str = m_pattern.substr(i + 1);
}
}
}
if(fmt_status == 0) {
if(!nstr.empty()) { // Parse []
vec.push_back(std::make_tuple(nstr, std::string(), 0));
nstr.clear();
}
vec.push_back(std::make_tuple(str, fmt, 1));
i = n - 1; // i start from} and traverse again
} else if(fmt_status == 1) {
std::cout << "pattern parse error: " << m_pattern << " - " << m_pattern.substr(i) << std::endl;
m_error = true;
vec.push_back(std::make_tuple("<<pattern_error>>", fmt, 0));
}
}
if(!nstr.empty()) {
vec.push_back(std::make_tuple(nstr, "", 0));
}
static std::map<std::string, std::function<FormatItem::ptr(const std::string& str)> > s_format_items = {
#define XX(str, C) \
{#str, [](const std::string& fmt) { return FormatItem::ptr(new C(fmt));}}
XX(m, MessageFormatItem), //m: News
XX(p, LevelFormatItem), //p: Log level
XX(r, ElapseFormatItem), //r: Cumulative milliseconds
XX(c, NameFormatItem), //c: Log name
XX(t, ThreadIdFormatItem), //t: Thread id
XX(n, NewLineFormatItem), //n: Line feed
XX(d, DateTimeFormatItem), //d: Time
XX(f, FilenameFormatItem), //f: File name
XX(l, LineFormatItem), //l: Line number
XX(T, TabFormatItem), //T:Tab
XX(F, FiberIdFormatItem), //F: Synergetic id
XX(N, ThreadNameFormatItem), //N: Thread name
#undef XX
};
for(auto& i : vec) {
if(std::get<2>(i) == 0) {
m_items.push_back(FormatItem::ptr(new StringFormatItem(std::get<0>(i)))); // Put []
} else {
auto it = s_format_items.find(std::get<0>(i));
if(it == s_format_items.end()) {
m_items.push_back(FormatItem::ptr(new StringFormatItem("<<error_format %" + std::get<0>(i) + ">>")));
m_error = true;
std::cout << "error" << std::endl;
} else {
m_items.push_back(it->second(std::get<1>(i)));
}
}
// std::cout << "(" << std::get<0>(i) << ") - (" << std::get<1>(i) << ") - (" << std::get<2>(i) << ")" << std::endl;
}
// std::cout << m_items.size() << std::endl;
}