This series of articles translates the official version 8.0 document of ES. Corresponding content address of this chapter Slow Log

Due to the limited level, errors and omissions are inevitable in the translation content. You are welcome to point out

Click to jump to the catalog of this series

Slow log

Query slow log

The slow query log at fragment level (which can be divided into query node and acquisition stage) can be written to specific files, and the threshold can be set for different stages, as shown below

index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms

index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms

All configurations can be dynamically configured for each index, as follows

PUT /my-index-000001/_settings
{
  "index.search.slowlog.threshold.query.warn": "10s",
  "index.search.slowlog.threshold.query.info": "5s",
  "index.search.slowlog.threshold.query.debug": "2s",
  "index.search.slowlog.threshold.query.trace": "500ms",
  "index.search.slowlog.threshold.fetch.warn": "1s",
  "index.search.slowlog.threshold.fetch.info": "800ms",
  "index.search.slowlog.threshold.fetch.debug": "500ms",
  "index.search.slowlog.threshold.fetch.trace": "200ms"
}

The default value is - 1, which means it is not turned on.

The log is used at the slice level, that is, the log records the operation log of a specific slice, which is not all time-consuming for this query, because the request may be broadcast to multiple or all slices for query. The advantage of this is that the recorded time is not only related to the query conditions, but also related to the performance of a specific machine.

Find the reason for slow query

It is usually necessary and beneficial to determine what causes the slow query. If a request carries an x-opera-id request header, an ID field is not added to the recorded log, and the content is the request header, as shown below

{
  "type": "index_search_slowlog",
  "timestamp": "2030-08-30T11:59:37,786+02:00",
  "level": "WARN",
  "component": "i.s.s.query",
  "cluster.name": "distribution_run",
  "node.name": "node-0",
  "message": "[index6][0]",
  "took": "78.4micros",
  "took_millis": "0",
  "total_hits": "0 hits",
  "stats": "[]",
  "search_type": "QUERY_THEN_FETCH",
  "total_shards": "1",
  "source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",
  "id": "MY_USER_ID",
  "cluster.uuid": "Aq-c-PAeQiK3tfBYtig9Bw",
  "node.id": "D7fUYfnfTLa2D7y-xw6tZg"
}

Slow log of write operations

The slow log of write operation is similar to the slow log of query. The file name of log log is_ index_indexing_slowlog.json end. The threshold setting of logging is similar to the setting of query operation, as follows:

index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms
index.indexing.slowlog.source: 1000

All the above settings can be dynamically set by updating the API interface, as shown below

PUT /my-index-000001/_settings
{
  "index.indexing.slowlog.threshold.index.warn": "10s",
  "index.indexing.slowlog.threshold.index.info": "5s",
  "index.indexing.slowlog.threshold.index.debug": "2s",
  "index.indexing.slowlog.threshold.index.trace": "500ms",
  "index.indexing.slowlog.source": "1000"
}

By default, Elasticsearch records_ The first 1000 characters of source can be transferred to the slow log file through the configuration item index indexing. slowlog. Source to modify the field length. If it is set to false or 0, the item will not be recorded. If it is set to true, the complete data will be recorded_ The source data will be reformatted to display on one line. If you have to keep the original format, you can set index indexing. slowlog. Reformat is false. The log adopts log4j2 Properties configuration file.

Slow log level

You can turn off some long and useless logs by setting the log level. For example, we need index indexing. slowlog. Level = log above info level, then we can set index indexing. slowlog. threshold. index. Debug and index indexing. slowlog. threshold. index. Trace is - 1