In fact, all operations recorded by MySQL binlog have corresponding event types. For example, DML operations in STATEMENT format correspond to QUERY_EVENT type, and DML operations in ROW format correspond to ROWS_EVENT type.
QUERY_EVENT
QUERY_EVENT records the operation of a transaction in the form of text.
Events of type QUERY_EVENT are usually used in the following situations:
1. BEGIN operations performed at the beginning of a transaction.
2. DML operations in STATEMENT format
3. DDL operations in ROW format
For example:
mysql> show binlog events in 'mysql-bin.000021'; +------------------+-----+-------------+-----------+-------------+-----------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+-------------+-----------+-------------+-----------------------------------------------+ | mysql-bin.000021 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000021 | 120 | Query | 1 | 195 | BEGIN | | mysql-bin.000021 | 195 | Query | 1 | 298 | insert into test.t1 values(1,'a') | | mysql-bin.000021 | 298 | Xid | 1 | 329 | COMMIT /* xid=25 */ | | mysql-bin.000021 | 329 | Query | 1 | 408 | BEGIN | | mysql-bin.000021 | 408 | Query | 1 | 515 | use `test`; insert into test.t1 values(2,'b') | | mysql-bin.000021 | 515 | Xid | 1 | 546 | COMMIT /* xid=33 */ | +------------------+-----+-------------+-----------+-------------+-----------------------------------------------+
FORMAT_DESCRIPTION_EVENT
FormAT_DESCRIPTION_EVENT was introduced in binlog version 4 to replace the START_EVENT_V3 event in previous versions. It's the first event in the binlog file, and it only occurs once in the binlog. MySQL parses other events according to the definition of FORMAT_DESCRIPTION_EVENT.
It usually specifies the version of MySQL Server, the version of binlog, and the creation time of the binlog file.
For example:
# at 4 #160817 11:00:10 server id 1 end_log_pos 120 CRC32 0x03010da1 Start: binlog v 4, server v 5.6.31-log created 160817 11:00:10 # Warning: this binlog is either in use or was not closed properly.
mysql> show binlog events in 'mysql-bin.000021'; +------------------+-----+-------------+-----------+-------------+-----------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+-------------+-----------+-------------+-----------------------------------------------+ | mysql-bin.000021 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | ...
ROWS_EVENT
For binlog in ROW format, all DML statements are recorded in ROWS_EVENT.
ROWS_EVENT is divided into three types: WRITE_ROWS_EVENT, UPDATE_ROWS_EVENT and DELETE_ROWS_EVENT, which correspond to insert, update and delete operations respectively.
For insert operations, WRITE_ROWS_EVENT contains the data to be inserted
For update operations, UPDATE_ROWS_EVENT contains not only modified data, but also pre-modified values.
For delete operations, you only need to specify the deleted primary key (all columns are assigned in the absence of the primary key)
For QUERY_EVENT events, DML operations are recorded in text form. For ROWS_EVENT events, they are not text-based, so when viewing binlog based on ROW format through mysqlbinlog, you need to specify - vv --base64-output=decode-rows.
For example:
mysql> show binlog events in 'mysql-bin.000027'; +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | mysql-bin.000027 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000027 | 120 | Query | 1 | 188 | BEGIN | | mysql-bin.000027 | 188 | Table_map | 1 | 236 | table_id: 80 (test.t1) | | mysql-bin.000027 | 236 | Write_rows | 1 | 278 | table_id: 80 flags: STMT_END_F | | mysql-bin.000027 | 278 | Xid | 1 | 309 | COMMIT /* xid=198 */ | | mysql-bin.000027 | 309 | Query | 1 | 377 | BEGIN | | mysql-bin.000027 | 377 | Table_map | 1 | 425 | table_id: 80 (test.t1) | | mysql-bin.000027 | 425 | Update_rows | 1 | 475 | table_id: 80 flags: STMT_END_F | | mysql-bin.000027 | 475 | Xid | 1 | 506 | COMMIT /* xid=199 */ | | mysql-bin.000027 | 506 | Query | 1 | 574 | BEGIN | | mysql-bin.000027 | 574 | Table_map | 1 | 622 | table_id: 80 (test.t1) | | mysql-bin.000027 | 622 | Delete_rows | 1 | 664 | table_id: 80 flags: STMT_END_F | | mysql-bin.000027 | 664 | Xid | 1 | 695 | COMMIT /* xid=200 */ | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ rows in set (0.00 sec)
XID_EVENT
When a transaction is committed, either STATEMENT or binlog in ROW format, an XID_EVENT event is added at the end to represent the end of the transaction. This event records the ID of the transaction, and when MySQL crashes recovery, it decides whether to submit a transaction whose state is prepared in the storage engine based on the transaction's submission in the binlog.
ROTATE_EVENT
When the size of the binlog file reaches the value of max_binlog_size or when the flush logs command is executed, the binlog switch occurs, at which time a ROTATE_EVENT event is added to the current binlog log log to specify the name and location of the next log.
mysql> show binlog events in 'mysql-bin.000028'; +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | mysql-bin.000028 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000028 | 120 | Rotate | 1 | 167 | mysql-bin.000029;pos=4 | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ rows in set (0.00 sec)
# at 120 #160817 12:34:26 server id 1 end_log_pos 167 CRC32 0xd965567c Rotate to mysql-bin.000029 pos: 4
GTID_LOG_EVENT
When GTID mode is enabled, MySQL actually allocates a GTID for each transaction
For example:
# at 448 #160818 5:37:32 server id 1 end_log_pos 496 CRC32 0xaeb24aac GTID [commit=yes] SET @@SESSION.GTID_NEXT= 'cad449f2-5d4f-11e6-b353-000c29c64704:3'/*!*/; # at 496 #160818 5:37:32 server id 1 end_log_pos 571 CRC32 0x042ca092 Query thread_id=2 exec_time=0 error_code=0 SET TIMESTAMP=1471469852/*!*/; BEGIN /*!*/; # at 571 #160818 5:37:32 server id 1 end_log_pos 674 CRC32 0xa35beb37 Query thread_id=2 exec_time=0 error_code=0 SET TIMESTAMP=1471469852/*!*/; insert into test.t1 values(2,'b') /*!*/; # at 674 #160818 5:37:32 server id 1 end_log_pos 705 CRC32 0x1905d8c6 Xid = 12 COMMIT/*!*/;
mysql> show binlog events in 'mysql-bin.000033'; +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ | mysql-bin.000033 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000033 | 120 | Previous_gtids | 1 | 191 | cad449f2-5d4f-11e6-b353-000c29c64704:1 | | mysql-bin.000033 | 191 | Gtid | 1 | 239 | SET @@SESSION.GTID_NEXT= 'cad449f2-5d4f-11e6-b353-000c29c64704:2' | | mysql-bin.000033 | 239 | Query | 1 | 314 | BEGIN | | mysql-bin.000033 | 314 | Query | 1 | 417 | insert into test.t1 values(1,'a') | | mysql-bin.000033 | 417 | Xid | 1 | 448 | COMMIT /* xid=11 */ | | mysql-bin.000033 | 448 | Gtid | 1 | 496 | SET @@SESSION.GTID_NEXT= 'cad449f2-5d4f-11e6-b353-000c29c64704:3' | | mysql-bin.000033 | 496 | Query | 1 | 571 | BEGIN | | mysql-bin.000033 | 571 | Query | 1 | 674 | insert into test.t1 values(2,'b') | | mysql-bin.000033 | 674 | Xid | 1 | 705 | COMMIT /* xid=12 */ | | mysql-bin.000033 | 705 | Rotate | 1 | 752 | mysql-bin.000034;pos=4 | +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ rows in set (0.00 sec)
PREVIOUS_GTIDS_LOG_EVENT
When GTID mode is turned on, each binlog begins with a PREVIOUS_GTIDS_LOG_EVENT event whose value is PRVIOUS_GTIDS_LOG_EVENT+GTID_LOG_EVENT of the previous binlog. In fact, when the database restarts, it needs to refill the value of gtid_executed, which is PRVIOUS_GTIDS_LOG_EVENT of the latest binlog. + GTID_LOG_EVENT.
For example:
mysql> show binlog events in 'mysql-bin.000033'; +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ | mysql-bin.000033 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000033 | 120 | Previous_gtids | 1 | 191 | cad449f2-5d4f-11e6-b353-000c29c64704:1 | | mysql-bin.000033 | 191 | Gtid | 1 | 239 | SET @@SESSION.GTID_NEXT= 'cad449f2-5d4f-11e6-b353-000c29c64704:2' | | mysql-bin.000033 | 239 | Query | 1 | 314 | BEGIN | | mysql-bin.000033 | 314 | Query | 1 | 417 | insert into test.t1 values(1,'a') | | mysql-bin.000033 | 417 | Xid | 1 | 448 | COMMIT /* xid=11 */ | | mysql-bin.000033 | 448 | Gtid | 1 | 496 | SET @@SESSION.GTID_NEXT= 'cad449f2-5d4f-11e6-b353-000c29c64704:3' | | mysql-bin.000033 | 496 | Query | 1 | 571 | BEGIN | | mysql-bin.000033 | 571 | Query | 1 | 674 | insert into test.t1 values(2,'b') | | mysql-bin.000033 | 674 | Xid | 1 | 705 | COMMIT /* xid=12 */ | | mysql-bin.000033 | 705 | Rotate | 1 | 752 | mysql-bin.000034;pos=4 | +------------------+-----+----------------+-----------+-------------+-------------------------------------------------------------------+ rows in set (0.00 sec) mysql> show binlog events in 'mysql-bin.000034'; +------------------+-----+----------------+-----------+-------------+------------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+----------------+-----------+-------------+------------------------------------------+ | mysql-bin.000034 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000034 | 120 | Previous_gtids | 1 | 191 | cad449f2-5d4f-11e6-b353-000c29c64704:1-3 | +------------------+-----+----------------+-----------+-------------+------------------------------------------+ rows in set (0.00 sec)
Previous_gtids in the log of mysql-bin.000033 is cad449f2-5d4f-5d4f-11e6-b353-000c29c29c29c64704:1, GTID cad449f2-5d4f-11e6-b354-5d4f-11e6-b353-000c29c64704:2 and cad449f2-5d4f-5d4f-11e6-11e6-b353-000mysql-bin.000033 log Previous_gtids in the log of mysql-bin.000033 is cad449f2-5d4f2-5d4f-5d4f-5f-5d4f-5f-11d4f 353-000c29c64704:1-3.
# at 120 #160818 5:39:38 server id 1 end_log_pos 191 CRC32 0x4e84f3b5 Previous-GTIDs # cad449f2-5d4f-11e6-b353-000c29c64704:1-3
STOP_EVENT
When the MySQL database stops, a STOP_EVENT event is added at the end of the current binlog to indicate that the database stops.
For example:
mysql> show binlog events in 'mysql-bin.000030'; +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ | mysql-bin.000030 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.31-log, Binlog ver: 4 | | mysql-bin.000030 | 120 | Stop | 1 | 143 | | +------------------+-----+-------------+-----------+-------------+---------------------------------------+ rows in set (0.04 sec)
# at 120 #160818 5:18:04 server id 1 end_log_pos 143 CRC32 0xf20ddc85 Stop
For the execution of binlog in ROW format on slave side:
1.SQL processes read events from relaylog logs
2. For Table_map events, the SQL process extracts table information and saves the definition of the table.
3. Lock the table to be changed and check whether the structure on master and slave is consistent
4. If the table structure is inconsistent, stop copying, or continue executing until STMT_END_F finishes copying the event.
For Update_rows and Delete_rows events, the SQL process first locates the specific rows, and the search steps are as follows:
1. Primary Key Priority
Priority is given to the primary key on the slave. When a matching primary key value is found, it is assumed that a matching row has been found and the contents of the other columns of the row do not match.
2. Non-null unique index scan, only comparing key values
3. Other indexes or full table scans: Compare the consistency of the data of the whole row on master and slave (which can ensure the consistency of master-slave replication, but also the slowest).
Recently, the company built backup data from the database (data is inconsistent), resulting in a period of time after the operation of the slave library, error reporting, resulting in interruption of the SQL process:
Last_Errno: 1032 Last_Error: Could not execute Delete_rows event on table mdb.item_info; Can't find record in 'item_info', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.001430, end_log_pos 12893999