MySQL Redo Log & Two-Phase Commit Guide
MySQL Redo Log & Two-Phase Commit Guide
○ Mixed Format: MySQL will be logging binlogs in STATEMENT mode , however, for statements with UUID() or
now() which is unsafe for data consistency , the bin logging mode will change to ROW mode.
● Binlogs and Replication
Threads involved in replication - 3 of those [ 2 in Source, 1 in Replica] Dump Thread IO Thread & SQL Thread (DIS):
○ Source : Dump thread - reads binlog , decodes events and sends it across to replication, Dump Thread:
■ Is created by Source upon a replica connecting to Source.
■ Knows the last log entry being applied to repilca from Replica.
■ Reads the binlog and sends events following the last log applied to replica.
○ Replica : IO thread (or Receiver thread)
■ Connects to primary & receives events from the binlog sent by the Dump thread.
■ The received events are written to a log file RELAY LOG in the Replica instance byte by byte.
○ Replica : SQL thread
■ Reads the RELAY LOGs and applies those byte-by-byte to the data in replica and thus keeps the
REPLICA in sync with the PRIMARY.
● REDO LOG - Physiological logging
○ Physiological = Physical + logical
○ Physical: The entries in the REDO LOG operate at page level.
○ Logical: Specifies the changes to be made in the exact location in the page. The before and after image of
the page is not recorded [ which is physical logging]
○ Advantage of Physiological Logging :
■ Logical logging [no full page image ]: space efficiency.
■ Physical Logging: Safe recovery [avoid data quality for say auto_increment , UUID generation etc..]
○ Challenge with Physiological Logging:
■ Database crash midway during page write (dirty page flush) results in fractured or torn database
pages. Applying physiological log entries which picks a page and updates bytes at an offset will
result in data corruption for the page if it was flushed from memory to disk partially due a MySQL
Server crash midway as the page was being flushed. The page is called “Fractured Page” or a “Torn
Page”.
■ Addressing the Challenge
● This is addressed by DoubleWrite Buffer. Every page changed in the MySQL database is
written by InnoDB to a contiguous,sequential area in the disk “DoubleWrite Buffer” and then
issued an FSYNC. Only after that, the corresponding dirty buffers are flush to disk.
○ Crash Recovery & Idempotency of Redo log application
■ Log Sequence Number: 64 bit ever increasing number generated for every entry is made to redo
log. LSN is recorded in:
● Log entry in the Redo log
● Headers of the changed pages in memory , which are eventually flushed to disk.
■ During crash recovery , redo log is scanned , each redo log entry will have among others:
● Page Id (File ID , Page ID) - Page Id will lead MySQL to find the page in memory.
● Log Sequence Number - Will let MySQL do the following:
○ LSN of Page >= LSN of Redo log entry -> skip applying
○ LSN of Page < LSN of Redo log -> apply the log entry.
● Hence , if a redo log is scanned and re-applied, the changes will be idempotent because the
entries in the Redo log are written in sequence.
Idempotency Achieved via Log Sequence Achieved by executing events in order; GTIDs
Number (LSN) comparison prevent re-execution
on pages 2
Primary Use Automatic recovery after an Propagating data changes to replica servers 12
Case unexpected server
shutdown 7
SECTION 2
The Two-Phase Commit: Orchestrating Consistency
The Two-Phase Commit: Orchestrating InConsistency:
Primary have the change vectors Change vector not applied to Replica Commit to Redo log,Crash,No commit
applied to Bin log
Primary do not have the change Change vector applied to Replica Commit to Bin Log,Crash,No commit to
vectors applied Redo log
2.1 The Necessity of Co-ordination: The “Split Brain" problem
- MySQL inter XA transaction [extended] architecture transaction - spans both Storage Engine and binary log.
- XA Transaction architecture :
- Binary Log subsystem = transaction co-ordinator
- Storage Engine (which contains redo log) -> participant in the co-ordination
- Storage Engine takes direction from Binary Log Subsystem
Step Action/Function Call Redo Log State Binary Log State Durability Point
1 Client issues COMMIT Transaction changes are No entry for this In-memory
in the in-memory redo transaction. only.
log buffer.
3 Phase 1: Redo log records All transaction changes No entry for this DURABLE in
written and fsync()'d and a "prepare" record transaction. Redo Log.
are written to the redo
log file.
->End of Phase 1 Snapshot state (a) Redo log - events persisted followed by “prepare” record (b) Binary Log - NA.
SECTION 3
1. InnoDB : Redo Log Scan : InnoDB Engine scan from last known checkpoint and apply redo log:roll forward
2. InnoDB : “In Doubt” Transaction identification - XID with ‘PREPARE’ but no corresponding ‘COMMIT’ in redo log.
Acquired lock for these doubt transactions.
3. Server :The thread for Binary Log (arbitrarer)
a. Co-ordinator Led Recovery : Latest binary log file - improperly closed - binlog-based recovery.
b. Cross-referencing : XID list from binary log before crash compared to in-doubt trx list from 2.
c. Final Decision
i. XID of in-doubt transaction in Binary Log - hence , co-ordinator instructs InnoDB to commit the
prepared transaction through innobase_commit.
ii. XID of in-doubt transaction not in Binary log - co-ordinator instructs InnoDB to rollback the
in-doubt/prepared transaction.
SECTION 4
Here is a detailed report on their relationship, the contents of binlogs, and how they are used for replication.
● Redo Log (InnoDB): This is a physical log that belongs to the storage engine layer (specifically InnoDB). Its
primary purpose is to provide crash safety and durability. It records low-level physical changes to data pages
(e.g., "write these bytes at this specific offset in this data file"). This allows InnoDB to recover its state after a server
crash by replaying the changes that hadn't yet been flushed from memory to the main data files on disk.
● Binary Log (Binlog): This is a logical log that belongs to the MySQL server layer (the SQL layer). Its primary
purpose is replication and point-in-time recovery. It records events that change data, such as SQL statements
or row changes.
1. Prepare Phase: When you issue a COMMIT, InnoDB first writes the transaction changes to the redo log in memory
and marks it as "prepared". It then notifies the MySQL server that it is ready.
2. Commit Phase:
○ The MySQL server, upon receiving the "prepared" signal, writes the transaction to the binary log.
○ Once the transaction is successfully written to the binlog on disk, the server tells InnoDB to finalize the
commit.
○ InnoDB then marks the transaction in the redo log as "committed".
This coordination is critical. If the server crashes after the "prepare" step but before the binlog is written, the transaction is
considered uncommitted and will be rolled back during recovery. This prevents a scenario where a change is applied to
the source database but never sent to the replica, which would cause them to become inconsistent.
1. STATEMENT-based Format
This was the original format. It logs the exact SQL statements that modify data.
● Content: An event contains the INSERT, UPDATE, or DELETE statement that was executed.
● Pros: Very compact. A single statement that updates millions of rows is logged as just that one statement, saving
significant disk space and network bandwidth.
● Cons: Not always safe. Statements using non-deterministic functions like UUID() or NOW() can produce
different results on the replica than on the source, leading to data inconsistency.
2. ROW-based Format
This is the default format in modern MySQL versions. It logs the changes to individual rows.
● Content: For each row that is changed, the binlog records an event containing its "before-image" (the values
before the change) and its "after-image" (the values after the change). It logs the effect of the statement, not the
statement itself.
● Pros: Extremely reliable and safe. Since it logs the exact row changes, replication is deterministic and
guaranteed to produce the same result on the replica.
● Cons: Can be very verbose. An UPDATE that affects 1 million rows will generate 1 million row-change events,
potentially consuming a large amount of disk space and network bandwidth.
3. MIXED Format
● Content: It uses statement-based logging by default for efficiency. However, it automatically switches to
row-based logging for any statement that it identifies as unsafe for statement-based replication (e.g., one using
UUID()).
● Pros: A good balance between performance and data safety.
● Cons: The behavior can be less predictable since you don't always know which format will be used for a given
statement.
========================================================================
A critical prerequisite for physiological logging is the guarantee that the underlying page writes to the data files are
themselves atomic. A crash during a 16 KB page write could result in a "torn page," where only part of the page is
updated on disk. Applying a physiological redo log record to a torn page would result in data corruption. To prevent this,
InnoDB uses a doublewrite buffer. Before writing a dirty page to its final location, InnoDB first writes it to a contiguous,
sequential area on disk called the doublewrite buffer and issues an fsync. Only after this is successful does it write the
page to the actual data file. In the event of a crash during the data file write, InnoDB can recover the pristine, correct
version of the page from the doublewrite buffer, ensuring that redo log replay always operates on a consistent page
image.2
To manage the state of pages and log records, InnoDB uses a globally increasing 64-bit integer called the Log
Sequence Number (LSN).2 Every byte written to the redo log increments the LSN. When a page in the buffer pool is
modified, it is stamped with the LSN of the corresponding redo log record. This LSN is also persisted as part of the
page's header when it is eventually flushed to disk.2
This mechanism is central to making crash recovery idempotent, meaning that re-applying a log record multiple times
will not cause further changes. During recovery, InnoDB scans the redo log starting from the LSN of the last successful
checkpoint.3 For each log record it encounters, it examines the corresponding data page on disk. If the LSN on the page
is greater than or equal to the LSN of the log record, it means the change has already been successfully flushed to disk
at some point before the crash. In this case, the log record is skipped. If the page's LSN is lower, the change is
re-applied.2 This simple comparison ensures that each modification is applied exactly once, bringing the database to a
consistent state.
Physical Structure
Historically, the redo log was implemented as a set of fixed-size, circular files, named ib_logfile0 and ib_logfile1 by
default.9 InnoDB would write to these files in a round-robin fashion. When the write position caught up to the checkpoint
position (the LSN up to which all changes have been flushed to data files), the system would have to stall writes while it
aggressively flushed dirty pages to advance the checkpoint and free up space in the redo log.4 Resizing these files was
a cumbersome offline operation.1
Starting with MySQL 8.0.30, this architecture was overhauled to be more dynamic and manageable. The redo log now
resides in a dedicated #innodb_redo directory and is managed by a single capacity setting, innodb_redo_log_capacity.1
InnoDB attempts to maintain approximately 32 individual redo log files within this directory, with each file's size being
roughly 1/32 of the total capacity. This new design allows for dynamic resizing of the total redo log capacity without a
server restart and simplifies space management by treating the log space more like a queue where older, fully
checkpointed files can be purged or reused.1
1.2 The MySQL Binary Log: The Server's Chronicle for Replication and Recovery
While the redo log ensures durability for a single InnoDB instance, the MySQL binary log (binlog) serves a completely
different purpose. It is a server-level log that provides a canonical, logical record of data changes, primarily for two
critical functions: replication and point-in-time recovery (PITR).12 Unlike the redo log, it is not specific to any single
storage engine; any data modification, whether to an InnoDB, MyISAM, or other table type, is recorded in the binary log
if it is enabled.4
Structure and Content
The binary log is not a single file but a series of numbered files (e.g., mysql-bin.000001, mysql-bin.000002) and an
index file that tracks all the log files in the set.10 The server writes events to the current log file sequentially. A new file is
created when the server starts, when the logs are flushed manually, or when the current file reaches the size defined by
max_binlog_size.12 Each binary log file contains a stream of "events," where each event describes a single database
modification. This could be a DDL statement (
CREATE TABLE), or a DML transaction, which is typically bracketed by a BEGIN (or GTID) event and a COMMIT (or XID)
event.12
The format of the events within the binary log is governed by the binlog_format system variable, and the choice of
format has profound consequences for the reliability and efficiency of replication.16
● Statement-Based Replication (SBR): The original format, SBR logs the exact SQL statements that produced the
data change (e.g., UPDATE my_table SET col = col + 1 WHERE id = 5;).17
○ Advantages: This format is extremely space-efficient, as a single statement that updates millions of rows is
logged as just that one statement. The logs are also human-readable and can serve as an audit trail of
executed queries.19
○ Disadvantages: SBR's critical flaw is its inability to reliably replicate non-deterministic statements. A query using
UUID() or NOW() will generate different values on the source and replica, leading to data inconsistency.
Similarly, an UPDATE... LIMIT 1 without an ORDER BY on a unique key could affect different rows on the source
and replica if the physical row order differs. Because of these safety issues, SBR is rarely recommended for
modern systems.17
● Row-Based Replication (RBR): In this format, the binary log records the actual changes to individual rows. For an
UPDATE, it logs the "before image" (the values of the row before the change) and the "after image" (the values
after the change) for the primary key and modified columns. For an INSERT, it logs the new row; for a DELETE, it
logs the row that was removed.17 RBR has been the default format since MySQL [Link]
○ Advantages: RBR is the safest and most reliable form of replication. It guarantees that the exact same rows are
changed on the replica, regardless of non-deterministic functions or query execution plans. It is a physical
representation of the logical change, ensuring data consistency.19
○ Disadvantages: The primary drawback is log size. A single UPDATE statement that modifies one million rows will
generate one million row-change events in the binary log, potentially consuming a vast amount of disk space
and network bandwidth for replication. The logs are also not easily human-readable without tools like
mysqlbinlog with verbose decoding options.16
● Mixed Logging: This format attempts to offer the best of both worlds. It defaults to the more efficient
statement-based logging but intelligently and automatically switches to row-based logging for any statement that it
identifies as unsafe for SBR, such as those involving non-deterministic functions.17 While it provides a good
balance, the deterministic safety of pure RBR has made it the preferred default for most use cases.
The divergent designs of the redo log and the binary log are a direct consequence of their distinct purposes. The redo
log is an internal, high-performance mechanism for instance-level crash recovery. Its physiological format is optimized
for speed and for the specific needs of the InnoDB storage engine. It is not designed to be portable or understood by
any other component. Conversely, the binary log is a server-wide, portable, logical record designed for interoperability.
Its primary function is to serve as a universally understandable stream of changes for replication and recovery tools.
MySQL cannot simply use the redo log for replication because its content—low-level page modification vectors—would
be meaningless to a replica, which may have a different physical data layout or even be using a different storage engine
entirely. This fundamental division of responsibility necessitates two separate, highly specialized logging systems, which
in turn creates the need for a protocol to ensure they remain perfectly synchronized.
Primary Purpose Instance crash recovery, ensuring durability (ACID) Replication, point-in-time recovery
5
(PITR) 12
Log Format Physiological (physical page address + logical Logical (Statement, Row, or Mixed) 17
change) 3
Content Low-level changes to data pages (byte-level diffs) High-level data modification events
3
(SQL statements or row images) 12
Idempotency Achieved via Log Sequence Number (LSN) Achieved by executing events in
comparison on pages 2 order; GTIDs prevent re-execution
Primary Use Case Automatic recovery after an unexpected server Propagating data changes to replica
shutdown 7 servers 12
With two independent logging systems—one for engine durability and one for server-level history—a critical challenge
arises: how to guarantee that a transaction is committed to both logs or neither log, even in the event of a server crash.
A failure to ensure this atomicity would lead to a catastrophic "split-brain" scenario, irrevocably corrupting the
replication topology. MySQL solves this problem by implementing an internal two-phase commit (2PC) protocol, a
distributed transaction algorithm that coordinates the commit process between the storage engine (the participant)
and the binary log (the transaction coordinator).
The danger of unsynchronized commits can be illustrated with two failure scenarios:
● Scenario A: Commit to Redo Log, Crash, No Commit to Binlog. In this case, a transaction's changes are made
durable within the InnoDB redo log. If the server crashes at this instant, upon restart, InnoDB's crash recovery will
roll forward and commit this transaction, making its changes permanent and visible on the source server. However,
because the transaction never made it to the binary log, it will never be sent to the replicas. This results in a state of
permanent inconsistency where the source contains data that is missing from all its replicas.
● Scenario B: Commit to Binlog, Crash, No Commit to Redo Log. Here, the transaction is successfully written to
the binary log and is therefore sent to all replicas, which will apply the change. However, if the server crashes
before the commit is made durable in the InnoDB redo log, InnoDB's crash recovery will see an uncommitted
transaction and roll it back. This leads to the opposite inconsistency: the replicas have committed a transaction that
has been undone on the source.
Both scenarios violate the fundamental promise of replication and are unacceptable. The two-phase commit protocol,
implemented within MySQL as an internal XA (eXtended Architecture) transaction, prevents these outcomes by creating
a single, atomic operation that spans both the storage engine and the binary log.23 In this model, the binary log
subsystem acts as the transaction coordinator, directing the storage engines, which act as participants.23
When a client connected to a MySQL server with binary logging enabled issues a COMMIT statement, it does not trigger
a simple, single-step commit. Instead, it initiates a carefully orchestrated, two-phase sequence.
The first phase is about getting all participating storage engines to a state where they can guarantee, or promise, that
the transaction can be committed successfully, no matter what happens next.
1. Coordinator Request: The MySQL server's SQL layer, upon receiving the COMMIT, passes control to the
transaction coordinator, which is the binary log module (MYSQL_BIN_LOG). The coordinator initiates the prepare
phase by calling MYSQL_BIN_LOG::prepare.23
2. Engine Preparation: The prepare call is propagated down to all storage engines that participated in the
transaction. For InnoDB, this ultimately invokes the innobase_xa_prepare function.23
3. Redo Log Action: Inside innobase_xa_prepare, InnoDB performs the most critical action of this phase. It takes all
the redo log records generated by the transaction, which have been buffered in memory, and writes them to the
redo log files on disk. It then appends a special "prepare" record to the redo log, which includes the transaction's
unique XA transaction identifier (XID).23
4. Durability Guarantee: To make this promise durable, InnoDB then executes a system call like fsync() on the redo
log file. The strictness of this is controlled by the innodb_flush_log_at_trx_commit variable. For full ACID compliance
and to make the 2PC protocol safe, this variable must be set to 1, which forces a flush and sync to disk for every
transaction commit.3 Once this
fsync() completes, the transaction is in a persistent "prepared" or "in-doubt" state. From InnoDB's perspective, the
transaction is no longer ephemeral; it is durably recorded and can be either committed or rolled back in the future,
even after a crash. However, the transaction's locks are still held, and its changes are not yet visible to other
sessions.23
5. State Snapshot at the end of Phase 1:
○ Redo Log: Contains all change records for the transaction, plus a "prepare" record. This state is durable on
disk.
○ Binary Log: The transaction has not yet been written to the binary log.
Phase 2: The 'Commit' Phase (The Coordinator's Decision)
Only after all participating engines have successfully returned from the prepare phase does the coordinator proceed to
the second phase, where the final, irrevocable decision to commit is made and recorded.
1. Write to Binary Log: The transaction coordinator, via MYSQL_BIN_LOG::commit, writes all the events that
constitute the transaction into the binary log file on disk.23 This is the point of no return. Once the transaction is in
the binary log, it is considered committed from the perspective of the entire replication topology.
2. Binary Log Durability: The durability of this write is controlled by the sync_binlog system variable. If
sync_binlog=1, the server will perform an fsync() on the binary log file after every transaction is written.8 This
ensures that the transaction is physically on disk before the commit operation returns success to the client,
providing the highest level of safety but at a significant performance cost, as it introduces a second
fsync() into the commit path (the first being for the redo log prepare). If sync_binlog is set to a value greater than 1,
the fsync() will only occur after that many transactions have been written, trading some durability for higher
throughput.
3. Engine Commit: After the transaction is durably recorded in the binary log, the coordinator instructs the storage
engines to finalize the commit. For InnoDB, this invokes the innobase_commit function.23
4. Final Redo Log Action: Compared to the prepare step, innobase_commit is a very fast, lightweight operation. It
simply writes a final "commit" record to the redo log for the corresponding XID. This action signifies that the
transaction is complete, and at this point, InnoDB releases all locks held by the transaction, making its changes
visible to other sessions.27 There is typically no immediate
fsync() required for this final commit record, as the durability of the transaction was already guaranteed by the
fsync() in the prepare phase.
5. State Snapshot at the end of Phase 2:
○ Redo Log: Contains a final "commit" record for the transaction.
○ Binary Log: Contains all events for the transaction. The transaction is now fully committed, visible, and
guaranteed to be replicated.
Step Action/Function Call Redo Log State Binary Log State Durability Point
1 Client issues COMMIT Transaction changes are No entry for this In-memory
in the in-memory redo transaction. only.
log buffer.
3 Phase 1: Redo log records All transaction changes No entry for this DURABLE in
written and fsync()'d and a "prepare" record transaction. Redo Log.
are written to the redo
log file.
The true value of the two-phase commit protocol is realized not during normal operation, but in its ability to provide a
clear and deterministic path to a consistent state after an unexpected server crash. The protocol's design ensures that
even if a failure occurs at the most inopportune moment, the system can recover without data loss or replication
inconsistency.
The most vulnerable point in a transaction's lifecycle is the window between the completion of Phase 1 and the
completion of Phase 2. Specifically, a crash is most problematic after the InnoDB redo log has been fsync'd with the
"prepare" record, but before the transaction has been durably written to the binary log.23 In this state:
● InnoDB has made a durable promise on disk that it is ready to commit the transaction.
● The binary log, which dictates what is sent to replicas, has not yet recorded this transaction.
Without a proper recovery protocol, this "in-doubt" transaction would be an ambiguity that could lead to the split-brain
scenarios described earlier.
When a MySQL server restarts after a crash, a multi-stage recovery process is initiated automatically before any new
client connections are accepted.
1. InnoDB Redo Log Scan: The first step is InnoDB's internal crash recovery. The engine scans its redo log files,
starting from the last known checkpoint, and re-applies all logged changes to the data pages (a process known as
roll-forward).21 This brings the data files to a physically consistent state as of the moment of the crash.
2. Identifying "In-Doubt" Transactions: During this scan, InnoDB builds a list of all transactions that have a
"prepare" record in the redo log but lack a corresponding "commit" record.21 These are the XA transactions that
were in the critical failure window. InnoDB does not unilaterally decide their fate; instead, it re-acquires the locks for
these transactions and holds them in the "in-doubt" state, awaiting a final decision from the transaction
coordinator.
3.3 The Binary Log as the Source of Truth
With InnoDB holding the in-doubt transactions, the recovery process moves to the server level, where the binary log
takes on its role as the ultimate arbiter of commit decisions.
1. Coordinator-Led Recovery: The MySQL server inspects the last binary log file. An improperly closed file is a
definitive sign of a crash, triggering the binlog-based recovery phase.23
2. Cross-Referencing: The server scans the contents of the binary log to find the XIDs of all transactions that were
successfully logged before the crash. It then compares this list of committed XIDs against the list of "in-doubt"
XIDs provided by InnoDB.23
3. The Final Decision: The fate of each in-doubt transaction is decided by a simple rule:
○ If the XID of an in-doubt transaction is found in the binary log: This proves that the decision to commit was
made and durably recorded for replication just before the crash. The server therefore instructs InnoDB to
commit the transaction by calling innobase_commit. This action ensures that the state of the source server will
match what has been or will be sent to the replicas.23
○ If the XID of an in-doubt transaction is not found in the binary log: This proves that the crash occurred
before the transaction could be recorded for replication. To maintain consistency, the transaction must not
exist on the source. The server instructs InnoDB to roll back the prepared transaction.24
This recovery logic reveals a crucial design principle in MySQL's architecture when replication is active. The decision to
finalize a transaction is not based on the storage engine's state but is dictated entirely by the contents of the binary log.
A replication topology is, by its nature, a distributed system. The primary contract of this system is that the state of the
replicas will eventually converge with the state of the source. If a transaction has been recorded in the binary log, it has
been promised to the replicas. Allowing the source to unilaterally roll back such a transaction would fundamentally
break this contract and lead to irreparable data divergence. Therefore, the binary log is elevated to become the "log of
record" or the "source of truth" for the entire replication cluster. InnoDB's prepared state is a powerful but subservient
mechanism; its purpose is to ensure that the storage engine can fulfill the commit decision made by the binary log
coordinator, even in the face of a crash. This architecture prioritizes the consistency of the distributed system over the
local state of any single node, and the two-phase commit is the linchpin that makes this prioritization both possible and
safe.
Once the two-phase commit protocol has ensured that a transaction is atomically and durably recorded in both the
InnoDB redo log and the server's binary log, the system is in a consistent state. The next challenge is to propagate this
state to other servers in the replication topology. The binary log is the sole vehicle for this propagation.
MySQL's standard replication architecture is based on a source-replica (historically master-slave) model.22 The source
server is the single point of writes for the dataset. It records all data-modifying events into its binary log. One or more
replica servers connect to the source, read this stream of events, and apply them to their own local copy of the data,
thereby staying in sync.22
The flow of data from source to replica is managed by a set of dedicated threads, with two on the replica and one on
the source for each connection.20
● Source: The Binary Log Dump Thread: When a replica connects to a source and requests replication, the source
server spawns a dedicated thread known as the "dump thread." This thread's sole responsibility is to read events
from the source's binary log files, starting from the position requested by the replica, and transmit them over the
network to that replica.20
● Replica: The I/O (Receiver) Thread and the Relay Log: On the replica side, an "I/O thread" is responsible for
establishing and maintaining the connection to the source. It receives the stream of binary log events sent by the
source's dump thread and writes them, byte for byte, into a set of local files on the replica called the relay log.20
The relay log is structurally identical to the binary log and serves as a durable buffer. This design decouples the
process of fetching events from the network from the process of applying them, which is beneficial for
performance and resilience. If the replica server is stopped or crashes, the I/O thread can resume fetching events
from where it left off without having to re-request them from the source, as its progress is recorded in the relay log.
● Replica: The SQL (Applier) Thread: A second thread on the replica, the "SQL thread" (or "applier thread"), reads
events sequentially from the relay log and executes them against the replica's database.20 This is the step where
the data changes that occurred on the source are finally replayed on the replica. In older versions of MySQL, this
was a single-threaded process, which could become a bottleneck if the source had a high-concurrency write
workload. Modern versions support parallel replication, where multiple SQL threads (
replica_parallel_workers) can be configured to apply transactions from the relay log concurrently, significantly
improving replica throughput.
The timing of the commit acknowledgment relative to the replication stream defines the consistency and durability
guarantees of the topology.
● Asynchronous Replication (Default): In the default asynchronous mode, the source server considers a
transaction committed and returns success to the client as soon as the 2PC process is complete on the source
itself. It does not wait for any acknowledgment from any replica.22 This configuration offers the highest
performance and lowest latency for write operations on the source. However, it introduces "replication lag"—a
delay between when a transaction is committed on the source and when it becomes visible on the replica.
Furthermore, it carries a risk of data loss. If the source server suffers a catastrophic failure after committing a
transaction but before its dump thread has sent that transaction to the replicas, that transaction will be lost forever
in a failover scenario.
● Semi-Synchronous Replication: To address the data loss risk of asynchronous replication, semi-synchronous
replication can be enabled.15 In this mode, the source server's commit process is altered. After completing the 2PC
locally, it will not immediately return success to the client. Instead, it will wait until at least one replica sends an
acknowledgment that it has received the transaction's events and successfully written them to its own relay log.20
Only after receiving this acknowledgment does the source return success to the client. This provides a much
stronger durability guarantee, ensuring that every committed transaction exists on at least two separate machines.
The trade-off is increased latency for every write transaction on the source, as it must wait for a network round-trip
to a replica.32
Connecting the high-level concepts of the two-phase commit to the underlying implementation requires a brief
examination of the key functions within the MySQL C++ source code. The commit process traverses multiple layers of
the server, from the generic SQL transaction handling down to the specific implementations within the binary log
module and the InnoDB storage engine handler.
The orchestration of a transaction commit, especially when the binary log is enabled, is not a single function call but a
sequence of interactions managed by the server's handlerton architecture and the binary log acting as a transaction
coordinator.
● trans_commit() (in sql/[Link]): This function serves as the high-level entry point when a client executes
a COMMIT statement. Its primary role is to perform initial state checks and then delegate the actual work of the
commit to the handler layer by calling ha_commit_trans.33
● ha_commit_trans() (in sql/[Link]): This is the core function at the handler interface level responsible for
committing a transaction across all participating storage engines.34 When the binary log is disabled, this function
would simply iterate through the engines and call their respective
commit methods. However, when the binary log is enabled, its behavior changes fundamentally. It recognizes the
binary log as the transaction coordinator (TC) and yields control to it. The TC then drives the 2PC process, calling
back into the storage engine handlers at the appropriate prepare and commit stages.36
The redo log is an engine-internal, high-performance mechanism focused on providing instance-level durability and
enabling rapid, automatic recovery from crashes. Its physiological format and reliance on the Write-Ahead Logging
principle are optimizations tailored for speed and the physical realities of disk I/O, ensuring that the 'D' in ACID is upheld
with minimal performance impact on transaction throughput.
The binary log, in contrast, is a server-wide, portable, logical history of events designed for the distributed systems
context of replication and the forensic needs of point-in-time recovery. Its logical format—whether statement-based or
row-based—ensures that the sequence of changes can be understood and replayed by any other MySQL server,
regardless of its underlying storage engine or physical data layout.
The necessity of the two-phase commit protocol arises directly from the existence of these two separate logs. It is the
essential bridge that synchronizes the physical, engine-level commit with the logical, server-level commit. By creating
an atomic operation that spans both the redo log (in its 'prepare' phase) and the binary log (in its 'commit' phase), 2PC
guarantees that the state which is durable for local crash recovery is identical to the state which is recorded for
replication. The crash recovery procedure, which designates the binary log as the ultimate source of truth for "in-doubt"
transactions, underscores a critical design choice: when replication is enabled, the consistency of the entire distributed
topology takes precedence.
Ultimately, the architecture represents a sophisticated balance of trade-offs. The performance of a single-node
database, optimized by mechanisms like the redo log, is carefully balanced against the stringent consistency guarantees
required to operate a reliable distributed system. Enabling the binary log fundamentally alters the commit path,
introducing the overhead of the 2PC protocol. This is not a mere performance penalty but a necessary cost for the
crucial guarantee that a source server and its replicas will never permanently diverge due to an unexpected failure,
thereby ensuring the integrity of the entire data ecosystem.
Works cited
1. Effortless MySQL Redo Log Resizing: A Guide to InnoDB in 8.0.30 - Mydbops, accessed September 18,
2025, [Link]
2. An In-Depth Analysis of REDO Logs in InnoDB - Alibaba Cloud Community, accessed September 18, 2025,
[Link]
3. How InnoDB handles REDO logging - Percona, accessed September 18, 2025,
[Link]
4. MySQL Day 19: Atomicity/Durability — MTR and Redo Logs in InnoDB - Medium, accessed September 18,
2025,
[Link]
5. Innodb redo log - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
6. Dynamic InnoDB Redo Log in MySQL 8.0 - Oracle Blogs, accessed September 18, 2025,
[Link]
7. MySQL 8.4 Reference Manual :: 17.6.5 Redo Log, accessed September 18, 2025,
[Link]
8. MySQL Limitations Part 2: The Binary Log - Percona, accessed September 18, 2025,
[Link]
9. Fun MySQL fact of the day: redo logs - Richard Burnison, accessed September 18, 2025,
[Link]
10.Difference between transaction log and redo log in MySQL, accessed September 18, 2025,
[Link]
mysql
11. Mysql: binary log vs transaction log [duplicate] - DBA Stack Exchange, accessed September 18, 2025,
[Link]
12.MySQL 8.4 Reference Manual :: 7.4.4 The Binary Log, accessed September 18, 2025,
[Link]
13.MySQL Archive/Binary Logs: Replication, Recovery, and More | Tessell, accessed September 18, 2025,
[Link]
14.MySQL Binary Log - IBM, accessed September 18, 2025,
[Link]
15.MySQL Replication - Oracle, accessed September 18, 2025,
[Link]
df
16.[Link] Usage of Row-Based Logging and Replication - MySQL :: Developer Zone, accessed September
18, 2025, [Link]
17.MySQL 8.4 Reference Manual :: [Link] Binary Logging Formats, accessed September 18, 2025,
[Link]
18.Difference between row-based and statement-based replication in MySQL?, accessed September 18,
2025,
[Link]
eplication-in-mysql
19.5.1.1 Advantages and Disadvantages of Statement-Based and Row-Based Replication, accessed
September 18, 2025, [Link]
20.Introduction to MySQL Replication: Solutions - Percona, accessed September 18, 2025,
[Link]
21.MySQL 8.4 Reference Manual :: 17.18.2 InnoDB Recovery, accessed September 18, 2025,
[Link]
22.A Beginners Guide to MySQL Replication Part 1 - Redgate Software, accessed September 18, 2025,
[Link]
23.Fun MySQL fact of the day: everything is two-phase - Richard Burnison, accessed September 18, 2025,
[Link]
24.Rollback Prepared Transactions Asynchronously During Binlog Crash Recovery - MariaDB, accessed
September 18, 2025,
[Link]
-recovery/
25.WL#5223: Group Commit of Binary Log - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
26.Group Commit and Transaction Dependency Tracking - Hack MySQL, accessed September 18, 2025,
[Link]
27.Fixing MySQL group commit (part 2) - Kristian Nielsen, accessed September 18, 2025,
[Link]
28.14.19.2 InnoDB Recovery, accessed September 18, 2025,
[Link]
29.1 Replication - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
30.1 Replication - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
31.Demystifying MySQL Replication: From Theory to Implementation - Saigon Technology, accessed
September 18, 2025, [Link]
32.Question about Semi-Synchronous Replication: the Answer with All the Details, accessed September 18,
2025,
[Link]
all-the-details/
33.sql/[Link] · 778c525ff8f47aa56027d185a3f271d34cdb66ef · nexedi / MariaDB · GitLab, accessed
September 18, 2025,
[Link]
[Link]
34.sql/[Link] File Reference - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
35.[Maria-developers] Updated (by Knielsen): Efficient group commit for binary log (116) - developers -
[Link], accessed September 18, 2025,
[Link]
J6D4WJ6QG6/
36.源码分析| MySQL 的commit 是怎么commit 的? - 爱可生开源社区, accessed September 18, 2025,
[Link]
84-commit-%E6%98%AF%E6%80%8E%E4%B9%88-commit-%E7%9A%84%EF%BC%9F/
37.sql/handler.h Source File - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
38.MYSQL_BIN_LOG::LOCK_commit Impact on Performance - MinervaDB, accessed September 18, 2025,
[Link]
39.How many fsync / sec FusionIO can handle - Percona, accessed September 18, 2025,
[Link]
40.Fixing MySQL group commit (part 3) - Kristian Nielsen, accessed September 18, 2025,
[Link]
41.mysql5.5.31 log0log.c中log_write_up_to 函数代码原创 - CSDN博客, accessed September 18, 2025,
[Link]
42.innodb重做日志实现原理(上)1.概念介绍重做日志主要用来故障恢复数据的修复纠正,保证实现事务的持久
性。为了实现持 - 稀土掘金, accessed September 18, 2025, [Link]
43.Redo log buffer - MySQL :: Developer Zone, accessed September 18, 2025,
[Link]
44.InnoDB deep dive : commit phase - Medium, accessed September 18, 2025,
[Link]