Every MySQL replication runbook starts the same way: run SHOW REPLICA STATUS, read Seconds_Behind_Source, decide whether to panic. I have watched that number say 0 while a replica was minutes behind, say 4000 while the replica was seconds from catching up, and flap between 0 and a huge value every few seconds while a single large transaction applied. It is not a useless metric. It is a metric that answers a narrower question than most people think it does.
This article is about measuring replication lag on MySQL 8.0 and 8.4 in ways that survive contact with production: what Seconds_Behind_Source actually computes, how to get a trustworthy end-to-end lag number with a heartbeat table, how multi-threaded appliers change the diagnosis, and why one big transaction can make every graph lie at once.
What Seconds_Behind_Source actually measures
Seconds_Behind_Source is computed on the replica as the difference between the replica's clock and the timestamp carried by the event the SQL (applier) thread is currently processing. That definition has three sharp edges.
First, it depends on the I/O (receiver) thread being caught up. If the receiver thread has fetched everything the source sent, and the applier has applied everything in the relay log, the value is 0 — even if the network stalled and the source has seconds of binlog the replica has not yet received. A replica cut off from its source degrades gracefully to a confident-looking 0 until the connection error surfaces.
Second, it is NULL whenever the applier thread is not running, which is exactly the moment you most want a number.
Third, event timestamps come from statement execution time on the source. A transaction that ran for ten minutes on the source before committing arrives on the replica already carrying old timestamps, so lag jumps the instant the replica starts applying it. The graph shows a cliff that never actually happened as a slow drift.
None of this makes the metric wrong. It makes it a measure of applier position relative to received relay log, not a measure of how stale the data a client reads from this replica is. Those are different questions, and the second one is the one your product cares about. The stale reads on read replicas comparison goes deeper on why that distinction bites application teams.
Measure true lag with a heartbeat table
The boring, reliable fix is a heartbeat table, the pattern pt-heartbeat made standard. A job on the source updates one row with the current timestamp every second. On the replica, you read that row through the normal replication stream and compare it to the replica's clock. The result is end-to-end lag: network transfer, relay log write, and apply, all included, immune to the timestamp quirks above.
-- On the source, once:
CREATE TABLE ops.heartbeat (
id TINYINT UNSIGNED NOT NULL PRIMARY KEY,
ts TIMESTAMP(6) NOT NULL
);
INSERT INTO ops.heartbeat (id, ts) VALUES (1, NOW(6));
-- On the source, every second (event scheduler or external job):
UPDATE ops.heartbeat SET ts = NOW(6) WHERE id = 1;
-- On the replica, to measure lag:
SELECT TIMESTAMPDIFF(MICROSECOND, ts, NOW(6)) / 1000000.0 AS lag_seconds
FROM ops.heartbeat
WHERE id = 1;Two operational notes. Run the updater from an external scheduler rather than the MySQL event scheduler if you can, because you want the heartbeat to stop when the source is genuinely unhealthy, not to be the last thing still working. And make sure source and replica clocks are NTP-disciplined, because the measurement is only as good as clock agreement. A heartbeat also gives you something Seconds_Behind_Source never can: a lag number that keeps working during a replication error, because the timestamp simply stops advancing and the computed lag grows honestly.
Use performance_schema to see where the lag lives
Once you know lag exists, the next question is which stage owns it: receiving, queueing, or applying. MySQL 8.0 added original and immediate commit timestamps to the replication instrumentation, and the performance_schema replication tables expose them per worker.
SELECT worker_id,
last_applied_transaction,
TIMESTAMPDIFF(SECOND,
last_applied_transaction_original_commit_timestamp,
last_applied_transaction_end_apply_timestamp) AS source_to_applied_s,
applying_transaction,
TIMESTAMPDIFF(SECOND,
applying_transaction_original_commit_timestamp,
NOW(6)) AS current_txn_age_s
FROM performance_schema.replication_applier_status_by_worker
ORDER BY worker_id;Read it like this. If source_to_applied_s is small but the heartbeat lag is large, the applier is fine and the receiver thread is behind: look at network throughput and source binlog generation rate. If current_txn_age_s is large on one worker while others sit idle, one transaction is dominating: usually a large batch write or a DDL. If all workers show steady but growing apply delay, the replica genuinely cannot keep up with the write rate and you have a capacity or parallelism problem.
Multi-threaded replicas: the knobs that matter
Since MySQL 8.0.27, replicas default to a multi-threaded applier with replica_parallel_workers=4 and replica_preserve_commit_order=ON. In 8.4, multi-threading is no longer optional in the old sense: replica_parallel_type was removed and LOGICAL_CLOCK scheduling is simply how the applier works.
The knob that actually moves throughput is replica_parallel_workers, but only up to the parallelism available in the binlog stream. Workers can only apply transactions concurrently when the source marked them as non-conflicting. On MySQL 8.0 that marking is controlled on the source by binlog_transaction_dependency_tracking: COMMIT_ORDER (the old default) only parallelizes transactions that committed together in a group, while WRITESET tracks actual row-level write sets and exposes far more parallelism, especially for workloads with many small single-row transactions. In 8.4 that variable is gone and WRITESET behavior is always on, which is a quiet but meaningful reason replica apply throughput often improves on upgrade.
So the tuning sequence on 8.0 is: set binlog_transaction_dependency_tracking=WRITESET on the source first, then raise replica_parallel_workers on the replica in steps while watching apply lag under real load. Raising workers against a COMMIT_ORDER binlog is the classic no-op tune: sixteen workers, one of them busy.
Keep replica_preserve_commit_order=ON unless you have a very specific reason not to. It prevents the replica from externalizing transactions in a different order than the source, which matters for anything reading from the replica and for consistent GTID gaps on crash.
Big transactions break every model
A single 5 GB transaction — a backfill UPDATE, a data migration, an ORM bulk delete — defeats all of the machinery above. It cannot be parallelized, because it is one transaction. It stalls commit-order progress for every worker behind it. Seconds_Behind_Source sits near 0 while the replica receives the transaction, then leaps to the transaction's full duration when apply starts. The heartbeat freezes for the whole apply window, which is the honest signal.
The mitigation is unglamorous: chunk large writes on the source into bounded transactions (tens of thousands of rows, not tens of millions), and alert on transaction size before it becomes replica lag. The binary log is where the evidence lands first; the binlog vs WAL comparison covers why MySQL replicas serialize on transaction boundaries in a way PostgreSQL physical replicas do not.
What to alert on
A lag alerting baseline that has survived several fleets: alert on heartbeat lag against a per-replica threshold set by what reads from it, not a global number; alert on replica_io_running or replica_sql_running going off, immediately, because lag metrics go quiet exactly then; alert on heartbeat lag rate-of-change, because a replica losing one second per second never catches up; and record applier worker saturation from performance_schema so capacity trends are visible before the incident. Treat Seconds_Behind_Source as a cheap corroborating signal, never the primary one.
Where MonPG fits
An honest note on tooling. MonPG today is a PostgreSQL monitoring platform — query history, lock chains, replication lag, vacuum evidence for Postgres fleets. We are currently building MySQL monitoring (coming soon), and replication lag is exactly the kind of signal we are designing it around: heartbeat-based true lag, applier-stage breakdown, and big-transaction detection rather than a single Seconds_Behind_Source graph. Nothing on this page requires MonPG; it works with stock MySQL. If your team also runs PostgreSQL, you can start there today and the MySQL side will land in the same workflow when it ships.