MySQL9 min read

MySQL Semi-Sync Replication Pitfalls in Production

Semi-synchronous replication promises that a commit is acknowledged by at least one replica before the client sees success. The fine print — silent fallback to async, wait points, and what an ack actually means — is where production incidents live.

Teams usually adopt semi-synchronous replication after a bad day: a source failed, the freshest replica was missing the last few seconds of commits, and someone had to explain to the business which writes were gone. Semi-sync is the standard answer. The source waits, on commit, until at least one replica acknowledges receipt of the transaction before returning success to the client. Deployed carefully, it turns "we probably lost a little data" into "we can prove we lost none."

Deployed casually, it provides a durability guarantee that quietly turns itself off under exactly the conditions where you need it. This article is about the fine print on MySQL 8.0 and 8.4: the silent fallback to async, what the wait point setting changes, and the gap between what semi-sync guarantees and what people believe it guarantees.

The moving parts, briefly

Semi-sync ships as plugins, not core server behavior. Since MySQL 8.0.26 the components use source/replica naming — semisync_source.so on the source, semisync_replica.so on replicas — with variables to match (rpl_semi_sync_source_enabled, rpl_semi_sync_replica_enabled, and friends). The old master/slave-named plugins were deprecated then and are gone in 8.4, so config management that still installs semisync_master.so breaks on upgrade. That is worth checking today, not during the upgrade window.

The variables that define your actual durability posture are a short list: rpl_semi_sync_source_enabled, rpl_semi_sync_source_timeout, rpl_semi_sync_source_wait_for_replica_count, and rpl_semi_sync_source_wait_point. Every pitfall below traces back to one of them.

Pitfall 1: the silent fallback to async

rpl_semi_sync_source_timeout defaults to 10000 milliseconds. If no replica acknowledges a commit within that window, the source does not fail the transaction. It commits anyway, flips itself to asynchronous replication, and keeps going. Clients notice nothing except one slow commit. The status variable Rpl_semi_sync_source_status goes from ON to OFF, and semi-sync resumes only when a replica catches up and acks again.

Read that again with an operator's eye: the degradation happens precisely when replicas are unhealthy or the network is bad — the exact circumstances in which the source is most likely to fail next, and in which you were counting on the guarantee. Semi-sync's default posture is availability over durability, and it will not ask your permission.

So the single most important semi-sync monitoring rule is: alert on the fallback itself, immediately, at incident priority.

-- On the source: is the guarantee actually in force right now?
SHOW GLOBAL STATUS LIKE 'Rpl_semi_sync_source_status';

-- How often are commits acked vs falling through?
SHOW GLOBAL STATUS LIKE 'Rpl_semi_sync_source_yes_tx';
SHOW GLOBAL STATUS LIKE 'Rpl_semi_sync_source_no_tx';

-- How many replicas are currently ack-capable?
SHOW GLOBAL STATUS LIKE 'Rpl_semi_sync_source_clients';

A growing Rpl_semi_sync_source_no_tx counter is a ledger of commits that were acknowledged to clients without the replica guarantee. If you cannot say how many hours last month your fleet actually spent in semi-sync mode, you do not have semi-sync; you have async with extra latency on good days. Some teams set the timeout to an enormous value to effectively refuse async fallback — that is a legitimate choice, but it converts replica trouble into source write stalls, so it must be made with eyes open and paired with rpl_semi_sync_source_wait_for_replica_count sized against how many replicas can be down at once.

Pitfall 2: the wait point, and why AFTER_SYNC is the default

rpl_semi_sync_source_wait_point controls where in the commit sequence the source pauses to wait for the ack, and it changes failure semantics more than any other setting. With AFTER_SYNC — the default, often called lossless semi-sync — the source writes and syncs the transaction to its binary log, then waits for the replica ack, and only after the ack completes the commit in the storage engine and returns to the client. With AFTER_COMMIT, the source commits in the engine first, then waits for the ack before returning to the client.

The difference sounds academic until a source crashes mid-wait. Under AFTER_COMMIT, the transaction was already committed and therefore already visible: other sessions may have read that data, acted on it, and had their own writes replicate — while the crashed transaction itself may exist on no replica. Fail over, and you have externalized a write that the new source never had. This is the phantom-read failover problem, and it is why AFTER_COMMIT semi-sync does not deliver the guarantee most people deploy semi-sync to get. Under AFTER_SYNC, nothing becomes visible to other sessions until a replica holds the event, so a crash before the ack loses only work that no client was ever told succeeded.

The residual cost of AFTER_SYNC shows up on crash recovery of the old source: transactions that were synced to its binlog but never acknowledged get rolled forward locally on restart, so the old source can come back with more data than the promoted replica. That is one of several reasons a failed-over source should be re-seeded, not blindly rejoined — a topic the Orchestrator vs Patroni comparison treats in more depth.

Pitfall 3: an ack is receipt, not apply

The replica sends its acknowledgment after the transaction has been received and flushed to its relay log. Not after it has been applied. This has two consequences that surprise people regularly.

First, semi-sync is not synchronous replication. A semi-sync replica can be minutes behind on apply while cheerfully acking every commit, because the receiver thread is fast even when the applier is drowning. Reads from that replica are as stale as any async replica's — semi-sync does nothing for read-your-writes consistency, a distinction that matters if anyone routes reads by "it's semi-sync, it's current" logic; the stale reads comparison unpacks that failure mode.

Second, failover after a source loss still involves waiting for the promoted replica to finish applying its relay log. The data is durable in the relay log, but not yet queryable. Budget that apply time into your recovery time objective, and keep replica apply lag monitored independently of semi-sync status.

-- On a replica: ack capability and apply progress are separate facts.
SHOW GLOBAL STATUS LIKE 'Rpl_semi_sync_replica_status';

SELECT channel_name,
       service_state,
       count_transactions_remote_in_applier_queue AS queued_txns
FROM performance_schema.replication_applier_status;

Pitfall 4: counting replicas wrong

rpl_semi_sync_source_wait_for_replica_count defaults to 1: one ack from any semi-sync replica satisfies the commit. With several semi-sync-enabled replicas, the ack usually comes from whichever is fastest — often the one in the same availability zone, which is also the one most likely to die with the source. If your durability story is "the data survives a zone loss," the ack has to come from a replica outside the zone, which means either raising the count or being deliberate about which replicas run the semi-sync replica plugin at all. Enabling it everywhere feels safer but mostly guarantees the nearest replica acks first.

Raising the count to 2 tightens durability and raises the odds of hitting the timeout during replica maintenance. That trade is the whole game with semi-sync: every setting is a position on the durability-versus-availability line, and the honest move is to pick the position on purpose. It is also worth saying that binlog durability on the source still matters — sync_binlog=1 and innodb_flush_log_at_trx_commit=1 are what make the binlog the source of truth that semi-sync then extends to another host; the binlog vs WAL comparison covers that layer.

A monitoring baseline for semi-sync

Distilled: alert when Rpl_semi_sync_source_status leaves ON, at page-someone priority. Track Rpl_semi_sync_source_no_tx as a counter and treat any increase as a durability audit event. Alert when Rpl_semi_sync_source_clients drops below wait_for_replica_count plus your maintenance headroom. Watch Rpl_semi_sync_source_avg_net_wait_time for creep, because commit latency now includes a network round trip and slow drift here is capacity planning information. And graph replica apply lag next to all of it, because acked is not applied.

Where MonPG fits

Honesty about where we stand: MonPG is a PostgreSQL monitoring platform, and MySQL support is being built now — see MySQL monitoring (coming soon). Semi-sync state tracking is on the shortlist for it precisely because the failure mode is silent: the difference between semi-sync ON and OFF appears in no query latency graph, only in the status variables above. Until then, everything in this article is doable with stock MySQL and your existing metrics pipeline. If you also run PostgreSQL, MonPG can start there today, where we track the equivalent synchronous_commit and replication guarantees.