We turned on SERIALIZABLE because an audit found a real race: two concurrent requests could each pass a balance check and together overdraw an account. The fix worked. Tests passed, staging was calm, and for two weeks in production nothing happened. Then the nightly batch window started overlapping evening traffic, and our error tracker filled with SQLSTATE 40001 — could not serialize access due to concurrent update. Support tickets followed: users retried payments that had failed, and some retried ones that had not. The isolation level was behaving exactly as designed. Our application, which treated a failed transaction as an unexpected event instead of a routine scheduling decision, was not.
Serializable snapshot isolation — SSI — is one of the best ideas in PostgreSQL. You keep snapshot isolation's non-blocking reads, and the engine watches for the specific shape of anomaly that snapshot isolation allows: a cycle of rw-antidependencies, where one transaction reads a version another overwrites and vice versa. When it sees that dangerous structure about to close, it aborts one of the participants. No locking readers, no blocking writers, just a referee who occasionally blows the whistle. The part the documentation cannot make visceral is that the whistle is the feature. A 40001 is not PostgreSQL failing; it is PostgreSQL succeeding at detecting a possible anomaly, on evidence that is sometimes approximate.
What does SERIALIZABLE cost when nothing conflicts?
Less than its reputation suggests: the steady-state overhead is memory and bookkeeping, not waiting. Every transaction takes SIREAD locks — predicate locks that record what you read, at tuple granularity when possible. They block nobody; they exist so the conflict detector can notice when a later write intersects an earlier read. They are held until the transaction commits and, briefly, until overlapping transactions finish, because the dangerous structure can complete after a commit. On workloads I have measured — order processing, a few hundred transactions per second, mostly short — the throughput difference versus REPEATABLE READ was in the low single digits, inside run-to-run noise.
The real cost is variance, and it arrives in bursts. Conflict probability scales with overlap: longer transactions, hotter rows, more concurrency on the same data. That is why staging lies to you. A staging environment has no overlap, so it has no conflicts, so SERIALIZABLE looks free. Production has the batch job and the web tier touching the same account rows at 21:00, and suddenly three percent of transactions are being aborted and resubmitted. Budget for the failure path, not the happy path.
Why do you get 40001 errors on rows you never touched?
Because the abort decision is about dependency structure, not about specific rows, and the tracking granularity is coarser than a tuple more often than you think. The canonical false positive: a transaction that scans a whole table — a report, a SUM, a reconciliation — holds SIREAD locks across every page it visited, and any concurrent writer on any of those pages creates an rw-edge. Two such edges forming a cycle, and somebody gets aborted even though a perfectly valid serial order existed. The detector answers the question "could this schedule be equivalent to some serial execution" with a heuristic, and heuristics have false positives.
Granularity promotion is the multiplier. Predicate locks start at the tuple, but the lock pool is finite, so PostgreSQL promotes: enough tuple locks on one page collapse into one page-level lock, and enough page-level locks on one relation collapse into one relation-level lock. Each promotion widens the net — a page-level SIREAD lock conflicts with writes to any tuple on that page, including tuples your transaction never looked at. Long transactions make everything worse twice: they hold locks longer, so they overlap more writers, and they accumulate more locks, so they promote further. The 40001 you cannot reproduce locally was almost certainly a promoted page or relation lock plus an unlucky overlap window.
How should the retry loop be written?
Assume every SERIALIZABLE transaction can fail with 40001 at any point including COMMIT, and retry the whole transaction from the beginning, with a bound and a jittered backoff. The whole transaction, not the failing statement: the abort invalidates the snapshot, so values you read earlier must be re-read inside the retry, and any decision derived from them must be re-derived. If you computed the transfer amount from a balance you selected before the conflict, replaying the old statements with old numbers is how you build the exact anomaly SERIALIZABLE exists to prevent.
The discipline details are where teams get hurt. Retry only on 40001 (and treat 40P01 deadlocks the same way), never on constraint violations or syntax errors. Cap attempts at three to five and then fail loudly — an unbounded retry loop under contention is a retry storm, and a retry storm is a throughput amplifier that turns a busy minute into an outage. Back off with jitter so aborted transactions do not re-collide in lockstep. Keep transactions short, because conflict probability grows with duration and retry cost is proportional to work already done. Do not perform non-transactional side effects — emails, webhooks, file writes — inside the retry boundary; queue them and dispatch after a successful commit. And for the big read-only reports that scan everything, declare them up front: a READ ONLY DEFERRABLE transaction waits for a snapshot guaranteed to be safe, trading startup delay for never aborting mid-report.
BEGIN ISOLATION LEVEL SERIALIZABLE;
UPDATE accounts SET balance = balance - 100 WHERE id = 7;
UPDATE accounts SET balance = balance + 100 WHERE id = 9;
COMMIT; -- can still fail here with 40001: retry the whole block
BEGIN TRANSACTION ISOLATION LEVEL SERIALIZABLE READ ONLY DEFERRABLE;
SELECT sum(balance) FROM accounts;
COMMIT;What does max_pred_locks_per_transaction actually change?
It sizes the shared predicate-lock pool, not a per-transaction quota: the lock table is built to hold this setting times max_connections plus max_prepared_transactions locks, as an average, and any single transaction may hold more than its share while the pool has room. Raising it therefore costs memory, not speed on hot paths. The default is 64, and note what 64 is not: it is not the threshold at which your tuple locks start collapsing into page locks. That decision belongs to the promotion knobs, and they are far more trigger-happy — which is why range reads go page-wide long before the pool itself is in danger.
I raise it to 256 or 512 on any cluster that runs SERIALIZABLE in earnest, and the memory line item has never mattered — the arithmetic is the setting times connections times a small struct, so 512 with a few hundred connections is spare change next to one gigabyte of shared_buffers. The companion knobs matter too: max_pred_locks_per_page controls how eagerly tuple locks on a page promote (the default of 2 is aggressive), and max_pred_locks_per_relation does the same job one level up. You can watch promotion happen directly in pg_locks, where predicate locks show up with mode SIReadLock and the granularity is visible in what they reference.
SELECT relation::regclass AS rel, mode, locktype, count(*)
FROM pg_locks
WHERE mode = 'SIReadLock'
GROUP BY 1, 2, 3
ORDER BY 4 DESC;
SELECT name, setting FROM pg_settings
WHERE name LIKE 'max_pred_locks%';A relation-level SIReadLock with a big count on your hottest table, alongside a climbing 40001 rate, is the signature that your reports and your writers are fighting through promoted locks. Fix it with the budget knobs, with shorter transactions, or by moving the report to READ ONLY DEFERRABLE — usually in that order of effort.
When is REPEATABLE READ plus explicit locking the calmer choice?
When the invariants you care about live on a small number of hot rows, blocking beats failing, and explicit locks block. An inventory decrement, a seat reservation, a balance column: these are places where contention is guaranteed and the conflict surface is one or two rows. A SELECT ... FOR UPDATE — or FOR NO KEY UPDATE when you are not changing key columns, which keeps the lock lighter for foreign-key checks — serializes the contenders with waits. Waits show up as latency; aborts show up as errors, and errors get retried, and retries multiply load at the exact moment the system is least able to absorb it.
My rule of thumb: if the retry rate on a hot path climbs past roughly ten percent, you are paying for every unit of work twice and the curve is getting worse, not better. Move that path to REPEATABLE READ with explicit row locks or a single atomic UPDATE ... RETURNING, and keep SERIALIZABLE for the wide, subtle invariants — the ledger must balance across forty tables — where enumerating the locks by hand would be its own bug farm. The lock-wait half of this trade is covered in the notes on lock manager contention. SERIALIZABLE is the right tool for invariants you cannot enumerate; it is an expensive way to protect one counter.
Watching serialization failures with MonPG
MonPG monitors PostgreSQL in production today, and the numbers around SSI are the kind it graphs well: SIReadLock counts by relation out of pg_locks, the age of the oldest active transaction, and lock-wait accumulation, so a promotion-driven conflict buildup is visible before your application logs fill with 40001s. Pair those graphs with an application-side counter on serialization failures per endpoint — the database sees the locks, but only your code sees the retries. The PostgreSQL monitoring surface details how the database-side numbers are collected. Tune the knobs, keep the retry loop honest, and SERIALIZABLE goes back to being what it should be: quiet, cheap, and boring.