The most dangerous minute in MySQL operations is the one right after a restart, when the port opens and the application discovers a database with an empty buffer pool. I once watched a routine reboot for a kernel patch turn into a twenty-minute incident: every popular query suddenly read from storage, p99 latency went from 40 ms to over a second, connections piled up toward max_connections, and the retry logic in the application made everything worse. The database was perfectly healthy. It was just cold.
MySQL 8.0 ships with a built-in answer to this, the buffer pool dump and load feature, and it is on by default. But the defaults have gaps, and the difference between a restart nobody notices and a restart that pages the on-call is usually how deliberately you configured and monitored the warmup. This is how I run it.
What a cold pool actually costs
The buffer pool is where InnoDB keeps the pages your queries touch. When it is empty, every logical read becomes a physical read, and physical reads are where latency budgets go to die. On local NVMe the penalty is bad; on network-attached storage with per-I/O latency and IOPS caps, it is brutal. A query plan that was perfectly fine at a hundred thousand logical reads per second becomes a different animal when each of those reads costs a storage round trip.
The secondary effects hurt more than the slow queries themselves. Statements take longer, so connections stay open longer, so the connection count climbs. If you run without a connection limit strategy, you hit max_connections and start refusing logins. Application threads block, upstream load balancers mark the service unhealthy, and retry logic, especially the kind without jitter, turns one cold database into a self-inflicted denial of service. I have seen a cold pool take down an otherwise healthy system not because of the database, but because everything around it lost patience at the same moment.
Organic warmup is the slow path. Left alone, the pool refills as traffic touches pages, but on a 100 GB pool with a 40 GB working set that can take the better part of an hour, and you pay full storage latency the entire time.
How the dump and load mechanism works
InnoDB can record which pages were in the pool and reload them in bulk at startup. Two settings control it, and both default to ON in MySQL 8.0 and 8.4: innodb_buffer_pool_dump_at_shutdown writes the list when the server shuts down cleanly, and innodb_buffer_pool_load_at_startup reads that list during startup and fetches the pages. The dump file is ib_buffer_pool in the data directory by default, movable with innodb_buffer_pool_filename if you care.
The key detail that makes this cheap: the dump stores only page identifiers, the tablespace ID and page number, never page contents. The file is small, writing it takes seconds even on a huge pool, and because the identifiers are sorted, the reload is largely sequential I/O rather than random. Loading reads the current version of each page from the data files, so a stale dump is not a correctness problem; it is just a slightly less accurate picture of what was hot.
innodb_buffer_pool_dump_pct controls how much of each pool instance gets recorded, expressed as the percentage of the most recently used pages. It defaults to 25. On a latency-sensitive system I raise it, 40 or 50, because the most-recently-used quarter of the pool is often not the whole hot set. The tradeoff is a bigger dump file and a longer load, but the load is bounded and the cold-cache alternative is not. None of this touches crash recovery semantics: the dump is a hint about what to cache, and InnoDB's redo-based recovery after a crash works exactly the same whether the dump exists or not.
Why shutdown-only dumps are not enough
Here is the gap in the defaults: the dump happens at shutdown, but the restarts that hurt most are the ones where shutdown never runs. A crash, an OOM kill, a kill -9 from a frustrated operator, a hypervisor eviction; none of these write a fresh dump. If your last clean shutdown was three weeks ago, your startup load restores the working set from three weeks ago, which for a growing application is closer to a random sample than a cache.
The fix is almost embarrassingly simple. Trigger a dump on a schedule:
-- take a snapshot of the current hot-page list, in the background
SET GLOBAL innodb_buffer_pool_dump_now = ON;
-- confirm what the feature is configured to do
SELECT @@innodb_buffer_pool_dump_at_shutdown AS dump_at_shutdown,
@@innodb_buffer_pool_load_at_startup AS load_at_startup,
@@innodb_buffer_pool_dump_pct AS dump_pct,
@@innodb_buffer_pool_filename AS dump_file;
SHOW STATUS LIKE 'Innodb_buffer_pool_dump_status';I run that from cron every five to fifteen minutes on every production node, primaries and replicas alike. The dump is written by a background thread, the status variable confirms progress, and the file cost is trivial. After this change, any restart, clean or otherwise, starts from a page list that is minutes old instead of weeks old. It is the cheapest insurance I know of in MySQL operations, and it surprises me how many shops leave it off because the defaults looked fine.
Watching the load and gating traffic
During startup the load runs while the server comes up, and you can watch it:
SHOW STATUS LIKE 'Innodb_buffer_pool_load_status';The value is human-readable, something like "Loaded 82045 of 131071 pages", and it flips to a completion message when finished. Two operational uses. First, if the load is crawling because storage is slow, you learn that before the application does, and you can abort it with SET GLOBAL innodb_buffer_pool_load_abort = ON if the box is needed for write traffic more than it needs a warm cache. Second, and more important: use it as a readiness gate.
A rebooted node should not receive full traffic the moment the port accepts connections. Wire your readiness check, whether that is a Kubernetes probe, a load balancer health check, or a proxy script, to require the load to be finished, or at least past a high percentage, before restoring full weight. Most failover tooling and proxies can run an arbitrary SQL check; this is the one to run. If you cannot gate on it, at least ramp traffic gradually for the first minutes after any restart.
Warmup after failover
Failover is where this really pays. When a replica gets promoted, its buffer pool contains whatever the replica workload touched, which on a replica that only applied binlog and served a handful of read queries may be very different from the primary's write working set. I have watched promotions where the new primary's pool was technically warm but warm with the wrong pages, and the effect was nearly as bad as cold.
Scheduled dumps solve half of it: the replica's dump reflects its actual recent page access, so the page list on disk is minutes old. But be precise about the mechanics. A load runs automatically only at server startup, through innodb_buffer_pool_load_at_startup; a promotion by itself loads nothing. If the promoted replica crashed and restarted, the startup load already ran, but in a clean promotion where the server never restarts, your failover tooling has to fire SET GLOBAL innodb_buffer_pool_load_now = ON on the new primary and then gate traffic on the load status. Combine that with a sensible dump_pct and a readiness gate, and a promotion stops being a latency event. The residual gap is real, though: the replica's dump reflects the replica's workload, not necessarily the primary's write working set, and that is the one place synthetic warmup queries, replaying the hottest read paths against the new primary before opening full traffic, still earn their keep. They are fragile, so drive them from the real query mix, the same discipline I described in the slow query log digest workflow, not from a hand-written list of queries someone thought were important.
One sizing note: warmup assumes the pool can hold the working set at all. If your pool is chronically too small, the dump just pre-loads a too-small cache. Sizing is its own topic, and I covered it in the InnoDB buffer pool sizing field guide. Warmup is the second half of that story: sizing decides what can be cached, dump and load decide how fast you get there after the inevitable restart.
What I'd monitor after a restart
Full disclosure before the pitch: MonPG monitors PostgreSQL today, not MySQL, and the MySQL monitoring is coming soon, still in active development. I want it for exactly the questions this article runs on: is the post-restart load actually progressing, how fresh is the dump this node would restore from, and how long did p99 take to normalize after the last promotion, none of which should require hand-rolled cron and a readiness probe you wired yourself at 2am. If PostgreSQL runs anywhere in your estate, that evidence-first treatment already ships for it on the MonPG PostgreSQL platform, and the blog carries the rest of this MySQL series. Until the MySQL product lands, the scheduled dump and the readiness gate above are the whole play, and they cost almost nothing.