The question arrives the same way every time: mysqld's resident memory is 20 GB on a host where the buffer pool is set to 12 GB, and someone wants to know where the other 8 GB went. Sometimes the question arrives more violently, as a kernel OOM kill at 3 a.m. and a database that restarted with cold caches.
MySQL memory has a reputation for being unknowable, and that reputation is out of date. Since 8.0, memory instrumentation in performance_schema is enabled by default and accounts for most of what the server allocates. You can ask the server where its memory went and get a mostly honest answer. The remaining gap between the instrumented total and the operating system's view has explanations too.
This guide maps the memory landscape for MySQL 8.x, shows the queries I actually run, and ends with the postmortem workflow for an OOM kill.
Three kinds of memory, not one
Mentally splitting MySQL memory into three buckets makes every diagnosis faster.
Global, mostly fixed allocations are dominated by the InnoDB buffer pool, plus the redo log buffer, table caches, the dictionary cache, performance_schema itself, and binary log caches. These are sized at startup or grow to a configured ceiling and then sit there. The buffer pool alone is usually 60 to 80 percent of a dedicated server's RAM, and sizing it is its own discipline - the InnoDB buffer pool sizing field guide covers that. InnoDB also carries overhead beyond innodb_buffer_pool_size for page metadata and internal structures, so the pool costs somewhat more than its setting.
Per-session allocations exist for each connection: thread stack, connection buffers, and session state. Individually small, they scale with Threads_connected. Replication adds its own per-session line item that audits routinely miss: every session on a binlog-enabled server gets a binlog cache (binlog_cache_size, 32 KB by default) that spills to temporary files for large transactions, and replicas carry relay log and applier buffers on top.
Per-operation allocations are the dangerous bucket. Sort buffers, join buffers, read buffers, and in-memory temporary tables are allocated as queries execute. A single query can allocate a join buffer per join that lacks an index and a sort buffer per sort. These allocations scale with active concurrency times query nastiness, which is why memory incidents so often coincide with a bad deploy rather than gradual growth.
The per-session buffer trap
The classic mistake is setting sort_buffer_size or join_buffer_size to hundreds of megabytes globally because one reporting query benefited. Those settings are per allocation, not per server. Under a concurrency burst, fifty sessions each grabbing a 256 MB sort buffer is 12.8 GB of transient demand the buffer pool sizing never budgeted for. Keep the global defaults modest and raise them per session for the specific job that needs it.
Internal temporary tables deserve their own paragraph because 8.0 changed the model. In-memory internal temp tables use the TempTable engine, which draws from a shared pool capped by temptable_max_ram (1 GB by default) rather than a strictly per-table limit, and spills beyond that. The spill behavior and its disk consequences are covered in MySQL internal temp tables on disk; the memory consequence is that GROUP BY-heavy workloads can hold that shared pool fully allocated for long stretches.
Ask the server: performance_schema memory instrumentation
MySQL 8.0 enables memory instruments by default, so every instrumented allocation is tallied in the memory summary tables. The raw table is memory_summary_global_by_event_name, and the columns worth reading are current bytes used and high-water marks.
SELECT EVENT_NAME,
CURRENT_NUMBER_OF_BYTES_USED / 1024 / 1024 AS current_mb,
HIGH_NUMBER_OF_BYTES_USED / 1024 / 1024 AS high_mb
FROM performance_schema.memory_summary_global_by_event_name
WHERE CURRENT_NUMBER_OF_BYTES_USED > 0
ORDER BY CURRENT_NUMBER_OF_BYTES_USED DESC
LIMIT 15;Event names are self-describing: memory/innodb/buf_buf_pool is the buffer pool, memory/temptable/physical_ram is the TempTable engine, memory/sql/thd::main_mem_root is per-session statement memory, and so on. The high-water columns are the underrated part - after a memory scare, they tell you which allocator spiked even if it has since released.
The sys schema wraps the same data more readably. sys.memory_global_by_current_bytes gives the ranked view, sys.memory_global_total gives one number to compare against RSS, and the per-thread view answers "which connection is eating memory right now":
SELECT thread_id,
user,
current_allocated,
current_max_alloc
FROM sys.memory_by_thread_by_current_bytes
ORDER BY current_allocated DESC
LIMIT 10;Two caveats. First, these are aggregates since instrument enablement, effectively since restart on a default 8.0 config; high-water marks reset then too. Second, instrumentation covers most but not all allocations, so the instrumented total will run below the process RSS. Keeping the instrumentation overhead sane is a solved problem - the defaults are fine, and the low-overhead performance_schema setup covers what to trim if you are squeezed.
When RSS is bigger than the sum of the parts
If sys.memory_global_total says 14 GB and the OS says mysqld is 17 GB resident, do not assume a leak. The usual explanations, in order of likelihood: allocator fragmentation and retention, since glibc malloc holds freed memory in per-thread arenas and returns it to the OS reluctantly (many high-connection-count fleets run jemalloc or tcmalloc partly for this); uninstrumented allocations, including memory allocated by some plugins or before instruments initialize; and OS-level accounting quirks around huge pages if you use them.
A steady gap is normal. A growing gap with flat instrumented totals points at fragmentation; a growing gap that tracks a specific instrumented event points at a real workload change. Graphing RSS and the instrumented total together turns this from philosophy into a diff.
The OOM-kill postmortem workflow
When the kernel kills mysqld, work the evidence in this order.
First, confirm it was the OOM killer: journalctl -k or dmesg will show the oom-killer invocation with mysqld's RSS and the score at kill time. Record that RSS number; it is your budget-overrun figure. Check whether the host has swap and what vm.swappiness is - a database host with no swap converts every transient overshoot into a kill.
Second, reconstruct the budget. Buffer pool setting plus InnoDB overhead plus temptable_max_ram plus a realistic per-connection allowance times Max_used_connections. If that arithmetic already exceeds host RAM, the config was a time bomb and the postmortem writes itself.
Third, find the trigger. The MySQL error log shows the restart but rarely the cause. Your monitoring history is the real witness: look for a step in Threads_running, a new query family, or a temp-table spike in the minutes before the kill. This is the strongest argument for collecting the memory summary views continuously - after a restart, the in-server counters are wiped, and only external history can tell you which allocator was climbing.
Fourth, decide the fix by category: shrink the buffer pool if the budget never fit; fix the query or its indexes if per-operation buffers spiked; cap or pool connections if session memory scaled past the plan. "Add RAM" is only the right answer when the budget math says the workload legitimately grew.
Fifth, harden against the recurrence. On a dedicated database host, adjust the OOM score for mysqld (OOMScoreAdjust in the systemd unit) so the kernel prefers to kill almost anything else first - an OOM-killed sidecar is an annoyance, an OOM-killed database is an outage with crash recovery attached. Keep a modest amount of swap as a shock absorber for transient overshoot; the latency hit of brief swapping is nearly always cheaper than a cold buffer pool. And alert on mysqld RSS as a percentage of host RAM with enough headroom - firing at 90 percent - that a human sees the trend before the kernel does.
Monitoring memory, and where MonPG is headed
The practical loop: know your three buckets, keep per-session buffer settings boring, graph RSS against sys.memory_global_total, and retain the top memory events over time so an OOM kill leaves evidence behind.
MonPG is a PostgreSQL monitoring platform today, and MySQL support is under active development - the memory views above, collected continuously with history that survives restarts, are exactly the kind of evidence it is being built to keep. It does not monitor MySQL yet; see MySQL monitoring (coming soon) for the roadmap. Teams that also operate PostgreSQL can start there now and get the same workflow when MySQL lands.