The worst plan I ever shipped to production came from a number I had never looked at. A query the planner estimated at about a thousand rows actually matched 400,000, it chose a nested loop whose inner side executed 400,000 times, and the API p99 went from 90 ms to 11 seconds on a Tuesday morning. The fix was not an index, not a rewrite, not more hardware: one ALTER COLUMN ... SET STATISTICS and one ANALYZE. The estimate snapped from 1,100 rows to 380,000, the planner flipped to a hash join, and latency went back to baseline. The number I had never looked at was default_statistics_target, and the column was quietly sampling thirty thousand rows out of three hundred million.
What the target actually controls
default_statistics_target defaults to 100, and despite the name it is not a count of anything the planner keeps directly. It is a sampling dial. ANALYZE reads 300 times the target in randomly sampled rows — 30,000 at the default, one sample shared by every column being analyzed, with the largest per-column target setting the size — and from that sample builds everything you see in pg_stats: the most-common-value list, the histogram bounds, the n_distinct estimate, the correlation number. The target also caps how deep those structures go: at 100, ANALYZE tracks at most 100 common values and 100 histogram buckets per column. Raise the target and two things improve together — the sample is bigger, and the structures built from it are deeper. The cost is ANALYZE time and plan churn, and I will get to both, because they are real and they are why the answer is not "set it to 10,000 everywhere".
n_distinct: the estimator that skew breaks
Open pg_stats and n_distinct is the first column people misread. A positive value is an absolute estimate of distinct values in the table. A negative value is a fraction of rows: -0.05 on a 300-million-row table means roughly 15 million distinct. Minus one means the analyzer believes the column is unique. The estimator extrapolates from the sample — it counts how many distinct values appear in 30,000 draws and how many appear exactly once, then projects the population. On roughly uniform data it does well. On skewed data — Zipfian columns, which is to say most real-world foreign keys — the sample is dominated by hot values, the tail is mostly invisible, and the projection can be wrong by an order of magnitude in either direction. My incident column, a merchant ID with brutal skew, was estimated at 12,000 distinct values; the real count was over 400,000.
Why it matters: for any value not in the most-common-value list, the planner estimates equality selectivity by spreading the leftover probability mass across the estimated distinct count. Underestimate n_distinct by 30x and that mass is divided thirty ways too few — every rare merchant looks more common than it is — while the merchants who should have held measured slots in the MCV list get priced from a residual guess that can land orders of magnitude off in either direction. My incident was the low side: the planner priced the join at a handful of rows, picked nested loops, and the API paid the difference in production while every dashboard insisted the database was fine.
most_common_vals depth decides equality estimates
For values that do make the most-common-value list, the planner skips the math and uses the observed frequency directly — which is why the depth of that list quietly decides plan quality on skewed columns. At target 100, only the hundred hottest values get measured; everything else falls back to the residual estimate above. A status column with three values never notices. A tenant column where the top hundred tenants hold 60 percent of the rows but tenant number 450 still owns half a percent — that tenant lands outside the list, gets a residual estimate that can be wildly low, and the plan for that tenant's queries is built on a fiction. Raising the target on that column deepens the list until the values you actually filter on are measured, not guessed.
-- What does the planner know about this column today?
SELECT tablename, attname, n_distinct, most_common_vals, most_common_freqs
FROM pg_stats
WHERE schemaname = 'public'
AND tablename = 'orders'
AND attname = 'merchant_id';The two pg_stats columns nobody reads
While you are in there, two more columns earn their keep. histogram_bounds drives range selectivity: the planner interpolates between bucket edges to estimate BETWEEN and inequality filters, so shallow histograms mean coarse range estimates on skewed distributions. correlation measures how closely physical row order matches logical value order. Near 1 or -1, an index range scan reads mostly adjacent heap pages and the planner prices it cheap; near 0 — random UUIDs are the poster child — the same range scan is priced as scattered random I/O and the planner may refuse the index on wide ranges entirely. Neither number is wrong at target 100, but both get sharper with a bigger sample, and sharper is what flips borderline plans.
Raising the target the surgical way
You can raise default_statistics_target globally, and I recommend you do not — at least not first. A global raise multiplies ANALYZE cost on every column of every table, including the text columns and timestamps that were fine, and it invalidates plans fleet-wide the next time autoanalyze runs. The surgical tool is per-column:
ALTER TABLE orders ALTER COLUMN merchant_id SET STATISTICS 1000;
ANALYZE orders;Target 1000 means a 300,000-row sample and up to a thousand most-common-value and histogram slots for that one column. Then verify the estimate actually moved before declaring victory: run EXPLAIN on the problem query and compare estimated rows against the actual rows from EXPLAIN ANALYZE. If the estimate is now sane and the plan still stinks, your problem was never the statistics — and you learned that with a metadata change, not a schema change.
When a bigger sample changes nothing
Spend the ANALYZE budget where sampling is the bottleneck. Uniform columns gain nothing from a deeper sample. Tiny tables are read nearly whole already, so their target is moot. And correlated columns are a different failure entirely — no per-column statistic can express that zip_code implies city, which is what extended statistics exist for; the notes on extended statistics for correlated columns own that topic. The honest cost side: on our 300-million-row events table, ANALYZE at the default ran in about eight seconds; at target 1000 on two columns it ran close to a minute, roughly linear in sampled rows plus the sort for the deeper histograms. Autoanalyze inherits the per-column setting, so a big target raises the price of every future autoanalyze on that table — one more reason to set it where it pays, not everywhere. For that autoanalyze half of the tradeoff, see the notes on autoanalyze and stale statistics.
Watching estimates with MonPG
Estimate drift is the slowest-moving disaster in PostgreSQL: same query, same data shape, plans quietly rotting as the data skews and the statistics stay stale. MonPG monitors PostgreSQL in production today, and its pg_stat_statements trends are how I catch the regression before the postmortem — same normalized query, mean time climbing after an analyze cycle is the fingerprint. The PostgreSQL monitoring surface covers what is collected, and the field guide to planner statistics and estimates pairs well with everything above. Set the target where it pays, verify in pg_stats, and stop treating 100 as a law of nature — it is a default, chosen when tables were smaller than your memory.