The plan was textbook: add created_by and updated_by audit columns to the customers table as INVISIBLE — a feature MariaDB has shipped since 10.3.3 — so that no application code, no ORM, and no SELECT * anywhere would notice them while we rolled out the writer changes over two weeks. It worked exactly as designed, which is precisely what hurt. The nightly reconciliation loader, a ten-year-old script that ended its run with INSERT INTO customers_archive SELECT * FROM customers joined to a VALUES-list insert, started failing at 02:40 with ERROR 1136 (21S01): Column count doesn't match value count at row 1. The archive table did not have the new columns; the live table did — and invisible columns count fully for INSERT statements that omit the column list. Hiding the columns from readers had been effortless. Hiding them from writers was impossible, and the loader was a writer.
Invisible columns are the best schema-evolution tool MariaDB has that MySQL users only got in 8.0, and they are routinely misunderstood as "a column nothing can see." The truth is narrower: invisibility is a default projection behavior, not a permission, and the operations that break are not the ones you expect. Here is the exact semantics, the incident classes I have cataloged, and the rollout pattern that now lets us add columns to hot tables without a maintenance window.
What exactly does INVISIBLE hide — and from whom?
An invisible column is excluded from SELECT * and from the implicit column list of an INSERT without named columns — and from nothing else. The column is fully present in storage, fully indexed if you index it, fully visible to anyone who names it, and fully counted by every statement that reasons about column count instead of column names. You can read it the moment you ask for it by name:
-- add an audit column nothing sees by default
ALTER TABLE customers
ADD COLUMN updated_by VARCHAR(64) NULL INVISIBLE;
SELECT * FROM customers LIMIT 1; -- updated_by absent
SELECT id, email, updated_by FROM customers LIMIT 1; -- there it is
-- what the information_schema reports
SELECT COLUMN_NAME, EXTRA, IS_NULLABLE, COLUMN_DEFAULT
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'app' AND TABLE_NAME = 'customers'
ORDER BY ORDINAL_POSITION; -- EXTRA shows INVISIBLE
-- the trap: column count includes invisible columns
INSERT INTO customers_archive VALUES (?); -- ERROR 1136 if counts differTwo properties follow that people trip over in opposite directions. First, because SELECT * drops invisible columns, a SELECT * INTO or INSERT ... SELECT * between tables with different invisibility sets silently misaligns or fails — our loader incident, exactly. Second, because the column is genuinely there, tooling that diffs schemas by SELECT * output will report the table as unchanged while the physical row format already carries the new field; our schema-drift detector missed the columns entirely until we taught it to diff information_schema instead of result sets. A table must also always have at least one visible column, so you cannot build a table that is entirely hidden, and INVISIBLE has no effect on replication — the column replicates like any other, in row-based and statement-based modes alike.
Why does INSERT without a column list still count invisible columns?
Because invisibility is defined at the projection layer, not the storage layer: an INSERT without named columns must supply values for every column in the table, visible or not, unless the invisible column is nullable or has a DEFAULT. This asymmetry is deliberate — MariaDB chose to make SELECT * safe while keeping INSERT ... VALUES honest about the physical row — and it produces the failure taxonomy worth memorizing. ERROR 1136, the column-count mismatch, is what you get when the receiving table lacks the columns, like our archive loader. The quieter variant is the opposite misalignment: an INSERT INTO t VALUES (a, b, c) against a table that gained an invisible column succeeds if the new column has a default, and now your positional values have shifted — value c lands in the wrong column if the invisible column was added anywhere but last. Always append invisible columns at the end of the table, and treat any positional INSERT in the codebase as a bug to fix during the rollout, not after.
The ORM layer deserves its own warning. Frameworks that generate INSERTs with explicit column lists — which is most of them, most of the time — are immune. Framework features that reflect the table structure to build statements, and any code doing INSERT ... SELECT *, are not. Before we add an invisible column now, we grep the application and job repos for INSERT statements lacking a column list on the target table; the count is never zero. Dumps behave themselves, at least: mariadb-dump emits explicit column lists and includes invisible columns in the output, so logical backups and restores are unaffected — one less thing to rehearse in the restore drills from the mariabackup operational guide.
How do invisible columns compare to generated columns for rollouts?
They solve different halves of the schema-evolution problem, and the combination is stronger than either. Invisible columns hide storage you are not ready to expose; virtual and persistent generated columns derive values without changing how the row is written at all — the indexing trade-offs of which I covered in the virtual and persistent columns notes. The pattern that has survived three major rollouts for us: add the new physical column INVISIBLE and NULL-able with a DEFAULT, deploy writers that populate it by name, backfill in chunks, validate with explicit-column queries, and only then ALTER the column VISIBLE as the final, separate deploy. Each step is independently reversible, and the VISIBLE flip is a metadata-only change with no table rebuild. What invisible columns do not give you is a way to change existing columns: for that, the instant-DDL machinery from the instant add column notes is the right tool, and the two compose cleanly — an instant ADD ... INVISIBLE on a 400-million-row table takes milliseconds, because it is the same metadata-only path.
The one genuinely new capability invisible columns unlocked beyond rollouts is the soft primary key pattern: a legacy table with no natural key gets an invisible AUTO_INCREMENT column, giving it a stable row identity for replication and tooling without changing any SELECT * behavior the legacy code depends on. We used this to move a 1990s-era table onto row-based replication without touching the application that owned it. It is the closest thing MariaDB has to a free lunch, and it costs one ALTER TABLE.
Where MonPG fits
The signals worth trending around schema evolution are the failure counts and the drift: ERROR 1136 rates after any column addition, positional INSERTs found by static audit trending to zero, and a schema inventory that diffs information_schema — including the INVISIBLE flag — rather than SELECT * output. Full disclosure, as in every article of this series: I work on MonPG, which monitors PostgreSQL in production today and does not monitor MariaDB yet. MariaDB support is coming soon and in active development — the /mariadb-monitoring page tracks where it stands — and post-deploy error-rate spikes are exactly the signal it is being built to surface. Until that ships, the audit queries above are your kit. If PostgreSQL is also in your fleet, that monitoring is live today — see the PostgreSQL overview, or browse more field notes on the blog.