diff --git a/docs/audit/AUDIT-2026-06-15-full-repo.md b/docs/audit/AUDIT-2026-06-15-full-repo.md index ffd54f1..f831084 100644 --- a/docs/audit/AUDIT-2026-06-15-full-repo.md +++ b/docs/audit/AUDIT-2026-06-15-full-repo.md @@ -567,3 +567,50 @@ unwrapped at `ingest.py:183` — a Nougat 5xx aborts PDF ingest. **Closure:** every `codex/` module is deep-reviewed; the two runtime-confirmable HIGHs are empirically reproduced. The **audit / finding-identification pass is complete**. Remediation planning is the open next phase. + +--- + +# Migration Incident — 2026-06-15 (M-1 confirmed live, new privilege dimension) + +While running the C-7 re-ingest migration (`infra/reingest_canonical_ids.sh`), +M-1 manifested exactly as predicted — and exposed a dimension the static audit +did **not** capture. + +**What happened.** The helper ran `TRUNCATE papers CASCADE` (corpus wiped to 0), +then `ingest_all.sh` failed on the first chunk insert: +`UndefinedColumn: column "section" of relation "chunks" does not exist`. The live +`chunks` table predates the F-16 `section` column, and nothing had ever applied +that DDL to the live DB (M-1: `apply_schema` is never called; `schema.sql` is not +re-applyable). The ingest code was ahead of the live schema. + +**New finding — privilege separation (extends M-1).** The fix +`ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT` then failed with +`InsufficientPrivilege: must be owner of table chunks`. The live ownership: + +| table | owner | +|-------|-------| +| papers, chunks, citations, code_links, formulas, figures | `postgres` | +| paper_identifiers | `researcher` (app user) | + +The app role `researcher` has DML + `TRUNCATE` but **not** DDL on the +postgres-owned core tables. So a `codex migrate` / `apply_schema` invoked with the +application's `DATABASE_URL` would fail on every `ALTER` / `CREATE INDEX` against +those tables. **Implication for the M-1 fix:** making `schema.sql` idempotent is +necessary but *not sufficient* — the migration must run under a privileged role +(owner/superuser), via a separate admin connection, not the least-privilege app +user. This is now part of M-1's remediation scope. + +**Consequences observed.** +- Corpus left wiped (papers = 0) until the owner adds the column and re-ingest is + re-run — a destructive-then-fail sequence. +- Helper hardened (`fix(infra)` on `fix/audit-wave-2`): a **preflight** verifies + `chunks.section` exists *before* the `TRUNCATE`, aborting early with the + owner-level `ALTER` to run, so a schema gap can no longer cost the corpus. + +**Unblock (owner action):** +`sudo -u postgres psql -d papers -c "ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT;"` +then re-run `bash ingest_all.sh`. + +This incident is the strongest possible validation of M-1 and should anchor its +Wave-3 fix: **idempotent `schema.sql` + a `codex migrate` command that connects as +a privileged role**, plus a documented owner/grant model for the app user.