docs+chore: audit findings + tooling (base of the remediation stack) #15
@@ -567,3 +567,50 @@ unwrapped at `ingest.py:183` — a Nougat 5xx aborts PDF ingest.
|
||||
**Closure:** every `codex/` module is deep-reviewed; the two runtime-confirmable
|
||||
HIGHs are empirically reproduced. The **audit / finding-identification pass is
|
||||
complete**. Remediation planning is the open next phase.
|
||||
|
||||
---
|
||||
|
||||
# Migration Incident — 2026-06-15 (M-1 confirmed live, new privilege dimension)
|
||||
|
||||
While running the C-7 re-ingest migration (`infra/reingest_canonical_ids.sh`),
|
||||
M-1 manifested exactly as predicted — and exposed a dimension the static audit
|
||||
did **not** capture.
|
||||
|
||||
**What happened.** The helper ran `TRUNCATE papers CASCADE` (corpus wiped to 0),
|
||||
then `ingest_all.sh` failed on the first chunk insert:
|
||||
`UndefinedColumn: column "section" of relation "chunks" does not exist`. The live
|
||||
`chunks` table predates the F-16 `section` column, and nothing had ever applied
|
||||
that DDL to the live DB (M-1: `apply_schema` is never called; `schema.sql` is not
|
||||
re-applyable). The ingest code was ahead of the live schema.
|
||||
|
||||
**New finding — privilege separation (extends M-1).** The fix
|
||||
`ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT` then failed with
|
||||
`InsufficientPrivilege: must be owner of table chunks`. The live ownership:
|
||||
|
||||
| table | owner |
|
||||
|-------|-------|
|
||||
| papers, chunks, citations, code_links, formulas, figures | `postgres` |
|
||||
| paper_identifiers | `researcher` (app user) |
|
||||
|
||||
The app role `researcher` has DML + `TRUNCATE` but **not** DDL on the
|
||||
postgres-owned core tables. So a `codex migrate` / `apply_schema` invoked with the
|
||||
application's `DATABASE_URL` would fail on every `ALTER` / `CREATE INDEX` against
|
||||
those tables. **Implication for the M-1 fix:** making `schema.sql` idempotent is
|
||||
necessary but *not sufficient* — the migration must run under a privileged role
|
||||
(owner/superuser), via a separate admin connection, not the least-privilege app
|
||||
user. This is now part of M-1's remediation scope.
|
||||
|
||||
**Consequences observed.**
|
||||
- Corpus left wiped (papers = 0) until the owner adds the column and re-ingest is
|
||||
re-run — a destructive-then-fail sequence.
|
||||
- Helper hardened (`fix(infra)` on `fix/audit-wave-2`): a **preflight** verifies
|
||||
`chunks.section` exists *before* the `TRUNCATE`, aborting early with the
|
||||
owner-level `ALTER` to run, so a schema gap can no longer cost the corpus.
|
||||
|
||||
**Unblock (owner action):**
|
||||
`sudo -u postgres psql -d papers -c "ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT;"`
|
||||
then re-run `bash ingest_all.sh`.
|
||||
|
||||
This incident is the strongest possible validation of M-1 and should anchor its
|
||||
Wave-3 fix: **idempotent `schema.sql` + a `codex migrate` command that connects as
|
||||
a privileged role**, plus a documented owner/grant model for the app user.
|
||||
|
||||
Reference in New Issue
Block a user