user2595/codex-py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(infra): preflight schema check before TRUNCATE in migration helper

Browse Source 
The helper TRUNCATEd papers CASCADE and only then ran ingest_all.sh, which
failed on the missing chunks.section column (audit M-1) — leaving the corpus
wiped. Add a preflight that verifies chunks.section exists BEFORE the
destructive step and aborts early (no TRUNCATE) with the owner-level ALTER to
run, since the app user cannot alter postgres-owned tables.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-15 15:37:35 +02:00
parent 1ff8052b69
commit 9f172f5c68
1 changed files with 29 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
infra/reingest_canonical_ids.sh
View File

@@ -58,6 +58,35 @@ run_sql "SELECT count(*) AS papers, count(*) FILTER (WHERE starts_with(id, 'http
echo " sample ids:" echo " sample ids:"
run_sql "SELECT id FROM papers ORDER BY added_at LIMIT 3" run_sql "SELECT id FROM papers ORDER BY added_at LIMIT 3"
# ── 2b. Preflight: verify the schema is re-ingestable BEFORE wiping anything ──
# ingest writes chunks.section (F-16). On a live DB that predates it the column
# is missing, and the app user cannot ALTER tables owned by 'postgres' (audit
# M-1). Check FIRST so we never TRUNCATE and then fail mid-re-ingest.
echo "── Preflight: schema readiness ───────────────────────────────"
SECTION_OK="$("$PY" - <<'PYEOF'
import os, psycopg
try:
with psycopg.connect(os.environ["DATABASE_URL"], connect_timeout=10) as c:
cols = [r[0] for r in c.execute(
"SELECT column_name FROM information_schema.columns WHERE table_name='chunks'").fetchall()]
print("yes" if "section" in cols else "no")
except Exception:
print("error")
PYEOF
)"
if [[ "$SECTION_OK" != "yes" ]]; then
echo "ABORT — schema not ready: chunks.section is missing (audit M-1)."
echo "The app user cannot add it (chunks is owned by 'postgres'). Apply it as"
echo "the table owner, then re-run this script:"
echo ""
echo " ssh alfred@192.168.178.103 \\"
echo " \"sudo -u postgres psql -d papers -c \\\\\"ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT;\\\\\"\""
echo ""
echo "Nothing was changed — no TRUNCATE was issued."
exit 1
fi
echo " chunks.section present — schema OK."
# ── 3. Confirmation gate (destructive) ─────────────────────────────────────── # ── 3. Confirmation gate (destructive) ───────────────────────────────────────
echo "" echo ""
echo "This TRUNCATEs papers CASCADE (papers/chunks/citations/formulas/figures/" echo "This TRUNCATEs papers CASCADE (papers/chunks/citations/formulas/figures/"