docs+chore: audit findings + tooling (base of the remediation stack) #15

user2595 · 2026-06-15T20:51:49Z

user2595 commented

2026-06-15 20:51:49 +00:00

Foundation of the audit-remediation stack

This is the base the three remediation PRs build on:
main ← this ← #12 ← #13 ← #14.
It carries no production-code changes — only the audit record + rescue/tooling.

Audit findings — docs/audit/AUDIT-2026-06-15-loop-1.md (F-15 / dev-loop-1)
and AUDIT-2026-06-15-full-repo.md (whole-repo, parts A–G + live-DB & runtime
validation + the M-1 migration-incident addendum). 45 findings, finding-ID
register continued across both.
Data-quality handoff — docs/audit/DATA-QUALITY-2026-06-15.md: the second
audit axis (is the information good?), self-contained for a fresh session
(DQ-1…DQ-4; DQ-1 = 12/29 papers have zero citations).
Rescue / hygiene — ruff format normalization to line-length=100;
.gitignore hardened to ignore .env.* (keep .env.example) so DB-credential
profiles can never be staged (audit S-1); ingest_all.sh batch-ingest helper.

Merge order

Merge this first, then #12 → #13 → #14 (each is stacked on the previous and shows
only its own diff).

🤖 Generated with Claude Code

## Foundation of the audit-remediation stack This is the **base** the three remediation PRs build on: `main` ← **this** ← [#12](../pulls/12) ← [#13](../pulls/13) ← [#14](../pulls/14). It carries no production-code changes — only the audit record + rescue/tooling. ### Contents - **Audit findings** — `docs/audit/AUDIT-2026-06-15-loop-1.md` (F-15 / dev-loop-1) and `AUDIT-2026-06-15-full-repo.md` (whole-repo, parts A–G + live-DB & runtime validation + the M-1 migration-incident addendum). 45 findings, finding-ID register continued across both. - **Data-quality handoff** — `docs/audit/DATA-QUALITY-2026-06-15.md`: the *second* audit axis (is the information good?), self-contained for a fresh session (DQ-1…DQ-4; DQ-1 = 12/29 papers have zero citations). - **Rescue / hygiene** — `ruff format` normalization to line-length=100; `.gitignore` hardened to ignore `.env.*` (keep `.env.example`) so DB-credential profiles can never be staged (audit S-1); `ingest_all.sh` batch-ingest helper. ### Merge order Merge this first, then #12 → #13 → #14 (each is stacked on the previous and shows only its own diff). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

user2595 added 9 commits 2026-06-15 20:51:50 +00:00

style: ruff format normalization to line-length=100 9726042964

Committed code predated the line-length=100 ruff config; this brings the
five drifted files into compliance. No logic change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore(ops): ignore .env.* profiles + add batch ingest script 0c89eebeb8

- .gitignore: ignore .env.* (keep .env.example) so local DB-credential
  profiles like .env.jetson-ingest can never be staged (audit S-1).
- ingest_all.sh: batch-ingest helper writing to the Jetson DB via SSH tunnel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): record development-loop-1 findings a0ad069b1d

Audit of main @5dfa6e4: 4 HIGH (S-1 secret-in-tree, M-1 missing migration
path, C-1 unpropagated ID-mismatch fix in discover.py, D-1 stale ADR-F15
validation, T-1 untested resolution SQL) plus MED/LOW findings. Findings
only; remediation planning deferred to the next loop phase.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): whole-repo audit parts A-C (grounding, sources, parsing) 7682ff112f

Closes request (a) — the second pass over loop-1 §8 open items:
- Part A grounding guard: C-4 (HIGH, zero-claim pages bypass quarantine),
  C-5/R-1 MED, C-6/R-2/R-3 LOW.
- Part B sources/: C-7 (HIGH, papers.id stored in OpenAlex URL form vs the
  bare-ID contract), T-3 MED, R-4/R-5/C-9 LOW.
- Part C parsing/: C-10 MED (formulas/figures key on filename not paper.id),
  R-6/R-7 LOW — all latent (live corpus is .txt).

Running tally: 30 findings, 7 HIGH. Common root: un-normalized identity.
Parts D-G (synthesis/wiki/embed/cli) still open.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): complete whole-repo audit (parts D-G + exec summary) 2d1b0b5e23

- Part D synthesis: C-11 HIGH (lead-ID collision → silent overwrite), R-8 MED, D-4 LOW.
- Part E wiki-rest: R-9 MED (cross-ref injection corrupts LaTeX math), C-12/C-13/R-10/R-11 LOW.
- Part F embed/quality/models: C-14/R-12 LOW + C-7 corroboration. Cleanest modules.
- Part G cli/config: C-15 MED (embedding_dim vs hardcoded vector(1024)), R-13/S-4/R-14/D-5 LOW.

Executive summary: 45 findings total (8 HIGH, 11 MED, 26 LOW). Five root themes;
dominant one is un-normalized identity (C-1/C-7/C-10/M-1). Whole repo reviewed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): live-DB verification — confirm C-7 & C-1, refine M-1 33fb91d478

Read-only queries vs the live Jetson corpus (papers=36, citations=811):
- C-7 CONFIRMED: 35/36 papers.id are DOI-URL form (https://doi.org/...),
  incl. arXiv papers as arXiv-DOI URLs. Bare-ID lookups fail for 35/36.
- C-1 CONFIRMED: discover.py mis-flags 13 ingested papers as dangling;
  graph.py correctly resolves 57 cross-cite edges → the two paths diverge.
- M-1 REFINED HIGH→MED: paper_identifiers exists (39 rows), so no live crash;
  structural migration-tooling gap remains.
- C-2 partially refuted: 39 alias rows > 35 → ~4 diverging aliases, not inert.

Adjusted tally: 7 HIGH, 12 MED, 26 LOW.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): runtime-validate C-4 & C-11, close nougat.py review 8af9f9ad0a

Exercised the two runtime-confirmable HIGHs against real code paths (injected
fake LLM, no Ollama/DB):
- C-11 CONFIRMED: 4 distinct leads -> 2 files; connection+gap L-0001 silently
  overwritten by improvement L-0001 (2/4 leads lost).
- C-4 CONFIRMED: zero-claim (uncited) page published to wiki/ + marked compiled,
  not quarantined (grounding_rate=0.0 < 0.5 but total_claims>0 guard skips it).
- nougat.py now fully reviewed -> folds into R-6 (only retries ConnectError,
  wait_fixed(0), unwrapped in ingest). No new finding.

Audit/finding-identification pass complete; remediation planning still open.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): record M-1 migration incident + privilege dimension b4cf924bf1

The C-7 re-ingest hit M-1 live: TRUNCATE succeeded, then ingest failed on the
missing chunks.section column (schema.sql never applied to the live DB). Adding
the column then failed with 'must be owner' — the app user 'researcher' has DML
+ TRUNCATE but not DDL on the postgres-owned core tables. M-1's fix therefore
needs a privileged migration role, not just an idempotent schema. Records the
unblock and the helper's new preflight.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(audit): data-quality findings + handoff for a fresh session 64e3ee188c

Second audit axis (data, not code): is the information in the DB actually good?
Baseline scan of the live 29-paper corpus + four open investigation items
prepared self-contained for a cold session.

- Good: chunks 0/1507 fail the F-16 re-gate, healthy sizing, 0 dups.
- DQ-1 (HIGH): 12/29 papers have zero citations; 11 of them have an openalex_id,
  so OpenAlex returned empty referenced_works (source-coverage limit, not a bug)
  — propose an S2 references supplement.
- DQ-2 (MED): 3 papers no abstract; 1911.00966 fully metadata-less (OpenAlex 404).
- DQ-3 (MED): chunk-vs-source fidelity not yet verified.
- DQ-4 (MED): retrieval relevance not yet measured.

Includes connection steps, exact ids, priorities, and a reproduce appendix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

user2595 merged commit fb06c6f65a into main

2026-06-16 04:52:59 +00:00

user2595 referenced this issue from a commit

2026-06-16 04:53:00 +00:00

Merge pull request 'docs+chore: audit findings + tooling (base of the remediation stack)' (#15) from audit/loop-1 into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: user2595/codex-py#15