docs+chore: audit findings + tooling (base of the remediation stack) #15
@@ -483,10 +483,11 @@ cluster into five root themes, not random scatter.
|
||||
- **Then:** MED batch (R-1, R-8, R-9, C-15, C-10) and the LOW polish.
|
||||
|
||||
### Coverage honesty
|
||||
Every `codex/` module was read. Not deep-reviewed: `nougat.py` (54-line `.pdf`
|
||||
wrapper, skimmed), and the exact runtime DB state (URL-form ids in C-7, and the
|
||||
`paper_identifiers` existence in M-1) is **inferred from code** and flagged
|
||||
"confirm on the Jetson DB" — I could not query the live database from here.
|
||||
Every `codex/` module was read. After the follow-up pass **all are deep-reviewed**
|
||||
(`nougat.py` included — see Runtime Validation). Runtime state that was initially
|
||||
*inferred from code* (C-7 id-form, M-1 table existence, and the C-4/C-11
|
||||
behaviours) has since been **empirically confirmed** — see the *Live DB
|
||||
Verification* and *Runtime Validation* sections below.
|
||||
|
||||
---
|
||||
|
||||
@@ -535,3 +536,34 @@ HIGH **8 → 7** (M-1 → MED). C-7 and C-1 are now **confirmed**, not inferred
|
||||
should lead remediation. The dominant root theme (**un-normalized identity**) is
|
||||
empirically validated: 35/36 ids in the "wrong" form, 13 papers mis-classified by
|
||||
the older code path, 57 edges rescued by the newer one.
|
||||
|
||||
---
|
||||
|
||||
# Runtime Validation — 2026-06-15 (C-4 and C-11 reproduced end-to-end)
|
||||
|
||||
The two HIGH findings flagged "code-certain, runtime-confirmable" were exercised
|
||||
against the **real** code paths (injected fake LLM; no Ollama, no DB needed).
|
||||
|
||||
### C-11 — CONFIRMED ✅ (lead-ID collision → silent loss; stays HIGH)
|
||||
Reproduced the exact CLI assembly (`cli.py:421` `connections + gaps +
|
||||
improvements`, each stage numbered from `L-0001`): **4 distinct leads** (2
|
||||
connection, 1 gap, 1 improvement) → `write_leads` produced **2 files**.
|
||||
`grounded/L-0001.md` survived as the **improvement** ("IMPROVEMENT-one"); the
|
||||
connection and gap `L-0001` were silently overwritten. **2 of 4 leads lost.**
|
||||
|
||||
### C-4 — CONFIRMED ✅ (zero-claim page bypasses quarantine; stays HIGH)
|
||||
Drove the real `compile_all` with a fake LLM returning confident **uncited** prose
|
||||
(→ 0 parseable claims). With `wiki_min_grounding_rate = 0.5` and an actual
|
||||
grounding rate of `0.0`, the page was **published to `wiki/` and marked
|
||||
compiled** (`report.compiled=['test-concept']`, `report.quarantined=[]`) — not
|
||||
quarantined to `wiki/draft/`. The `total_claims > 0` guard (`wiki.py:730`) is the
|
||||
exact bypass; the bug fires precisely when grounding evidence is weakest.
|
||||
|
||||
### `nougat.py` — now fully reviewed (no new finding)
|
||||
Folds into **R-6** (PDF-path resilience). Detail: `pdf_to_markdown` retries **only**
|
||||
`httpx.ConnectError` with `wait_fixed(0)` (no 5xx retry, no back-off) and is
|
||||
unwrapped at `ingest.py:183` — a Nougat 5xx aborts PDF ingest.
|
||||
|
||||
**Closure:** every `codex/` module is deep-reviewed; the two runtime-confirmable
|
||||
HIGHs are empirically reproduced. The **audit / finding-identification pass is
|
||||
complete**. Remediation planning is the open next phase.
|
||||
|
||||
Reference in New Issue
Block a user