codex-py/docs/audit/AUDIT-2026-06-15-loop-1.md

AUDIT — Development Loop 1 (main branch)

Auditor: Audit-Loop (Opus) Date: 2026-06-15 Scope: entire main branch (HEAD 5dfa6e4) Method: static review of source + schema + tests, branch-wide pattern scan (ruff, mypy, defect-class greps), full test run (330 passed).

Status legend — HIGH = fix before relying on the feature · MED = fix soon, correctness/maintainability · LOW = polish · INFO = accepted risk / no action, documented for the record.

This document only records findings. Remediation planning is a separate step (next phase of the audit loop).

---

0. Headline

The most recent dev-loop work (F-15 citation graph + the follow-up fixes cited_id→papers.id resolution and the paper_identifiers alias table) is functionally green but unverified where it matters: the SQL that is the fix has zero real-database coverage, the live-corpus validation in the ADR predates the fix, and the same ID-mismatch bug the fix solves still lives in a parallel code path (discover.py). Separately, a live DB password sits in an un-ignored file and the new table has no migration path onto the live DB.

Test suite: 330 passed, ruff check clean, ruff format --check clean, mypy clean on reviewed modules.

---

1. Security

S-1 — Live DB credential in an un-ignored file · HIGH

• Evidence: .env.jetson-ingest contains DATABASE_URL=postgresql://researcher:05761b87…@localhost:5433/papers. .gitignore ignores only the literal .env (line: # Environment secrets — NEVER commit → .env). git check-ignore .env.jetson-ingest → NOT IGNORED.
• Impact: one git add -A && git commit leaks a real Postgres password to history. The file is currently untracked (??), so the leak has not happened yet — this is a near-miss, not an incident.
• Note: the same gap exposes any future .env.<profile> file.

S-2 — Non-constant-time token comparison (MCP HTTP) · LOW

• Evidence: codex/mcp_server.py:340 — if auth != f"Bearer {token}":.
• Impact: theoretical timing side-channel on the Bearer token. Low on a trusted LAN, but trivially fixable with secrets.compare_digest.

S-3 — DNS-rebinding protection disabled · INFO (accepted)

• Evidence: codex/mcp_server.py:35-37, TransportSecuritySettings(enable_dns_rebinding_protection=False) (commit d29dcf2).
• Assessment: documented and mitigated — HTTP transport requires a Bearer token (_TokenAuth) and is firewalled to LAN IPs (UFW). A rebound browser request lacks the token. Residual risk only if the firewall assumption fails. No action; recorded so the trade-off stays visible.

---

2. Correctness

C-1 — discover.py still has the ID-format-mismatch bug F-15 fixed · HIGH

• Evidence: codex/discover.py:20 — WHERE cited_id NOT IN (SELECT id FROM papers); same pattern in cocited_papers (discover.py:52-61), citing_papers, cited_by.
• Root cause: citations.cited_id is a mix of OpenAlex W-IDs (OpenAlex API path, ingest.py:229-231 ← sources/openalex.py:155) and DOIs/arXiv (GROBID/S2 paths). papers.id is the canonical DOI/arXiv. An ingested paper cited by its OpenAlex ID is not in (SELECT id FROM papers), so it is wrongly reported as a discovery lead.
• Impact: codex discover leads, codex discover cocited, and the MCP discover_leads tool over-report — foundational papers the user has already ingested show up as "cited but not ingested". This is exactly the D-05-class issue the F-15 graph path resolves via the papers.openalex_id JOIN (graph.py:48-56) — but the fix was not propagated to discover.py. Two divergent "dangling" implementations now disagree on what "ingested" means.

C-2 — paper_identifiers JOIN is effectively inert · MED

• Evidence: graph.py:50-56 second LEFT JOIN; populated only at ingest.py:153-161, which inserts the same paper.openalex_id that papers.openalex_id already holds.
• Impact: because ingest never records an alias that differs from papers.openalex_id, the pi JOIN can only resolve what the p JOIN already resolves. It adds value only in the narrow case where a paper's openalex_id changes across re-ingests (old alias retained via ON CONFLICT DO NOTHING). The table + unique index + seed INSERT + migration burden buy near-zero behaviour today — speculative generality. No test exercises an alias that diverges from papers.openalex_id.

C-3 — MCP wiki_read returns the slug as its own "sources" · MED

• Evidence: codex/mcp_server.py:138-149 — comment says "Collect cited bibkeys", code sets sources = [concept_slug].
• Impact: the sources field is a placeholder masquerading as data; any MCP client trusting it to list cited bibkeys gets the concept's own slug.

---

3. Documentation / Code Drift

D-1 — ADR-F15 validation predates the shipped graph · HIGH (integrity)

• Evidence: ADR-F15 (commit a1a6f45) lands before the resolution fixes 664906f (cited_id→papers.id) and 5dfa6e4 (paper_identifiers).
• Stale claims:
  • "Spike Result": 478 dangling nodes, specific PageRank scores, Bobenko/Springborn rank 7, 5.7 % spread — all measured on the pre-resolution (bipartite DOI→OpenAlex) graph. Post-fix, in-KB cited papers resolve to DOIs, so dangling count and PageRank distribution change.
  • "Decision → Graph Construction": "inter-KB citations are structurally invisible" — now false.
  • "Known Limitation: ID-Format Mismatch": "cross-citations… not represented" — now resolved by the code, contradicting the section.
• Impact: the GO decision rests on numbers the shipping code no longer produces. Validation must be re-run on the post-fix graph (or the ADR marked validation-pending).

D-2 — graph_cite_boost_alpha doc describes the fixed bug · MED

• Evidence: config.py:296-300 describes Final score = dense_score * (1 + alpha * pagerank_score) — a multiply. Actual code (cli.py:107) divides distance: boosted = distance / (1.0 + alpha * pr_score). The ADR explicitly states the multiply form was the "critical bug corrected during the F-15 review gate".
• Impact: the config docstring still documents the inverted (buggy) formula.

D-3 — paper_identifiers tagged "F-17" in schema, shipped as F-15 fix · LOW

• Evidence: infra/schema.sql:129 comment "F-17 Paper identifier aliases", but the table is introduced by the F-15 graph follow-up (5dfa6e4) and the ADR-F15 work. Feature-tag inconsistency; ADR-F15 doesn't mention the table at all.

---

4. Schema / Migration

M-1 — New table has no migration path onto the live DB · HIGH (operational)

• Evidence chain:
  1. apply_schema docstring (db.py:46-49) claims "all DDL statements use CREATE … IF NOT EXISTS" — false: papers, chunks, citations, code_links and their indexes use bare CREATE TABLE/INDEX (schema.sql:16,33,56,68,30,41,…). Only F-09/F-16/F-17 DDL is idempotent.
  2. apply_schema is never called anywhere in the app — no init/migrate CLI command (grep apply_schema → only db.py + a mocked unit test).
  3. ingest_all.sh does not apply the schema; it only runs codex ingest.
• Impact: re-running infra/schema.sql against the existing Jetson DB fails on the first CREATE TABLE papers (already exists) and rolls back the whole script in one implicit transaction — so the new paper_identifiers table (and its seed INSERT) is never created on a DB that predates it. build_citation_graph then fails with relation "paper_identifiers" does not exist on the live corpus. The migration step is currently manual and undocumented.

---

5. Test Integrity

T-1 — The F-15 resolution SQL has zero real coverage · HIGH

• Evidence: every test in tests/graph/test_graph.py and tests/graph/test_cli.py builds the graph from a MagicMock conn (_make_conn, _make_conn_with_paper_ids). test_cross_citation_uses_doi_when_in_kb (test_graph.py:83-99) pre-resolves the rows in Python and asserts the graph builds — it tests the mock, not the COALESCE(p.id, pi.paper_id, …) + two LEFT JOINs that are the fix.
• Impact: the substance of the two most recent commits is unverified. 330 green tests give false confidence about the resolution logic, the p.id IS NULL correlated guard, and the paper_identifiers unique-index behaviour. Needs an integration test against a real (or pytest-managed) Postgres.

T-2 — apply_schema idempotency untested · MED

• Evidence: tests/scaffold/test_db.py:24-32 mocks the conn and only checks execute/commit are called. The (false) idempotency claim in M-1 is never exercised.

---

6. Design / Usability

U-1 — graph report prints bare IDs, no titles · LOW

• cli.py:582-588 emits score <id> and bare dangling IDs — a mix of DOIs and W… OpenAlex IDs with no title/year. The ADR's readable author/title table is not what the CLI produces; output is hard to interpret.

U-2 — --cite-boost can only reorder within the top-limit page · LOW

• cli.py:84-94 applies ORDER BY … LIMIT limit in SQL before the boost (cli.py:96-109). A high-PR paper just outside the semantic cut can never be pulled in. With α=0.3 and PR≈0.002 the effect is ~0.06 % — effectively a within-page tie-breaker. Consistent with ADR intent, but worth stating as a limitation (the flag does far less than "weight results by PageRank" implies).

U-3 — graph report --json drops the small-corpus warning · LOW

• cli.py:557-570 returns before the graph_min_corpus_size warning (cli.py:572-577). JSON consumers get neither the warning nor a flag for it.

U-4 — graph.find_co_cited implemented + tested but not wired to a CLI · LOW

• graph.find_co_cited (graph.py:142) is unit-tested but exposed by no graph sub-command (only the separate SQL-based discover cocited is wired). ADR-F15 R-43's co-citation is only half-surfaced.

---

7. State of Tree (INFO — no defect)

• The uncommitted diff (wiki.py, synthesis.py, semanticscholar.py, tests/synthesis/test_cli.py, tests/wiki/test_compile.py) is a legitimate ruff format normalization to line-length=100. ruff format --check reports all 25 files formatted with these changes applied, meaning the committed code was non-compliant. These edits should be committed, not reverted. (Not a finding; clarifies the working-tree churn.)

---

8. Coverage of this audit (honesty ledger)

Deep-reviewed: graph.py, db.py, infra/schema.sql, ingest.py, discover.py, provenance.py (scan), mcp_server.py, cli.py (search + graph), config.py (graph settings), F-15 tests, ADR-F15; branch-wide pattern scan (SQL injection, bare except, eval/exec/subprocess, secrets, timeouts).

Not yet deep-reviewed (no findings asserted):

• synthesis.py grounding guard / claim parser (_run_grounding_guard, _parse_claims) — the "zero-fact-leak (N-02)" core. Prompts reviewed; guard logic not yet traced.
• wiki.py grounding guard + conflict detection (only the format diff seen).
• sources/ (arxiv, openalex, semanticscholar) beyond citation shapes.
• parsing/ (grobid, nougat, mathpix, figures, tex) — error handling in mathpix.py/figures.py uses broad except Exception without noqa (mathpix.py:109,151, figures.py:125,152) — flagged for a later pass.
• embed.py, quality.py, models.py.

A second pass should close these before the audit is declared complete.