Second audit axis (data, not code): is the information in the DB actually good?
Baseline scan of the live 29-paper corpus + four open investigation items
prepared self-contained for a cold session.
- Good: chunks 0/1507 fail the F-16 re-gate, healthy sizing, 0 dups.
- DQ-1 (HIGH): 12/29 papers have zero citations; 11 of them have an openalex_id,
so OpenAlex returned empty referenced_works (source-coverage limit, not a bug)
— propose an S2 references supplement.
- DQ-2 (MED): 3 papers no abstract; 1911.00966 fully metadata-less (OpenAlex 404).
- DQ-3 (MED): chunk-vs-source fidelity not yet verified.
- DQ-4 (MED): retrieval relevance not yet measured.
Includes connection steps, exact ids, priorities, and a reproduce appendix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per review: the two latent upsert conflicts should be observable even where not
auto-fixed.
- bibkey collision (auto-suffixed) now logs a warning so a paper landing as
'BobenkoLutz2023a' instead of '…2023' isn't silent.
- openalex_id collision (two papers claiming the same OpenAlex work — a real data
conflict, not auto-renamable) now raises a clear ValueError naming the
openalex_id and the offending paper, instead of the raw psycopg UniqueViolation.
Regression test added for the openalex_id error path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-ran the citation graph after the C-7 re-ingest (29/29 papers canonical/bare):
489 nodes, 590 edges, 472 dangling, 5.4% spread. The resolution fix's signature:
Pinkall/Polthier 1993 resolves to its DOI and is now the rank-1 *in-KB* hub
(was a dangling OpenAlex node); Bobenko/Springborn 2007 holds rank 7. Foundational
hubs still surface — GO re-affirmed on the shipping topology. Updates the ADR
spike table with before/after numbers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The id-keyed upsert (ON CONFLICT (id)) does not catch the papers.bibkey UNIQUE
constraint, so a second paper whose auto-generated key matches an existing one
(two same-author/year works → e.g. 'BobenkoLutz2023') failed the whole ingest
with a UniqueViolation. Surfaced by the canonical-id re-ingest: 2305.10988
collided with 2310.17529.
ingest now catches a bibkey UniqueViolation, rolls back, and retries the upsert
with an a/b/c… suffix. Non-bibkey violations re-raise unchanged. Mocks don't
raise, so the existing happy-path tests are untouched; a new test drives the
collision-and-retry path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-14: embed.py header now states that only the dense path is wired (search
'hybrid' is dense + Postgres FTS); encode_sparse/encode are reserved for a
future sparse-retrieval layer, not yet consumed.
- U-2: --cite-boost help clarifies it re-ranks within the top results (a
tie-breaker), not a hard re-ranking — matching the small alpha effect.
Accepted (documented heuristics, no change): R-3 conflict detection is a
keyword-only signal labelled as such; R-12 classify_section is mostly 'body'
because chunks are word-windows that rarely start at a section header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-6: the content-5-gram check used 'gram in src' on space-joined strings, so the
first/last gram token could match a prefix/suffix of a longer chunk word
('set' inside 'subset'). Both gram and chunk are now space-padded, so a match
requires whole-token alignment. Regression test added.
- R-2: the reference-list filter's author pattern ('Surname, I.') also matched
'Theorem A.'/'Lemma B.', so theorem-dense chunks could be dropped as a
bibliography. The pattern now excludes common math-structure words.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- U-1: 'graph report' shows the paper title next to each in-KB hub (dangling
OpenAlex-id hubs are flagged), instead of bare ids.
- U-3: '--json' now includes a 'small_corpus_warning' field (null unless below
graph_min_corpus_size) instead of silently dropping the warning.
- C-9: new 'codex discover recommend <id>' wires semanticscholar.fetch_recommendations
(was implemented + tested but unreachable).
- U-4: new 'codex graph cocited <id>' wires graph.find_co_cited (likewise).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- R-4: arxiv.fetch_source had a try/except httpx.RequestError that only re-raised
— removed (no behaviour change, less noise).
- R-5: semanticscholar URLs now quote(paper_id, safe=':') so a legacy-arXiv '/'
(arXiv:math/0603097) doesn't corrupt the path and silently return no references.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
database_url, migration_database_url, mcp_auth_token and the mathpix app
id/key are now pydantic SecretStr, so they render as '**********' in any
repr/log/traceback instead of leaking the plaintext credential. Consumers
(db.get_conn, cli.migrate, mcp._require_http_token, mathpix.extract_formulas)
call .get_secret_value() at the point of use. Tests updated to the SecretStr
type.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The grounding guard tokenized with .lower().split() and split sentences on bare
.!?, so faithful claims were wrongly marked ungrounded: 'volume,' != 'volume',
and a decimal like 'V = 1.5' split into a 1-word fragment '5' that fell below
the 5-content-word floor.
- _content_words now strips edge sentence-punctuation (.,;:!?"') from tokens
while preserving math delimiters ()=+; claim and chunk are normalised
identically so matching stays consistent.
- _SENTENCE_SPLIT_RE splits on .!? only when followed by whitespace/end, so
decimals no longer create spurious sentence boundaries.
Regression test: a 5-word claim with a trailing comma now grounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-3 (MED): wiki_read returned 'sources' = [concept_slug] (a placeholder), not
the cited sources. Now parses the page's inline [BibKey #loc] citations and
returns the actual set of cited bibkeys.
- S-2 (LOW): _TokenAuth compared the Bearer header with != (length/equality
timing side-channel). Use secrets.compare_digest for constant-time comparison.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
find_gaps defaulted its probe topics to paper titles, so a title — which
retrieves mostly its own single bibkey — fell below synthesis_gap_min_coverage
and flagged almost every paper as a 'gap'. Coverage-map gaps now run only for
explicitly-requested topics; the CLI gains a repeatable --topic option. The
code-vs-corpus gap path (precise) is unchanged and still default.
Tests: the two cases that relied on default-title coverage gaps now pass explicit
topics; added a regression that no coverage gaps appear without --topic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- R-9 (MED): _inject_cross_refs now protects LaTeX math spans ($…$, $$…$$,
\(…\), \[…\]) like inline code, so a concept name inside a formula is no longer
rewritten to a [[slug]] link and corrupting the LaTeX. Regression test added.
- C-13 (LOW): mark ungrounded claims on the raw body BEFORE injecting cross-refs
(render then inject), so a concept name inside a claim cannot stop the ⚠ regex
from matching.
- C-12 (LOW): _try_embed_formulas is now a true no-op — it previously issued a
per-concept information_schema round-trip that did nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
README step 4 used an inline apply_schema over the app DATABASE_URL, which fails
under the privilege model (least-privilege app role cannot run DDL). Replace it
with 'codex migrate' and document MIGRATION_DATABASE_URL (owner/superuser) in the
README env table and .env.example, so the privileged migrate path is the
standard for future schema upgrades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Now that 'codex migrate' applies the idempotent schema via a privileged role,
recommend it (with MIGRATION_DATABASE_URL) as the primary schema-sync fix in the
preflight abort message, keeping the manual owner-ALTER as a fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M-1: schema.sql could not be re-applied (leading CREATE TABLE/INDEX lacked
IF NOT EXISTS), apply_schema was never called, and the live migration just
failed on a missing chunks.section column. Fixes:
- schema.sql: all CREATE TABLE/INDEX now use IF NOT EXISTS — the whole file is
re-applyable as a no-op.
- codex migrate: new CLI command that applies schema.sql. Connects via
MIGRATION_DATABASE_URL (falls back to DATABASE_URL) and catches
InsufficientPrivilege with guidance — because the app role is DML-only and
core tables are owned by 'postgres' (the privilege dimension found during the
live migration incident).
- config: migration_database_url (optional privileged connection).
- db: apply_schema docstring corrected (idempotency now true) + privilege note.
- T-2: static test that schema.sql is fully idempotent; migrate privilege-error
test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The C-7 re-ingest hit M-1 live: TRUNCATE succeeded, then ingest failed on the
missing chunks.section column (schema.sql never applied to the live DB). Adding
the column then failed with 'must be owner' — the app user 'researcher' has DML
+ TRUNCATE but not DDL on the postgres-owned core tables. M-1's fix therefore
needs a privileged migration role, not just an idempotent schema. Records the
unblock and the helper's new preflight.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The helper TRUNCATEd papers CASCADE and only then ran ingest_all.sh, which
failed on the missing chunks.section column (audit M-1) — leaving the corpus
wiped. Add a preflight that verifies chunks.section exists BEFORE the
destructive step and aborts early (no TRUNCATE) with the owner-level ALTER to
run, since the app user cannot alter postgres-owned tables.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-time, guarded migration for the C-7 id-format change: a plain re-ingest
would duplicate every paper (new bare-id row beside the old URL-id PK), so this
wipes (TRUNCATE papers CASCADE) and rebuilds via ingest_all.sh.
Safety: requires the SSH tunnel, prints a BEFORE snapshot, gates the TRUNCATE
behind an explicit 'MIGRATE' confirmation, then prints AFTER verification —
C-7 (url_form_ids should be 0) and C-1 via the real resolver-based
discovery_leads() (ingested papers leaked should be 0). Uses .venv psycopg
(psql is not installed); DATABASE_URL is sourced, never echoed.
Joins PR #13 (Wave 2). Read-only verification SQL validated against the live DB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ADR's GO rested on a live-corpus spike (478 dangling, Bobenko/Springborn
rank 7, 5.7% spread) measured before the resolution + canonicalisation landed.
Add an audit addendum: mark the 'Known Limitation: ID-Format Mismatch' as
resolved (C-1 shared resolver + C-7 canonical ids), and flag the Spike Result
numbers as pending re-measurement after the C-7 re-ingest. History preserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
extract_formulas/extract_figures derive paper_id from Path(pdf_path).stem, which
need not equal papers.id (and won't, given canonical ids). Inserting that stem
violated the formulas/figures -> papers(id) FK on rich (.pdf) ingest. The DELETE
already used paper.id; the INSERTs now do too.
Strengthened the rich-ingest test: the fake formula/figure carry a filename-stem
paper_id distinct from papers.id, and the test asserts the inserted rows are
keyed on papers.id.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
openalex.fetch_paper fills paper.id from OpenAlex's URL-form doi/id
(e.g. https://doi.org/10.48550/arxiv.math/0603097), and ingest upserted that as
papers.id. Live DB confirmed 35/36 ids were URL-form — breaking bare-id lookups
(graph related / discover citing / search by id) and leaving cited_id matching
to rely solely on openalex_id.
ingest now sets paper.id to the caller-supplied id (the arXiv id / DOI the CLI
and ingest scripts use); OpenAlex's work id is retained as openalex_id and in
paper_identifiers. Regression test: fetch_paper returns a URL-form id, papers.id
upsert uses the bare caller id.
T-3: the OpenAlex test fixture used a bare 'doi' (not the URL form the API
emits) and never asserted paper.id, which masked this. Fixture now uses the URL
form and the test pins fetch_paper's mapping.
NOTE: existing rows keep their URL-form ids until re-ingested — run ingest_all.sh
to migrate the 36-paper corpus (chosen migration path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- D-2 config.py: graph_cite_boost_alpha described the OLD inverted formula
(score = dense * (1 + alpha*pr)); the code divides distance. Document the
actual boosted_distance = distance / (1 + alpha*pr).
- D-4 synthesis.py: write_leads claimed an 'append-only' INDEX.md that keeps
previously-recorded ids; _update_index actually rebuilds from on-disk glob.
Docstring now matches the implementation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
discover.py compared the raw OpenAlex cited_id against papers.id (a DOI/arXiv
id), so an ingested paper cited by its OpenAlex id was mis-reported as a
discovery lead. graph.py already resolved this via papers.openalex_id, but the
fix was never propagated — two 'dangling' implementations disagreed.
Extract the resolution into one shared constant RESOLVED_CITATIONS_SQL in
graph.py; build_citation_graph and all four discover.py queries
(discovery_leads / citing_papers / cited_by / cocited_papers) now go through it,
so the 'what counts as ingested' rule cannot drift again.
Live-DB validated: before, 13 ingested papers leaked into discovery_leads;
after, the real discovery_leads() returns 0 ingested papers among its leads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each stage (find_connections/gaps/improvements/propose_conjectures) numbered its
leads from seq=1, so connection L-0001, gap L-0001 and improvement L-0001 all
resolved to grounded/L-0001.md. The CLI concatenates them
(connections + gaps + improvements) and write_leads writes in order, so later
kinds silently overwrote earlier ones — survivors = max(per-kind count), not the
sum. Reproduced: 4 distinct leads -> 2 files.
_make_lead_id now takes the kind and emits L-C-/L-G-/L-I-/L-X- prefixes, keeping
same-seq ids distinct across stages. Regression tests cover the format and the
no-collision invariant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compile_all's quarantine gate was guarded by `total_claims > 0`, so a page
with no parseable citations — including the LLM-outage case where
compile_concept returns an empty page — fell through to the publish branch and
was written to wiki/ + marked compiled. Such a page carries zero grounding
evidence.
Now: total_claims == 0 (or grounding_rate below threshold) quarantines and does
NOT update the compile-state, so a transient LLM failure retries next run
instead of freezing an empty/uncited page as 'compiled'. Empty bodies are
recorded but not written as draft files.
Regression test reproduces the exact bypass (uncited LLM output -> quarantined,
not published).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Part D synthesis: C-11 HIGH (lead-ID collision → silent overwrite), R-8 MED, D-4 LOW.
- Part E wiki-rest: R-9 MED (cross-ref injection corrupts LaTeX math), C-12/C-13/R-10/R-11 LOW.
- Part F embed/quality/models: C-14/R-12 LOW + C-7 corroboration. Cleanest modules.
- Part G cli/config: C-15 MED (embedding_dim vs hardcoded vector(1024)), R-13/S-4/R-14/D-5 LOW.
Executive summary: 45 findings total (8 HIGH, 11 MED, 26 LOW). Five root themes;
dominant one is un-normalized identity (C-1/C-7/C-10/M-1). Whole repo reviewed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes request (a) — the second pass over loop-1 §8 open items:
- Part A grounding guard: C-4 (HIGH, zero-claim pages bypass quarantine),
C-5/R-1 MED, C-6/R-2/R-3 LOW.
- Part B sources/: C-7 (HIGH, papers.id stored in OpenAlex URL form vs the
bare-ID contract), T-3 MED, R-4/R-5/C-9 LOW.
- Part C parsing/: C-10 MED (formulas/figures key on filename not paper.id),
R-6/R-7 LOW — all latent (live corpus is .txt).
Running tally: 30 findings, 7 HIGH. Common root: un-normalized identity.
Parts D-G (synthesis/wiki/embed/cli) still open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit of main @5dfa6e4: 4 HIGH (S-1 secret-in-tree, M-1 missing migration
path, C-1 unpropagated ID-mismatch fix in discover.py, D-1 stale ADR-F15
validation, T-1 untested resolution SQL) plus MED/LOW findings. Findings
only; remediation planning deferred to the next loop phase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- .gitignore: ignore .env.* (keep .env.example) so local DB-credential
profiles like .env.jetson-ingest can never be staged (audit S-1).
- ingest_all.sh: batch-ingest helper writing to the Jetson DB via SSH tunnel.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Committed code predated the line-length=100 ruff config; this brings the
five drifted files into compliance. No logic change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes preprint/journal duplicate problem: arXiv papers were ingested
under their preprint OpenAlex ID, but corpus citations reference the
published-version OpenAlex ID → those citations were invisible in the
graph (appeared as dangling).
Changes:
- infra/schema.sql: CREATE TABLE paper_identifiers (paper_id, openalex_id)
with UNIQUE index; seeded from papers.openalex_id on migration
- codex/graph.py: build_citation_graph LEFT JOINs paper_identifiers as
a second resolution path: COALESCE(p.id, pi.paper_id, c.cited_id)
- codex/ingest.py: every ingest inserts openalex_id into paper_identifiers
(ON CONFLICT DO NOTHING) — aliases can be added manually alongside
Live DB: 35 rows seeded + 4 published-version aliases added:
math/0503219 → W2163787581 (DCG 2007)
math/0306167 → W2136126748 (Commun Analysis Geom 2004)
math/0203250 → W1567166970 (Trans AMS 2003)
1005.2698 → W1511400044 (Geom & Topol 2015)
Effect: dangling 574→570; Bobenko+Springborn 2007 now IN-KB rank 9.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
build_citation_graph now does:
SELECT c.citing_id, COALESCE(p.id, c.cited_id) AS cited_id
FROM citations c
LEFT JOIN papers p ON p.openalex_id = c.cited_id
Before: cited_id (OpenAlex IDs) never matched papers.id (DOIs) →
13 cross-citations between ingested papers were invisible in graph.
After: in-KB papers use their canonical DOI key; dangling references
keep their raw OpenAlex ID unchanged.
Test: test_cross_citation_uses_doi_when_in_kb (42 passed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
D-04: add _make_bibkey(paper) — ConcatSurnames+Year, ≤3 authors full list,
>3 authors FirstEtAlYear. Applied in ingest_paper when paper.bibkey is
None. ON CONFLICT now fills NULL bibkeys via COALESCE(papers.bibkey,
EXCLUDED.bibkey) while preserving manually-set values.
D-05b: add codex/__main__.py so `python -m codex` dispatches to the CLI.
D-05c: fix test_ingest.py:75 — rename var to raw_api_citations (the raw
OpenAlex payload with citing_id=openalex_id) and add regression
assertion: citing_id written to DB must equal paper.id (DOI), not
paper.openalex_id.
D-05a (proxy): add TestEntryPoint.test_python_m_codex_help — runs
`sys.executable -m codex --help` via subprocess, covering the
non-uv-run entry path. 253 tests green, ruff+mypy clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Closes audit N-02: docs/adr/ADR-F13-synthesis-engine.md was missing,
blocking R-34. Documents spike result (GO, Opus APPROVE 2 passes),
four-stage lead taxonomy, grounding discipline, three-level
zero-fact-leak invariant, and falsification-gate in _parse_conjecture_output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
