Per review: the two latent upsert conflicts should be observable even where not
auto-fixed.
- bibkey collision (auto-suffixed) now logs a warning so a paper landing as
'BobenkoLutz2023a' instead of '…2023' isn't silent.
- openalex_id collision (two papers claiming the same OpenAlex work — a real data
conflict, not auto-renamable) now raises a clear ValueError naming the
openalex_id and the offending paper, instead of the raw psycopg UniqueViolation.
Regression test added for the openalex_id error path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-ran the citation graph after the C-7 re-ingest (29/29 papers canonical/bare):
489 nodes, 590 edges, 472 dangling, 5.4% spread. The resolution fix's signature:
Pinkall/Polthier 1993 resolves to its DOI and is now the rank-1 *in-KB* hub
(was a dangling OpenAlex node); Bobenko/Springborn 2007 holds rank 7. Foundational
hubs still surface — GO re-affirmed on the shipping topology. Updates the ADR
spike table with before/after numbers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The id-keyed upsert (ON CONFLICT (id)) does not catch the papers.bibkey UNIQUE
constraint, so a second paper whose auto-generated key matches an existing one
(two same-author/year works → e.g. 'BobenkoLutz2023') failed the whole ingest
with a UniqueViolation. Surfaced by the canonical-id re-ingest: 2305.10988
collided with 2310.17529.
ingest now catches a bibkey UniqueViolation, rolls back, and retries the upsert
with an a/b/c… suffix. Non-bibkey violations re-raise unchanged. Mocks don't
raise, so the existing happy-path tests are untouched; a new test drives the
collision-and-retry path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-14: embed.py header now states that only the dense path is wired (search
'hybrid' is dense + Postgres FTS); encode_sparse/encode are reserved for a
future sparse-retrieval layer, not yet consumed.
- U-2: --cite-boost help clarifies it re-ranks within the top results (a
tie-breaker), not a hard re-ranking — matching the small alpha effect.
Accepted (documented heuristics, no change): R-3 conflict detection is a
keyword-only signal labelled as such; R-12 classify_section is mostly 'body'
because chunks are word-windows that rarely start at a section header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-6: the content-5-gram check used 'gram in src' on space-joined strings, so the
first/last gram token could match a prefix/suffix of a longer chunk word
('set' inside 'subset'). Both gram and chunk are now space-padded, so a match
requires whole-token alignment. Regression test added.
- R-2: the reference-list filter's author pattern ('Surname, I.') also matched
'Theorem A.'/'Lemma B.', so theorem-dense chunks could be dropped as a
bibliography. The pattern now excludes common math-structure words.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- U-1: 'graph report' shows the paper title next to each in-KB hub (dangling
OpenAlex-id hubs are flagged), instead of bare ids.
- U-3: '--json' now includes a 'small_corpus_warning' field (null unless below
graph_min_corpus_size) instead of silently dropping the warning.
- C-9: new 'codex discover recommend <id>' wires semanticscholar.fetch_recommendations
(was implemented + tested but unreachable).
- U-4: new 'codex graph cocited <id>' wires graph.find_co_cited (likewise).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- R-4: arxiv.fetch_source had a try/except httpx.RequestError that only re-raised
— removed (no behaviour change, less noise).
- R-5: semanticscholar URLs now quote(paper_id, safe=':') so a legacy-arXiv '/'
(arXiv:math/0603097) doesn't corrupt the path and silently return no references.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
database_url, migration_database_url, mcp_auth_token and the mathpix app
id/key are now pydantic SecretStr, so they render as '**********' in any
repr/log/traceback instead of leaking the plaintext credential. Consumers
(db.get_conn, cli.migrate, mcp._require_http_token, mathpix.extract_formulas)
call .get_secret_value() at the point of use. Tests updated to the SecretStr
type.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The grounding guard tokenized with .lower().split() and split sentences on bare
.!?, so faithful claims were wrongly marked ungrounded: 'volume,' != 'volume',
and a decimal like 'V = 1.5' split into a 1-word fragment '5' that fell below
the 5-content-word floor.
- _content_words now strips edge sentence-punctuation (.,;:!?"') from tokens
while preserving math delimiters ()=+; claim and chunk are normalised
identically so matching stays consistent.
- _SENTENCE_SPLIT_RE splits on .!? only when followed by whitespace/end, so
decimals no longer create spurious sentence boundaries.
Regression test: a 5-word claim with a trailing comma now grounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- C-3 (MED): wiki_read returned 'sources' = [concept_slug] (a placeholder), not
the cited sources. Now parses the page's inline [BibKey #loc] citations and
returns the actual set of cited bibkeys.
- S-2 (LOW): _TokenAuth compared the Bearer header with != (length/equality
timing side-channel). Use secrets.compare_digest for constant-time comparison.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
find_gaps defaulted its probe topics to paper titles, so a title — which
retrieves mostly its own single bibkey — fell below synthesis_gap_min_coverage
and flagged almost every paper as a 'gap'. Coverage-map gaps now run only for
explicitly-requested topics; the CLI gains a repeatable --topic option. The
code-vs-corpus gap path (precise) is unchanged and still default.
Tests: the two cases that relied on default-title coverage gaps now pass explicit
topics; added a regression that no coverage gaps appear without --topic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- R-9 (MED): _inject_cross_refs now protects LaTeX math spans ($…$, $$…$$,
\(…\), \[…\]) like inline code, so a concept name inside a formula is no longer
rewritten to a [[slug]] link and corrupting the LaTeX. Regression test added.
- C-13 (LOW): mark ungrounded claims on the raw body BEFORE injecting cross-refs
(render then inject), so a concept name inside a claim cannot stop the ⚠ regex
from matching.
- C-12 (LOW): _try_embed_formulas is now a true no-op — it previously issued a
per-concept information_schema round-trip that did nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
README step 4 used an inline apply_schema over the app DATABASE_URL, which fails
under the privilege model (least-privilege app role cannot run DDL). Replace it
with 'codex migrate' and document MIGRATION_DATABASE_URL (owner/superuser) in the
README env table and .env.example, so the privileged migrate path is the
standard for future schema upgrades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Now that 'codex migrate' applies the idempotent schema via a privileged role,
recommend it (with MIGRATION_DATABASE_URL) as the primary schema-sync fix in the
preflight abort message, keeping the manual owner-ALTER as a fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M-1: schema.sql could not be re-applied (leading CREATE TABLE/INDEX lacked
IF NOT EXISTS), apply_schema was never called, and the live migration just
failed on a missing chunks.section column. Fixes:
- schema.sql: all CREATE TABLE/INDEX now use IF NOT EXISTS — the whole file is
re-applyable as a no-op.
- codex migrate: new CLI command that applies schema.sql. Connects via
MIGRATION_DATABASE_URL (falls back to DATABASE_URL) and catches
InsufficientPrivilege with guidance — because the app role is DML-only and
core tables are owned by 'postgres' (the privilege dimension found during the
live migration incident).
- config: migration_database_url (optional privileged connection).
- db: apply_schema docstring corrected (idempotency now true) + privilege note.
- T-2: static test that schema.sql is fully idempotent; migrate privilege-error
test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
