fix(ingest): canonicalise papers.id to the caller id (audit C-7, T-3)
openalex.fetch_paper fills paper.id from OpenAlex's URL-form doi/id (e.g. https://doi.org/10.48550/arxiv.math/0603097), and ingest upserted that as papers.id. Live DB confirmed 35/36 ids were URL-form — breaking bare-id lookups (graph related / discover citing / search by id) and leaving cited_id matching to rely solely on openalex_id. ingest now sets paper.id to the caller-supplied id (the arXiv id / DOI the CLI and ingest scripts use); OpenAlex's work id is retained as openalex_id and in paper_identifiers. Regression test: fetch_paper returns a URL-form id, papers.id upsert uses the bare caller id. T-3: the OpenAlex test fixture used a bare 'doi' (not the URL form the API emits) and never asserted paper.id, which masked this. Fixture now uses the URL form and the test pins fetch_paper's mapping. NOTE: existing rows keep their URL-form ids until re-ingested — run ingest_all.sh to migrate the 36-paper corpus (chosen migration path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -98,6 +98,14 @@ def ingest_paper(
|
||||
if paper is None:
|
||||
raise ValueError(f"Paper not found: {paper_id}")
|
||||
|
||||
# Canonical identity is the caller-supplied id — what the CLI / ingest scripts
|
||||
# use and what papers.id is contracted to hold (arXiv id or DOI). OpenAlex's
|
||||
# fetch_paper fills paper.id with the URL-form doi/id (e.g.
|
||||
# "https://doi.org/10.48550/arxiv.math/0603097"), which broke bare-id lookups
|
||||
# and cross-citation matching (audit C-7). The OpenAlex work id is retained
|
||||
# separately as openalex_id and in the paper_identifiers alias table.
|
||||
paper.id = paper_id.strip()
|
||||
|
||||
# Auto-generate bibkey when source has none (D-04).
|
||||
if paper.bibkey is None:
|
||||
paper.bibkey = _make_bibkey(paper)
|
||||
|
||||
Reference in New Issue
Block a user