fix(ingest): canonicalise papers.id to the caller id (audit C-7, T-3)
openalex.fetch_paper fills paper.id from OpenAlex's URL-form doi/id (e.g. https://doi.org/10.48550/arxiv.math/0603097), and ingest upserted that as papers.id. Live DB confirmed 35/36 ids were URL-form — breaking bare-id lookups (graph related / discover citing / search by id) and leaving cited_id matching to rely solely on openalex_id. ingest now sets paper.id to the caller-supplied id (the arXiv id / DOI the CLI and ingest scripts use); OpenAlex's work id is retained as openalex_id and in paper_identifiers. Regression test: fetch_paper returns a URL-form id, papers.id upsert uses the bare caller id. T-3: the OpenAlex test fixture used a bare 'doi' (not the URL form the API emits) and never asserted paper.id, which masked this. Fixture now uses the URL form and the test pins fetch_paper's mapping. NOTE: existing rows keep their URL-form ids until re-ingested — run ingest_all.sh to migrate the 36-paper corpus (chosen migration path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -15,7 +15,8 @@ from codex.sources.openalex import _resolve_id
|
||||
|
||||
_SAMPLE_WORK = {
|
||||
"id": "https://openalex.org/W2741809807",
|
||||
"doi": "10.1145/3592430",
|
||||
# Real OpenAlex returns the DOI as a full URL, not a bare DOI (audit T-3).
|
||||
"doi": "https://doi.org/10.1145/3592430",
|
||||
"title": "Conformal Prediction: A Review",
|
||||
"publication_year": 2023,
|
||||
"authorships": [
|
||||
@@ -90,6 +91,10 @@ def test_fetch_paper_success(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
assert paper.authors == ["Alice Smith", "Bob Jones"]
|
||||
assert paper.abstract == "Conformal prediction is great"
|
||||
assert paper.openalex_id == "https://openalex.org/W2741809807"
|
||||
# fetch_paper maps id from the (URL-form) doi; ingest later canonicalises
|
||||
# papers.id to the caller's bare id (audit C-7). Pin the real mapping here so
|
||||
# the fixture cannot silently misrepresent OpenAlex again (audit T-3).
|
||||
assert paper.id == "https://doi.org/10.1145/3592430"
|
||||
|
||||
|
||||
def test_fetch_paper_arxiv_id_prefixed(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
|
||||
Reference in New Issue
Block a user