merge: reconcile data-quality roadmap with audit-remediation main (#12-15)

Resolved 4 conflicts: cli.py (R-D citing-coverage warning + main's small-corpus JSON/hub-title changes both kept); ingest.py (C-7 paper.id=caller-id pin first, then DQ-2 abstract recovery); test_ingest.py + test_openalex.py (kept both branches' tests).

C-7/DQ-5 overlap (both canonicalise papers.id): kept both layers — openalex._canonical_id normalizes fetch_paper's id (DQ-5); ingest pins papers.id to the caller's bare id (C-7); they converge on the bare canonical id. Fixed the auto-merged test_fetch_paper_success that had contradictory bare/URL assertions. 402 tests pass; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-22 15:11:26 +02:00
29 changed files with 900 additions and 193 deletions

View File

@@ -118,10 +118,7 @@ def fetch_source(arxiv_id: str) -> str | None:
file is present (signals Nougat fallback).
"""
url = f"{_BASE}/src/{arxiv_id}"
try:
response = httpx.get(url, timeout=60, follow_redirects=True)
except httpx.RequestError:
raise
response = httpx.get(url, timeout=60, follow_redirects=True)
if response.status_code == 404:
logger.debug("arXiv 404 for source id=%s", arxiv_id)
return None