GROBID is only needed during reference extraction (ra_grobid_backfill.py),
not for normal corpus operations (search, embeddings, wiki). Idle it still
holds ~3.5GB on the 8GB Jetson, so keep it stopped by default.
- infra/docker-compose.yml: grobid restart unless-stopped -> "no" (no auto-start
on boot; db keeps unless-stopped). Start/stop papers-grobid around an R-A run.
- docs/infra/grobid-jetson.md: add "Run on-demand" section with the start/stop
workflow, and record the refextract head-to-head evaluation on the Jetson that
led to keeping GROBID: far worse DOI recall on this math corpus (lutz thesis
101 usable IDs vs refextract 2) and slower per PDF despite a smaller footprint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enable roadmap R-A (GROBID reference extraction) end-to-end and run it
against local Jetson infra instead of a Mac/Rosetta emulation.
Backfill:
- Add scripts/ra_grobid_backfill.py: references-only, idempotent citation
backfill (dry-run default). Deliberately not ingest_paper(source_path=pdf),
which re-OCRs and replaces the DQ-3-clean .txt chunks; this touches only the
citations table (ON CONFLICT DO NOTHING). 7 tests.
- Make extract_references timeout configurable (large theses exceed the 60s
default, especially against a slower GROBID).
Jetson infra:
- Bump grobid/grobid 0.8.2 -> 0.9.0-crf. The -crf tag is the only GROBID
variant published as an arm64 multi-arch manifest; every 0.8.x tag and the
full deep-learning image are amd64-only, which is why GROBID never ran on the
aarch64 Jetson. Enable the 4g memory cap + init/ulimits per GROBID docs.
- Add docs/infra/grobid-jetson.md runbook: arm64 image rationale, the two host
prerequisites (docker-group membership + the Compose v2 CLI plugin, both
missing on the Jetson), service-scoped deploy, and the GET /api/isalive check.
Verified live 2026-06-17: native arm64 pull, isalive=true, end-to-end
extract_references on lutz-2024-thesis.pdf = 116 refs (101 with DOI/arXiv).
docs(audit): mark R-A core done (citing coverage 27/29 -> 29/29; edges 920 -> 1022).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes preprint/journal duplicate problem: arXiv papers were ingested
under their preprint OpenAlex ID, but corpus citations reference the
published-version OpenAlex ID → those citations were invisible in the
graph (appeared as dangling).
Changes:
- infra/schema.sql: CREATE TABLE paper_identifiers (paper_id, openalex_id)
with UNIQUE index; seeded from papers.openalex_id on migration
- codex/graph.py: build_citation_graph LEFT JOINs paper_identifiers as
a second resolution path: COALESCE(p.id, pi.paper_id, c.cited_id)
- codex/ingest.py: every ingest inserts openalex_id into paper_identifiers
(ON CONFLICT DO NOTHING) — aliases can be added manually alongside
Live DB: 35 rows seeded + 4 published-version aliases added:
math/0503219 → W2163787581 (DCG 2007)
math/0306167 → W2136126748 (Commun Analysis Geom 2004)
math/0203250 → W1567166970 (Trans AMS 2003)
1005.2698 → W1511400044 (Geom & Topol 2015)
Effect: dangling 574→570; Bobenko+Springborn 2007 now IN-KB rank 9.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
