feat(grobid): R-A reference backfill + native arm64 GROBID on Jetson
Enable roadmap R-A (GROBID reference extraction) end-to-end and run it against local Jetson infra instead of a Mac/Rosetta emulation. Backfill: - Add scripts/ra_grobid_backfill.py: references-only, idempotent citation backfill (dry-run default). Deliberately not ingest_paper(source_path=pdf), which re-OCRs and replaces the DQ-3-clean .txt chunks; this touches only the citations table (ON CONFLICT DO NOTHING). 7 tests. - Make extract_references timeout configurable (large theses exceed the 60s default, especially against a slower GROBID). Jetson infra: - Bump grobid/grobid 0.8.2 -> 0.9.0-crf. The -crf tag is the only GROBID variant published as an arm64 multi-arch manifest; every 0.8.x tag and the full deep-learning image are amd64-only, which is why GROBID never ran on the aarch64 Jetson. Enable the 4g memory cap + init/ulimits per GROBID docs. - Add docs/infra/grobid-jetson.md runbook: arm64 image rationale, the two host prerequisites (docker-group membership + the Compose v2 CLI plugin, both missing on the Jetson), service-scoped deploy, and the GET /api/isalive check. Verified live 2026-06-17: native arm64 pull, isalive=true, end-to-end extract_references on lutz-2024-thesis.pdf = 116 refs (101 with DOI/arXiv). docs(audit): mark R-A core done (citing coverage 27/29 -> 29/29; edges 920 -> 1022). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -26,6 +26,7 @@ def _text(element: ET.Element | None) -> str:
|
||||
def extract_references(
|
||||
pdf_path: str,
|
||||
grobid_url: str | None = None,
|
||||
timeout: float = 60.0,
|
||||
) -> list[dict[str, str]]:
|
||||
"""Extract a structured reference list from a PDF via GROBID.
|
||||
|
||||
@@ -35,6 +36,9 @@ def extract_references(
|
||||
Path to the PDF file on disk.
|
||||
grobid_url:
|
||||
Base URL of the GROBID server. Defaults to ``Settings().grobid_url``.
|
||||
timeout:
|
||||
HTTP client timeout in seconds. Large PDFs (e.g. theses) can exceed the
|
||||
60s default, especially against a slower/emulated GROBID; raise it then.
|
||||
|
||||
Returns
|
||||
-------
|
||||
@@ -46,7 +50,7 @@ def extract_references(
|
||||
if grobid_url is None:
|
||||
grobid_url = Settings().grobid_url
|
||||
|
||||
with open(pdf_path, "rb") as fh, httpx.Client(timeout=60.0) as client:
|
||||
with open(pdf_path, "rb") as fh, httpx.Client(timeout=timeout) as client:
|
||||
response = client.post(
|
||||
f"{grobid_url}/api/processReferences",
|
||||
files={"input": (pdf_path, fh, "application/pdf")},
|
||||
|
||||
Reference in New Issue
Block a user