Flatten multi-file arXiv LaTeX (\input/\include) in arxiv.fetch_source so the full body is assembled rather than just the primary file's include skeleton, and add section-aware chunking (tex.chunk_sections) so .tex chunks never span a \section boundary and are labelled by their real heading. The ingest .tex path now produces (section, chunk) pairs, quality-gated together so labels stay aligned; the stored 'section' column gains genuine signal instead of mostly 'body' (serves DQ-3 fidelity + audit R-12).
Adds scripts/rc_tex_reingest.py to re-ingest arXiv papers from .tex (dry-run by default; replaces chunks). Tests: flatten_inputs, chunk_sections, multi-file fetch_source, section-label ingest, is_arxiv_id. Full suite 365 passed; ruff + mypy clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enable roadmap R-A (GROBID reference extraction) end-to-end and run it
against local Jetson infra instead of a Mac/Rosetta emulation.
Backfill:
- Add scripts/ra_grobid_backfill.py: references-only, idempotent citation
backfill (dry-run default). Deliberately not ingest_paper(source_path=pdf),
which re-OCRs and replaces the DQ-3-clean .txt chunks; this touches only the
citations table (ON CONFLICT DO NOTHING). 7 tests.
- Make extract_references timeout configurable (large theses exceed the 60s
default, especially against a slower GROBID).
Jetson infra:
- Bump grobid/grobid 0.8.2 -> 0.9.0-crf. The -crf tag is the only GROBID
variant published as an arm64 multi-arch manifest; every 0.8.x tag and the
full deep-learning image are amd64-only, which is why GROBID never ran on the
aarch64 Jetson. Enable the 4g memory cap + init/ulimits per GROBID docs.
- Add docs/infra/grobid-jetson.md runbook: arm64 image rationale, the two host
prerequisites (docker-group membership + the Compose v2 CLI plugin, both
missing on the Jetson), service-scoped deploy, and the GET /api/isalive check.
Verified live 2026-06-17: native arm64 pull, isalive=true, end-to-end
extract_references on lutz-2024-thesis.pdf = 116 refs (101 with DOI/arXiv).
docs(audit): mark R-A core done (citing coverage 27/29 -> 29/29; edges 920 -> 1022).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- mathpix.py + figures.py: wrap fitz page-loop in try/finally so
doc.close() is guaranteed even on exception (no file-handle leak)
- ingest.py: DELETE FROM formulas/figures WHERE paper_id before re-insert
so --rich re-ingest is idempotent (BIGSERIAL has no natural UNIQUE key;
ON CONFLICT DO NOTHING was a no-op)
Gate: 158 passed, ruff clean, mypy clean
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Review-Gate finding: GROBID emits type="arXiv" (camel-case), not "arxiv".
XPath literal match silently returned empty strings for all real responses.
Fix: iterate idno elements and compare .lower() == "arxiv".
Test fixture updated to reflect real GROBID output (type="arXiv").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
