docs(audit): add R-F (richer section labels) resolution — 10% to 95% non-body

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-18 09:25:03 +02:00
parent 1a879d4827
commit 5c60a7eda4

View File

@@ -454,6 +454,31 @@ is self-contained so a cold session can pick it up.
violates most publisher terms and can get the whole institution's access violates most publisher terms and can get the whole institution's access
revoked.** Do not wire a publisher login into the ingest pipeline. revoked.** Do not wire a publisher login into the ingest pipeline.
### R-F — Richer section labels: store real `\section` titles · **DONE 2026-06-17**
**Resolution (what was done):**
- **Why:** R-C labelled `.tex` chunks by mapping the real `\section` heading through
`classify_section`'s fixed vocab, so descriptively-titled Math sections collapsed
to `body` (only **34/329 = 10 %** non-`body`). A research pass confirmed the
`section` column is **write-only** (nothing filters on the vocab — `mcp_server.py`
and `wiki.py` don't even SELECT it), so storing free-text titles is safe.
- **Change:** new `quality.section_label(title, content)` (+ `_clean_title`) — keep the
controlled bucket (intro/theorem/proof/abstract/bibliography) when the heading maps
to one, else store the cleaned real title ("the flip algorithm", "main results", …).
The `.tex` ingest path uses it; `.pdf`/`.txt`/`run_quality_pass` keep
`classify_section` unchanged. `_clean_title` strips LaTeX commands/accents/ties +
leading numbering, lower-cases, truncates at a word boundary. ~1 helper + 1 ingest
line; no schema change (column already free `TEXT`).
- **Result (live):** re-applied to the 13 arXiv papers → non-`body` **34/329 (10 %) →
314/329 (95 %)**. Only `math/0306167` (old AMS-TeX, no `\section{}`) stays `body`.
- **Tests:** `section_label` / `_clean_title` units + `.tex` ingest stores a
descriptive heading verbatim. Full suite 377 passed; ruff/mypy clean. Branch
`feat/section-labels` (off `feat/tex-ingest`).
- **Pending (infra, not code):** an accent-cleanup polish (`m\"obius`→`mobius`) landed
in `_clean_title`, and a quick in-place `UPDATE` re-clean (no re-embed) was started,
but the Jetson went offline (ping 100 % loss) before it finished — a few live labels
still show LaTeX accents/ties. Re-run the in-place re-clean when the Jetson is back.
--- ---
## Priority for the next session ## Priority for the next session