feat(quality): store real section titles for .tex chunks (R-F)

Add quality.section_label(title, content): for section-aware .tex ingest, keep the controlled bucket (intro/theorem/proof/abstract/bibliography) when the real \section heading maps to one, else store the cleaned title verbatim (e.g. 'preliminaries', 'rigidity') instead of collapsing to 'body'. Math papers title most sections descriptively, so R-C's vocab-only mapping left ~90% as 'body'; this lifts the section signal toward near-full. .pdf/.txt/run_quality_pass keep content-based classify_section unchanged. Safe because the section column is write-only (no consumer filters on the vocab).

New _clean_title (strip LaTeX/numbering, lower-case, truncate). Tests: section_label bucket/verbatim/fallback paths + _clean_title; .tex ingest stores a descriptive heading as its title. Full suite 377 passed; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-18 09:03:46 +02:00
parent c22c0db3d0
commit c70ff9aa1e
4 changed files with 82 additions and 7 deletions

View File

@@ -128,6 +128,7 @@ def test_ingest_paper_source_tex(tmp_path: Any) -> None:
section_pairs = [
("Introduction", "Body of the introduction goes here."),
("Proof of the Main Theorem", "Body of the proof goes here."),
("3.2 Discrete Conformal Maps", "Body about discrete conformal maps."),
]
with (
@@ -151,9 +152,11 @@ def test_ingest_paper_source_tex(tmp_path: Any) -> None:
chunk_sql: str = chunk_insert_call[0][0]
assert "INSERT INTO chunks" in chunk_sql
# Section label comes from the heading: "Introduction"intro, "Proof of…" → proof.
# R-F: heading → controlled bucket where it maps ("Introduction"intro,
# "Proof of…"→proof), else the cleaned real title ("discrete conformal maps")
# instead of collapsing to "body".
chunk_rows = chunk_insert_call[0][1]
assert [row[4] for row in chunk_rows] == ["intro", "proof"]
assert [row[4] for row in chunk_rows] == ["intro", "proof", "discrete conformal maps"]
assert result.chunks_upserted == len(section_pairs)