user2595/codex-py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(F-09): rich parsing — formula + figure extraction #1

Merged
user2595 merged 6 commits from feat/F-09-rich-parsing into main 2026-06-14 01:52:24 +00:00
Conversation 0 Commits 6 Files Changed 20 +3596 -26
user2595 commented 2026-06-14 01:45:06 +00:00
Owner
Copy Link

Summary

  • codex/parsing/mathpix.py: pix2tex (lokal, CPU) als primärer Pfad + MathPix API als optionales Cloud-Upgrade; Singleton-Model-Cache; Bbox-Heuristik h>15px / math-char-count>5; try/finally für fitz-Doc-Close
  • codex/parsing/figures.py: pymupdf Embedded-Image → PNG-Export; Caption-Proximity-Detection (60pt, "Figure/Fig./Abbildung"-Prefix); try/finally für fitz-Doc-Close
  • codex/models.py: FormulaChunk + FigureChunk Dataclasses
  • codex/ingest.py: --rich Flag; DELETE-before-INSERT für Idempotenz bei Re-Ingest
  • codex/cli.py: search_app Sub-Typer (paper + formula Subcommands); --rich on ingest
  • codex/config.py: mathpix_app_id/key, pix2tex_fallback, figures_dir
  • infra/schema.sql: formulas + figures Tabellen mit HNSW pgvector-Index
  • pyproject.toml: pymupdf>=1.24, pix2tex>=0.1.4
  • tests/parsing/test_mathpix.py + test_figures.py: 31 Tests (mock pix2tex + MathPix HTTP)

Requirements

R-10 R-11 R-12 R-13 R-14 → done

Test plan

  • uv run pytest tests/ -q — 158 passed, 0 failed
  • uv run ruff check . — clean
  • uv run mypy codex/ — clean (20 files, strict)
  • Review-Gate (Opus, kalt) — APPROVE; 2 Findings inline fixiert (resource leak + idempotency)

🤖 Generated with Claude Code

## Summary - `codex/parsing/mathpix.py`: pix2tex (lokal, CPU) als primärer Pfad + MathPix API als optionales Cloud-Upgrade; Singleton-Model-Cache; Bbox-Heuristik h>15px / math-char-count>5; `try/finally` für fitz-Doc-Close - `codex/parsing/figures.py`: pymupdf Embedded-Image → PNG-Export; Caption-Proximity-Detection (60pt, "Figure/Fig./Abbildung"-Prefix); `try/finally` für fitz-Doc-Close - `codex/models.py`: `FormulaChunk` + `FigureChunk` Dataclasses - `codex/ingest.py`: `--rich` Flag; DELETE-before-INSERT für Idempotenz bei Re-Ingest - `codex/cli.py`: `search_app` Sub-Typer (paper + formula Subcommands); `--rich` on ingest - `codex/config.py`: `mathpix_app_id/key`, `pix2tex_fallback`, `figures_dir` - `infra/schema.sql`: `formulas` + `figures` Tabellen mit HNSW pgvector-Index - `pyproject.toml`: `pymupdf>=1.24`, `pix2tex>=0.1.4` - `tests/parsing/test_mathpix.py` + `test_figures.py`: 31 Tests (mock pix2tex + MathPix HTTP) ## Requirements R-10 R-11 R-12 R-13 R-14 → done ## Test plan - [x] `uv run pytest tests/ -q` — 158 passed, 0 failed - [x] `uv run ruff check .` — clean - [x] `uv run mypy codex/` — clean (20 files, strict) - [x] Review-Gate (Opus, kalt) — APPROVE; 2 Findings inline fixiert (resource leak + idempotency) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
user2595 added 6 commits 2026-06-14 01:45:07 +00:00
chore(env): add NOUGAT_URL to .env.example (F-11 / D-02) c6eaf999a1 
Documents the Nougat HTTP server endpoint. Resolves D-02 drift finding.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(wiki): concept schema loader + concepts.yaml (F-12) aee00515f4 
Adds wiki/concepts.yaml (5 curated seeds: discrete-conformal-map,
circle-packing, lobachevsky-function, discrete-yamabe-flow,
hyperideal-tetrahedron) and codex/wiki.py with full load_concepts()
implementation (Concept dataclass, YAML parser, graceful empty list).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(cli): wiki compile/list/check command group (F-12) 4d361fd8dc 
Adds codex wiki compile/list/check Typer subgroup to cli.py.
Adds wiki_dir, wiki_llm_model, wiki_llm_url, wiki_top_k settings
to config.py (F-12 section only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(cli): remove F-09 spillover from F-12 branch, keep only wiki group d2f9141a5c 
cli.py and config.py were accidentally contaminated with uncommitted F-09
changes (search_app restructure, rich ingest, mathpix/figures settings).
This restores both files to main-baseline + F-12-only additions:
- cli.py: wiki_app group (compile/list/check), restore @app.command search
- config.py: wiki_dir, wiki_llm_model, wiki_llm_url, wiki_top_k only

All 127 tests green, ruff+mypy clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(F-09): rich parsing — formula + figure extraction 1df9be6563 
- codex/parsing/mathpix.py: pix2tex (local, CPU) primary + MathPix API
  optional; bbox heuristic h>15px, math-char-count>5; singleton model cache
- codex/parsing/figures.py: pymupdf embedded-image extraction → PNG;
  caption detection via proximity + "Figure/Fig./Abbildung" prefix
- codex/models.py: FormulaChunk + FigureChunk dataclasses (R-10/R-11)
- codex/ingest.py: --rich flag wires formula+figure extraction post-ingest
- codex/cli.py: search_app sub-typer (paper + formula subcommands),
  --rich flag on ingest; wiki_app from F-12 preserved intact
- codex/config.py: mathpix_app_id/key, pix2tex_fallback, figures_dir
- infra/schema.sql: formulas + figures tables with HNSW pgvector indexes
- pyproject.toml: pymupdf>=1.24, pix2tex>=0.1.4
- tests/parsing/test_mathpix.py + test_figures.py: 31 tests (mock pix2tex
  + MathPix HTTP, real pymupdf on synthetic PDF)

Gate: 158 passed, ruff clean, mypy clean (20 files)
Requirements: R-10 R-11 R-12 R-13 R-14 → done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(F-09): address review-gate findings — resource leak + idempotency cd42e9abc8 
- mathpix.py + figures.py: wrap fitz page-loop in try/finally so
  doc.close() is guaranteed even on exception (no file-handle leak)
- ingest.py: DELETE FROM formulas/figures WHERE paper_id before re-insert
  so --rich re-ingest is idempotent (BIGSERIAL has no natural UNIQUE key;
  ON CONFLICT DO NOTHING was a no-op)

Gate: 158 passed, ruff clean, mypy clean

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
user2595 merged commit 946a80418e into main 2026-06-14 01:52:24 +00:00
user2595 deleted branch feat/F-09-rich-parsing 2026-06-14 01:52:25 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
user2595
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: user2595/codex-py#1
No description provided.
Delete Branch "feat/F-09-rich-parsing"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?