user2595/codex-py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
33 Commits 5 Branches 0 Tags
0202a2266e190d10d033c7922b6d88cd6c22cd7f
Commit Graph

16 Commits

Author SHA1 Message Date
Tarik Moussa
0202a2266e fix(wiki): pass chunks explicitly to compile_concept (single retrieve) 
compile_concept now accepts chunks as a positional parameter; compile_all
retrieves chunks once and passes them directly, avoiding double-retrieve.
Also separates all_concepts (full list) from compile_concepts (filtered),
so cross-ref injection always uses the complete concept list regardless of
--concept filter (Fix 6 + Fix 8).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 03:58:02 +02:00
Tarik Moussa
60e40f6f36 fix(wiki): exclude URL-shaped bibkeys from claim parsing 
_parse_claims now skips matches where bibkey contains spaces or starts
with "http", preventing Markdown hyperlinks like [text](https://...) from
being misidentified as citations. Add two tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 03:58:02 +02:00
Tarik Moussa
1df9be6563 feat(F-09): rich parsing — formula + figure extraction 
- codex/parsing/mathpix.py: pix2tex (local, CPU) primary + MathPix API
  optional; bbox heuristic h>15px, math-char-count>5; singleton model cache
- codex/parsing/figures.py: pymupdf embedded-image extraction → PNG;
  caption detection via proximity + "Figure/Fig./Abbildung" prefix
- codex/models.py: FormulaChunk + FigureChunk dataclasses (R-10/R-11)
- codex/ingest.py: --rich flag wires formula+figure extraction post-ingest
- codex/cli.py: search_app sub-typer (paper + formula subcommands),
  --rich flag on ingest; wiki_app from F-12 preserved intact
- codex/config.py: mathpix_app_id/key, pix2tex_fallback, figures_dir
- infra/schema.sql: formulas + figures tables with HNSW pgvector indexes
- pyproject.toml: pymupdf>=1.24, pix2tex>=0.1.4
- tests/parsing/test_mathpix.py + test_figures.py: 31 tests (mock pix2tex
  + MathPix HTTP, real pymupdf on synthetic PDF)

Gate: 158 passed, ruff clean, mypy clean (20 files)
Requirements: R-10 R-11 R-12 R-13 R-14 → done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 01:16:24 +02:00
Tarik Moussa
aee00515f4 feat(wiki): concept schema loader + concepts.yaml (F-12) 
Adds wiki/concepts.yaml (5 curated seeds: discrete-conformal-map,
circle-packing, lobachevsky-function, discrete-yamabe-flow,
hyperideal-tetrahedron) and codex/wiki.py with full load_concepts()
implementation (Concept dataclass, YAML parser, graceful empty list).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-13 20:22:51 +02:00
Tarik Moussa
65918cc80a feat(tests): scaffold + integration smoke tests (config, db, models, pipeline) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-05 07:40:25 +02:00
Tarik Moussa
4c177a0d29 fix(cli): use datetime() for CodeLink.added_at in tests (mypy strict) 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-05 07:34:51 +02:00
Tarik Moussa
2acb2ee213 feat(cli): wire all commands via Typer (ingest, search, discover, provenance, ask-stub) 
- Implement all CLI commands as specified in F-07: ingest, search, and discover subcommands (leads, citing, cited-by, cocited), and provenance subcommands (scan, add-link, links, bib).
- Add ask command stub that exits with code 1 (not yet implemented).
- Implement comprehensive tests with mocked dependencies (no real DB/API calls).

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
 2026-06-05 07:22:03 +02:00
Tarik Moussa
7675f63188 feat(discover,provenance): graph discovery queries + @cite scan + code_links + bib export 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-05 07:13:01 +02:00
Tarik Moussa
5344975bb1 feat(ingest): end-to-end idempotent ingest pipeline 
Implements codex/ingest.py with:
- OpenAlex primary / Semantic Scholar fallback metadata fetch
- Abstract dense embedding (zero-vector for missing abstracts)
- Idempotent paper upsert (ON CONFLICT DO UPDATE)
- Source file parsing (.tex via latex_to_text, .pdf via nougat + grobid)
- Chunk deletion + bulk re-insert with dense embeddings
- Citation merge from OpenAlex/S2 API and GROBID PDF refs (dedup via set)
- 7 tests covering basic, tex, pdf, not-found, idempotent, arxiv-fallback,
  and no-abstract scenarios (all mocked, offline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-05 07:04:12 +02:00
Tarik Moussa
fbf8949579 feat: merge F-04 embed layer (BGE-M3 dense+sparse via FlagEmbedding) 2026-06-05 06:56:17 +02:00
Tarik Moussa
c33fd9d697 feat: merge F-03 parsing layer (LaTeX, Nougat, GROBID) 
Review-Gate: APPROVE (Opus, cold, 2 passes)
23 tests, ruff+mypy clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-05 06:48:49 +02:00
Tarik Moussa
52737abe3d fix(sources): correct OpenAlex ID prefixing, citations endpoint, S2 rate-limit 
Review-Gate findings:
- openalex: bare DOIs/arXiv IDs need doi:/arxiv: prefix (bare IDs 404);
  add _resolve_id(); fix fetch_citations to use referenced_works field
  instead of non-existent /works/{id}/references endpoint.
- semanticscholar: wait_fixed(1) was inter-retry only; add per-request
  monotonic rate-limiter (_rate_limit()) before each httpx call.
  Add _is_retryable() filter so 404s don't burn 5 retry slots.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-04 23:49:35 +02:00
Tarik Moussa
1698f7dcad feat(embed): BGE-M3 dense+sparse via FlagEmbedding (ADR-0002) 
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-04 23:48:06 +02:00
Tarik Moussa
d1ffea1c1c fix(parsing): case-insensitive arXiv idno match; fix tmp_path types; add extract_structure test 
Review-Gate finding: GROBID emits type="arXiv" (camel-case), not "arxiv".
XPath literal match silently returned empty strings for all real responses.
Fix: iterate idno elements and compare .lower() == "arxiv".
Test fixture updated to reflect real GROBID output (type="arXiv").

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-04 23:47:18 +02:00
Tarik Moussa
b7f19b6e2c feat(sources): OpenAlex, SemanticScholar, arXiv API clients 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-04 23:31:15 +02:00
Tarik Moussa
c6a428d335 feat(parsing): LaTeX, Nougat, GROBID parsers with chunking 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-04 23:30:58 +02:00