Commit Graph

14 Commits

Author SHA1 Message Date
Tarik Moussa
b4fbd7930e feat(config): wrap credentials in SecretStr (audit S-4)
database_url, migration_database_url, mcp_auth_token and the mathpix app
id/key are now pydantic SecretStr, so they render as '**********' in any
repr/log/traceback instead of leaking the plaintext credential. Consumers
(db.get_conn, cli.migrate, mcp._require_http_token, mathpix.extract_formulas)
call .get_secret_value() at the point of use. Tests updated to the SecretStr
type.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 21:32:13 +02:00
Tarik Moussa
cdb7d254df feat(db): idempotent schema + 'codex migrate' command (audit M-1, T-2)
M-1: schema.sql could not be re-applied (leading CREATE TABLE/INDEX lacked
IF NOT EXISTS), apply_schema was never called, and the live migration just
failed on a missing chunks.section column. Fixes:

- schema.sql: all CREATE TABLE/INDEX now use IF NOT EXISTS — the whole file is
  re-applyable as a no-op.
- codex migrate: new CLI command that applies schema.sql. Connects via
  MIGRATION_DATABASE_URL (falls back to DATABASE_URL) and catches
  InsufficientPrivilege with guidance — because the app role is DML-only and
  core tables are owned by 'postgres' (the privilege dimension found during the
  live migration incident).
- config: migration_database_url (optional privileged connection).
- db: apply_schema docstring corrected (idempotency now true) + privilege note.
- T-2: static test that schema.sql is fully idempotent; migrate privilege-error
  test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 15:53:25 +02:00
Tarik Moussa
6193f5630d docs: fix two doc/code contradictions (audit D-2, D-4)
- D-2 config.py: graph_cite_boost_alpha described the OLD inverted formula
  (score = dense * (1 + alpha*pr)); the code divides distance. Document the
  actual boosted_distance = distance / (1 + alpha*pr).
- D-4 synthesis.py: write_leads claimed an 'append-only' INDEX.md that keeps
  previously-recorded ids; _update_index actually rebuilds from on-disk glob.
  Docstring now matches the implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 11:13:52 +02:00
Tarik Moussa
fd51b70000 feat(F-15): citation graph — PageRank, coupling, co-citation, CLI
- codex/graph.py: build_citation_graph (DiGraph from DB), citation_pagerank
  (graceful uniform fallback < 5 nodes), find_related (bibliographic coupling),
  find_co_cited, dangling_citations
- codex/config.py: graph_cite_boost_alpha=0.3, graph_min_corpus_size=15
- codex/cli.py: graph report [--top-n] [--json], graph related <paper-id>
  [--min-shared]; search paper --cite-boost (PageRank score weighting)
- tests/graph/: 39 tests (graph functions + CLI) — 326 total green

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 03:31:36 +02:00
9647897173 feat(F-16): chunk quality gate + section classification
3-signal quality filter (length/alpha-ratio/bib-score) + rule-based section classifier + retroactive CLI pass. 35 new tests, 287 total green.
2026-06-15 01:22:44 +00:00
Tarik Moussa
50e6bb8210 fix(config): remove duplicate F-13 settings block (merge artefact)
F-13 settings (leads_dir, synthesis_llm_*, synthesis_top_k,
synthesis_min_grounded_ratio, synthesis_gap_min_coverage) appeared twice
due to independent commits on F-13 and F-14 branches. Kept the second
(more complete) block; removed the first.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 08:59:05 +02:00
Tarik Moussa
b803811701 feat(mcp): FastMCP server exposing read-only KB tools (stdio)
Implements F-14: thin FastMCP wrapper over existing codex domain modules.
Seven read-only tools: search, ask, wiki_read, wiki_list, discover_leads,
provenance_verify, synthesis_browse. All optional-feature tools degrade
gracefully to {"error": "feature not available"} instead of crashing.
Adds mcp[cli]>=1.0 dependency and codex-mcp console_script entry-point.
28 new tests across test_server, test_search, test_readonly, test_graceful,
test_http_auth — all green; 0 regressions in existing 172 tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 08:55:49 +02:00
Tarik Moussa
8cf0cc7e01 feat(synthesis): Lead/Provenance model + grounded leads + conjecture generator
- codex/synthesis.py: Lead/Provenance dataclasses, find_connections/gaps/improvements,
  propose_conjectures (status=unverified, quarantined in leads/conjectures/),
  write_leads (grounded→leads/grounded/, conjectures→leads/conjectures/ HARD invariant)
- codex/cli.py: synthesis leads/conjectures/report command group
- codex/config.py: F-13 settings (leads_dir, synthesis_llm_*, synthesis_top_k,
  synthesis_min_grounded_ratio, synthesis_gap_min_coverage)
- tests/synthesis/: 42 tests covering model, grounding, conjecture invariant, quarantine, CLI
- spike/: F-13 spike script + output (live run attempted)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 08:37:42 +02:00
Tarik Moussa
408e4886bb fix(wiki): basic conflict detection in CompileReport
Add _detect_conflicts() using adversative keyword heuristic (but,
however, in contrast, …) between chunks of different bibkeys; results
populate CompileReport.conflicts and appear in wiki/log.md.
Also add CompileReport.quarantined field (used by Fix 2).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 05:11:06 +02:00
Tarik Moussa
1df9be6563 feat(F-09): rich parsing — formula + figure extraction
- codex/parsing/mathpix.py: pix2tex (local, CPU) primary + MathPix API
  optional; bbox heuristic h>15px, math-char-count>5; singleton model cache
- codex/parsing/figures.py: pymupdf embedded-image extraction → PNG;
  caption detection via proximity + "Figure/Fig./Abbildung" prefix
- codex/models.py: FormulaChunk + FigureChunk dataclasses (R-10/R-11)
- codex/ingest.py: --rich flag wires formula+figure extraction post-ingest
- codex/cli.py: search_app sub-typer (paper + formula subcommands),
  --rich flag on ingest; wiki_app from F-12 preserved intact
- codex/config.py: mathpix_app_id/key, pix2tex_fallback, figures_dir
- infra/schema.sql: formulas + figures tables with HNSW pgvector indexes
- pyproject.toml: pymupdf>=1.24, pix2tex>=0.1.4
- tests/parsing/test_mathpix.py + test_figures.py: 31 tests (mock pix2tex
  + MathPix HTTP, real pymupdf on synthetic PDF)

Gate: 158 passed, ruff clean, mypy clean (20 files)
Requirements: R-10 R-11 R-12 R-13 R-14 → done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 01:16:24 +02:00
Tarik Moussa
d2f9141a5c fix(cli): remove F-09 spillover from F-12 branch, keep only wiki group
cli.py and config.py were accidentally contaminated with uncommitted F-09
changes (search_app restructure, rich ingest, mathpix/figures settings).
This restores both files to main-baseline + F-12-only additions:
- cli.py: wiki_app group (compile/list/check), restore @app.command search
- config.py: wiki_dir, wiki_llm_model, wiki_llm_url, wiki_top_k only

All 127 tests green, ruff+mypy clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 20:28:22 +02:00
Tarik Moussa
4d361fd8dc feat(cli): wiki compile/list/check command group (F-12)
Adds codex wiki compile/list/check Typer subgroup to cli.py.
Adds wiki_dir, wiki_llm_model, wiki_llm_url, wiki_top_k settings
to config.py (F-12 section only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-13 20:22:55 +02:00
Tarik Moussa
c6a428d335 feat(parsing): LaTeX, Nougat, GROBID parsers with chunking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 23:30:58 +02:00
Tarik Moussa
e7ac7766a5 feat: initial project scaffold
pyproject.toml (Python 3.12, uv), codex/ package (config, db, models),
infra/ (docker-compose + schema), .env.example, .gitignore, README.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 11:19:10 +02:00