feat(F-12): wiki-compile — grounded concept pages over RAG substrate #2

Merged
user2595 merged 7 commits from feat/F-12-wiki-compile into main 2026-06-14 05:58:25 +00:00

7 Commits

Author SHA1 Message Date
Tarik Moussa
967e6baa59 feat(wiki): ADR-F12-wiki-compile.md — GO decision + grounding measurement
Document Spike result (grounding ~0.95, hallucination ≤5% on 3 concepts),
content-5-gram guard design, quarantine mechanism, LLM graceful degradation,
cross-ref injection rules, and conflict detection MVP.
R-27 satisfied.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 05:11:06 +02:00
Tarik Moussa
7d338d29ca fix(wiki): sentence-level claim granularity + content-5-gram grounding guard
Replace 3-gram substring match with:
- _last_sentence(): only the last sentence before a citation is checked
  (prevents a hallucinated paragraph grounding via a phrase at its end)
- _content_words(): stopwords removed from both claim and chunk before
  n-gram comparison (prevents bypass via "the discrete conformal map")
- Content-5-gram: require 5 consecutive non-stopword tokens from last
  sentence to appear in the chunk's content-word stream
- Claims with <5 content words in last sentence → ungrounded by default

Add adversarial tests: stopword-3-gram bypass → grounded=False;
legitimate content-5-gram → grounded=True.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 05:11:06 +02:00
Tarik Moussa
408e4886bb fix(wiki): basic conflict detection in CompileReport
Add _detect_conflicts() using adversative keyword heuristic (but,
however, in contrast, …) between chunks of different bibkeys; results
populate CompileReport.conflicts and appear in wiki/log.md.
Also add CompileReport.quarantined field (used by Fix 2).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 05:11:06 +02:00
Tarik Moussa
ef46d14aed fix(wiki): protect inline-code spans from cross-ref injection
_inject_cross_refs temporarily replaces backtick spans with null-byte
placeholders before injecting [[slug]] cross-refs, then restores them.
This prevents concept titles inside `code` from being rewritten.
Add tests for inline-code protection and full-list cross-ref injection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 03:58:02 +02:00
Tarik Moussa
0202a2266e fix(wiki): pass chunks explicitly to compile_concept (single retrieve)
compile_concept now accepts chunks as a positional parameter; compile_all
retrieves chunks once and passes them directly, avoiding double-retrieve.
Also separates all_concepts (full list) from compile_concepts (filtered),
so cross-ref injection always uses the complete concept list regardless of
--concept filter (Fix 6 + Fix 8).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 03:58:02 +02:00
Tarik Moussa
60e40f6f36 fix(wiki): exclude URL-shaped bibkeys from claim parsing
_parse_claims now skips matches where bibkey contains spaces or starts
with "http", preventing Markdown hyperlinks like [text](https://...) from
being misidentified as citations. Add two tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 03:58:02 +02:00
Tarik Moussa
89228a1f62 fix(wiki): LLM error handling — graceful degradation on Ollama unavailable
Wrap llm.generate() in compile_concept with a broad try/except;
log a warning and return an empty ConceptPage (no crash) when the
LLM endpoint is unreachable. Add logging module import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 03:58:02 +02:00