fix(wiki): token-boundary grounding match + sharper ref-list filter (C-6, R-2)

- C-6: the content-5-gram check used 'gram in src' on space-joined strings, so the
  first/last gram token could match a prefix/suffix of a longer chunk word
  ('set' inside 'subset'). Both gram and chunk are now space-padded, so a match
  requires whole-token alignment. Regression test added.
- R-2: the reference-list filter's author pattern ('Surname, I.') also matched
  'Theorem A.'/'Lemma B.', so theorem-dense chunks could be dropped as a
  bibliography. The pattern now excludes common math-structure words.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-15 21:46:02 +02:00
parent 92928bab99
commit f422b0fb84
2 changed files with 37 additions and 5 deletions

View File

@@ -230,6 +230,30 @@ def test_grounding_guard_robust_to_trailing_comma() -> None:
assert result[0].grounded is True
def test_grounding_guard_no_substring_bleed_across_tokens() -> None:
"""A 5-gram must match whole tokens, not bleed into a longer chunk word (audit C-6).
The chunk contains 'subset'; a claim whose first token is 'set' must NOT ground
by substring-matching the suffix of 'subset'.
"""
chunks = [
{
"id": 99,
"paper_id": "p",
"ord": 0,
"bibkey": "TestBib",
"content": "the subset conformal map energy functional is minimized here",
}
]
claim = Claim(
text="the set conformal map energy functional",
bibkey="TestBib",
locator="chunk 0",
)
result = _run_grounding_guard([claim], chunks)
assert result[0].grounded is False
# ---------------------------------------------------------------------------
# _inject_cross_refs
# ---------------------------------------------------------------------------