fix(wiki): token-boundary grounding match + sharper ref-list filter (C-6, R-2)
- C-6: the content-5-gram check used 'gram in src' on space-joined strings, so the
first/last gram token could match a prefix/suffix of a longer chunk word
('set' inside 'subset'). Both gram and chunk are now space-padded, so a match
requires whole-token alignment. Regression test added.
- R-2: the reference-list filter's author pattern ('Surname, I.') also matched
'Theorem A.'/'Lemma B.', so theorem-dense chunks could be dropped as a
bibliography. The pattern now excludes common math-structure words.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -230,6 +230,30 @@ def test_grounding_guard_robust_to_trailing_comma() -> None:
|
||||
assert result[0].grounded is True
|
||||
|
||||
|
||||
def test_grounding_guard_no_substring_bleed_across_tokens() -> None:
|
||||
"""A 5-gram must match whole tokens, not bleed into a longer chunk word (audit C-6).
|
||||
|
||||
The chunk contains 'subset'; a claim whose first token is 'set' must NOT ground
|
||||
by substring-matching the suffix of 'subset'.
|
||||
"""
|
||||
chunks = [
|
||||
{
|
||||
"id": 99,
|
||||
"paper_id": "p",
|
||||
"ord": 0,
|
||||
"bibkey": "TestBib",
|
||||
"content": "the subset conformal map energy functional is minimized here",
|
||||
}
|
||||
]
|
||||
claim = Claim(
|
||||
text="the set conformal map energy functional",
|
||||
bibkey="TestBib",
|
||||
locator="chunk 0",
|
||||
)
|
||||
result = _run_grounding_guard([claim], chunks)
|
||||
assert result[0].grounded is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _inject_cross_refs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user