docs: clarify sparse-not-wired and cite-boost tie-breaker (audit C-14, U-2)
- C-14: embed.py header now states that only the dense path is wired (search 'hybrid' is dense + Postgres FTS); encode_sparse/encode are reserved for a future sparse-retrieval layer, not yet consumed. - U-2: --cite-boost help clarifies it re-ranks within the top results (a tie-breaker), not a hard re-ranking — matching the small alpha effect. Accepted (documented heuristics, no change): R-3 conflict detection is a keyword-only signal labelled as such; R-12 classify_section is mostly 'body' because chunks are word-windows that rarely start at a section header. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -102,7 +102,8 @@ def search_paper(
|
|||||||
cite_boost: bool = typer.Option(
|
cite_boost: bool = typer.Option(
|
||||||
False,
|
False,
|
||||||
"--cite-boost",
|
"--cite-boost",
|
||||||
help="Weight results by citation PageRank (F-15). Graceful when corpus < 5 papers.",
|
help="Re-rank the top results by citation PageRank — a within-page "
|
||||||
|
"tie-breaker, not a hard re-ranking (F-15). Graceful when corpus < 5 papers.",
|
||||||
),
|
),
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Semantic similarity search over paper abstracts."""
|
"""Semantic similarity search over paper abstracts."""
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Hybrid dense + sparse embeddings via BGE-M3 (ADR-0002).
|
"""Dense (+ optional sparse) embeddings via BGE-M3 (ADR-0002).
|
||||||
|
|
||||||
Uses :class:`FlagEmbedding.BGEM3FlagModel` for encoding because the
|
Uses :class:`FlagEmbedding.BGEM3FlagModel` for encoding because the
|
||||||
``return_dense`` / ``return_sparse`` kwargs are part of the FlagEmbedding
|
``return_dense`` / ``return_sparse`` kwargs are part of the FlagEmbedding
|
||||||
@@ -7,6 +7,11 @@ vanilla ``SentenceTransformer`` load of ``BAAI/bge-m3`` (same weights,
|
|||||||
same model); sparse output is a list of ``{token_id: weight}`` dicts per
|
same model); sparse output is a list of ``{token_id: weight}`` dicts per
|
||||||
text.
|
text.
|
||||||
|
|
||||||
|
Status: only the **dense** path is wired into ingest/search today; the search
|
||||||
|
"hybrid" is dense + Postgres FTS. :meth:`Embedder.encode_sparse` / :meth:`encode`
|
||||||
|
are provided for a future sparse-retrieval layer but are not yet consumed by the
|
||||||
|
pipeline (audit C-14).
|
||||||
|
|
||||||
Notes for callers
|
Notes for callers
|
||||||
-----------------
|
-----------------
|
||||||
* Empty input is handled explicitly — no model call is issued.
|
* Empty input is handled explicitly — no model call is issued.
|
||||||
|
|||||||
Reference in New Issue
Block a user