docs: clarify sparse-not-wired and cite-boost tie-breaker (audit C-14, U-2)
- C-14: embed.py header now states that only the dense path is wired (search 'hybrid' is dense + Postgres FTS); encode_sparse/encode are reserved for a future sparse-retrieval layer, not yet consumed. - U-2: --cite-boost help clarifies it re-ranks within the top results (a tie-breaker), not a hard re-ranking — matching the small alpha effect. Accepted (documented heuristics, no change): R-3 conflict detection is a keyword-only signal labelled as such; R-12 classify_section is mostly 'body' because chunks are word-windows that rarely start at a section header. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
"""Hybrid dense + sparse embeddings via BGE-M3 (ADR-0002).
|
||||
"""Dense (+ optional sparse) embeddings via BGE-M3 (ADR-0002).
|
||||
|
||||
Uses :class:`FlagEmbedding.BGEM3FlagModel` for encoding because the
|
||||
``return_dense`` / ``return_sparse`` kwargs are part of the FlagEmbedding
|
||||
@@ -7,6 +7,11 @@ vanilla ``SentenceTransformer`` load of ``BAAI/bge-m3`` (same weights,
|
||||
same model); sparse output is a list of ``{token_id: weight}`` dicts per
|
||||
text.
|
||||
|
||||
Status: only the **dense** path is wired into ingest/search today; the search
|
||||
"hybrid" is dense + Postgres FTS. :meth:`Embedder.encode_sparse` / :meth:`encode`
|
||||
are provided for a future sparse-retrieval layer but are not yet consumed by the
|
||||
pipeline (audit C-14).
|
||||
|
||||
Notes for callers
|
||||
-----------------
|
||||
* Empty input is handled explicitly — no model call is issued.
|
||||
|
||||
Reference in New Issue
Block a user