feat(F-16): chunk quality gate + section classification
- codex/quality.py: 3-signal filter (length / alpha-ratio / bib-score) + rule-based section classifier + run_quality_pass retroactive DB pass - codex/ingest.py: promote quality imports to module level; apply filter_chunks before embedding; store section column in chunks INSERT - codex/config.py: CHUNK_MIN_CHARS / CHUNK_MIN_ALPHA_RATIO / CHUNK_MAX_BIB_SCORE - infra/schema.sql: ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT - .env.example: document F-16 quality thresholds - codex/cli.py: quality run sub-command (scope by --paper-id or all papers) - tests/quality/: 35 new tests covering all quality functions - tests/ingest/: patch filter_chunks in source-path tests to isolate ingest Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -122,3 +122,6 @@ CREATE TABLE IF NOT EXISTS figures (
|
||||
CREATE INDEX IF NOT EXISTS figures_emb_idx
|
||||
ON figures USING hnsw (embedding vector_cosine_ops);
|
||||
CREATE INDEX IF NOT EXISTS figures_paper_idx ON figures (paper_id);
|
||||
|
||||
-- F-16 Chunk Quality Gate: section classification
|
||||
ALTER TABLE chunks ADD COLUMN IF NOT EXISTS section TEXT;
|
||||
|
||||
Reference in New Issue
Block a user