12/29 papers had zero citations: OpenAlex indexes arXiv preprints and theses
without a parsed reference list (verified referenced_works_count=0 server-side
for all 11 with an openalex_id). Not an ingest bug — a source-coverage limit.
- ingest.py: when OpenAlex returns no references, fall back to a Semantic
Scholar reference supplement (_s2_reference_supplement); citing_id rewritten
to canonical papers.id, DOI cited-ids lowercased. Removed a now-redundant
discarded S2 probe in the OpenAlex-404 arXiv branch.
- semanticscholar.py: fix TypeError on S2's HTTP-200 {"data": null} no-refs
responses (.get("data", []) returns None when the key is present-but-null).
- tests: regression test for the OpenAlex-empty -> S2 path.
- Live DB backfilled idempotently: +330 citations (590->920), zero-out-edge
papers 12->2, citing coverage 17->27/29. Only the 2 TU-Berlin depositonce
theses remain (no references in any source).
- docs: DATA-QUALITY-2026-06-15.md — DQ-1 resolution, DQ-4 result, and a
free-source acquisition roadmap (R-A..R-E).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second audit axis (data, not code): is the information in the DB actually good?
Baseline scan of the live 29-paper corpus + four open investigation items
prepared self-contained for a cold session.
- Good: chunks 0/1507 fail the F-16 re-gate, healthy sizing, 0 dups.
- DQ-1 (HIGH): 12/29 papers have zero citations; 11 of them have an openalex_id,
so OpenAlex returned empty referenced_works (source-coverage limit, not a bug)
— propose an S2 references supplement.
- DQ-2 (MED): 3 papers no abstract; 1911.00966 fully metadata-less (OpenAlex 404).
- DQ-3 (MED): chunk-vs-source fidelity not yet verified.
- DQ-4 (MED): retrieval relevance not yet measured.
Includes connection steps, exact ids, priorities, and a reproduce appendix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The C-7 re-ingest hit M-1 live: TRUNCATE succeeded, then ingest failed on the
missing chunks.section column (schema.sql never applied to the live DB). Adding
the column then failed with 'must be owner' — the app user 'researcher' has DML
+ TRUNCATE but not DDL on the postgres-owned core tables. M-1's fix therefore
needs a privileged migration role, not just an idempotent schema. Records the
unblock and the helper's new preflight.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Part D synthesis: C-11 HIGH (lead-ID collision → silent overwrite), R-8 MED, D-4 LOW.
- Part E wiki-rest: R-9 MED (cross-ref injection corrupts LaTeX math), C-12/C-13/R-10/R-11 LOW.
- Part F embed/quality/models: C-14/R-12 LOW + C-7 corroboration. Cleanest modules.
- Part G cli/config: C-15 MED (embedding_dim vs hardcoded vector(1024)), R-13/S-4/R-14/D-5 LOW.
Executive summary: 45 findings total (8 HIGH, 11 MED, 26 LOW). Five root themes;
dominant one is un-normalized identity (C-1/C-7/C-10/M-1). Whole repo reviewed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes request (a) — the second pass over loop-1 §8 open items:
- Part A grounding guard: C-4 (HIGH, zero-claim pages bypass quarantine),
C-5/R-1 MED, C-6/R-2/R-3 LOW.
- Part B sources/: C-7 (HIGH, papers.id stored in OpenAlex URL form vs the
bare-ID contract), T-3 MED, R-4/R-5/C-9 LOW.
- Part C parsing/: C-10 MED (formulas/figures key on filename not paper.id),
R-6/R-7 LOW — all latent (live corpus is .txt).
Running tally: 30 findings, 7 HIGH. Common root: un-normalized identity.
Parts D-G (synthesis/wiki/embed/cli) still open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit of main @5dfa6e4: 4 HIGH (S-1 secret-in-tree, M-1 missing migration
path, C-1 unpropagated ID-mismatch fix in discover.py, D-1 stale ADR-F15
validation, T-1 untested resolution SQL) plus MED/LOW findings. Findings
only; remediation planning deferred to the next loop phase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
