merge: reconcile data-quality roadmap with audit-remediation main (#12-15)
Resolved 4 conflicts: cli.py (R-D citing-coverage warning + main's small-corpus JSON/hub-title changes both kept); ingest.py (C-7 paper.id=caller-id pin first, then DQ-2 abstract recovery); test_ingest.py + test_openalex.py (kept both branches' tests). C-7/DQ-5 overlap (both canonicalise papers.id): kept both layers — openalex._canonical_id normalizes fetch_paper's id (DQ-5); ingest pins papers.id to the caller's bare id (C-7); they converge on the bare canonical id. Fixed the auto-merged test_fetch_paper_success that had contradictory bare/URL assertions. 402 tests pass; ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,10 @@
|
||||
# ADR-F15 — Literature Graph / Citation PageRank
|
||||
|
||||
**Status:** IMPL (GO after live-corpus spike 2026-06-15)
|
||||
**Status:** IMPL — GO after live-corpus spike 2026-06-15. See **Audit Update
|
||||
(2026-06-15)** at the end: the ID-format mismatch is now resolved in code
|
||||
(audit C-1, C-7) and the Spike Result numbers below predate that resolution.
|
||||
**Owners:** F-15 Worker (Sonnet)
|
||||
**Last updated:** 2026-06-15
|
||||
**Last updated:** 2026-06-15 (audit addendum)
|
||||
|
||||
---
|
||||
|
||||
@@ -138,6 +140,12 @@ If the `citations` table is empty: graph has 0 nodes; `graph report` prints
|
||||
|
||||
## Known Limitation: ID-Format Mismatch
|
||||
|
||||
> **RESOLVED (audit 2026-06-15) — see Audit Update below.** This no longer holds:
|
||||
> `build_citation_graph` resolves OpenAlex `cited_id`s to the canonical
|
||||
> `papers.id` via the shared `RESOLVED_CITATIONS_SQL` (C-1) and ingest stores
|
||||
> canonical ids (C-7), so inter-KB citations *are* now edges. Original text kept
|
||||
> for the record.
|
||||
|
||||
`cited_id` uses OpenAlex IDs; `papers.id` uses DOIs. Cross-citations
|
||||
between ingested papers are therefore not represented as graph edges.
|
||||
This is a D-05-class issue (same root cause as the `citing_id`/`openalex_id`
|
||||
@@ -155,3 +163,42 @@ small while the corpus is < 100 papers.
|
||||
- R-44: `codex graph report [--top-n N] [--json]` ✓
|
||||
- R-45: graceful degradation < 5 nodes: uniform scores, warning ✓
|
||||
- R-46: **this document** — GO; Bobenko+Springborn 2007 at rank 7 validates hub detection; score flatness expected at corpus size 29; increase to > 100 papers for stronger PageRank differentiation
|
||||
|
||||
---
|
||||
|
||||
## Audit Update (2026-06-15)
|
||||
|
||||
A post-hoc audit of the shipped F-15 code found that the live-corpus spike above
|
||||
was measured on a graph the current code no longer produces:
|
||||
|
||||
- **ID-format mismatch resolved.** `build_citation_graph` resolves OpenAlex
|
||||
`cited_id`s to the canonical `papers.id` through the shared
|
||||
`RESOLVED_CITATIONS_SQL`, now also used by `codex.discover` so the two cannot
|
||||
diverge (audit C-1). Ingest stores the caller's canonical id as `papers.id`
|
||||
instead of OpenAlex's URL-form doi (audit C-7). Inter-KB citations are
|
||||
therefore represented as edges — the "Known Limitation" above is obsolete.
|
||||
- **Spike re-measured after migration (2026-06-15).** The corpus was re-ingested
|
||||
onto canonical bare ids (29 papers, all bare — verified `url_form_ids = 0`), and
|
||||
the graph re-run on the post-migration corpus:
|
||||
|
||||
| Metric | Original spike (pre-fix) | Post-migration (re-measured) |
|
||||
|--------|--------------------------|------------------------------|
|
||||
| Ingested papers | 29 | 29 |
|
||||
| Graph nodes | 495 | 489 |
|
||||
| Graph edges | 590 | 590 |
|
||||
| Dangling nodes | 478 | 472 |
|
||||
| Top-10 PR spread | 5.7 % | 5.4 % |
|
||||
| Rank-1 hub | W1974956622 (dangling) | **Pinkall/Polthier 1993 — in-KB (resolved)** |
|
||||
| Bobenko/Springborn 2007 | rank 7 | rank 7 |
|
||||
|
||||
The resolution fix's signature is visible: Pinkall/Polthier 1993 now resolves to
|
||||
its canonical DOI and rises to **rank 1 as an in-KB hub** (previously a dangling
|
||||
OpenAlex node). Bobenko/Springborn 2007 holds rank 7, and the foundational works
|
||||
(Pinkall/Polthier, Yamabe, Schoen, Bobenko/Springborn) still surface. **GO
|
||||
re-affirmed** on the shipping topology.
|
||||
|
||||
Migration footnotes: two papers initially survived in URL form because the
|
||||
id-keyed upsert did not catch a `papers.openalex_id` collision, and `2305.10988`
|
||||
failed on a `papers.bibkey` collision with `2310.17529` (both →
|
||||
`BobenkoLutz2023`). Same upsert gap — resolved by the bibkey-retry (audit C-15)
|
||||
plus a delete + re-ingest of the two stragglers; corpus is now 29/29 canonical.
|
||||
|
||||
Reference in New Issue
Block a user