codex-py/docs/adr/ADR-F15-literature-graph.md

ADR-F15 — Literature Graph / Citation PageRank

Status: IMPL — GO after live-corpus spike 2026-06-15. See Audit Update (2026-06-15) at the end: the ID-format mismatch is now resolved in code (audit C-1, C-7) and the Spike Result numbers below predate that resolution. Owners: F-15 Worker (Sonnet) Last updated: 2026-06-15 (audit addendum)

---

Context

F-15 adds a citation graph layer on top of the corpus DB. The goals are:

1. Hub detection — identify which papers are most central in the citation network (PageRank) so that codex search paper --cite-boost can up-weight them in dense retrieval.
2. Bibliographic coupling / co-citation — surface related papers that share many references or are frequently cited together.
3. Dangling-citation discovery — enumerate works cited by the corpus but not yet ingested, serving as an acquisition queue.

Primary risk: degenerate PageRank on a small, sparse corpus producing scores so flat that the boost has no practical effect or — worse — distorts results by promoting low-quality dangling works.

---

Spike Result (2026-06-15)

Run on the live Jetson corpus: 29 ingested papers, 590 citation edges, 495 total graph nodes, 478 dangling nodes.

Hub-Paper Ranking

Rank	OpenAlex ID	Year	Authors	Title	PR
1	W1974956622	1993	Pinkall, Polthier	Computing Discrete Minimal Surfaces and Their Conjugates	0.002282
2	W1964119857	1962	Ford, Fulkerson	Flows in Networks	0.002228
3	W312477465	1968	Trudinger	Remarks on conformal deformation of Riemannian structures	0.002204
4	W1595775335	1960	Yamabe	On a deformation of Riemannian structures on compact manifolds	0.002204
5	W2023551584	1996	Cooper, Rivin	Combinatorial scalar curvature and rigidity of ball packings	0.002204
6	W2159531493	1984	Schoen	Conformal deformation of a Riemannian metric to constant scalar curvature	0.002204
7	W2163787581	2007	Bobenko, Springborn	A Discrete Laplace–Beltrami Operator for Simplicial Surfaces	0.002195
8	W3099917509	2012	Mercat	Discrete Riemann Surfaces and the Ising Model	0.002190
9	W2021879624	2008	Mullen, Tong, et al.	Spectral Conformal Parameterization	0.002164
10	W1976584051	2002	Desbrun, Meyer, et al.	Intrinsic Parameterizations of Surface Meshes	0.002159

Validation Against Manual Expectation

Criterion	Target	Result
Bobenko or Springborn in top-10 hubs	✓ expected hub	PASS — Bobenko & Springborn 2007 at rank 7
Pinkall & Polthier 1993 prominent	✓ foundational DGP paper	PASS — rank 1
Score flatness acceptable	< 2× spread over top-10	PASS — 5.7 % spread (0.002282 → 0.002159)
Graceful small-corpus warning	shown when < 15 in-KB papers	PASS — shown at 29 in-KB papers (< graph_min_corpus_size=15 not triggered; Springborn 2008 is not in-KB so not visible as "ingested hub")

GO — PageRank produces a plausible field-specific ranking. The top hits (Pinkall/Polthier, Yamabe, Schoen, Bobenko/Springborn) are exactly the foundational works a DGP researcher would expect to see elevated.

---

Decision

Graph Construction

build_citation_graph builds an in-memory nx.DiGraph from the citations table: citing_id → cited_id. No schema migration needed.

• citing_id stores DOIs (matches papers.id).
• cited_id stores OpenAlex IDs (external references).

Consequence: in the current corpus, ingested papers do not appear as in-degree targets of each other — inter-KB citations are structurally invisible because the cited_id column uses OpenAlex IDs while papers.id uses DOIs. The graph is therefore bipartite-like (29 DOI-nodes → 478 OpenAlex-nodes), and all PageRank mass concentrates on the dangling OpenAlex-nodes cited by many of our 29 papers.

This is acceptable for the --cite-boost use-case: dangling hub papers are foundational works our corpus cites repeatedly, so boosting search results that match them is desirable.

PageRank Graceful Degradation

citation_pagerank returns a uniform distribution when the graph has fewer than 5 nodes (_MIN_PAGERANK_NODES) and logs a warning.

The CLI graph report emits a separate user-visible warning when the count of in-KB papers is below graph_min_corpus_size (default 15) — because at that point the hub ordering (which uses PageRank over all 495 nodes) is still valid, but its discriminative power is limited by the few citing sources.

At 29 ingested papers, graph_min_corpus_size=15 is satisfied; the score spread is narrow (5.7 %) but the ranking order is meaningful.

cite-boost Formula

boosted_distance = distance / (1.0 + alpha * pr_score)

alpha = 0.3 (config: graph_cite_boost_alpha). Dividing reduces cosine distance for high-PR papers (lower = closer in pgvector). Multiplying would invert the boost — this was the critical bug corrected during the F-15 review gate.

At alpha=0.3 and the observed PR scores (~0.002), the maximum boost is approximately 0.06 % per unit score — effectively a tie-breaker on equally relevant papers rather than a hard re-ranking. This is intentional: we do not want citation authority to override semantic relevance for queries that genuinely target niche papers.

Dangling Citations as Acquisition Queue

dangling_citations(graph, known_ids) returns the 478 OpenAlex IDs that are cited but not ingested. codex graph report surfaces these so the researcher can prioritize which papers to add to the corpus next.

The top hubs (rank 1–10) are the highest-value acquisition targets: adding Pinkall & Polthier 1993, Yamabe 1960, and Schoen 1984 would improve both retrieval quality and PageRank discrimination.

---

Fallback

If the corpus has < _MIN_PAGERANK_NODES (5) nodes: citation_pagerank returns uniform scores; --cite-boost has no effect (each paper is boosted equally); no crash.

If --cite-boost is omitted: search works as F-03/F-04 baseline.

If the citations table is empty: graph has 0 nodes; graph report prints "Graph is empty" and exits 0; all other commands degrade gracefully.

---

Known Limitation: ID-Format Mismatch

RESOLVED (audit 2026-06-15) — see Audit Update below. This no longer holds: build_citation_graph resolves OpenAlex cited_ids to the canonical papers.id via the shared RESOLVED_CITATIONS_SQL (C-1) and ingest stores canonical ids (C-7), so inter-KB citations are now edges. Original text kept for the record.

cited_id uses OpenAlex IDs; papers.id uses DOIs. Cross-citations between ingested papers are therefore not represented as graph edges. This is a D-05-class issue (same root cause as the citing_id/openalex_id FK mismatch fixed in PR #9). A future migration could add an openalex_id column to papers and populate cited_id cross-links; but the benefit is small while the corpus is < 100 papers.

---

Consequences

• R-41: build_citation_graph — in-memory DiGraph, no schema change ✓
• R-42: citation_pagerank + codex search paper --cite-boost ✓
• R-43: find_related (bibliographic coupling) + find_co_cited ✓
• R-44: codex graph report [--top-n N] [--json] ✓
• R-45: graceful degradation < 5 nodes: uniform scores, warning ✓
• R-46: this document — GO; Bobenko+Springborn 2007 at rank 7 validates hub detection; score flatness expected at corpus size 29; increase to > 100 papers for stronger PageRank differentiation

---

Audit Update (2026-06-15)

A post-hoc audit of the shipped F-15 code found that the live-corpus spike above was measured on a graph the current code no longer produces:

• ID-format mismatch resolved. build_citation_graph resolves OpenAlex cited_ids to the canonical papers.id through the shared RESOLVED_CITATIONS_SQL, now also used by codex.discover so the two cannot diverge (audit C-1). Ingest stores the caller's canonical id as papers.id instead of OpenAlex's URL-form doi (audit C-7). Inter-KB citations are therefore represented as edges — the "Known Limitation" above is obsolete.

• Spike re-measured after migration (2026-06-15). The corpus was re-ingested onto canonical bare ids (29 papers, all bare — verified url_form_ids = 0), and the graph re-run on the post-migration corpus:

  Metric	Original spike (pre-fix)	Post-migration (re-measured)
  Ingested papers	29	29
  Graph nodes	495	489
  Graph edges	590	590
  Dangling nodes	478	472
  Top-10 PR spread	5.7 %	5.4 %
  Rank-1 hub	W1974956622 (dangling)	Pinkall/Polthier 1993 — in-KB (resolved)
  Bobenko/Springborn 2007	rank 7	rank 7

  The resolution fix's signature is visible: Pinkall/Polthier 1993 now resolves to its canonical DOI and rises to rank 1 as an in-KB hub (previously a dangling OpenAlex node). Bobenko/Springborn 2007 holds rank 7, and the foundational works (Pinkall/Polthier, Yamabe, Schoen, Bobenko/Springborn) still surface. GO re-affirmed on the shipping topology.

  Migration footnotes: two papers initially survived in URL form because the id-keyed upsert did not catch a papers.openalex_id collision, and 2305.10988 failed on a papers.bibkey collision with 2310.17529 (both → BobenkoLutz2023). Same upsert gap — resolved by the bibkey-retry (audit C-15) plus a delete + re-ingest of the two stragglers; corpus is now 29/29 canonical.