user2595/codex-py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ingest): D-04 bibkey heuristic + D-05a/b/c regression tests + __main__

Browse Source 
D-04: add _make_bibkey(paper) — ConcatSurnames+Year, ≤3 authors full list,
      >3 authors FirstEtAlYear. Applied in ingest_paper when paper.bibkey is
      None. ON CONFLICT now fills NULL bibkeys via COALESCE(papers.bibkey,
      EXCLUDED.bibkey) while preserving manually-set values.

D-05b: add codex/__main__.py so `python -m codex` dispatches to the CLI.

D-05c: fix test_ingest.py:75 — rename var to raw_api_citations (the raw
       OpenAlex payload with citing_id=openalex_id) and add regression
       assertion: citing_id written to DB must equal paper.id (DOI), not
       paper.openalex_id.

D-05a (proxy): add TestEntryPoint.test_python_m_codex_help — runs
       `sys.executable -m codex --help` via subprocess, covering the
       non-uv-run entry path. 253 tests green, ruff+mypy clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Tarik Moussa
2026-06-15 03:07:06 +02:00
parent 335ca2c923
commit b87087d3c6
4 changed files with 139 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
codex/ingest.py
View File

@@ -27,6 +27,25 @@ class IngestResult:
figures_upserted: int = 0
def _make_bibkey(paper: Paper) -> str | None:
"""Generate a BibTeX-style key from paper metadata.
Format: ConcatSurnames + Year (e.g. "BobenkoPinkallSpringborn2015").
Surnames are the last whitespace-separated token of each author name.
>3 authors: FirstSurname + "EtAl" + Year.
Returns None when authors or year are missing.
"""
if not paper.authors or not paper.year:
return None
surnames = [name.split()[-1] for name in paper.authors if name.strip()]
if not surnames:
return None
year = str(paper.year)
if len(surnames) <= 3:
return "".join(surnames) + year
return surnames[0] + "EtAl" + year
def ingest_paper(
paper_id: str,
source_path: str | None = None,
@@ -61,6 +80,7 @@ def ingest_paper(
# ---------------------------------------------------------------
paper: Paper | None = openalex.fetch_paper(paper_id)
if paper is None:
# Detect arXiv IDs: numeric pattern like "2301.07041" or "arxiv:..." prefix
pid_lower = paper_id.lower()
@@ -78,6 +98,10 @@ def ingest_paper(
if paper is None:
raise ValueError(f"Paper not found: {paper_id}")
# Auto-generate bibkey when source has none (D-04).
if paper.bibkey is None:
paper.bibkey = _make_bibkey(paper)
# ---------------------------------------------------------------
# 2. Embed abstract (dense only — schema has one vector column)
# ---------------------------------------------------------------
@@ -102,6 +126,7 @@ def ingest_paper(
%(abstract)s, %(source_path)s, %(abstract_emb)s)
ON CONFLICT (id) DO UPDATE SET
openalex_id = EXCLUDED.openalex_id,
bibkey = COALESCE(papers.bibkey, EXCLUDED.bibkey),
title = EXCLUDED.title,
authors = EXCLUDED.authors,
year = EXCLUDED.year,
@@ -181,7 +206,10 @@ def ingest_paper(
raw = openalex.fetch_citations(paper.openalex_id)
# OpenAlex returns citing_id=openalex_id, but papers.id uses the DOI.
# Rewrite citing_id to paper.id so the FK constraint is satisfied.
api_citations = [Citation(citing_id=paper.id, cited_id=c.cited_id, context=c.context) for c in raw]
api_citations = [
Citation(citing_id=paper.id, cited_id=c.cited_id, context=c.context)
for c in raw
]
else:
api_citations = semanticscholar.fetch_references(paper_id)