b2ef8f2575
Documents MCP server setup for Claude Code (.mcp.json) and Claude Desktop (claude_desktop_config.json), HTTP transport config, and available tools table. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
180 lines
4.8 KiB
Markdown
180 lines
4.8 KiB
Markdown
# codex — Personal Paper Knowledge Base
|
|
|
|
A self-hostable tool for managing scientific papers: ingest PDFs and arXiv
|
|
sources, build a citation graph, and link C++ implementations back to the
|
|
papers that describe them.
|
|
|
|
---
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# 1. Start Postgres (pgvector) + GROBID
|
|
docker compose -f infra/docker-compose.yml up -d
|
|
|
|
# 2. Copy and edit environment variables
|
|
cp .env.example .env
|
|
$EDITOR .env
|
|
|
|
# 3. Install Python dependencies (requires uv)
|
|
uv sync
|
|
|
|
# 4. Apply the database schema (first run only)
|
|
uv run python -c "
|
|
from codex.db import get_conn, apply_schema
|
|
with get_conn() as conn:
|
|
apply_schema(conn)
|
|
print('Schema applied.')
|
|
"
|
|
```
|
|
|
|
---
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `DATABASE_URL` | `postgresql://researcher:change_me@localhost:5432/papers` | libpq connection string |
|
|
| `GROBID_URL` | `http://localhost:8070` | GROBID HTTP API base URL |
|
|
| `OLLAMA_BASE_URL` | `http://localhost:11434` | Local Ollama endpoint (optional) |
|
|
| `EMBEDDING_MODEL` | `BAAI/bge-m3` | sentence-transformers model name |
|
|
| `EMBEDDING_DIM` | `1024` | Embedding vector dimension (must match model) |
|
|
| `OPENALEX_MAILTO` | *(empty)* | E-mail for OpenAlex Polite Pool (required for automated use) |
|
|
|
|
See `.env.example` for a documented template.
|
|
|
|
---
|
|
|
|
## Three-layer data model
|
|
|
|
All data lives in a single Postgres instance with the pgvector extension.
|
|
|
|
### Layer 1 — Semantics (`papers`, `chunks`)
|
|
|
|
Papers are stored with metadata and an abstract-level dense embedding
|
|
(BGE-M3, 1024 dimensions). Full-text is split into `chunks`, each with its
|
|
own dense embedding and a Postgres full-text (GIN) index.
|
|
Hybrid search combines nearest-neighbour vector search with keyword (FTS)
|
|
retrieval for robust handling of exact mathematical terminology.
|
|
|
|
### Layer 2 — Citations (`citations`)
|
|
|
|
Directed edges in the citation graph. The `cited_id` column has **no
|
|
foreign-key constraint** on purpose: edges pointing to papers that have not
|
|
yet been ingested are kept as-is. Those "dangling" targets are your
|
|
**discovery leads** — papers frequently cited by your collection that you
|
|
have not yet read.
|
|
|
|
```sql
|
|
-- Top discovery leads
|
|
SELECT cited_id, count(*) AS pull
|
|
FROM citations
|
|
WHERE cited_id NOT IN (SELECT id FROM papers)
|
|
GROUP BY cited_id
|
|
ORDER BY pull DESC
|
|
LIMIT 20;
|
|
```
|
|
|
|
### Layer 3 — Provenance (`code_links`)
|
|
|
|
Maps C++ symbols (qualified names or `file.cpp:line` references) to the
|
|
papers they implement. The workflow:
|
|
|
|
1. Tag C++ source with `@cite <bibkey>` in Doxygen comments.
|
|
2. Run `codex provenance sync --lib-path <path>` to scan and resolve tags.
|
|
3. Run `codex provenance export-bib <out.bib>` to generate a `.bib` file
|
|
containing **only the cited subset** of your collection.
|
|
|
|
The exported `.bib` is a derived view of the master catalogue — regenerate
|
|
it at any time; it is not the source of truth.
|
|
|
|
---
|
|
|
|
## CLI reference (coming in F-07)
|
|
|
|
```
|
|
codex ingest <id> # ingest one paper by arXiv ID or DOI
|
|
codex ingest-file <ids.txt> # bulk ingest from a file of IDs
|
|
codex search "<query>" [--hybrid]
|
|
codex discover leads
|
|
codex provenance sync --lib-path <path>
|
|
codex provenance export-bib <out.bib>
|
|
codex ask "<question>" # optional LLM Q&A via Ollama
|
|
```
|
|
|
|
---
|
|
|
|
## MCP Server (F-14)
|
|
|
|
codex exposes its KB as read-only MCP tools for LLM clients (Claude Code, Claude Desktop, etc.).
|
|
|
|
### Start (stdio)
|
|
|
|
```bash
|
|
# directly
|
|
python -m codex.mcp_server
|
|
|
|
# or via the installed script
|
|
codex-mcp
|
|
```
|
|
|
|
### Client registration
|
|
|
|
**Claude Code** — add to `.mcp.json` in your project root:
|
|
|
|
```json
|
|
{
|
|
"mcpServers": {
|
|
"codex": {
|
|
"command": "uv",
|
|
"args": ["run", "codex-mcp"],
|
|
"cwd": "/path/to/codex-py"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Claude Desktop** — add to `~/Library/Application Support/Claude/claude_desktop_config.json`:
|
|
|
|
```json
|
|
{
|
|
"mcpServers": {
|
|
"codex": {
|
|
"command": "uv",
|
|
"args": ["--directory", "/path/to/codex-py", "run", "codex-mcp"]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### HTTP transport (optional, config-gated)
|
|
|
|
```bash
|
|
MCP_TRANSPORT=http MCP_AUTH_TOKEN=<secret> codex-mcp
|
|
```
|
|
|
|
HTTP transport requires `MCP_AUTH_TOKEN` — the server refuses to start without it.
|
|
Requests must carry `Authorization: Bearer <token>`.
|
|
|
|
### Available tools (read-only)
|
|
|
|
| Tool | Description |
|
|
|---|---|
|
|
| `search` | Chunk-level hybrid search (dense BGE-M3) |
|
|
| `ask` | RAG answer (returns "not available" until implemented) |
|
|
| `wiki_read` | Compiled concept page (F-12) |
|
|
| `wiki_list` | All concept pages + freshness |
|
|
| `discover_leads` | Dangling-citation leads |
|
|
| `provenance_verify` | Scan C++ sources for @cite tags |
|
|
| `synthesis_browse` | Leads/conjectures from leads/ (F-13) |
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
```bash
|
|
uv run ruff check . && uv run ruff format --check . # lint
|
|
uv run mypy codex/ # type-check
|
|
uv run pytest # tests
|
|
```
|