@alextis59/athena 1.0.0

package/docs/mcp-integration.md

@@ -0,0 +1,154 @@
+# Athena MCP Integration
+
+Athena exposes a read-only Model Context Protocol server through `athena mcp`.
+Use it when an external coding agent needs citation-aware project
+documentation retrieval without taking over Athena's own agent loop.
+
+## Preconditions
+
+Build or install Athena, then index the documentation project that the external
+agent should read:
+
+```sh
+npm run build
+node dist/cli/main.js index --root /absolute/path/to/project
+```
+
+For an installed package, replace `node dist/cli/main.js` with `athena` or
+`npx @alextis59/athena`.
+
+## OpenCode
+
+OpenCode configures MCP servers under the `mcp` key. A local MCP entry uses
+`type: "local"` and a `command` array, with optional `cwd`, `environment`,
+`enabled`, and `timeout` fields.
+
+Development checkout example:
+
+```json
+{
+  "$schema": "https://opencode.ai/config.json",
+  "mcp": {
+    "athena": {
+      "type": "local",
+      "command": [
+        "node",
+        "/absolute/path/to/athena/dist/cli/main.js",
+        "mcp",
+        "--root",
+        "."
+      ],
+      "cwd": "/absolute/path/to/project",
+      "enabled": true,
+      "timeout": 10000
+    }
+  }
+}
+```
+
+Installed package example:
+
+```json
+{
+  "$schema": "https://opencode.ai/config.json",
+  "mcp": {
+    "athena": {
+      "type": "local",
+      "command": ["npx", "--yes", "@alextis59/athena", "mcp", "--root", "."],
+      "cwd": "/absolute/path/to/project",
+      "enabled": true,
+      "timeout": 10000
+    }
+  }
+}
+```
+
+After saving the config, run `opencode mcp list` to check that OpenCode can
+start the server and discover its tools.
+
+## Exposed Tools
+
+Athena exposes only read-only retrieval tools:
+
+- `searchDocs`
+- `readSourceRange`
+- `readPdfSource`
+- `grepDocs`
+- `listDocs`
+- `relatedChunks`
+
+It does not expose write, edit, delete, shell, indexing, chat, or package
+management tools through MCP. External agents can retrieve context, but they do
+not own Athena's model loop. Use `athena ask` or the local UI when you want
+Athena itself to plan tool calls and validate final answer citations.
+
+## Citation And Trace Shape
+
+MCP tool results are returned as text content containing JSON:
+
+```json
+{
+  "result": {},
+  "trace": {
+    "seenChunkIds": [],
+    "readLocators": [],
+    "searchScores": []
+  }
+}
+```
+
+`result` is the normal Athena retrieval payload. Search results include chunk
+IDs, paths, snippets, score details, and text or PDF locators. Source-read
+tools return exact line or PDF page/region locators.
+
+`trace` mirrors the runtime trace shape used by the UI and Athena's internal
+agent loop. Use it to inspect which chunks were surfaced, which source locators
+were read, and how retrieval scores were assigned during the MCP session.
+
+## Manual Smoke Test
+
+From this repository, build and index a small fixture project:
+
+```sh
+tmpdir="$(mktemp -d)"
+mkdir -p "$tmpdir/docs"
+cat > "$tmpdir/docs/admin-guide.md" <<'EOF'
+# Admin Guide
+
+## Indexing
+
+Set `ATHENA_INDEX_BATCH_SIZE=64` before indexing large docsets.
+EOF
+
+npm run build
+node dist/cli/main.js index --root "$tmpdir" --json
+```
+
+Then send a finite stdio MCP session:
+
+```sh
+printf '%s\n' \
+  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"manual-smoke","version":"1.0.0"}}}' \
+  '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
+  '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
+  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"searchDocs","arguments":{"query":"ATHENA_INDEX_BATCH_SIZE","pathPrefix":"docs/"}}}' \
+  '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"readSourceRange","arguments":{"path":"docs/admin-guide.md","startLine":3,"endLine":5}}}' \
+  | node dist/cli/main.js mcp --root "$tmpdir"
+```
+
+Expected output:
+
+- one `initialize` response with `serverInfo.name` set to `athena`;
+- one `tools/list` response with the six read-only retrieval tools;
+- one `searchDocs` response whose JSON text has
+  `result[0].path == "docs/admin-guide.md"` and non-empty
+  `trace.seenChunkIds`;
+- one `readSourceRange` response whose JSON text includes
+  `ATHENA_INDEX_BATCH_SIZE` and a `trace.readLocators` entry for
+  `docs/admin-guide.md`.
+
+## References
+
+- [MCP 2025-03-26 transports](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports)
+- [MCP 2025-03-26 tools](https://modelcontextprotocol.io/specification/2025-03-26/server/tools)
+- [OpenCode MCP servers](https://opencode.ai/docs/mcp-servers)

package/docs/validation-corpus.md

@@ -0,0 +1,71 @@
+# Athena Validation Corpus
+
+Date: 2026-07-03
+
+## Purpose
+
+The validation corpus gives Athena realistic documents early in the
+implementation. It is used to validate discovery, parsing, chunking, retrieval,
+citations, PDF extraction, UI source previews, and later evaluation runs.
+
+## Directory Layout
+
+- `test/corpus/manifest.json` is the locked corpus manifest.
+- `test/corpus/manifest.schema.json` documents the manifest shape.
+- `test/corpus/ground-truth.json` contains deterministic validation questions.
+- `test/corpus/synthetic/` contains committed synthetic controls.
+- `test/corpus/cache/` contains downloaded third-party artifacts and is ignored
+  by git.
+
+The cache is intentionally outside `.athena/` so tests can distinguish the
+validation corpus from runtime indexes. The script still treats the cache as
+local runtime state, not source code.
+
+## Licensing and Redistribution
+
+Synthetic controls are committed under this project's license-compatible test
+fixture policy and are marked `redistribution: "committed"`.
+
+Third-party documents are marked `redistribution: "download-only"`. Athena
+commits their URL, license metadata, retrieval date, expected size, and SHA-256
+hash, but it does not commit the downloaded artifacts. This avoids copying
+third-party documentation into the repository while keeping the validation set
+reproducible.
+
+## Retrieval
+
+Download or refresh the corpus:
+
+```bash
+npm run corpus:retrieve
+```
+
+Verify an already-cached corpus without network access:
+
+```bash
+npm run corpus:health
+```
+
+The retrieval script refuses changed downloads. If an upstream document changes
+intentionally, refresh the cache, review the changed file, update the manifest
+hash and size in the same commit, and describe why the change is acceptable.
+
+## Coverage
+
+The manifest covers:
+
+- Markdown README, API reference, and changelog samples;
+- MDX with imports, exports, and embedded JSX that must not execute;
+- plain text and extensionless documentation;
+- reStructuredText;
+- AsciiDoc;
+- HTML with navigation and visible documentation content;
+- born-digital PDF;
+- table-heavy PDF;
+- encrypted hard-PDF warning behavior;
+- synthetic exact-citation, ambiguous-heading, exact-symbol, semantic-only,
+  hybrid-comparison, and stale-document controls.
+
+Every required document type has at least one ground-truth question. Negative
+questions require Athena to refuse or mark an answer unsupported when the corpus
+does not contain the requested fact.

package/package.json

@@ -0,0 +1,79 @@
+{
+  "name": "@alextis59/athena",
+  "version": "1.0.0",
+  "description": "Local-first agentic documentation RAG command.",
+  "type": "module",
+  "private": false,
+  "bin": {
+    "athena": "dist/cli/main.js"
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "LICENSE",
+    "docs"
+  ],
+  "keywords": [
+    "rag",
+    "documentation",
+    "search",
+    "sqlite",
+    "mcp",
+    "pdf"
+  ],
+  "homepage": "https://github.com/alextis59/athena#readme",
+  "bugs": {
+    "url": "https://github.com/alextis59/athena/issues"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/alextis59/athena.git"
+  },
+  "license": "ISC",
+  "author": "Athena contributors",
+  "engines": {
+    "node": ">=22.0.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.json",
+    "check": "npm run format:check && npm run lint && npm run typecheck && npm test",
+    "check:publish": "npm run check && npm run eval && npm run build",
+    "corpus:health": "tsx scripts/corpus.ts health --offline",
+    "corpus:retrieve": "tsx scripts/corpus.ts retrieve",
+    "eval": "tsx scripts/eval.ts validate --offline",
+    "format:check": "prettier --check .",
+    "lint": "eslint .",
+    "prepack": "npm run build",
+    "prepublishOnly": "npm run check:publish",
+    "test": "node --import tsx --test \"test/**/*.test.ts\"",
+    "typecheck": "tsc -p tsconfig.test.json --noEmit"
+  },
+  "devDependencies": {
+    "@eslint/js": "^10.0.1",
+    "@types/better-sqlite3": "^7.6.13",
+    "@types/node": "^26.1.0",
+    "eslint": "^10.6.0",
+    "prettier": "^3.9.4",
+    "tsx": "^4.23.0",
+    "typescript": "^6.0.3",
+    "typescript-eslint": "^8.62.1"
+  },
+  "dependencies": {
+    "@huggingface/transformers": "^4.2.0",
+    "better-sqlite3": "^12.11.1",
+    "fast-glob": "^3.3.3",
+    "jiti": "^2.7.0",
+    "pdf-2-llm": "^1.1.0"
+  }
+}