@alextis59/athena 1.2.0 → 1.3.0

package/docs/index.md

@@ -1,61 +0,0 @@
-# Documentation Index
-
-Start with the root files, then use the task-specific docs below.
-
-## Agent Entry Points
-
-- [../AGENTS.md](../AGENTS.md): repository instructions for coding agents.
-- [../ARCHITECTURE.md](../ARCHITECTURE.md): current package, module, runtime,
-  data-flow, and invariant map.
-- [../README.md](../README.md): user-facing install, command, config, MCP, and
-  development quickstart.
-
-## Product And Implementation
-
-- [agentic-rag-specifications.md](agentic-rag-specifications.md): product and
-  engineering requirements for Athena's local-first documentation RAG behavior.
-- [agentic-rag-study.md](agentic-rag-study.md): source study behind the
-  architecture choices.
-- [implementation-plan.md](implementation-plan.md): milestone checklist and
-  historical proof notes for implementation work.
-- [mcp-integration.md](mcp-integration.md): OpenCode, Codex, and generic MCP
-  setup, exposed read-only tools, and manual stdio smoke test.
-- [validation-corpus.md](validation-corpus.md): corpus layout, licensing,
-  retrieval, cache, and coverage rules.
-
-## Harness Operations
-
-- [SECURITY.md](SECURITY.md): security model, local-first boundaries, secrets,
-  network, filesystem, MCP, and server exposure notes.
-- [QUALITY.md](QUALITY.md): validation ladder for docs, source changes,
-  retrieval/parser changes, corpus changes, package release checks, and manual
-  smoke tests.
-- [tech-debt-tracker.md](tech-debt-tracker.md): practical deferred work that is
-  visible from the current code and docs.
-- [exec-plans/template.md](exec-plans/template.md): reusable plan template for
-  larger changes.
-
-## Decisions
-
-- [adr/0001-sqlite-driver.md](adr/0001-sqlite-driver.md): why Athena uses
-  `better-sqlite3` for the local SQLite layer.
-
-## Source Map
-
-- `src/cli/`: command parsing and dispatch.
-- `src/config/`: default config, config loading, validation, and
-  `defineConfig()`.
-- `src/storage/`: `.athena/` paths, migrations, schema checks, and
-  transactions.
-- `src/indexer/`: discovery, hashing, parsing, chunking, embedding, and index
-  writes.
-- `src/parsers/`: text-like parsers, PDF parsing, chunking, and normalization.
-- `src/embeddings/`: local, Ollama, and remote embedding providers.
-- `src/retrieval/`: read-only retrieval tools and trace capture.
-- `src/agent/`: Athena-owned agent loop, model providers, tools, citations, and
-  turn persistence.
-- `src/server/`: local HTTP UI/API server.
-- `src/ui/`: static web UI.
-- `src/mcp/`: stdio MCP server.
-- `scripts/`: corpus and evaluation commands.
-- `test/`: Node test suites and validation corpus fixtures.

package/docs/mcp-integration.md

@@ -1,227 +0,0 @@
-# Athena MCP Integration
-
-Athena exposes a read-only Model Context Protocol server through `athena mcp`.
-Use it when an external coding agent needs citation-aware project
-documentation retrieval without taking over Athena's own agent loop.
-
-## Preconditions
-
-Build or install Athena, then index the documentation project that the external
-agent should read:
-
-```sh
-npm run build
-node dist/cli/main.js index --root /absolute/path/to/project
-```
-
-For an installed package, replace `node dist/cli/main.js` with `athena` or
-`npx @alextis59/athena`.
-
-## Agent Provider Option
-
-OpenCode is the default external-agent provider. Use `--agent-provider codex`
-when the MCP server is configured for Codex:
-
-```sh
-athena mcp --root /absolute/path/to/project --agent-provider codex
-```
-
-The same choice can be made durable in `athena.config.ts`,
-`athena.config.mjs`, or `athena.config.json`:
-
-```ts
-import { defineConfig } from "@alextis59/athena";
-
-export default defineConfig({
-  agent: {
-    provider: "codex"
-  }
-});
-```
-
-The option changes client-specific initialization guidance only. The MCP server
-continues to expose the same read-only retrieval tools for every provider.
-
-## OpenCode
-
-OpenCode configures MCP servers under the `mcp` key. A local MCP entry uses
-`type: "local"` and a `command` array, with optional `cwd`, `environment`,
-`enabled`, and `timeout` fields.
-
-Development checkout example:
-
-```json
-{
-  "$schema": "https://opencode.ai/config.json",
-  "mcp": {
-    "athena": {
-      "type": "local",
-      "command": [
-        "node",
-        "/absolute/path/to/athena/dist/cli/main.js",
-        "mcp",
-        "--root",
-        "."
-      ],
-      "cwd": "/absolute/path/to/project",
-      "enabled": true,
-      "timeout": 10000
-    }
-  }
-}
-```
-
-Installed package example:
-
-```json
-{
-  "$schema": "https://opencode.ai/config.json",
-  "mcp": {
-    "athena": {
-      "type": "local",
-      "command": ["npx", "--yes", "@alextis59/athena", "mcp", "--root", "."],
-      "cwd": "/absolute/path/to/project",
-      "enabled": true,
-      "timeout": 10000
-    }
-  }
-}
-```
-
-After saving the config, run `opencode mcp list` to check that OpenCode can
-start the server and discover its tools.
-
-## Codex
-
-Codex stores MCP servers in `config.toml`. You can configure Athena in the
-user-level `~/.codex/config.toml` or in a trusted project-scoped
-`.codex/config.toml`.
-
-Installed package example:
-
-```toml
-[mcp_servers.athena]
-command = "npx"
-args = [
-  "--yes",
-  "@alextis59/athena",
-  "mcp",
-  "--root",
-  ".",
-  "--agent-provider",
-  "codex"
-]
-cwd = "/absolute/path/to/project"
-enabled = true
-startup_timeout_sec = 10
-tool_timeout_sec = 60
-```
-
-Development checkout example:
-
-```toml
-[mcp_servers.athena]
-command = "node"
-args = [
-  "/absolute/path/to/athena/dist/cli/main.js",
-  "mcp",
-  "--root",
-  ".",
-  "--agent-provider",
-  "codex"
-]
-cwd = "/absolute/path/to/project"
-enabled = true
-startup_timeout_sec = 10
-tool_timeout_sec = 60
-```
-
-After saving the config, start Codex in the project and use `/mcp` in the
-terminal UI to check that the `athena` server is connected.
-
-## Exposed Tools
-
-Athena exposes only read-only retrieval tools:
-
-- `searchDocs`
-- `readSourceRange`
-- `readPdfSource`
-- `grepDocs`
-- `listDocs`
-- `relatedChunks`
-
-It does not expose write, edit, delete, shell, indexing, chat, or package
-management tools through MCP. External agents can retrieve context, but they do
-not own Athena's model loop. Use `athena ask` or the local UI when you want
-Athena itself to plan tool calls and validate final answer citations.
-
-## Citation And Trace Shape
-
-MCP tool results are returned as text content containing JSON:
-
-```json
-{
-  "result": {},
-  "trace": {
-    "seenChunkIds": [],
-    "readLocators": [],
-    "searchScores": []
-  }
-}
-```
-
-`result` is the normal Athena retrieval payload. Search results include chunk
-IDs, paths, snippets, score details, and text or PDF locators. Source-read
-tools return exact line or PDF page/region locators.
-
-`trace` mirrors the runtime trace shape used by the UI and Athena's internal
-agent loop. Use it to inspect which chunks were surfaced, which source locators
-were read, and how retrieval scores were assigned during the MCP session.
-
-## Manual Smoke Test
-
-From this repository, build and index a small fixture project:
-
-```sh
-tmpdir="$(mktemp -d)"
-mkdir -p "$tmpdir/docs"
-cat > "$tmpdir/docs/admin-guide.md" <<'EOF'
-# Admin Guide
-
-## Indexing
-
-Set `ATHENA_INDEX_BATCH_SIZE=64` before indexing large docsets.
-EOF
-
-npm run build
-node dist/cli/main.js index --root "$tmpdir" --json
-```
-
-Then send a finite stdio MCP session:
-
-```sh
-printf '%s\n' \
-  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"manual-smoke","version":"1.0.0"}}}' \
-  '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
-  '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
-  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"searchDocs","arguments":{"query":"ATHENA_INDEX_BATCH_SIZE","pathPrefix":"docs/"}}}' \
-  '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"readSourceRange","arguments":{"path":"docs/admin-guide.md","startLine":3,"endLine":5}}}' \
-  | node dist/cli/main.js mcp --root "$tmpdir"
-```
-
-Expected output:
-
-- one `initialize` response with `serverInfo.name` set to `athena`;
-- one `tools/list` response with the six read-only retrieval tools;
-- one `searchDocs` response whose JSON text has
-  `result[0].path == "docs/admin-guide.md"` and non-empty
-  `trace.seenChunkIds`;
-- one `readSourceRange` response whose JSON text includes
-  `ATHENA_INDEX_BATCH_SIZE` and a `trace.readLocators` entry for
-  `docs/admin-guide.md`.
-
-## References
-
-- [MCP 2025-03-26 transports](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports)
-- [MCP 2025-03-26 tools](https://modelcontextprotocol.io/specification/2025-03-26/server/tools)
-- [OpenCode MCP servers](https://opencode.ai/docs/mcp-servers)

package/docs/tech-debt-tracker.md

@@ -1,21 +0,0 @@
-# Tech Debt Tracker
-
-Track deferred work that is visible from the current implementation. Keep this
-file practical: each item should have a reason, a likely validation path, and a
-status.
-
-| Item | Context | Validation | Owner | Status |
-| --- | --- | --- | --- | --- |
-| Align chat provider docs and adapters | Config validation recognizes `openai`, `anthropic`, and `gemini` chat providers, but `createConfiguredModelProvider()` currently implements Ollama only after validation. Either add adapters or keep user docs explicit that non-Ollama chat adapters are not ready. | Provider tests plus `athena ask` smoke for each supported adapter. | TBD | Open |
-| Decide authenticated server posture before non-loopback use | The HTTP server is intentionally local and unauthenticated today. Binding `--host` to a non-loopback address should remain an operator-controlled advanced use until there is an authenticated mode. | Server tests for auth behavior if added; security doc update. | TBD | Open |
-| Evaluate query embedding parity for remote/provider-specific indexes | Indexed vectors can come from provider-specific embeddings. Retrieval currently builds query vectors inside the retrieval layer rather than routing the query through the configured embedding provider. Confirm whether this is acceptable for release quality or add provider-aware query embedding. | Retrieval/eval tests comparing hybrid, FTS-only, vector-only, and provider-backed query behavior. | TBD | Open |
-| Harden package smoke automation | Package readiness currently relies on manual tarball and `npx --package` smoke commands documented in `docs/QUALITY.md`. A script could reduce release mistakes while preserving the same checks. | New script plus tarball CLI smoke in a temp project. | TBD | Open |
-| Expand parser warning summaries | Doctor reports total parser warnings, while PDF warning detail is available in parser artifacts and retrieval responses. A future doctor view could summarize warning codes by source. | Doctor tests and fixture with repeated PDF warnings. | TBD | Open |
-
-## Closing Criteria
-
-Close an item only after:
-
-- the implementation or documentation change lands;
-- the validation in the row passes;
-- related docs are updated when behavior or policy changes.

package/docs/validation-corpus.md

@@ -1,71 +0,0 @@
-# Athena Validation Corpus
-
-Date: 2026-07-03
-
-## Purpose
-
-The validation corpus gives Athena realistic documents early in the
-implementation. It is used to validate discovery, parsing, chunking, retrieval,
-citations, PDF extraction, UI source previews, and later evaluation runs.
-
-## Directory Layout
-
-- `test/corpus/manifest.json` is the locked corpus manifest.
-- `test/corpus/manifest.schema.json` documents the manifest shape.
-- `test/corpus/ground-truth.json` contains deterministic validation questions.
-- `test/corpus/synthetic/` contains committed synthetic controls.
-- `test/corpus/cache/` contains downloaded third-party artifacts and is ignored
-  by git.
-
-The cache is intentionally outside `.athena/` so tests can distinguish the
-validation corpus from runtime indexes. The script still treats the cache as
-local runtime state, not source code.
-
-## Licensing and Redistribution
-
-Synthetic controls are committed under this project's license-compatible test
-fixture policy and are marked `redistribution: "committed"`.
-
-Third-party documents are marked `redistribution: "download-only"`. Athena
-commits their URL, license metadata, retrieval date, expected size, and SHA-256
-hash, but it does not commit the downloaded artifacts. This avoids copying
-third-party documentation into the repository while keeping the validation set
-reproducible.
-
-## Retrieval
-
-Download or refresh the corpus:
-
-```bash
-npm run corpus:retrieve
-```
-
-Verify an already-cached corpus without network access:
-
-```bash
-npm run corpus:health
-```
-
-The retrieval script refuses changed downloads. If an upstream document changes
-intentionally, refresh the cache, review the changed file, update the manifest
-hash and size in the same commit, and describe why the change is acceptable.
-
-## Coverage
-
-The manifest covers:
-
-- Markdown README, API reference, and changelog samples;
-- MDX with imports, exports, and embedded JSX that must not execute;
-- plain text and extensionless documentation;
-- reStructuredText;
-- AsciiDoc;
-- HTML with navigation and visible documentation content;
-- born-digital PDF;
-- table-heavy PDF;
-- encrypted hard-PDF warning behavior;
-- synthetic exact-citation, ambiguous-heading, exact-symbol, semantic-only,
-  hybrid-comparison, and stale-document controls.
-
-Every required document type has at least one ground-truth question. Negative
-questions require Athena to refuse or mark an answer unsupported when the corpus
-does not contain the requested fact.