@alextis59/athena 1.2.0 → 1.3.0

package/README.md

@@ -133,10 +133,11 @@ available. Low-confidence PDF regions are surfaced as uncertain in the UI.
 - `listDocs`
 - `relatedChunks`
 
-See [docs/mcp-integration.md](docs/mcp-integration.md) for OpenCode, Codex, and
-generic MCP setup examples. OpenCode is the default external-agent provider;
-pass `athena mcp --agent-provider codex` or set `agent.provider: "codex"` when
-configuring Athena for Codex.
+See the
+[MCP integration guide](https://github.com/alextis59/athena/blob/main/docs/mcp-integration.md)
+for OpenCode, Codex, and generic MCP setup examples. OpenCode is the default
+external-agent provider; pass `athena mcp --agent-provider codex` or set
+`agent.provider: "codex"` when configuring Athena for Codex.
 
 ## Development
 

package/dist/parsers/pdf2llm.d.ts

@@ -2,8 +2,8 @@ import { convertPdfToMarkdown } from "pdf-2-llm/node";
 import type { PdfConfig } from "../config/index.js";
 import type { ParsedDocument } from "./types.js";
 export declare const PDF2LLM_PARSER_NAME = "pdf-2-llm";
-export declare const PDF2LLM_PACKAGE_VERSION = "1.1.0";
-export declare const PDF2LLM_PARSER_VERSION = "1.1.0";
+export declare const PDF2LLM_PACKAGE_VERSION = "1.2.0";
+export declare const PDF2LLM_PARSER_VERSION = "1.2.0";
 export type Pdf2LlmConverter = typeof convertPdfToMarkdown;
 export interface ParsePdfFileOptions {
     config: PdfConfig;

package/dist/parsers/pdf2llm.js

@@ -4,7 +4,7 @@ import { UserFacingError } from "../errors.js";
 import { parseMarkdownLikeLines } from "./line-parser.js";
 import { normalizeText } from "./normalize.js";
 export const PDF2LLM_PARSER_NAME = "pdf-2-llm";
-export const PDF2LLM_PACKAGE_VERSION = "1.1.0";
+export const PDF2LLM_PACKAGE_VERSION = "1.2.0";
 export const PDF2LLM_PARSER_VERSION = PDF2LLM_PACKAGE_VERSION;
 export async function parsePdfFile(absolutePath, options) {
     const result = await convertWithTimeout(absolutePath, options);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@alextis59/athena",
-  "version": "1.2.0",
+  "version": "1.3.0",
   "description": "Local-first agentic documentation RAG command.",
   "type": "module",
   "private": false,
@@ -10,8 +10,7 @@
   "files": [
     "dist",
     "README.md",
-    "LICENSE",
-    "docs"
+    "LICENSE"
   ],
   "keywords": [
     "rag",
@@ -49,6 +48,8 @@
     "build": "tsc -p tsconfig.json",
     "check": "npm run format:check && npm run lint && npm run typecheck && npm test",
     "check:publish": "npm run check && npm run eval && npm run build",
+    "benchmark:pdf": "tsx scripts/pdf-benchmark.ts run",
+    "benchmark:pdf:plan": "tsx scripts/pdf-benchmark.ts plan",
     "corpus:health": "tsx scripts/corpus.ts health --offline",
     "corpus:retrieve": "tsx scripts/corpus.ts retrieve",
     "eval": "tsx scripts/eval.ts validate --offline",
@@ -74,6 +75,6 @@
     "better-sqlite3": "^12.11.1",
     "fast-glob": "^3.3.3",
     "jiti": "^2.7.0",
-    "pdf-2-llm": "^1.1.0"
+    "pdf-2-llm": "^1.2.0"
   }
 }

package/docs/QUALITY.md

@@ -1,137 +0,0 @@
-# Quality
-
-Use the narrowest validation that proves the change, then expand when the
-change touches shared behavior, package output, retrieval quality, security
-boundaries, or user-facing workflows.
-
-## Fast Checks
-
-For documentation-only changes:
-
-```bash
-npm run format:check
-```
-
-For TypeScript source changes, start with the nearest targeted Node test when
-possible:
-
-```bash
-node --import tsx --test test/<area>.test.ts
-```
-
-Then run the relevant package checks from the sections below.
-
-## Standard Source Change Ladder
-
-Run this for most runtime changes:
-
-```bash
-npm run format:check
-npm run lint
-npm run typecheck
-npm test
-npm run build
-```
-
-`npm run check` combines the first four commands:
-
-```bash
-npm run check
-```
-
-Prefer the expanded form when you need to show the exact failing stage.
-
-## Area-Specific Tests
-
-- CLI behavior: `node --import tsx --test test/cli.test.ts`
-- Config loading and remote opt-ins: `node --import tsx --test test/config.test.ts`
-- Storage and migrations: `node --import tsx --test test/storage.test.ts`
-- Indexing and run metadata: `node --import tsx --test test/indexer.test.ts`
-- Parsers and PDF artifacts: `node --import tsx --test test/parsers.test.ts`
-- Embedding providers and cache keys: `node --import tsx --test test/embeddings.test.ts`
-- Retrieval tools and ranking: `node --import tsx --test test/retrieval.test.ts`
-- Agent loop and provider adapters: `node --import tsx --test test/agent-loop.test.ts test/agent-provider.test.ts`
-- Citation policy: `node --import tsx --test test/agent-citations.test.ts`
-- HTTP server and UI API contract: `node --import tsx --test test/server.test.ts test/ui.test.ts`
-- MCP protocol and tool exposure: `node --import tsx --test test/mcp.test.ts`
-- Corpus manifest and eval plan: `node --import tsx --test test/corpus.test.ts test/eval.test.ts`
-
-When changing cross-cutting types, exports, or shared contracts, run the full
-standard ladder after targeted tests pass.
-
-## Retrieval, Parser, And Citation Quality
-
-For changes that affect discovery, parsing, chunking, embeddings, retrieval,
-ranking, PDF handling, source locators, agent tool use, or citations, include:
-
-```bash
-npm run corpus:health
-npm run eval
-```
-
-`npm run eval` runs offline validation against the committed corpus manifest and
-ground truth. If corpus cache files are missing, use:
-
-```bash
-npm run corpus:retrieve
-npm run corpus:health
-```
-
-Only update corpus manifest hashes or sizes after reviewing the changed
-downloaded artifact and documenting why the refresh is acceptable.
-
-## Manual CLI Smoke
-
-Use a temporary project root when validating real CLI behavior:
-
-```bash
-tmpdir="$(mktemp -d)"
-mkdir -p "$tmpdir/docs"
-printf '# Admin Guide\n\nSet ATHENA_INDEX_BATCH_SIZE=64 for large docsets.\n' > "$tmpdir/docs/admin-guide.md"
-npm run build
-node dist/cli/main.js index --root "$tmpdir" --json
-node dist/cli/main.js doctor --root "$tmpdir"
-```
-
-For `serve`, prefer a test harness or short-lived manual run. The server waits
-for shutdown by default.
-
-## MCP Smoke
-
-For MCP changes, run the targeted MCP tests and use the finite stdio smoke in
-`docs/mcp-integration.md` when the wire format or exposed tools change.
-
-## Package And Publish Readiness
-
-For release or package-entrypoint changes:
-
-```bash
-npm run check:publish
-npm pack --dry-run --json
-```
-
-Then smoke the generated tarball from this checkout:
-
-```bash
-npm pack
-npx --yes --package ./alextis59-athena-1.0.0.tgz athena --help
-```
-
-Also smoke an index command against a temporary docs folder before publishing.
-Use `npm publish --dry-run --json` for final registry packaging validation.
-Remove local `*.tgz` artifacts after the smoke test.
-
-The npm package name is `@alextis59/athena`; the installed command must remain
-`athena`.
-
-## What To Record
-
-In final notes, include:
-
-- commands run;
-- pass/fail outcome;
-- any skipped checks and why;
-- manual smoke details when automation does not cover the behavior.
-
-Do not mark a plan item or debt item complete unless the matching validation has
-actually passed.

package/docs/SECURITY.md

@@ -1,140 +0,0 @@
-# Security
-
-Athena is a local developer tool for indexing and querying project
-documentation. Its default security posture is local-first operation, explicit
-remote document-text opt-in, read-only retrieval integrations, and no public
-service exposure.
-
-## Trust Model
-
-- Trusted operator: the local user running `athena`.
-- Trusted codebase: this package and installed runtime dependencies.
-- Trusted data root: the `--root` directory selected by the user.
-- Untrusted inputs: documentation content, user config files in projects being
-  indexed, model output, MCP client requests, HTTP query parameters, and remote
-  provider responses.
-
-Athena does not sandbox arbitrary project config execution. `athena.config.ts`
-and `athena.config.mjs` are loaded with `jiti`, so only use config files from
-repositories you are willing to execute locally.
-
-## Local-First Boundaries
-
-By default:
-
-- indexes are written under `.athena/index.sqlite`;
-- generated PDF assets are written under `.athena/blobs/`;
-- the default embedding provider is local Transformers.js with remote model
-  downloads disabled at runtime;
-- chat is disabled until the user configures a provider;
-- the external agent provider defaults to OpenCode and only changes
-  client-specific MCP initialization guidance;
-- the HTTP server binds to `127.0.0.1:4317`.
-
-Do not change defaults so that document text leaves the machine. Remote
-embedding providers must require `embeddings.allowRemoteDocumentText: true`.
-Remote chat providers must require `chat.allowRemoteDocumentText: true`.
-
-## Secrets
-
-Remote providers use environment variables:
-
-- `OPENAI_API_KEY`
-- `VOYAGE_API_KEY`
-- `COHERE_API_KEY`
-- `GEMINI_API_KEY`
-- `ANTHROPIC_API_KEY` for configured chat validation
-
-Do not log API key values, persist them in `.athena/`, commit them, or include
-them in doctor output. The current doctor command reports provider/model/cache
-state, not secret values.
-
-## Network
-
-Expected network surfaces:
-
-- optional remote embedding APIs when explicitly configured and allowed;
-- optional Ollama embedding/chat requests to `OLLAMA_HOST` or local defaults;
-- optional corpus retrieval through `npm run corpus:retrieve`;
-- npm/package operations run manually by maintainers.
-
-The local web server has no authentication or authorization layer. Keep it on
-loopback for normal use. If `--host` or config binds to a non-loopback address,
-the operator is responsible for network controls until an authenticated server
-mode exists.
-
-## Filesystem
-
-Indexing reads files discovered under the configured root using supported
-documentation include/exclude patterns. Default exclusions cover `.git/`,
-`node_modules/`, `.athena/`, `dist/`, `build/`, and `coverage/`.
-
-Important filesystem constraints:
-
-- discovery does not follow symbolic links;
-- retrieval source reads require indexed repository-relative paths;
-- citation validation rejects absolute paths and paths containing `..`;
-- server source preview rejects absolute paths, `..`, and paths outside the
-  selected root;
-- PDF sidecar asset reads are restricted to `.athena/blobs/pdf-assets/`;
-- downloaded validation corpus artifacts belong in ignored
-  `test/corpus/cache/`.
-
-Do not commit `.athena/`, `dist/`, `*.tgz`, model caches, `.env*`, or
-downloaded corpus cache files.
-
-## MCP
-
-`athena mcp` exposes read-only retrieval tools:
-
-- `searchDocs`
-- `readSourceRange`
-- `readPdfSource`
-- `grepDocs`
-- `listDocs`
-- `relatedChunks`
-
-It does not expose write, edit, delete, indexing, shell, chat, or package
-management tools. Keep MCP read-only unless a future change includes explicit
-product requirements, authentication/authorization design, and tests.
-
-`agent.provider` and `--agent-provider` may select OpenCode or Codex setup
-guidance, but they must not change the read-only tool set or let an external
-agent own Athena's model loop.
-
-## Model Output And Citations
-
-Treat model output as untrusted. Athena validates citations before accepting an
-answer:
-
-- source paths must be indexed and repository-relative;
-- text citation line ranges must exist;
-- PDF citations must reference PDF page labels;
-- cited locators must have been read during the turn;
-- substantive answers require citations.
-
-Do not weaken citation validation to make a model answer pass. Fix retrieval,
-tool use, or the answer policy instead.
-
-## Parser And Document Content
-
-Markdown, MDX, HTML, RST, AsciiDoc, text, and PDF inputs are parsed as
-documentation content. Embedded code blocks, MDX JSX, script tags, and style
-tags must not execute as part of parsing. PDF parsing is bounded by configured
-timeout and wrapped in user-facing errors.
-
-When changing parsers, preserve warning behavior and avoid turning malformed or
-hostile documents into process crashes.
-
-## Practical Review Checklist
-
-Before merging changes that affect config, providers, server routes, MCP,
-filesystem reads, parsing, or citations, verify:
-
-- remote document text still requires explicit opt-in;
-- no secret value is logged, persisted, or printed;
-- source and asset path traversal checks still exist;
-- MCP remains read-only;
-- the server is not documented as safe for public exposure;
-- citation validation still rejects unsupported or unread citations;
-- relevant tests from `docs/QUALITY.md` pass.

package/docs/adr/0001-sqlite-driver.md

@@ -1,23 +0,0 @@
-# ADR 0001: SQLite Driver
-
-Date: 2026-07-03
-
-## Decision
-
-Athena uses `better-sqlite3` for the initial local SQLite storage layer.
-
-## Rationale
-
-- Athena's index updates and retrieval reads are local, short-lived CLI/server
-  operations where synchronous transactions are simple and deterministic.
-- `better-sqlite3` provides mature SQLite and FTS5 support without the
-  `node:sqlite` ExperimentalWarning emitted by the current Node 22 runtime.
-- The storage boundary is isolated in `src/storage/`, so a future migration to a
-  different SQLite binding or vector extension can be handled behind the same
-  migration and transaction helpers.
-
-## Consequences
-
-- `better-sqlite3` is a runtime dependency.
-- Tests can inspect schema, indexes, and rollback behavior without a separate
-  database service.