@graphpilot-oss/graphpilot 0.0.1

package/docs/architecture.md

@@ -0,0 +1,311 @@
+# Architecture
+
+How GraphPilot turns a folder of source files into a refactor-safe,
+branch-aware code graph an agent can query in milliseconds — with every
+answer carrying a `file:line @ sha` evidence anchor.
+
+This doc is for contributors and evaluators. If you just want to use it,
+see [quickstart.md](quickstart.md).
+
+## Three load-bearing properties
+
+1. **Evidence anchors** — `src/provenance.ts` attaches `{file, line, sha,
+excerpt}` to every symbol and call edge in tool output. The git SHA is
+   captured at index time via the pure-fs helpers in `src/git.ts` (no
+   `child_process`; we read `.git/HEAD`, `refs/heads/*`, and `packed-refs`
+   directly). Old graphs without `indexedSha` still load — the field is
+   optional in the schema.
+2. **Differential impact** — `gp_impact` takes an optional `since:
+<commit|tag|branch>`. When set, `getChangedFiles()` (in `src/git.ts`,
+   backed by `isomorphic-git` — pure JS, no shell-out) computes the diff
+   between that ref and HEAD; `analyzeImpact` filters callers to that
+   file set. Scope a refactor to your branch in one tool call.
+3. **Worktree-aware roots** — `resolveIndexRoot()` walks up to the git
+   worktree top by default, so two `git worktree add`'d branches produce
+   two separate indexes (since `repoIdFor` hashes the absolute root).
+   Both the CLI and the MCP layer route through it; opt out with
+   `--no-worktree`.
+
+## Top-level view
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                     Your TypeScript / JS repo                        │
+└───────────────────────────────────┬─────────────────────────────────┘
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  indexer.ts                        │
+                  │  walk dir, ignore node_modules,    │
+                  │  followSymbolicLinks: false        │
+                  └─────────────────┬──────────────────┘
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  parser.ts                         │
+                  │  tree-sitter → AST                 │
+                  │  (5 MB file cap, iterative walk)   │
+                  └─────────────────┬──────────────────┘
+                                    │
+              ┌─────────────────────┴──────────────────────┐
+              │                                            │
+   ┌──────────▼──────────┐                      ┌──────────▼──────────┐
+   │  symbols.ts         │                      │  edges.ts           │
+   │  extract            │                      │  call sites +       │
+   │  func/class/method/ │                      │  same-file > global │
+   │  iface/type/enum    │                      │  resolver           │
+   └──────────┬──────────┘                      └──────────┬──────────┘
+              │                                            │
+              └─────────────────────┬──────────────────────┘
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  storage.ts                        │
+                  │  ~/.graphpilot/<repo-id>/          │
+                  │    graph.json        (mode 0600)   │
+                  │    interactions.jsonl (mode 0600)  │
+                  └─────────────────┬──────────────────┘
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  query.ts (GraphIndex)             │
+                  │  4 pre-computed maps:              │
+                  │  byName, byId, callers, callees    │
+                  └─────────────────┬──────────────────┘
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  mcp.ts                            │
+                  │  5 tools over stdio JSON-RPC       │
+                  │  validators.ts + interactions log  │
+                  └─────────────────┬──────────────────┘
+                                    │
+                            [MCP protocol]
+                                    │
+                  ┌─────────────────▼──────────────────┐
+                  │  Claude Code / Cursor / Cline /    │
+                  │  Windsurf / Continue / ...         │
+                  └────────────────────────────────────┘
+```
+
+Data flow is one-way: source → tree → symbols + edges → JSON → query →
+agent. Nothing flows back. GraphPilot never modifies your code.
+
+## The five-stage pipeline
+
+### Stage 1 — Walk the directory
+
+File: [`src/indexer.ts`](../src/indexer.ts)
+
+- Uses `fast-glob` over `**/*.{ts,tsx,js,jsx,mjs,cjs}`
+- Skips `node_modules/`, `dist/`, `build/`, `.git/`, `coverage/`,
+  `.next/`, `.nuxt/`, `.cache/`, `out/`, and `*.d.ts`
+- `followSymbolicLinks: false` + per-file realpath check — files whose
+  realpath escapes the indexed root are skipped (defends against
+  symlink-escape attacks)
+- Hard cap of `MAX_FILES_PER_INDEX = 50,000`. Throws above that.
+
+### Stage 2 — Parse each file
+
+File: [`src/parser.ts`](../src/parser.ts)
+
+- `tree-sitter` + `tree-sitter-typescript` (covers TS, TSX, JSX, and JS)
+- Pre-read stat check: files over `MAX_FILE_BYTES = 5 MB` are skipped
+- `walk()` is **iterative** (stack-based), not recursive — protects
+  against stack overflow on pathologically deep generated code
+
+### Stage 3 — Extract symbols + raw calls (per file)
+
+Files: [`src/symbols.ts`](../src/symbols.ts) and
+[`src/edges.ts`](../src/edges.ts)
+
+Symbols extracted:
+
+- `function_declaration`
+- arrow / function expressions assigned to consts → `variable` kind
+- `class_declaration` and its `method_definition` children
+- `interface_declaration` (TS)
+- `type_alias_declaration` (TS)
+- `enum_declaration` (TS)
+
+Each gets a stable id of the form:
+
+```
+<file>#<parent>.<name>@<line>
+```
+
+`<parent>` is the enclosing class name for methods; empty otherwise.
+
+Call extraction:
+
+- For every function-like symbol, walk its body subtree
+- **Stop at nested function boundaries** — calls inside an inline arrow
+  are attributed to the arrow, not the outer function
+- Emit a `RawCall` for every `call_expression` and `new_expression`
+- Callee name is the identifier or the `.property` of a member-expression
+
+### Stage 4 — Resolve + save
+
+File: [`src/edges.ts`](../src/edges.ts) (resolver) and
+[`src/storage.ts`](../src/storage.ts) (persistence)
+
+After all files are parsed, a second pass resolves each `RawCall`:
+
+1. Prefer a symbol with the matching name in the **same file**
+2. Otherwise pick the **first** global match
+3. Otherwise leave `toId: null`; preserve `toName` so the agent still
+   sees the call happened
+
+Save location:
+
+```
+~/.graphpilot/<repo-id>/graph.json
+```
+
+Where `<repo-id>` is the first 16 hex chars of
+`sha256(absolute_repo_path)`. File permissions: `0o600`. Directory:
+`0o700`.
+
+Schema is versioned (`version: 1`) so future migrations are clean.
+
+### Stage 5 — Serve queries
+
+Files: [`src/query.ts`](../src/query.ts) and [`src/mcp.ts`](../src/mcp.ts)
+
+When the MCP server starts, it lazy-loads `graph.json` for whichever
+repo path is being queried and builds a `GraphIndex`:
+
+- `byNameLower` — lowercase name → SymbolRecord[]
+- `byId` — full id → SymbolRecord
+- `callersOf` — target id → CallEdge[] (answers "who calls X")
+- `calleesOf` — source id → CallEdge[] (answers "what does X call")
+
+The index is cached per absolute path inside the process so repeated
+tool calls don't re-parse the JSON.
+
+Every tool call flows through:
+
+```
+MCP request → validator → tool handler → response
+                              ↓
+                         interaction log
+```
+
+## The five MCP tools
+
+| Tool         | Input                                                                               | Output                                                                                                                              |
+| ------------ | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
+| `gp_index`   | `{ path? }`                                                                         | Triggers re-indexing + saves graph; invalidates per-path cache                                                                      |
+| `gp_recall`  | `{ query, limit?, substring?, path? }`                                              | Symbols matching name (exact case-insensitive by default; `substring: true` opt-in)                                                 |
+| `gp_callers` | `{ symbol, direction?: 'callers' \| 'callees', limit?, includeUnresolved?, path? }` | Edges where the symbol is target (callers) or source (callees)                                                                      |
+| `gp_impact`  | `{ symbol, depth? (1–5, default 3), path? }`                                        | Blast-radius report: direct callers, transitive callers grouped by BFS depth, tests likely affected, public-API flag, summary stats |
+| `gp_stats`   | `{ path? }`                                                                         | Health check: repo id, indexedAt, file/symbol/edge counts                                                                           |
+
+Every input is validated by hand-rolled validators in
+[`src/validators.ts`](../src/validators.ts):
+
+- Reject unknown fields (`additionalProperties: false` defence in depth)
+- Type-check every field
+- Range-check numbers (`limit` capped at 50–100 depending on tool)
+- Length-cap strings (no 2 MB symbol names)
+- Strict enums for `direction`
+
+If any check fails, the tool returns `{ isError: true, content: ... }`
+with a clear message — the request never reaches the handler.
+
+## What lives where on disk
+
+```
+~/.graphpilot/
+   <repo-id-1>/
+      graph.json           ← structural index (mode 0600)
+      interactions.jsonl   ← append-only tool-call log (mode 0600)
+   <repo-id-2>/
+      graph.json
+      interactions.jsonl
+```
+
+Nothing else is written. Nothing leaves your machine.
+
+## The interaction log
+
+File: [`src/interactions.ts`](../src/interactions.ts)
+
+Every tool call appends one line to `interactions.jsonl`:
+
+```json
+{
+  "ts": "2026-05-18T20:45:00Z",
+  "tool": "gp_recall",
+  "input": { "query": "parseToken" },
+  "results": 1,
+  "durationMs": 3
+}
+```
+
+**What is logged:** tool name, sanitized input args, result count,
+duration, error (if any).
+
+**What is NOT logged:** source code, file contents, user prompts.
+
+**Sanitization** (defends against log-line forgery via crafted symbol
+names):
+
+- Strip control characters (e.g. newlines become spaces)
+- Cap strings at 500 chars
+- Cap whole-line size at 8 KB; oversize entries fall back to a marker
+- Disabled entirely with `GRAPHPILOT_NO_LOG=1`
+
+v0.1 doesn't _read_ this log. It exists from day one so future ranking
+and personalization have data to train on. Local-only, your data.
+
+## Process model
+
+- One process per MCP session
+- stdio transport: reads JSON-RPC from stdin, writes responses to stdout
+- Diagnostics go to **stderr** (stdout is reserved for the protocol)
+- The process stays alive as long as stdin is open; exits cleanly on
+  client disconnect (see the Day-10 stdio fix)
+- No daemon mode in v0.1. One process per `~/.claude.json` entry.
+
+## Security model
+
+See [SECURITY.md](../SECURITY.md) for the user-facing policy + how to
+report a vulnerability. Active defences in code:
+
+- `validateRootPath` refuses `/`, `/etc`, `/var`, `~`, `/Users`,
+  `/home`, Windows system paths, and macOS-resolved aliases like
+  `/private/etc`
+- File size cap (5 MB) and file count cap (50k)
+- Symlink protection (fast-glob `followSymbolicLinks: false` + realpath
+  bounds check per file)
+- Storage perms (`0o700` dir, `0o600` files)
+- No `child_process`, no `exec`, no network code anywhere in `src/`
+- Hand-rolled validators on every MCP tool input (zero deps)
+- Empty `additionalProperties: false` on every tool's input schema
+
+## Testing strategy
+
+| Test file                    | What it covers                                            | Tests  |
+| ---------------------------- | --------------------------------------------------------- | ------ |
+| `tests/parser.test.ts`       | Tree-sitter wiring + function detection                   | 3      |
+| `tests/symbols.test.ts`      | Per-kind symbol extraction + id format                    | 9      |
+| `tests/edges.test.ts`        | Raw call extraction + resolution + nested fns             | 10     |
+| `tests/security.test.ts`     | T1/T2/T7/T10 defences                                     | 10     |
+| `tests/query.test.ts`        | GraphIndex maps + edge cases                              | 18     |
+| `tests/validators.test.ts`   | Per-tool input validators                                 | 20     |
+| `tests/interactions.test.ts` | Sanitization + log file + env-var disable                 | 11     |
+| `tests/mcp.test.ts`          | Tools through InMemoryTransport                           | 14     |
+| `tests/mcp-stdio.test.ts`    | Real subprocess over stdio (catches the Day-10 bug class) | 3      |
+| **Total**                    |                                                           | **98** |
+
+`InMemoryTransport` is fast and covers tool logic. `mcp-stdio.test.ts`
+spawns the real binary and drives it over stdin/stdout — slower but
+catches the "server starts but never responds" regression class.
+
+## Extension points (where v0.2+ work plugs in)
+
+| Feature               | Where it would live                                | Effort |
+| --------------------- | -------------------------------------------------- | ------ |
+| ~~Watch mode~~        | Shipped: `src/watcher.ts` + `graphpilot watch` CLI | —      |
+| ~~`gp_impact` tool~~  | Shipped: `src/impact.ts` + handler in `mcp.ts`     | —      |
+| `.graphpilotignore`   | extend `DEFAULT_IGNORE` in indexer + watcher       | small  |
+| Cross-repo workspace  | new `src/workspace.ts` + workspace yaml loader     | medium |
+| Semantic search       | embedding pipeline + vector index                  | medium |
+| Stack-Graphs resolver | replace `resolveCallEdges` algorithm               | large  |
+| Python support        | new tree-sitter grammar wired through `parser.ts`  | medium |

package/docs/limitations.md

@@ -0,0 +1,156 @@
+# Limitations of v0.1
+
+GraphPilot v0.1 makes deliberate trade-offs to ship a small, sharp tool
+fast. Knowing what it doesn't do matters as much as knowing what it does.
+
+The list below is exhaustive as of v0.1.0. Items with a milestone have
+a planned fix; unmarked items are out of scope for v1.x.
+
+## Language coverage
+
+- **TypeScript, TSX, JavaScript, JSX only.**
+  - Python — deferred to **v0.2 / v0.3** (demand-gated)
+  - Rust / Go / Java — deferred to **v1.x**
+  - All other languages — not planned for v1
+- **`.d.ts` declaration files are skipped.** They mostly express types
+  that don't add structural information to the call graph.
+- **JSON, YAML, Markdown, configs:** not indexed (we are a _code_
+  index, not a project index).
+
+## Resolver accuracy
+
+GraphPilot uses a deliberately simple name-based resolver. The
+trade-offs:
+
+- **No import-path resolution.** `import { foo } from "./bar"` does
+  not get followed. If `foo` appears in multiple files, the resolver
+  picks **same-file first**, then the **first global match** — which
+  may be wrong.
+- **Re-exports may pick the wrong source.** A chain like
+  `index.ts → utils/index.ts → utils/string.ts` resolves to whichever
+  file the walker saw first.
+- **No type-based method dispatch.** `userRepo.save()` and
+  `productRepo.save()` both resolve to whichever `save` we saw first.
+- **No `super()` or constructor inheritance tracking** beyond name
+  match.
+- **Standard library calls show as unresolved.** `JSON.parse`,
+  `Date.now`, `console.log`, `Array.from`, fs/path/process — all have
+  `toId: null`. The agent still sees the call happened; it just doesn't
+  get a jump-to-definition pointer.
+- **Expected resolution rate:** roughly **25–35% of edges resolve** to
+  an in-repo symbol id; the rest are external. On GraphPilot's own code
+  it's 42/155 (27%). That's enough to materially reduce hallucinations
+  because the questions agents actually ask (_"who calls X in my
+  repo"_) are the ones the dumb resolver answers correctly.
+
+**Planned in v0.2:** import-path tracking, re-export resolution.
+
+## Indexing model
+
+- **Watch mode is per-file incremental.** `graphpilot watch` re-parses
+  only the file that changed and re-resolves edges across the symbol
+  table in ~3–10ms per save. Full re-index is only needed on first run
+  or after a `pnpm install` / branch switch that changes many files.
+- **Single-process.** No CPU parallelism in v0.1.
+- **No `.graphpilotignore`.** Defaults skip `node_modules`, `dist`,
+  `build`, `.git`, `coverage`, `.next`, `.nuxt`, `.cache`, `out`,
+  `*.d.ts`. To customize, hand-edit `src/indexer.ts` (and
+  `src/watcher.ts` for watch mode).
+- **Max 50,000 files per index** (`MAX_FILES_PER_INDEX`). Larger repos
+  error out — narrow the path or wait for v0.4 workspaces.
+- **Max 5 MB per file** (`MAX_FILE_BYTES`). Larger files (minified
+  bundles, generated code) are silently skipped.
+
+**Planned in v0.2:** watch mode. **v0.3+:** incremental updates,
+`.graphpilotignore`.
+
+## What we don't index (deliberate)
+
+- **Comments and docstrings.** Use `rg TODO` etc.
+- **String literals.** `process.env.FOO`, route paths, embedded SQL.
+- **Configuration files.** package.json, tsconfig.json, .env.
+- **Git history.** No blame, no diff awareness.
+
+## Scope
+
+- **Single repo per query.** Each `gp_*` tool call operates on one
+  indexed repo. For microservices, index each repo separately; the
+  agent must coordinate lookups.
+- **No workspace abstraction.** Cross-repo namespace resolution (e.g.
+  `@org/auth` imported in `@org/payments`) is not native in v0.1. See
+  the manual workaround in `quickstart.md`.
+
+**Planned in v0.4 / v1.x:** workspace.yaml-driven cross-repo.
+
+## Agent capabilities
+
+- **No impact analysis tool.** "What breaks if I change X" must be
+  composed from `gp_callers` results. A dedicated `gp_impact` tool is
+  planned for v0.3.
+- **No route detection.** Express, Fastify, NestJS, Hono handlers are
+  not recognized as routes in v0.1.
+- **No test-to-unit mapping.** `tests/auth.spec.ts` does _not_ link to
+  the symbols it tests. Planned for v0.3.
+- **No semantic search.** `gp_recall` is name-only (exact case-insensitive
+  or substring). "Find code similar to this snippet" — not supported.
+  Deferred until 30+ users request it.
+- **No public-API extraction.** Inferable from `exported: true` symbols
+  but not a first-class tool.
+
+## Privacy / data handling
+
+- **No telemetry, no remote calls.** Verifiable: `src/` has zero `http`,
+  `fetch`, `axios`, or analytics imports. A CI lint rule will enforce
+  this from v0.2.
+- **Source code never leaves your machine.** Only the structured graph
+  (names, locations, signatures, call relationships) lives in
+  `~/.graphpilot/`.
+- **Signatures may contain secrets if your code does.** If you have
+  `const API_KEY = "sk-..."` literally in source, that line ends up in
+  `graph.json`. We don't redact in v0.1. **Planned in v0.2:** secret-
+  pattern detection (matches against known formats like `sk-`, `ghp_`,
+  AWS keys, JWTs, PEM headers).
+
+## Platform support
+
+- **Linux:** tested, CI green
+- **macOS** (Intel + Apple Silicon): tested, CI green
+- **Windows:** experimental in v0.1. CI green but real-world testing is
+  light. The subprocess MCP test is currently skipped on Windows. File
+  bugs eagerly.
+
+## Performance ceiling
+
+Rough numbers on Apple Silicon (M1 Pro, 16 GB):
+
+| Repo size | Index time | Resident memory | graph.json |
+| --------- | ---------- | --------------- | ---------- |
+| 100 files | 80 ms      | ~30 MB          | ~100 KB    |
+| 1k files  | 800 ms     | ~80 MB          | ~1 MB      |
+| 10k files | 8 s        | ~300 MB         | ~10 MB     |
+| 50k files | 40 s       | ~1.2 GB         | ~50 MB     |
+
+Query latency on the pre-computed indexes: sub-millisecond even at 50k
+symbols.
+
+## What we deliberately don't build
+
+Not a value judgement — just clarity on scope:
+
+- **Not a coding agent.** Claude Code / Cursor / Aider generate code.
+  We provide them context.
+- **Not a security scanner.** CodeQL owns taint analysis.
+- **Not a build system.** We don't compile.
+- **Not a Sourcegraph clone.** No web UI for human browsing — the agent
+  is the user.
+- **Not a SaaS.** No accounts, no cloud, no enterprise tier in v1.
+
+If your use case needs one of those, GraphPilot is the wrong tool.
+
+## Reporting limits we missed
+
+The list above is intentionally exhaustive. If you hit a real limit
+that's not documented here,
+[open an issue](https://github.com/graphpilot-oss/graphpilot/issues) —
+"undocumented limitation" is a valid issue type and helps us keep this
+list honest.