@wwjd/pi-graphify 0.1.0

package/docs/DESIGN.md

@@ -0,0 +1,288 @@
+# Design Document: pi-graphify
+
+## Executive summary
+
+`pi-graphify` integrates the [Graphify](https://github.com/Graphify-Labs/graphify) knowledge graph ecosystem into [Pi](https://pi.dev) as a native extension. It exposes Graphify capabilities through Pi tools, slash commands, and session lifecycle hooks, and it abstracts over multiple Graphify interfaces (CLI and MCP) so that the Pi agent can use Graphify without caring how it is installed.
+
+The design is intentionally ecosystem-aware: it works alongside memory extensions like `pi-hermes-memory`, code intelligence extensions like `pi-fallow`, and any other Pi package without duplicating their responsibilities.
+
+For deeper detail, see:
+- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — backend abstraction, component design, and data flow
+- [docs/ECOSYSTEM.md](docs/ECOSYSTEM.md) — integration with other Pi extensions
+- [docs/VERSIONING.md](docs/VERSIONING.md) — Graphify dependency and compatibility strategy
+
+---
+
+## Vision
+
+Make Graphify a first-class Pi citizen. A user should be able to open a Pi session in a project that has Graphify installed, ask codebase questions, and have the agent automatically use the knowledge graph when it is the right tool. Manual Graphify commands should still be available, but they should not be the default path for common questions.
+
+The experience should feel similar to how `pi-hermes-memory` makes persistent memory feel native: present in the background, available through tools, and surfaced automatically when relevant.
+
+---
+
+## Design principles
+
+1. **Agent-first**
+   The LLM should discover and call Graphify capabilities through typed tools. The user should not have to remember command syntax.
+
+2. **Backend abstraction**
+   The extension supports both the Graphify CLI and the Graphify MCP server behind a single interface. Callers do not choose the backend; the extension picks the best available one.
+
+3. **Defensive compatibility**
+   Graphify is an external tool with its own release cycle. The extension detects the installed version, compares it to a supported range, and degrades gracefully when there is a mismatch.
+
+4. **Progressive automation**
+   Start with manual tools, then add context hints, then background rebuilds, then optional interception. Each layer is configurable and can be disabled.
+
+5. **Safe defaults**
+   Do not auto-run expensive operations. Code-only incremental rebuilds are safe; LLM-based semantic extraction is explicit. Do not auto-update Graphify without user confirmation.
+
+6. **Ecosystem composable**
+   Work with, not against, other Pi extensions. Store durable lessons in memory extensions. Coexist with code intelligence extensions. Respect project trust and provider settings.
+
+7. **Integration-ready**
+   Third-party integrations (e.g., with `pi-hermes-memory`) are optional and not part of the core implementation. The extension exposes well-defined hooks so that other extensions can integrate with it without tight coupling.
+
+8. **Full ecosystem coverage**
+   Expose the useful Graphify surface: build, query, path, explain, affected, add, watch, reflect, and diagnostics. Do not build a thin wrapper around only `query`.
+
+---
+
+## User experience
+
+### Ideal flow
+
+1. User opens a project in Pi.
+2. `pi-graphify` detects `graphify-out/graph.json` and the installed Graphify version.
+3. If a graph exists, the system prompt gets a hint to prefer graph tools for codebase questions.
+4. User asks: “How does auth flow to the database?”
+5. The agent calls `graphify_query` and receives a scoped subgraph.
+6. While the user edits code, the graph is incrementally rebuilt in the background (if enabled).
+7. The user rarely types `/graphify`.
+
+### Fallback flow
+
+1. User opens a project with no graph or an outdated Graphify version.
+2. The extension explains the situation and offers `/graphify-build` or `/graphify-status`.
+3. The agent or user triggers the build.
+4. The extension reports success or failure clearly.
+
+### Version mismatch flow
+
+1. Extension detects Graphify 1.x when 2.x is expected.
+2. It notifies the user and disables features that require 2.x.
+3. It provides the exact command to update Graphify.
+4. Basic query/explain still work if the CLI contract is compatible.
+
+---
+
+## Feature surface
+
+### Tools
+
+| Tool | Purpose |
+|---|---|
+| `graphify_status` | Check whether a graph exists and whether the CLI is compatible. |
+| `graphify_build` | Build or incrementally update the graph. |
+| `graphify_query` | Natural-language graph traversal. |
+| `graphify_path` | Shortest path between two concepts. |
+| `graphify_explain` | Explain a node and its connections. |
+| `graphify_affected` | Show blast radius of a change. |
+| `graphify_add` | Add a URL or document to the corpus. |
+| `graphify_watch` | Start or stop the background file watcher. |
+| `graphify_reflect` | Refresh LESSONS.md from saved query results. |
+| `graphify_version` | Report installed Graphify version and compatibility status. |
+
+### Commands
+
+| Command | Purpose |
+|---|---|
+| `/graphify-status` | Show graph status and compatibility. |
+| `/graphify-build` | Build or update the graph. |
+| `/graphify-query` | Run a query. |
+| `/graphify-path` | Trace a path. |
+| `/graphify-explain` | Explain a node. |
+| `/graphify-affected` | Show affected nodes. |
+| `/graphify-watch` | Toggle file watcher. |
+| `/graphify-install-hook` | Install a post-commit rebuild hook. |
+| `/graphify-diagnose` | Run graph health / extension diagnostics. |
+| `/graphify-version` | Show Graphify version and backend details. |
+
+### Events
+
+| Event | Behavior |
+|---|---|
+| `session_start` | Detect graph, detect Graphify version, start watcher if configured. |
+| `before_agent_start` | Inject graph hint if a graph exists. |
+| `session_shutdown` | Stop watcher, close MCP connection. |
+| `tool_call` (optional future) | Nudge read/grep toward graph query when appropriate. |
+
+---
+
+## Architecture
+
+See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full component design.
+
+At a high level, the extension is layered:
+
+```
+Pi Agent
+    │
+    ▼
+Extensions API  (extensions/index.ts)
+    │
+    ├─ Tool registration
+    ├─ Command registration
+    └─ Event handlers
+    ▼
+Coordinator     (src/coordinator.ts)
+    │
+    ├─ Backend selector (CLI vs MCP)
+    ├─ Version manager
+    ├─ State manager
+    └─ Feature capability map
+    ▼
+Graphify Backend
+    ├─ CliBackend    (src/backends/cli.ts)
+    └─ McpBackend    (src/backends/mcp.ts)
+```
+
+All Graphify operations go through the coordinator, which picks the backend, checks capabilities, and normalizes results. No tool or command talks directly to the CLI or MCP server.
+
+---
+
+## Graphify dependency
+
+See [docs/VERSIONING.md](docs/VERSIONING.md) for the detailed compatibility strategy.
+
+### Supported range
+
+The extension declares a supported Graphify version range (e.g., `>= 2.100.0`). This range is checked at session start and before each operation if the version is unknown.
+
+### Detection
+
+The extension finds Graphify by:
+1. Checking the `GRAPHIFY_PATH` environment variable.
+2. Searching `PATH` for the `graphify` executable (`which` / `where`).
+3. Checking common installation locations (uv tool, pipx, pip user bin).
+
+### Version command
+
+`graphify --version` or `graphify version` is used to determine the installed version. If the command fails, the extension falls back to trying CLI operations in compatibility mode.
+
+### Mismatch handling
+
+| Scenario | Behavior |
+|---|---|
+| Graphify missing | Disable tools, show install instructions, keep session alive. |
+| Version below minimum | Disable advanced features, warn user, suggest update command. |
+| Version above maximum | Disable features with known breaking changes, warn user, log telemetry. |
+| Version unknown | Run in best-effort mode; assume basic CLI compatibility. |
+| Feature missing | Disable that specific feature; other tools continue working. |
+
+### Updates
+
+The extension never auto-updates Graphify. It can:
+- Detect that an update is available.
+- Show the exact update command for the detected installation method.
+- Let the agent or user run the update via a tool/command if they choose.
+
+For the hybrid version-detection strategy (hardcoded minimum + runtime capability probes), see [docs/VERSIONING.md](docs/VERSIONING.md).
+
+---
+
+## Ecosystem integration
+
+See [docs/ECOSYSTEM.md](docs/ECOSYSTEM.md) for detailed integration points.
+
+### With `pi-hermes-memory`
+
+Complementary. Graphify provides project structure context; Hermes provides durable agent/user memory. The extension may write durable lessons to Hermes memory when graph queries produce reusable insights or corrections.
+
+This integration is **not part of the initial implementation**. It will be added later via the integration hook architecture described in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
+### With `pi-fallow`
+
+Coexistence. Fallow is useful for TypeScript/JavaScript code intelligence; Graphify is useful for project-wide structural knowledge. Both can run in the same session without conflict.
+
+### With provider/auth extensions
+
+The extension does not manage API keys or providers. It relies on Graphify to read `GEMINI_API_KEY` / `GOOGLE_API_KEY` for semantic extraction, and on Pi to manage LLM provider auth for the agent itself.
+
+---
+
+## Security model
+
+1. **Path validation** — All paths resolved relative to project root; no traversal.
+2. **No shell injection** — Use `execFile` with argument arrays.
+3. **User confirmation for expensive ops** — Full builds and semantic extraction require explicit action.
+4. **Output sanitization** — Graphify output is treated as text; never re-executed.
+5. **Project trust** — Project-local config and auto-watch only on trusted projects.
+6. **URL validation** — URLs are validated before `graphify add`.
+
+---
+
+## Error handling
+
+| Scenario | Behavior |
+|---|---|
+| Graphify missing | Clear install instructions; extension stays loaded. |
+| Version incompatible | Feature matrix adjusted; user notified. |
+| Graph missing | Offer to build; do not crash. |
+| Build fails | Return error to agent with stderr. |
+| Watcher fails | Log warning; disable watcher; continue. |
+| MCP connection fails | Fall back to CLI automatically. |
+| Query no results | Clear message; agent can try other tools. |
+
+---
+
+## Out of scope (for v1)
+
+- Rewriting Graphify in TypeScript.
+- Bundling Python or graphify dependencies.
+- Auto-running LLM-based semantic extraction on every save.
+- Auto-updating Graphify without user confirmation.
+- Replacing other memory or intelligence extensions.
+- Hosting a persistent public MCP server.
+- Cross-repo graph customization beyond what Graphify already supports.
+
+---
+
+## Roadmap
+
+### Phase 1 — Foundation ✅
+- Project setup, config loader, CLI discovery, status tool, version detection.
+
+### Phase 2 — Backend abstraction
+- Coordinator, `CliBackend`, backend interface, capability matrix.
+
+### Phase 3 — Core tools
+- Build, query, path, explain, affected, version tools.
+
+### Phase 4 — Commands
+- Slash commands for all core tools.
+
+### Phase 5 — Automation
+- File watcher, staleness detection, git hook helper, auto-hints.
+
+### Phase 6 — Advanced features
+- MCP backend, `add`, `watch`, `reflect`, optional read interception.
+
+### Phase 7 — Ecosystem polish
+- Hermes memory integration, telemetry/logging, diagnostics.
+
+### Phase 8 — 1.0 readiness
+- Tests, README, security review, stable config, npm publish, gallery.
+
+---
+
+## Success criteria for 1.0
+
+- [ ] User can install `pi-graphify` and use graph tools against an existing graph.
+- [ ] Agent automatically prefers graph tools for codebase questions when a graph exists.
+- [ ] Extension handles missing or outdated Graphify gracefully.
+- [ ] Graph can be rebuilt and queried without leaving Pi.
+- [ ] File watcher can optionally auto-update code-only graphs.
+- [ ] Backend abstraction supports CLI and MCP.
+- [ ] Full test coverage and passing CI.

package/docs/ECOSYSTEM.md

@@ -0,0 +1,130 @@
+# Ecosystem Integration Document: pi-graphify
+
+`pi-graphify` is designed to live in a Pi session alongside other extensions. **Direct third-party integrations are not the main focus of the initial implementation**, but the architecture is intentionally open so that integrations can be added later by us or by other extension authors.
+
+This document explains the design philosophy and the integration points that are planned for the future. For the technical hook design, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
+---
+
+## Core philosophy
+
+- **Do not duplicate.** If another extension already solves a problem, `pi-graphify` should interoperate with it rather than reimplement.
+- **Add project context.** Graphify’s unique value is structural knowledge of the current project. It should feed that context into the agent.
+- **Export durable lessons.** When graph queries produce reusable insights, those lessons can be stored in memory extensions.
+- **Stay out of the way.** If another extension is better suited for a task, `pi-graphify` should not interfere.
+
+---
+
+## pi-hermes-memory
+
+### Status
+
+This integration is **planned but not part of the initial implementation**. The hook architecture in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) makes it straightforward to add later without coupling the core extension to Hermes.
+
+### Relationship
+
+`pi-hermes-memory` stores durable facts, preferences, corrections, failures, and procedural skills across sessions. `pi-graphify` provides structural knowledge of the current project.
+
+They are **complementary**.
+
+### Integration points
+
+1. **Memory entries from graph lessons**
+   When a graph query reveals a durable fact (e.g., “The auth flow goes through `AuthMiddleware` → `TokenService` → `UserRepository`), `pi-graphify` can save that as a convention or insight memory if the user confirms it is useful.
+
+2. **Graphify conventions**
+   If the user consistently prefers `--directed` graphs or `--mode deep`, Hermes memory can store that preference, and `pi-graphify` can read it from `USER.md` or the `memory_search` tool.
+
+3. **Failure memory**
+   If a graph query gives a wrong answer and the user corrects it, the correction can be written to Hermes failure/correction memory so the agent does not repeat the mistake.
+
+### Implementation note
+
+Direct integration with Hermes memory will be optional. If `pi-graphify` detects that Hermes memory tools are available, it can use them. If not, it skips the integration silently.
+
+---
+
+## pi-fallow
+
+### Relationship
+
+`pi-fallow` provides deterministic TypeScript/JavaScript code intelligence: dead code, duplication, complexity, and architecture drift. `pi-graphify` provides project-wide structural knowledge across languages and file types.
+
+They are **complementary** and can coexist in the same session.
+
+### When to use which
+
+| Use case | Best tool |
+|---|---|
+| “Is this export used?” | Fallow |
+| “How does this function relate to files across the repo?” | Graphify |
+| “Where is code duplicated?” | Fallow |
+| “What is the shortest path from `AuthModule` to `Database`?” | Graphify |
+| “Is this component imported correctly?” | Fallow |
+| “What communities exist in this codebase?” | Graphify |
+
+### Integration points
+
+1. **Coexistence**
+   Both extensions register tools. The agent decides which to use based on the question. `pi-graphify` should not intercept Fallow tool calls or vice versa.
+
+2. **Shared project context**
+   If both extensions detect a project, they may both inject context hints. The system prompt should remain coherent. `pi-graphify` will use a clearly delimited `<graphify-context>` block to avoid clashing with Fallow’s hints.
+
+### Implementation note
+
+No direct code coupling. Integration is purely at the agent level: both tools are available, and the agent chooses.
+
+---
+
+## Other memory extensions
+
+Any Pi extension that provides a `memory` or `memory_search` tool can, in principle, receive durable lessons from `pi-graphify`. The integration is generic:
+
+1. Detect if a memory tool is registered.
+2. If a graph query produces a lesson, format it as a memory entry.
+3. Call the memory tool with the lesson.
+
+This is future work and will be configurable.
+
+---
+
+## Provider and auth extensions
+
+Extensions like `pi-claude-oauth-adapter` or custom provider packages manage how Pi talks to LLMs. `pi-graphify` does not interact with them directly.
+
+However, if the Graphify semantic extraction uses the user’s configured LLM provider (e.g., Gemini), the extension should respect the same environment variables Graphify already uses (`GEMINI_API_KEY`, `GOOGLE_API_KEY`). It should not attempt to read or manage those keys itself.
+
+---
+
+## Project trust extensions
+
+If a Pi extension customizes the project trust flow, `pi-graphify` should respect the final trust decision. Project-local config and auto-watch are only honored when the project is trusted.
+
+---
+
+## Ecosystem behavior rules
+
+1. **Use namespaced tool names.** All tools are `graphify_*` to avoid collisions.
+2. **Use namespaced commands.** All commands are `/graphify-*` to avoid collisions.
+3. **Do not intercept other extensions.** `pi-graphify` may intercept read/grep only when explicitly enabled and only to suggest graph tools, not block the original tool.
+4. **Export, do not import.** `pi-graphify` may write lessons to memory extensions, but it should not read their private state.
+5. **Respect user config.** If another extension stores a preference that affects Graphify (e.g., “use directed graphs”), read it through Pi’s normal channels if available.
+
+---
+
+## Future ecosystem integrations
+
+- **Skill extensions:** A Pi skill could teach the agent how to combine Fallow and Graphify for deep code reviews.
+- **Prompt template extensions:** A prompt template could provide a pre-canned “Graphify-assisted refactor” workflow.
+- **Theme extensions:** Not relevant to `pi-graphify` unless we want a custom TUI theme.
+
+---
+
+## Open questions
+
+1. Should `pi-graphify` explicitly detect `pi-hermes-memory` and offer to write graph lessons into it?
+2. Should `pi-graphify` read `USER.md` to discover user preferences like `--directed` graphs?
+3. Should there be a shared convention for “project context” blocks so multiple extensions can inject hints without clashing?
+
+These will be answered as we implement and test alongside other extensions.

package/docs/STANDARDS.md

@@ -0,0 +1,155 @@
+# Project Standards
+
+This document defines project-level standards for `pi-graphify`. It is public and tracked in the repo. User-specific conventions should go in an untracked [`AGENTS.local.md`](AGENTS.local.md) file.
+
+---
+
+## Project structure
+
+```
+extensions/         # Pi registration code only
+src/                # Core implementation
+  backends/         # Graphify CLI and MCP backends
+  commands/         # Slash command definitions
+  tools/            # LLM tool definitions
+  coordinator.ts    # Backend routing and normalization
+  config.ts         # Configuration loading
+  graphify.ts       # Graphify CLI helpers
+  state.ts          # Session state
+  version.ts        # Version detection and compatibility
+  watcher.ts        # File watcher
+.github/workflows/  # CI/CD
+```
+
+No build step is required. Pi loads TypeScript directly via jiti.
+
+---
+
+## Coding standards
+
+### TypeScript
+
+- Use strict TypeScript.
+- Prefer `type` imports for types.
+- Use `.js` extensions in imports (NodeNext module resolution).
+- Avoid `any`. Use `unknown` when the type is genuinely unknown.
+- Keep functions small and focused.
+
+### Formatting
+
+- 2-space indentation (enforced by Biome).
+- No trailing whitespace.
+- LF line endings in the repo (Git handles CRLF on Windows via `core.autocrlf`).
+
+### Linting
+
+- All code must pass `npm run lint`.
+- Prefer auto-fixing with `npm run lint:fix`.
+
+---
+
+## Dependencies
+
+### Pi core packages
+
+Pi core packages are `peerDependencies` with `"*"` range. They are provided by Pi at runtime.
+
+```json
+"peerDependencies": {
+  "@earendil-works/pi-agent-core": "*",
+  "@earendil-works/pi-ai": "*",
+  "@earendil-works/pi-coding-agent": "*",
+  "@earendil-works/pi-tui": "*",
+  "typebox": "*"
+}
+```
+
+They must be listed as optional peers:
+
+```json
+"peerDependenciesMeta": {
+  "@earendil-works/pi-coding-agent": { "optional": true }
+}
+```
+
+### Runtime dependencies
+
+Third-party packages used at runtime belong in `dependencies`.
+
+### Dev dependencies
+
+Type definitions, linters, and type-checking tools belong in `devDependencies`.
+
+---
+
+## Module boundaries
+
+- `extensions/` — Only Pi registration code. No business logic.
+- `src/coordinator.ts` — The only place backend selection happens.
+- `src/backends/` — Backend-specific code. No Pi imports.
+- `src/tools/` — Tool definitions. Each calls the coordinator.
+- `src/commands/` — Command definitions. Each calls the coordinator.
+- `src/config.ts` — Pure config loading. No Pi imports.
+- `src/state.ts` — Session state.
+- `src/watcher.ts` — File watcher.
+- `src/version.ts` — Version detection and compatibility.
+
+---
+
+## Conventional commits
+
+All commits must follow the standards defined in [RELEASE.md](../RELEASE.md). In summary, use [Conventional Commits](https://www.conventionalcommits.org/) with one of these prefixes:
+
+| Prefix | Use for |
+|---|---|
+| `feat:` | New user-facing tool, command, or feature |
+| `fix:` | Bug fix |
+| `docs:` | Documentation changes |
+| `refactor:` | Code restructuring with no behavior change |
+| `test:` | Tests |
+| `chore:` | Maintenance, dependency updates, cleanup |
+| `ci:` | CI/CD changes |
+
+Breaking changes use `!` or `BREAKING CHANGE:` in the body.
+
+---
+
+## Testing
+
+- All new tools and commands should have tests where practical.
+- Run `npm run typecheck` and `npm run lint` before committing.
+- Integration tests should use the `CliBackend` with a mock or stubbed CLI.
+
+---
+
+## Security
+
+- Use `execFile` with argument arrays, never `exec` with shell strings.
+- Validate all paths relative to the project root.
+- Do not auto-run expensive operations (full builds, semantic extraction).
+- Treat Graphify output as text; never re-execute it.
+- Respect project trust for project-local config and watchers.
+
+---
+
+## Documentation
+
+- Design docs live in [`docs/`](./).
+- Release standards live in [`RELEASE.md`](../RELEASE.md).
+- Agent context lives in [`AGENTS.md`](../AGENTS.md).
+- User-specific conventions live in untracked [`AGENTS.local.md`](../AGENTS.local.md).
+
+All public docs should be clear, accurate, and maintained as the project changes.
+
+---
+
+## Verification commands
+
+```bash
+npm run typecheck
+npm run lint
+npm run lint:fix
+npm audit --audit-level=moderate
+```
+
+Run all of them before opening a pull request or pushing to `main`.