kirograph 0.14.1 → 0.16.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Davide De Sio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -80,6 +80,49 @@ Enable via `kirograph install` or directly in `.kirograph/config.json`:
 
 See the [Architecture Analysis](#architecture-analysis-opt-in-1) section below for full details.
 
+### Memory (opt-in)
+
+When `enableMemory: true` is set, KiroGraph stores persistent observations across sessions — decisions, errors, patterns, and architecture notes. Inspired by [cavemem](https://github.com/JuliusBrussee/cavemem) by [Julius Brussee](https://www.linkedin.com/in/julius-brussee/). Observations are:
+
+- **Compressed** with the caveman grammar (if caveman mode is enabled) — deterministic, no LLM tokens spent
+- **Linked to code symbols** — identifiers in observation text are matched against the graph and stored as stable `qualified_name` references
+- **Embedded** with the configured semantic engine — enabling natural-language search over past observations
+- **Deduplicated** — SHA-256 content hash prevents storing the same observation twice
+
+Memory surfaces automatically in `kirograph_context` and `kirograph_impact` results when relevant observations are linked to the symbols being queried. The agent can also search memory directly via `kirograph_mem_search` or store new observations via `kirograph_mem_store`.
+
+Zero LLM tokens on write. ~150-350 tokens per search (vs ~2000-5000 tokens to re-discover context by reading files).
+
+Enable via `kirograph install` or directly in `.kirograph/config.json`:
+
+```json
+{
+  "enableMemory": true
+}
+```
+
+See the [Memory](#memory-requires-enablememory-true) section below for full details.
+
+### Documentation indexing (opt-in)
+
+When `enableDocs: true` is set, KiroGraph indexes project documentation by heading hierarchy and section structure. Instead of reading entire doc files, agents retrieve exactly the section they need via stable section IDs. Inspired by [jDocMunch-MCP](https://github.com/jgravelle/jdocmunch-mcp) by [J. Gravelle](https://www.linkedin.com/in/j-gravelle-2778223/).
+
+- **9 format parsers**: Markdown, MDX, reStructuredText, AsciiDoc, RDoc, Org-mode, HTML, plain text, OpenAPI/Swagger
+- **Code ↔ docs cross-references**: Backtick references, CamelCase identifiers, and snake_case patterns in docs are resolved against the code graph
+- **Section-level FTS search**: Independent from code search (`kirograph_docs_search`)
+- **Stable section IDs**: `{file_path}::{ancestor-chain/slug}#{level}` — stable across re-indexing
+- **Token savings**: 92–97% reduction vs reading full doc files (tracked in `kirograph_gain`)
+
+Enable via `kirograph install` or directly in `.kirograph/config.json`:
+
+```json
+{
+  "enableDocs": true
+}
+```
+
+See the [Documentation](#documentation-requires-enabledocs-true) section below for full details.
+
 ## Installation
 
 ### From npm (not yet available on npm registry)
@@ -179,7 +222,7 @@ kg install
 └───────────────────────────────────────────┘
 ```
 
-Kiro hooks mark the index dirty on every file save or create, then flush on agent idle, batching changes efficiently with no overhead during active editing.
+A single Kiro hook triggers on `agentStop` and asks the agent to sync the index if any source files were changed during the session. No per-file hooks, no background watcher — zero overhead during active editing.
 
 ## Using with Kiro
 
@@ -201,37 +244,36 @@ Registers the KiroGraph MCP server. Used by both the IDE and the CLI agent:
         "kirograph_status", "kirograph_files", "kirograph_dead_code",
         "kirograph_circular_deps", "kirograph_path", "kirograph_type_hierarchy",
         "kirograph_architecture", "kirograph_coupling", "kirograph_package",
-        "kirograph_hotspots", "kirograph_surprising", "kirograph_diff"
+        "kirograph_hotspots", "kirograph_surprising", "kirograph_diff",
+        "kirograph_exec", "kirograph_gain"
+        "kirograph_mem_search", "kirograph_mem_store",
+        "kirograph_mem_timeline", "kirograph_mem_status"
       ]
     }
   }
 }
 ```
 
-### IDE Auto-Sync Hooks (`.kiro/hooks/`)
-
-Four hooks keep the index fresh automatically in the Kiro IDE:
+### IDE Hooks (`.kiro/hooks/`)
 
-| Hook | Event | Action |
-|------|-------|--------|
-| `kirograph-mark-dirty-on-save.json` | `fileEdited` | `kirograph mark-dirty` |
-| `kirograph-mark-dirty-on-create.json` | `fileCreated` | `kirograph mark-dirty` |
-| `kirograph-sync-on-delete.json` | `fileDeleted` | `kirograph sync-if-dirty` |
-| `kirograph-sync-if-dirty.json` | `agentStop` | `kirograph sync-if-dirty --quiet` |
+Up to two hooks are installed (`.kiro.hook` extension):
 
-File changes are batched: saves and creates write a dirty marker; the actual sync runs when the agent stops. Deletes sync immediately. This means no overhead during active editing.
+| Hook file | Event | Type | Behavior |
+|-----------|-------|------|----------|
+| `kirograph-sync-if-dirty.kiro.hook` | `agentStop` | `runCommand` | Runs `kirograph sync --quiet` when the agent stops, syncing any file changes from the session. The sync command skips unchanged files via content hashing, so it's fast even when nothing changed. |
+| `kirograph-compress-hint.kiro.hook` | `preToolUse` (shell) | `askAgent` | Reminds the agent to use `kirograph_exec` for commands that benefit from token compression (git, gh, test, lint, build, docker, aws, grep). Only installed when shell compression is enabled. |
 
-Hooks fire for: `.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, `.rs`, `.java`, `.cs`, `.rb`, `.php`, `.swift`, `.kt`, `.dart`, `.ex`, `.exs`
+The sync hook replaces the previous per-file approach (mark-dirty-on-save, mark-dirty-on-create, sync-on-delete). A single `agentStop` hook handles all file changes in one pass with zero overhead during active editing.
 
 ### CLI Agent Config (`.kiro/agents/kirograph.json`)
 
-A custom agent for Kiro CLI that wires up the MCP server, inlines the steering instructions as a prompt, and handles sync in the CLI's own hook format. The CLI has no file-watch events, so syncing is handled at session boundaries instead:
+A custom agent for Kiro CLI that wires up the MCP server, references the steering file as a resource, and handles sync in the CLI's own hook format. The CLI has no file-watch events, so syncing is handled at session boundaries:
 
-| Hook | Event | Action |
-|------|-------|--------|
-| `agentSpawn` | Agent starts | `kirograph sync-if-dirty --quiet` (catches edits made between sessions) |
-| `userPromptSubmit` | Each prompt | `kirograph sync-if-dirty --quiet` (keeps graph fresh within a session) |
-| `stop` | End of each turn | `kirograph sync-if-dirty --quiet` (deferred flush, mirrors IDE `agentStop`) |
+| Event | Action |
+|-------|--------|
+| `agentSpawn` | `kirograph sync-if-dirty --quiet` (catches edits made between sessions) |
+| `userPromptSubmit` | `kirograph sync-if-dirty --quiet` (keeps graph fresh within a session) |
+| `stop` | `kirograph sync-if-dirty --quiet` (deferred flush, mirrors IDE `agentStop`) |
 
 Use it with:
 
@@ -531,6 +573,100 @@ Show token savings statistics from compressed command outputs.
 
 Returns total commands, original/compressed token counts, savings percentage, breakdown by command family, and recent command history.
 
+### `kirograph_mem_search` *(requires `enableMemory: true`)*
+
+Search project memory for past decisions, errors, patterns, and context.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `query` | string | required | Natural language search query |
+| `kind` | string | - | Filter: `decision`, `error`, `pattern`, `architecture`, `summary`, `note` |
+| `limit` | number | 10 | Max results |
+| `sessionId` | string | - | Filter to specific session |
+| `projectPath` | string | cwd | Project root path |
+
+**How it works:** Hybrid search combining FTS5 keyword matching and vector cosine similarity (using the configured semantic engine). Results are ranked by a blended score (configurable via `memorySearchAlpha`). Falls back to FTS-only if embeddings are disabled or model mismatch is detected.
+
+### `kirograph_mem_store` *(requires `enableMemory: true`)*
+
+Store an observation in project memory. Content is automatically compressed (if caveman mode is on) and linked to relevant code symbols.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `content` | string | required | Observation text |
+| `kind` | string | `note` | `decision`, `error`, `pattern`, `architecture`, `summary`, `note` |
+| `projectPath` | string | cwd | Project root path |
+
+**How it works:** Strips `<private>` blocks → applies caveman compression (if enabled) → computes SHA-256 hash for deduplication → stores in `mem_observations` → detects symbol identifiers and creates `mem_links` → embeds with the configured model. Zero LLM tokens consumed.
+
+### `kirograph_mem_timeline` *(requires `enableMemory: true`)*
+
+List recent sessions and their observations chronologically.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `limit` | number | 5 | Number of sessions to show |
+| `sessionId` | string | - | Show observations for a specific session |
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_mem_status` *(requires `enableMemory: true`)*
+
+Memory subsystem health: session count, observations, embedding coverage, model mismatch detection.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_docs_toc` *(requires `enableDocs: true`)*
+
+Get table of contents for a documentation file or the whole project. Returns section IDs, titles, levels, and summaries.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `file` | string | - | Filter to a specific doc file (relative path). Omit for project-wide TOC. |
+| `tree` | boolean | false | Return nested tree structure |
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_docs_search` *(requires `enableDocs: true`)*
+
+Search documentation sections by query. Returns matching sections ranked by relevance. Independent from `kirograph_search` (code-only).
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `query` | string | required | Search query (natural language or keywords) |
+| `file` | string | - | Narrow search to a specific doc file |
+| `limit` | number | 10 | Max results |
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_docs_section` *(requires `enableDocs: true`)*
+
+Retrieve full content of a documentation section by its stable ID. Use `context=true` to also get ancestor headings and child summaries.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `id` | string | required | Section ID (from `kirograph_docs_toc` or `kirograph_docs_search` results) |
+| `context` | boolean | false | Include ancestor heading chain and child summaries |
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_docs_outline` *(requires `enableDocs: true`)*
+
+Get the heading hierarchy for a single documentation file. Lighter than full TOC when you know which file is relevant.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `file` | string | required | Relative path to the doc file |
+| `projectPath` | string | cwd | Project root path |
+
+### `kirograph_docs_refs` *(requires `enableDocs: true`)*
+
+Find code symbols referenced by a doc section, or doc sections that reference a code symbol. Bidirectional lookup.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `sectionId` | string | - | Doc section ID (find code symbols it references) |
+| `nodeId` | string | - | Code symbol qualified name (find doc sections that reference it) |
+| `projectPath` | string | cwd | Project root path |
+
 ## CLI Reference
 
 ### Setup
@@ -809,6 +945,8 @@ The `kirograph_gain` MCP tool exposes the same stats to the agent.
 | `kirograph_dead_code` | Not feasible manually (read every file) | 5× output, min 15,000 |
 | `kirograph_hotspots` | Not feasible manually (count edges for every symbol) | 5× output, min 15,000 |
 | `kirograph_architecture` | Not feasible manually | 4× output, min 7,500 |
+| `kirograph_mem_search` | Re-read 3-5 files to recall past decisions + grep | ~5,800 tokens |
+| `kirograph_mem_timeline` | Ask user or re-read session history | ~2,300 tokens |
 
 Constants used: 1,500 tokens per average source file (~200 lines), 800 tokens per grep result set, 2,000 tokens per directory listing. These are conservative estimates; in practice agents often read more files, retry failed searches, and explore dead ends.
 
@@ -930,6 +1068,82 @@ kirograph path --format json          # JSON output
 
 The command resolves symbol names using the same fuzzy search as `kirograph query`, preferring real symbol kinds (class, function, method…) over import/file nodes. The result shows each hop with file and line.
 
+### Memory *(requires `enableMemory: true`)*
+
+Persistent cross-session observations — search, store, and manage project memory from the CLI.
+
+```bash
+# Search (mirrors kirograph_mem_search)
+kirograph mem search "payment retry"              # hybrid FTS + vector search
+kirograph mem search "auth bug" --kind error      # filter by kind
+kirograph mem search "refactor" --limit 5         # limit results
+kirograph mem search "token" --format json        # JSON output
+
+# Store (mirrors kirograph_mem_store)
+kirograph mem store "decided to use idempotency keys for payments"
+kirograph mem store "auth bug: token refresh missing" --kind error
+kirograph mem store --kind decision < decision.txt   # pipe from stdin
+
+# Timeline (mirrors kirograph_mem_timeline)
+kirograph mem timeline                    # last 5 sessions
+kirograph mem timeline --limit 10         # more sessions
+kirograph mem timeline --session <id>     # specific session
+kirograph mem timeline --format json
+
+# Status (mirrors kirograph_mem_status)
+kirograph mem status                      # health dashboard
+
+# Maintenance
+kirograph mem prune --older-than 90d      # cleanup old observations
+kirograph mem export --format jsonl       # machine-readable export (importable)
+kirograph mem export --format md          # human-readable export
+kirograph mem import backup.jsonl         # restore from backup (deduplicates)
+kirograph mem reembed                     # re-embed after model change
+kirograph mem reembed --batch 50          # control batch size
+kirograph mem lint                        # find stale links, model mismatch
+kirograph mem lint --fix                  # auto-repair issues
+```
+
+**How observations are stored:** Text → strip `<private>` blocks → caveman compress (if enabled) → SHA-256 dedup check → store → detect symbol identifiers → link to graph → embed. Zero LLM tokens.
+
+**How observations surface:** `kirograph_context` and `kirograph_impact` automatically include relevant memory observations (max 3, above relevance threshold 0.3) when memory is enabled. No extra tool call needed.
+
+### Documentation *(requires `enableDocs: true`)*
+
+Section-level documentation navigation — search, browse, and retrieve doc sections from the CLI.
+
+```bash
+# Table of contents
+kirograph docs toc                          # whole project
+kirograph docs toc README.md                # single file
+kirograph docs toc README.md --tree         # nested tree structure
+kirograph docs toc --json                   # JSON output
+
+# Search (mirrors kirograph_docs_search)
+kirograph docs search "authentication"
+kirograph docs search "config" --file docs/guide.md
+kirograph docs search "install" --limit 5
+
+# Retrieve a section (mirrors kirograph_docs_section)
+kirograph docs section "README.md::installation#1"
+kirograph docs section "README.md::installation#1" --context
+
+# Outline (mirrors kirograph_docs_outline)
+kirograph docs outline docs/api.md
+
+# Cross-references (mirrors kirograph_docs_refs)
+kirograph docs refs "docs/auth.md::oauth/token-refresh#2"
+
+# Maintenance
+kirograph docs reindex                      # force full re-index
+kirograph docs lint                         # health checks (broken refs, stale sections)
+kirograph docs reembed                      # re-embed with current model
+```
+
+**How sections are identified:** Each section gets a stable ID in the format `{file_path}::{ancestor-chain/slug}#{level}`. IDs remain stable across re-indexing as long as the file path, heading text, heading level, and parent chain don't change.
+
+**How code linking works:** When `docsLinkCode: true` (default), the indexer scans section content for backtick references (`` `functionName` ``), CamelCase identifiers, and snake_case patterns, then resolves them against the code graph. Matches are stored as `doc_code_refs` using `qualified_name` (stable across reindex).
+
 ### Graph Export
 
 Export the full graph as an interactive dashboard. three files served from a local directory, no server required, works offline.
@@ -1062,7 +1276,7 @@ By default, KiroGraph uses exact name lookup and full-text search. Enable semant
 }
 ```
 
-This generates vector embeddings for all functions, methods, classes, interfaces, type aliases, components, and modules using a local embedding model (downloaded automatically to `~/.kirograph/models/` on first use). Embeddings are kept in sync automatically via Kiro hooks. on every file save, create, or delete.
+This generates vector embeddings for all functions, methods, classes, interfaces, type aliases, components, and modules using a local embedding model (downloaded automatically to `~/.kirograph/models/` on first use). Embeddings are kept in sync automatically via the Kiro `agentStop` hook, which syncs the index (including embeddings) whenever files change during a session.
 
 Run `kirograph install` to be guided through model and engine selection interactively with arrow-key menus, or set the fields manually in `.kirograph/config.json`.
 
@@ -1448,6 +1662,11 @@ Detected frameworks are stored in config and used to improve symbol extraction a
 
 KiroGraph is inspired by [CodeGraph](https://github.com/colbymchenry/codegraph) by [Colby McHenry](https://www.linkedin.com/in/colby-mchenry/). the original concept of building a semantic code graph for AI coding agents comes from his work.
 
+### Inspirations
+
+- [cavemem](https://github.com/JuliusBrussee/cavemem) by [Julius Brussee](https://www.linkedin.com/in/julius-brussee/): the memory module's hook-based observation capture, deterministic compression, and SQLite storage pattern.
+- [jDocMunch-MCP](https://github.com/jgravelle/jdocmunch-mcp) by [J. Gravelle](https://www.linkedin.com/in/j-gravelle-2778223/): the documentation module's section-first retrieval approach, stable section IDs, and byte-offset addressing.
+
 ### Contributors
 
 - [Alessandro Franceschi](https://www.linkedin.com/in/alessandrofranceschi/). Claude Code and Codex integration, Elixir/Phoenix language and framework support.