sinapse-ai 9.2.0 → 9.4.0

package/squads/squad-claude/knowledge-base/claude-code-tools-reference.md

@@ -0,0 +1,313 @@
+# Claude Code Tools Reference
+
+> Complete inventory of all 184 tool modules across 35 subsystems. Based on source code analysis (v2.1.88 leak, April 2026).
+
+---
+
+## Tool System Architecture
+
+### Core Interface: `Tool<Input, Output, P>`
+
+Three generics: Zod input schema, output type, progress event shape.
+
+**Required members:**
+- `name` — primary identifier
+- `aliases` — legacy names
+- `inputSchema` — Zod schema (source of truth for validation)
+- `maxResultSizeChars` — trigger for disk persistence
+- `call()` — async execution, receives args, context, permission function, parent message
+- `checkPermissions()` — validation before execution
+- `isConcurrencySafe(input)` — can run in parallel?
+- `isReadOnly(input)` — read-only operation?
+- `isDestructive?()` — optional flag
+
+**`buildTool()` factory defaults (fail-closed):**
+```
+isConcurrencySafe: () => false    // assumes state mutation
+isReadOnly: () => false
+isDestructive: () => false
+checkPermissions: () => Promise.resolve({ behavior: 'allow' })
+```
+
+### 3-Tier Tool Pipeline
+
+1. **`getAllBaseTools()`** — exhaustive catalog with feature flags and env var gating
+2. **`getTools()`** — filter by mode (simple, REPL, deny rules, `isEnabled()`)
+3. **`assembleToolPool()`** — built-ins alphabetically as prefix, MCP tools alphabetically after (separation preserves cache breakpoints)
+
+### Concurrency Orchestration
+
+`partitionToolCalls()` groups consecutive safe tools:
+- Tools with `isConcurrencySafe(input) === true` run in parallel
+- Non-safe tools break batches and run serially
+- Ceiling: `CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY` (default 10)
+
+---
+
+## Complete Tool Catalog
+
+### File Operations (~40 registered, ~20 enabled by default)
+
+| Tool | Category | Default On | Description |
+|------|----------|-----------|-------------|
+| `FileReadTool` | File | Yes | Read file contents |
+| `FileEditTool` | File | Yes | Edit existing files (diff-based) |
+| `FileWriteTool` | File | Yes | Create/overwrite files |
+| `GlobTool` | File | Yes | File pattern matching |
+| `GrepTool` | File | Yes | Content search with regex |
+| `NotebookEditTool` | File | Yes | Jupyter notebook editing |
+
+### Shell Execution
+
+| Tool | Category | Default On | Notes |
+|------|----------|-----------|-------|
+| `BashTool` | Shell | Yes | 16 modules, AST-based security, errors abort siblings |
+| `PowerShellTool` | Shell | Windows | 14 modules, git safety |
+| `REPLTool` | Shell | No | Interactive REPL mode |
+
+**BashTool special treatment:**
+- Only tool whose errors abort siblings (implicit dependency chains)
+- Starts security classifier speculatively BEFORE hooks
+- 23 security checks in `bashSecurity.ts`
+- 18 Zsh builtins blocked
+
+### Agent System
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `AgentTool` | 20 | Core agent spawning, fork model, built-in agents |
+| `forkSubagent.ts` | — | Fork model for subagent creation |
+| `runAgent.ts` | — | Agent execution runtime |
+| `resumeAgent.ts` | — | Agent resumption |
+| `agentMemory.ts` | — | Agent-specific memory |
+| `agentMemorySnapshot.ts` | — | Memory snapshots |
+
+**Built-in agents:** `generalPurposeAgent`, `planAgent`, `verificationAgent`, `exploreAgent`, `clawCodeGuideAgent`
+
+### MCP Tools
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `MCPTool` | 4 | Core MCP protocol integration |
+| `ListMcpResourcesTool` | 3 | List available MCP resources |
+| `ReadMcpResourceTool` | 3 | Read MCP resources |
+| `McpAuthTool` | 1 | MCP authentication |
+
+**MCP tool naming convention:**
+```
+mcp__{server}__{tool}
+```
+Example: "weather tool" on "Example Server" → `mcp__Example_Server__weather_tool`
+
+### Task Management
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `TaskCreateTool` | 3 | Create background tasks |
+| `TaskGetTool` | 3 | Get task status |
+| `TaskListTool` | 3 | List all tasks |
+| `TaskUpdateTool` | 3 | Update task state |
+| `TaskStopTool` | 3 | Stop running tasks |
+| `TaskOutputTool` | 2 | Get task output |
+
+### Multi-Agent Team Tools
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `TeamCreateTool` | 4 | Create agent team |
+| `TeamDeleteTool` | 4 | Delete agent team |
+| `SendMessageTool` | 4 | Send message to teammate or broadcast |
+| `AskUserQuestionTool` | 2 | Ask user for input |
+
+### Planning & Navigation
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `EnterPlanModeTool` | 4 | Enter plan-only mode |
+| `ExitPlanModeTool` | 4 | Exit plan mode |
+| `EnterWorktreeTool` | 4 | Enter git worktree isolation |
+| `ExitWorktreeTool` | 4 | Exit worktree |
+| `BriefTool` | 5 | Create structured brief |
+
+### Web Tools
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `WebFetchTool` | 5 | Fetch web content |
+| `WebSearchTool` | 3 | Search the web |
+| `WebBrowserTool` | — | Browser automation (feature-flagged) |
+
+### IDE Integration
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `LSPTool` | 6 | Language Server Protocol integration |
+| `NotebookEditTool` | 4 | Jupyter notebook support |
+
+**LSP modules:** `LSPTool`, `UI`, `formatters`, `prompt`, `schemas`, `symbolContext`
+
+### Scheduling & Automation
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `ScheduleCronTool` | 5 | Create scheduled tasks |
+| `CronDeleteTool` | — | Delete scheduled tasks |
+| `CronListTool` | — | List scheduled tasks |
+| `RemoteTriggerTool` | 3 | Remote task triggering |
+
+### Utility Tools
+
+| Tool | Modules | Description |
+|------|---------|-------------|
+| `ToolSearchTool` | 3 | Search/discover available tools (deferred loading) |
+| `TodoWriteTool` | 3 | Task/todo tracking |
+| `ConfigTool` | 5 | Configuration management (internal) |
+| `SkillTool` | 4 | Skill invocation |
+| `SleepTool` | 1 | Wait/delay |
+| `SyntheticOutputTool` | 1 | Generate synthetic output |
+
+### KAIROS-Exclusive Tools (unreleased)
+
+| Tool | Description |
+|------|-------------|
+| `SendUserFileTool` | Send file to user via push notification |
+| `PushNotificationTool` | Push notification to user devices |
+| `SubscribePRTool` | Subscribe to GitHub PR events |
+
+### Internal/Debug Tools
+
+| Tool | Availability |
+|------|-------------|
+| `TungstenTool` | Internal Anthropic only (`USER_TYPE === 'ant'`) |
+| `SuggestBackgroundPRTool` | Internal |
+| `MonitorTool` | MCP monitoring |
+
+---
+
+## Tool Execution Pipeline
+
+`runToolUse()` orchestrates:
+
+```
+1. Zod validation — inputSchema.safeParse(input)
+2. Semantic validation — path traversal, size limits
+3. Speculative classifier — Bash commands start security check before hooks
+4. backfillObservableInput — shallow clone for hooks and canUseTool
+5. PreToolUse hooks — async generators yielding progress/permission
+6. canUseTool() — primary permission gate
+7. tool.call() — actual execution
+8. PostToolUse hooks — after completion
+9. Result serialization — size budget processing
+```
+
+**3 copies of input maintained:**
+1. API-bound original — for cache/serialization
+2. Backfilled observable clone — for hooks and canUseTool
+3. Hook-updated call input — potentially modified for execution
+
+---
+
+## Deferred Tool Loading (Tool Search)
+
+### Context Window Savings
+
+| Before (v2.1.68) | After (v2.1.69+) |
+|------------------|------------------|
+| ~14-16K tokens for tool definitions | ~968 tokens |
+| All tools upfront | Only ToolSearch + non-deferred |
+
+**93% reduction in tool definition tokens.**
+
+### How Tool Search Works
+
+1. Claude sees only `ToolSearchTool` + non-deferred tools at session start
+2. When task requires a specific tool, calls `ToolSearchTool` with keyword
+3. Returns 3-5 `tool_reference` blocks (most relevant matches)
+4. References auto-expanded throughout conversation history
+5. Tool now available for use in subsequent turns
+
+### Configuration
+
+```bash
+# Threshold-based (loads when context > 10% full) — recommended
+ENABLE_TOOL_SEARCH=auto
+
+# Always deferred
+ENABLE_TOOL_SEARCH=true
+
+# All upfront (pre-v2.1.69 behavior)
+ENABLE_TOOL_SEARCH=false
+```
+
+---
+
+## Safety Architecture
+
+### Safety Through Proximity
+
+"Safety rules appear in the immediate context of the action they govern" — embedded directly in tool descriptions rather than separate policy files. This makes them harder for LLMs to overlook.
+
+**Applied in SINAPSE:** Hook governance rules embedded in tool-specific hook scripts, not just in documentation.
+
+### Tool Authority Matrix
+
+| Agent Type | Tool Access |
+|-----------|-------------|
+| Built-in general-purpose | `['*']` — all tools |
+| Built-in Explore | Read-only tools only |
+| Built-in Plan | Read-only tools only |
+| Built-in verification | No Edit/Write; allows /tmp scripts |
+| Custom agents | Configured via `tools` + `disallowedTools` frontmatter |
+
+### Permission Escalation Flow
+
+```
+ReadOnly → WorkspaceWrite → DangerFullAccess → Prompt → Allow
+```
+
+- Tools default to requiring `DangerFullAccess` unless explicitly configured
+- Missing prompters deny escalation requests
+- `Prompt` mode triggers interactive user decision-making
+
+---
+
+## MCP Transport Types
+
+| Transport | Description | Use Case |
+|-----------|-------------|----------|
+| **Stdio** | Local process execution | Local CLI tools, scripts |
+| **SSE** | Server-Sent Events (deprecated → HTTP) | Remote endpoints |
+| **HTTP** | Remote endpoints with optional OAuth | Cloud services (recommended) |
+| **WebSocket** | Real-time bidirectional | Real-time data |
+| **SDK** | Built-in implementations | No external process needed |
+| **ManagedProxy** | Anthropic-hosted proxy | Managed services |
+
+**"Claude Code isn't built on top of MCP. It IS MCP — every capability, including Computer Use, runs as a tool call."**
+
+Computer Use is implemented as `@ant/computer-use-mcp` — a dedicated MCP server, not special-cased functionality.
+
+### MCP Tool Discovery: Deferred Loading
+
+MCP tools are deferred by default — only tool names consume context until used. Tool Search discovers them on-demand. `ENABLE_TOOL_SEARCH=auto` for threshold-based loading.
+
+---
+
+## SINAPSE Tool Selection Guidance
+
+### When to use each approach
+
+| Scenario | Tool | Reason |
+|----------|------|--------|
+| Read files | `Read` (native) | Faster, no overhead |
+| Write/edit files | `Write`/`Edit` (native) | Direct filesystem access |
+| Run shell commands | `Bash` (native) | Security validation included |
+| Search files | `Glob` (native) | Optimized pattern matching |
+| Search content | `Grep` (native) | Regex with line numbers |
+| Library documentation | Context7 (MCP) | Up-to-date docs |
+| Web research | EXA (MCP via Docker) | Current information |
+| Browser automation | Playwright (MCP) | Full browser control |
+| Database operations | Supabase (MCP) | Native SQL operations |
+| Code analysis | Nogic (MCP, essential) | Import chain, usage tracking |
+| Dependency graphs | Code-Graph (MCP, essential) | Circular dependency detection |
+
+**Rule:** Always prefer Tier 1 native tools. Only escalate to MCP when native tools cannot accomplish the task.

package/squads/squad-claude/knowledge-base/context-window-optimization.md

@@ -0,0 +1,334 @@
+# Context Window Optimization
+
+> Token diet, compaction strategies, and 1M context management. Based on Claude Code internals + MS-009 Agentic Second Brain research.
+
+---
+
+## The Context Window as a Resource
+
+**Karpathy (2025):** "Think of the LLM as a CPU, and its context window as RAM. Your job as an engineer is analogous to an OS: load exactly the right code and data for the task."
+
+**Willison (2025):** "Context engineering is what we do instead of fine-tuning."
+
+The context window is the scarcest resource in agentic systems. Every token has real cost. Every irrelevant token reduces quality.
+
+---
+
+## Context Budget Breakdown
+
+### Claude Code Session Anatomy
+
+```
+TOTAL WINDOW: ~200,000 tokens (Sonnet/Opus)
+  ├── System Prompt (static):   ~2,000 tokens (1%)       [CACHED]
+  ├── CLAUDE.md hierarchy:      1-10,000 tokens (0.5-5%) [CACHED]
+  ├── Rules files (loaded):     500-5,000 tokens (0.25-2.5%) [CACHED]
+  ├── Tool definitions:         ~968 tokens (0.5%)        [DEFERRED]
+  │   (after deferred loading; was 14-16K without)
+  ├── Conversation history:     grows per turn            [COMPACTED]
+  ├── Tool outputs:             variable                  [BUDGETED]
+  └── Response budget:          ~50-100K tokens           [RESERVED]
+```
+
+### Token Sources by Control Level
+
+| Source | Typical Size | Your Control |
+|--------|-------------|--------------|
+| System prompt | ~2-4K | None (fixed by CC) |
+| CLAUDE.md (all levels) | 1-10K | Full |
+| .claude/rules/ (loaded) | 0.5-5K | Full (conditional loading) |
+| Conversation history | Grows per turn | Partial |
+| Tool outputs | Variable | Partial (hooks) |
+| MCP tool schemas | ~100-500 per server | Partial |
+| Tool definitions | ~968 (deferred) or 14-16K (upfront) | Full |
+
+**CLAUDE.md and rules = highest leverage targets** (full control, loaded every turn).
+
+---
+
+## Claude Code Compaction System
+
+### 5-Layer Pipeline
+
+| Layer | Strategy | API Cost | When |
+|-------|----------|----------|------|
+| **Tool Result Budget** | Caps individual result sizes | 0 | Continuous |
+| **Snip Compact** | Removes unnecessary intermediate messages | 0 | As needed |
+| **MicroCompact** | Merges consecutive tool-result/user pairs | 0 | Per turn |
+| **Context Collapse** | Read-time projection over history | 0 | Near limit |
+| **AutoCompact** | Complete summarization via forked agent | 1 | ~95% capacity |
+
+**AutoCompact trigger:** ~95% context window capacity (25% remaining).
+
+### What Survives Compaction
+
+**Preserved:**
+- CLAUDE.md (re-read from disk after each compaction)
+- User requests and key code snippets
+- "Compact Instructions" from CLAUDE.md
+- Active plans and pending work
+
+**Lost:**
+- Instructions given in conversation (NOT in CLAUDE.md) — **put important instructions in CLAUDE.md, not chat**
+- Error messages, line numbers, variable values, stack traces
+- Reasoning behind decisions
+- Specific debugging details
+
+### Compact Instructions Section
+
+Add to CLAUDE.md to control what AutoCompact preserves:
+
+```markdown
+## Compact instructions
+- Preserve code paths and unresolved security questions
+- Preserve diff summaries and failed test output
+- Preserve current story ID and acceptance criteria
+- Preserve any pending tasks and their status
+```
+
+---
+
+## Token Reduction Techniques
+
+### 1. Deferred Tool Loading (93% reduction on tool definitions)
+
+The biggest single optimization since v2.1.69:
+
+```
+Before: ~14,000-16,000 tokens for all tool definitions
+After:  ~968 tokens (ToolSearch + non-deferred tools)
+```
+
+Tools are discovered on-demand via `ToolSearchTool`. Only activated tools add to context.
+
+### 2. Static/Dynamic Prompt Separation
+
+```
+[Static — before boundary]      ← Cached, shared across all users
+  Identity, guidelines, tasks,
+  actions, tools, tone, output
+  ─────────────────────────────
+[SYSTEM_PROMPT_DYNAMIC_BOUNDARY]
+  ─────────────────────────────
+[Dynamic — after boundary]      ← Not cached, session-specific
+  Project context, CLAUDE.md,
+  memory, env info, MCP instructions
+```
+
+**Cache hit rate** = biggest cost lever. A cache miss on static content = **5x cost for that turn**.
+
+### 3. Rules File Conditional Loading
+
+Without `paths:` frontmatter: rule loads globally (every interaction).
+With `paths:` frontmatter: rule loads only when editing matching files.
+
+```yaml
+---
+paths:
+  - "src/api/**/*.ts"
+---
+# API Route Rules
+Only loaded when editing API files.
+```
+
+**Strategy:** Convert global rules to scoped rules wherever possible.
+
+### 4. CLAUDE.md Token Diet
+
+**High-value content (include):**
+- Project identity (1 sentence)
+- Technology stack (table format)
+- Non-obvious directory structure
+- Conventions that differ from standard practices
+- Anti-patterns (prevents costly mistakes)
+- Common commands (build, test, lint)
+- Known gotchas
+
+**Low-value content (exclude):**
+- Generic programming advice (Claude already knows)
+- Standard framework conventions (React, Express, etc.)
+- Obvious file purposes
+- Long code examples (use one-liners or references)
+- Historical context (why decisions were made)
+- Duplicate information
+
+### 5. Document Sharding (BMAD Pattern)
+
+Large documents split into focused shards:
+
+| Format | Token Size |
+|--------|-----------|
+| Single large PRD | ~5,000 tokens |
+| Sharded (per section) | ~300 tokens each |
+| **Savings** | **74-90%** |
+
+Only load the shard relevant to current task.
+
+### 6. Image Optimization
+
+| Image Size | Token Cost |
+|-----------|-----------|
+| 200×200 pixels | 54 tokens |
+| 1000×1000 pixels | 1,334 tokens |
+| **Savings with cropping** | **25x** |
+
+Always crop/resize images to minimum necessary for the task.
+
+### 7. File Format Efficiency
+
+| Format | Token Efficiency vs JSON | Notes |
+|--------|------------------------|-------|
+| Markdown | -16% tokens | Best for instructions, docs |
+| YAML | Better for nested | 62% accuracy in nested structures |
+| JSON | Baseline | Standard for schemas, APIs |
+
+**Recommendation:**
+- CLAUDE.md: Markdown
+- Config: YAML frontmatter + Markdown body
+- Skills/Agents: YAML frontmatter in `.md` files
+- Settings: JSON
+- Files: < 300 lines each
+
+### 8. Model Routing (Tiered)
+
+Not all decisions require frontier-class inference:
+
+| Decision Type | Model | Cost Multiplier |
+|-------------|-------|----------------|
+| Planning complex tasks | Opus | 1.0x (baseline) |
+| Code implementation | Sonnet | 0.33x |
+| Permission checks | Haiku | 0.067x |
+| Frustration detection | Regex | ~0x |
+| Context compression (micro) | None (no API call) | 0x |
+
+**Savings:** 40-70% total cost reduction with proper routing.
+
+---
+
+## 14 Cache-Break Vectors
+
+Claude Code monitors 14 vectors that invalidate prompt cache:
+
+1. Adding an MCP tool
+2. Putting timestamp in system prompt (use in message instead)
+3. Switching model mid-session
+4. Changing images in prompt
+5. Modifying tool settings
+6. Adding/removing CLAUDE.md files
+7. Changing rule files
+8. Updating agent definitions
+9. Modifying hook configurations
+10. Changing environment variables in settings
+11. Adding new MCP server instructions
+12. Updating agent memory
+13. Modifying project structure detected by hooks
+14. Permission mode changes
+
+**Sticky latches:** `afkModeHeaderLatched` and `fastModeHeaderLatched` prevent cache busts when toggling settings mid-session. Once latched, remains latched for session duration.
+
+---
+
+## Context Engineering for Multi-Agent
+
+### Cache Sharing Between Subagents
+
+Fork children use **byte-identical system prompt prefixes** to share KV cache:
+
+```
+Parent system prompt (cached):
+  [System guidelines]
+  [Project context]
+  [CLAUDE.md]
+  [Tool definitions]
+  ────────────────────
+  [Agent-specific task] ← unique per agent
+```
+
+**Cost:** "Spawning five forked agents costs barely more than 1."
+
+**Requirement:** Agent prompts must be designed to maximize shared prefix length.
+
+### Agent Memory Context Budget
+
+For SINAPSE agents, recommended allocation:
+
+```
+200K token session:
+  Agent persona (CLAUDE.md + rules): ~5K (2.5%)
+  HOT memory (current task): ~20K (10%)
+  WARM memory (retrieved context): ~30K (15%)
+  Conversation history (compacted): ~20K (10%)
+  Tool outputs (last N results): ~50K (25%)
+  Response budget: ~75K (37.5%)
+```
+
+---
+
+## The `lost-in-the-middle` Effect
+
+Research finding: LLMs show significantly degraded performance when critical information appears in the **middle** of a long context. Performance peaks when information is at the beginning or end.
+
+**Mitigation strategies:**
+1. Put most important instructions in system prompt (beginning)
+2. Put current task at end of context (just before response)
+3. Use compaction to remove middle noise
+4. Structure long contexts with clear section headers
+5. Use explicit attention anchors ("IMPORTANT:", "CRITICAL:")
+
+---
+
+## Practical Optimization Checklist
+
+### Before Starting a Session
+
+- [ ] CLAUDE.md is under 200 lines
+- [ ] No global rules that should be scoped rules
+- [ ] Images cropped to necessary size
+- [ ] `ENABLE_TOOL_SEARCH=auto` configured
+- [ ] Compact instructions section in CLAUDE.md
+
+### Before Each Agent Turn
+
+- [ ] Is conversation history manageable? (< 50% of budget)
+- [ ] Have tool outputs been budgeted? (large outputs truncated)
+- [ ] Is the current task clearly specified in last message?
+- [ ] Any irrelevant context that should be cleared?
+
+### Red Flags (Memory Full Indicators)
+
+- Claude asking for information it should have from CLAUDE.md
+- Claude losing track of requirements mid-task
+- Responses getting shorter and less detailed
+- Claude proposing solutions already tried
+- `/compact` triggered automatically
+
+**Action on red flags:** Use `/compact` or `/clear` immediately. Or add key context to CLAUDE.md so it survives compaction.
+
+---
+
+## SINAPSE Context Architecture
+
+### System Prompt Allocation
+
+```
+~/.claude/CLAUDE.md (global):     ~2K tokens
+Project CLAUDE.md:                ~3K tokens
+.claude/rules/ (all):             ~5K tokens
+──────────────────────────────────
+Total baseline:                   ~10K tokens
+```
+
+### Agent Memory Allocation
+
+Each SINAPSE agent file: target < 300 lines / ~3K tokens.
+
+**Why:** Agents are loaded into context when activated. Large agent definitions increase baseline cost for every session turn.
+
+### KB Files in Context
+
+Knowledge base files are NOT automatically loaded. They are:
+- Referenced in CLAUDE.md or rules
+- Loaded on-demand when agent needs specific knowledge
+- Ideal for: reference material, patterns, examples
+
+**Recommendation:** Keep KB files as standalone reference documents. Link from CLAUDE.md with `see squads/squad-claude/knowledge-base/X.md` comments, not full @include.