joonecli 0.1.1 → 0.2.0

package/docs/02_edge_cases_and_mitigations.md

@@ -1,143 +0,0 @@
-# Edge Cases & Mitigations
-
-When building a coding agent with Prompt Caching + Middlewares, these are the primary edge cases to design around:
-
-## 1. Prompt Caching Edge Cases (Cost & Latency Traps)
-
-- **The "Leaky Timestamp" Cache Breaker:**
-  - _The Edge Case:_ If you inject dynamic data (like the current time, memory usage, or random UUIDs) into your Base System Prompt, you will achieve a **0% cache hit rate**. The cache relies on exact prefix matching.
-  - _Mitigation:_ Put all static, immutable instructions at the top. Any dynamic state must be injected via a `<system-reminder>` inside the _Messages_ array (which sits at the end of the context).
-- **The Mid-Session Model Switch:**
-  - _The Edge Case:_ Switching models mid-thread (e.g., cheap model for summarizing, smart model for coding) means the new model has an empty cache and must re-process the entire prompt prefix from scratch.
-  - _Mitigation:_ Avoid swapping models in the same thread. Span a "Sub-agent" thread and only pass minimum necessary context.
-- **Context Window Compaction (Amnesia):**
-  - _The Edge Case:_ Summarizing a long conversation and starting a new prompt causes you to lose your cached prefix AND the agent forgets specific constraints.
-  - _Mitigation:_ Implement **Cache-Safe Forking**. Keep the exact same System Prompt and Tool definitions. Start a new thread by passing the summary of the previous history as the first few messages, followed by the new task.
-
-## 2. Harness & Middleware Edge Cases (Logic Traps)
-
-- **The "Massive File" Blunder:**
-  - _The Edge Case:_ The agent reads a 10,000-line minified file. This floods the context window, pushes out important instructions, and ruins the session cache.
-  - _Mitigation:_ Harness-level Guardrails. Restrict `read_file` to return chunks or force the agent to use `grep_search` / `view_file_outline`.
-- **The "Blind Retry" Doom Loop:**
-  - _The Edge Case:_ The agent misses a space in a search-and-replace, fails, and tries the exact same edit endlessly.
-  - _Mitigation:_ Use `LoopDetectionMiddleware`. If the agent emits identical tool calls 3 times, intercept and inject: _"You have failed this 3 times. Stop trying this approach."_
-- **The "Fake Success" Verification:**
-  - _The Edge Case:_ The agent runs tests, they fail, but the agent hallucinates that the failure is acceptable and marks the task as Done. Older approaches relied on fragile string parsing (e.g., matching "failed" in output), which could easily be bypassed or confused by test output.
-  - _Mitigation:_ The harness must programmatically parse terminal exit codes. By explicitly surfacing structured tool metadata (e.g., `ToolResult.metadata.exitCode`) from execution sandboxes, the `PreCompletionMiddleware` reliably blocks the agent from exiting if tests don't pass (`exitCode !== 0`).
-- **Tool Schema Amnesia (with Lazy Loading):**
-  - _The Edge Case:_ An agent loads a complex tool lazily, uses it once, and then later forgets how to format its JSON schema.
-  - _Mitigation:_ If a tool is "discovered", it must remain in the "Messages" context as a system reminder so the schema is preserved.
-- **The "Ghost Tool Call" (Context Desync):**
-  - _The Edge Case:_ A model emits a tool call but occasionally forgets to attach a internal `tool_call_id` (this breaks the strict `AIMessage[tool_calls] -> ToolMessage[tool_call_id]` sequencing rules required by modern LangChain/Anthropic/OpenAI APIs). If you forge a fake ID or cast it as a string, the LLM rejects the context on the next turn.
-  - _Mitigation:_ The "Soft Fail" approach. Intercept the malformed tool call in the `ExecutionHarness`. Do not execute the tool and do not emit a `ToolMessage`. Instead, emit a corrective `HumanMessage` stating: _"You attempted to call tool X, but didn't provide a tool_call_id. Please try again."_ This prevents context poisoning.
-
-## 3. Security & Execution Edge Cases (Tool Exploits)
-
-- **Command Injection via Malicious Interpolation:**
-  - _The Edge Case:_ Passing user-provided arguments directly into shell commands (e.g., `agent-browser --url "${args.url}"` or `gemini --file "${args.path}"`) allows attackers to escape quotes and execute arbitrary commands in the sandbox (e.g., `url = '"; cat /etc/passwd; "'`).
-  - _Mitigation:_ Use strict Bash parameter escaping. All dynamic strings passed to shell commands are wrapped in single quotes, and any internal single quotes are escaped (`'\\''`).
-- **Host Filesystem Path Traversal (The "Escaped Workspace" Vulnerability):**
-  - _The Edge Case:_ Because `read_file` and `write_file` execute on the host machine to support live IDE syncing, a malicious prompt could instruct the agent to write to `~/.bashrc`, `C:\Windows\System32`, or `/.ssh/id_rsa`, compromising the user's host machine.
-  - _Mitigation:_ Implement strict Workspace Jail boundaries. Before any host I/O operation, the resolved path is evaluated against `process.cwd()`. If the path attempts to escape the root workspace, the tool immediately rejects the call returning a permissions error.
-- **Silently Swallowed CLI Errors:**
-  - _The Edge Case:_ A CLI tool (like OSV-Scanner) crashes due to a configuration error (exit code > 1) and prints an error to `stderr`. If the orchestration layer only checks for `stdout` and swallows non-zero exit codes silently falling back to another tool, the critical error trace is lost.
-  - _Mitigation:_ Enforce strict exit code verification (e.g., `exitCode === 1` means vulnerabilities found) and emit clear warnings with the full `stderr` trace before attempting any fallback strategies.
-- **The "Over-Eager Doom Loop" Reporter:**
-  - _The Edge Case:_ When detecting a doom loop (calling the same tool with identical args continuously), firing an alert during the active iteration causes redundant, spammy issue reports (e.g., reporting loop counts 3, 4, and 5 as separate critical issues).
-  - _Mitigation:_ Track the loop state continuously but defer pushing the `AnalysisIssue` to the report array until the loop is visibly broken by a differing action, or the trace ends.
-- **The "Parallel Tool Expansion" Bug (TUI Memory Corruption):**
-  - _The Edge Case:_ In a Terminal UI rendering loop, executing an array of tool calls _inside_ the UI rendering iteration causes the generated `ToolMessage` array to be appended to the conversation history $N$ times (for $N$ tools), massively inflating context usage with duplicated data.
-
-## 4. Persistent Session Edge Cases (State Management)
-
-- **File System Drift (Host Desync):**
-  - _The Edge Case:_ The agent edits a file, the session is paused. A human edits the file externally before the session is resumed. The agent resumes, unaware of the external edits, and attempts a line-based replacement that corrupts the file.
-  - _Mitigation:_ `SessionResumer` explicitly logs `mtime` file stats. Upon resumption, it flags recently modified workspace files and injects a "Wakeup Prompt" forcing the LLM to diff or re-read the file before acting.
-- **Sandbox Ephemerality (The Amnesia Problem):**
-  - _The Edge Case:_ A session running a background Express server in a cloud sandbox on Friday is resumed on Monday. The cloud provider killed the idle VM. The new VM lacks the running server, but the LLM’s context history believes it is still running.
-  - _Mitigation:_ Sandboxes are treated strictly statelessly. Upon string resumption, the agent is injected with a system message that the sandbox was recycled and it must manually restart required daemons/dev-servers.
-- **"Mid-Breath" Interruption State (Corrupt Serialization):**
-  - _The Edge Case:_ A forced exit (`SIGINT`/Power Loss) occurs exactly while the agent stream is halfway through emitting a JSON tool call chunk, serializing a broken `AIMessage` into history.
-  - _Mitigation:_ The `SessionStore` must only trigger a `saveSession()` at strict execution boundaries (e.g. after a complete LLM generation cycle or successfully parsed CLI execution), guaranteeing invalid mid-stream JSON chunks never touch the disk.
-- **Context Overflow (The Infinite Chat Log):**
-  - _The Edge Case:_ A persistent session spanning weeks scales the context past 200k tokens, hitting API limits and exponentially inflating the per-turn token costs.
-  - _Mitigation:_ Compaction is forced _before_ disk serialization. The session stringizes and compresses turns older than $N$ iterations into a dense system summary block before writing to `.jsonl`.
-- **Provider/Model Switching Mid-Task:**
-  - _The Edge Case:_ Starting a complex reasoning loop with Opus, pausing, and resuming with a lightweight local model like Llama 3 8B. The history is filled with complex schema usages that confused the smaller model.
-  - _Mitigation:_ Serialize the `.jsonl` lines with `provider/model` metadata blocks. Upon resumption, the CLI explicitly warns if a provider downgrade is detected.
-
-## 5. Error Recovery & Retry Edge Cases
-
-- **Transient LLM API Failure (429/5xx):**
-  - _The Edge Case:_ The LLM provider returns a rate-limit (429) or server error (500/502/503) mid-turn, crashing the entire session.
-  - _Mitigation:_ `retryWithBackoff()` wraps all LLM calls with exponential backoff (1s→2s→4s + jitter). Only `JooneError` instances with `retryable === true` trigger retries; auth failures (401/403) propagate immediately.
-- **Exhausted Retries (Self-Recovery):**
-  - _The Edge Case:_ After 3 retry attempts, the LLM API is still down. The session crashes and the user loses all progress.
-  - _Mitigation:_ Instead of crashing, `ExecutionHarness` injects the error's `toRecoveryHint()` as a `SystemMessage` into the conversation, returning a synthetic `AIMessage`. The agent can observe the error context and adapt (e.g., wait, simplify, or ask the user).
-- **Unclassified Provider Errors:**
-  - _The Edge Case:_ A new LLM provider throws a non-standard error with no HTTP status code, bypassing the retry classification.
-  - _Mitigation:_ `wrapLLMError()` inspects `.status`, `.statusCode`, `.code`, and `.response.status` on raw errors, covering the common patterns of LangChain, Axios, and native `fetch` errors.
-
-## 6. Human-in-the-Loop Edge Cases
-
-- **Permission Timeout (User Away):**
-  - _The Edge Case:_ The agent calls a dangerous tool (`bash`, `write_file`) while the user is away from the terminal. The agent blocks indefinitely waiting for permission.
-  - _Mitigation:_ `HITLBridge.requestPermission()` has a configurable timeout (default 5 minutes) that auto-denies and returns a short-circuit string, letting the agent try an alternative.
-- **Ask Question Timeout:**
-  - _The Edge Case:_ The agent asks the user a clarifying question via `ask_user_question`, but the user doesn't respond.
-  - _Mitigation:_ `HITLBridge.askUser()` resolves with `"[No response]"` after timeout, so the agent can proceed with a default assumption.
-- **Permission Mode Misconfiguration:**
-  - _The Edge Case:_ The user sets `"permissionMode": "ask_all"` and then every tool call — including harmless reads — triggers a prompt, making the agent unusable.
-  - _Mitigation:_ `PermissionMiddleware` maintains a hardcoded `SAFE_TOOLS` whitelist (`read_file`, `search_skills`, `ask_user_question`, etc.) that bypasses approval even in `ask_all` mode.
-
-## 7. Skills Sync Edge Cases
-
-- **Missing User Skills Directory:**
-  - _The Edge Case:_ `~/.joone/skills/` doesn't exist on the user's machine. The sync crashes trying to walk a nonexistent path.
-  - _Mitigation:_ `syncSkillsToSandbox()` checks `fs.existsSync()` before walking each skill directory and silently skips missing paths.
-- **Skill Name Collision (Project vs. User):**
-  - _The Edge Case:_ A user-level skill and a project-level skill have the same name. Both get synced to the sandbox, creating confusion.
-  - _Mitigation:_ `SkillLoader.discoverSkills()` deduplicates by name with project-level priority. `syncSkillsToSandbox()` only uploads `source: "user"` skills since project-level skills are already inside `projectRoot`.
-
-## 8. Slash Command Edge Cases (M11)
-
-- **Command Typos & Frustration:**
-  - _The Edge Case:_ User types `/modle` instead of `/model` and the agent treats it as a prompt, wasting LLM tokens and failing to switch the model.
-  - _Mitigation:_ Levenshtein distance check in `CommandRegistry`. If an unknown command is `< 3` edits away from a known command, the TUI intercepts it and suggests the correct command without calling the LLM.
-- **State Mutation While Processing:**
-  - _The Edge Case:_ User runs `/exit` or `/clear` while the agent is midway through generating a sequence of ToolCalls.
-  - _Mitigation:_ App-level UI blocks input while `isProcessing === true`. The commands are disabled.
-- **Model Switch to Non-Existent Model:**
-  - _The Edge Case:_ User runs `/model nonexistent`.
-  - _Mitigation:_ The command validates the model string against `ConfigManager`'s available models and securely rejects it before updating internal state.
-
-## 9. LLM-Powered Compaction Edge Cases (M12)
-
-- **Compaction Data Loss (Amnesia 2.0):**
-  - _The Edge Case:_ The LLM summarizes a 50-turn conversation but drops explicit file paths or tool choices, leaving the main agent blind when resuming.
-  - _Mitigation:_ The built-in Compact Prompt explicitly mandates a structured format: `Files Modified`, `Decisions Made`, `Tools Used`. A handoff prompt (`[CONTEXT HANDOFF]`) is injected into the bottom of the history to glue the summary back to the agent's persona.
-- **Double Compaction Fidelity Loss:**
-  - _The Edge Case:_ A session exists so long it must be compacted twice. A "summary of a summary" loses critical resolution.
-  - _Mitigation:_ `ConversationCompactor` detects prior summaries and includes them entirely in the eviction block, prompting the LLM to unify the old summary with the new evicted messages.
-
-## 10. Sub-Agent Orchestration Edge Cases (M13)
-
-- **The Sub-Agent Recursion Bomb:**
-  - _The Edge Case:_ A sub-agent uses the `spawn_agent` tool to spawn another sub-agent, creating an infinite nesting loop.
-  - _Mitigation:_ Hardcoded Depth-1 limit. Pre-configured sub-agents in `AgentRegistry` never include `spawn_agent` or `check_agent` in their allowed toolsets.
-- **Async Resource Contention:**
-  - _The Edge Case:_ The main agent loops over a directory and spawns 50 async `test_runner` agents concurrently.
-  - _Mitigation:_ `SubAgentManager` maintains a hard cap of 3 concurrent async tasks. Further spawn requests are queued or rejected with a backpressure error tool response.
-- **Stale Files in Sandbox:**
-  - _The Edge Case:_ The main agent edits a file on the host, then immediately spawns a `bash` sub-agent. The sub-agent runs in the sandbox before the new host file is synced.
-  - _Mitigation:_ The `SubAgentManager` shares the main harness's `FileSync` instance and always forces a `syncToSandbox()` pass _before_ the sub-agent takes its first step.
-
-## 11. Stability & Reliability Edge Cases (M14)
-
-- **Context Window Overflows (Instant Death):**
-  - _The Edge Case:_ Despite compaction thresholds, a single `read_file` returns 120k tokens string, instantly blowing past the 100% capacity mark. Compaction fails because the context is already overflowing.
-  - _Mitigation:_ `ContextGuard` has a 95% "Emergency Truncation" threshold. Before hitting the API, if tokens > 95%, it _bypasses_ LLM compaction and brutally slices all but the last 4 messages, inserting a loud warning message directly into the stream, guaranteeing survival.
-- **Process Death Serialization Tearing:**
-  - _The Edge Case:_ The `AutoSave` triggers at the exact millisecond the user presses `Ctrl+C`. The Node process terminates while `fs.writeFileSync` is mid-chunk, corrupting the JSONL session file irreversibly.
-  - _Mitigation:_ Atomic saves. `SessionStore.saveSession()` writes to an intermediate staging stream. On `process.on('SIGINT')`, a synchronous `forceSave()` is fired to cleanly flush state _before_ `process.exit(0)`.

package/docs/03_initial_implementation_plan.md

@@ -1,66 +0,0 @@
-# Initial Implementation Plan
-
-## Phase 1: Context Engine & Caching Layer
-
-Build a structured Prompt Builder that strictly enforces the Prefix Matching patterns so every task in a session enjoys a >90% cache hit rate.
-
-```mermaid
-graph TD
-    A[Base System Prompt] -->|Static Prefix| B
-    B[Tool Schemas] -->|Static Prefix| C
-    C[Project Memory e.g., README] -->|Project Prefix| D
-    D[Session Context e.g., OS Info] -->|Session Prefix| E
-    E[Conversation History] -->|Dynamic Appends| F[New User/Tool Message]
-
-    style A fill:#1e4620,stroke:#2b662e,color:#fff
-    style B fill:#1e4620,stroke:#2b662e,color:#fff
-    style C fill:#1e4620,stroke:#2b662e,color:#fff
-    style D fill:#2b465e,stroke:#3b6282,color:#fff
-    style E fill:#4a3219,stroke:#664422,color:#fff
-
-    subgraph Fully Cached Prefix
-    A
-    B
-    C
-    end
-```
-
-## Phase 2: Interoperable Tooling & Lazy Loading
-
-Implement tools as immutable objects for the session. Implement "Plan Mode" to alter agent rules without unloading tool schemas.
-
-- Define core tools: `read_file`, `write_file`, `bash_command`.
-- Implement dummy/stub tools for complex integrations.
-- Implement "Cache-Safe Forking" for compaction.
-
-## Phase 3: The Middleware Harness
-
-Implement pre-completion checks and loop detection via a middleware pipeline.
-
-```mermaid
-sequenceDiagram
-    participant Agent as LLM Agent
-    participant Harness as Execution Harness
-    participant Middle as Middleware Pipeline
-    participant Env as Environment (Bash/FS)
-
-    Agent->>Harness: Request: Edit target_file.py
-    Harness->>Middle: Emit: 'pre_tool_call'
-    Middle-->>Harness: Check LoopDetection (Fail if > 4 tries)
-    Harness->>Env: Execute Edit
-    Env-->>Harness: Return File Diff
-    Harness->>Agent: Send Tool Result
-
-    Agent->>Harness: Request: Submit/Exit
-    Harness->>Middle: Emit: 'pre_submit'
-    Middle->>Harness: Inject 'PreCompletionChecklist' (Wait, did you run tests?)
-    Harness->>Agent: System Reminder: "Please run tests to verify."
-    Agent->>Harness: Request: Run `pytest`
-```
-
-## Phase 4: Tracing & Feedback Loop
-
-Build an automated pipeline that sends JSON traces of failed agent runs into an evaluation database.
-
-- Hook LLM API calls to save traces.
-- Implement `TraceAnalyzer` subagent to review failures.

package/docs/04_tech_stack_proposal.md

@@ -1,20 +0,0 @@
-# Tech Stack
-
-The technology stack has been finalized. We are moving forward with a combination of the strong typing of TypeScript and the robust AI orchestration ecosystem of LangChain.
-
-## The Final Stack
-
-- **Language:** TypeScript (Node.js)
-  - Provides end-to-end type safety, especially crucial for tool schemas (Zod) and avoiding runtime errors in the execution loop.
-- **Orchestration / LLM Framework:** LangChain.js / LangGraph.js
-  - Using the TypeScript SDK for LangChain allows us to build complex, cyclical agent workflows (like Middlewares and self-correction loops) via LangGraph.
-- **Typing / Tool Schemas:** Zod
-  - Seamless integration with LangChain for structural output parsing and strict tool definition.
-- **Tracing:** LangSmith
-  - First-party integration with LangChain, providing deep visibility into token usage, prompt construction, and latency. Essential for debugging cache hit rates.
-- **CLI Framework (Optional):** Commander.js / Ink
-  - To be used if we build a robust terminal interface for the agent.
-
-## Why this combination?
-
-This marries the best of both originally proposed worlds. It gives us the frontend/backend interoperability and strict compile-time checks of TypeScript, while retaining the mature, graph-based agent orchestration and high-fidelity trace analysis typically dominated by Python's LangChain ecosystem.

package/docs/05_prd.md

@@ -1,87 +0,0 @@
-# Product Requirements Document (PRD)
-
-## 1. Product Overview
-
-**Joone** is a CLI-based autonomous coding agent that leverages **Prompt Caching** and **Harness Engineering** to achieve high autonomy and robustness in complex coding tasks while minimizing token cost and latency. It executes all generated code inside **E2B sandboxed microVMs**, isolating the host machine from any destructive operations.
-
-## 2. Target Audience
-
-- Software Engineers looking for an autonomous pair-programmer.
-- DevOps engineers looking to automate script fixes and verifications.
-- AI Researchers running benchmarks (e.g., Terminal Bench 2.0).
-
-## 3. Core Features
-
-### 3.1. CLI Interface & Provider Selection
-
-- **Installable CLI**: Packaged as an npm global binary (`npx joone` or `npm i -g joone`).
-- **Provider/Model Selection**: On first run (or via `joone config`), the user interactively selects their LLM provider and model. Stored at `~/.joone/config.json`.
-- **Supported Providers**: Anthropic, OpenAI, Google, Mistral, Groq, DeepSeek, Fireworks, Together AI, Ollama (local).
-- **Dynamic Provider Loading**: Provider packages are loaded on demand. If a package isn't installed, the CLI prints a helpful install command.
-- **Streaming Output**: Token-by-token streaming enabled by default for all providers. Tool calls are buffered until complete before execution.
-
-### 3.2. API Key Security (Tiered)
-
-- **Tier 1 (Default)**: API keys stored in `~/.joone/config.json` with restrictive file permissions (`chmod 600`). Masked input during `joone config`.
-- **Tier 2 (Planned)**: OS Keychain integration (Windows Credential Manager / macOS Keychain) via `keytar`.
-- **Tier 3 (Planned)**: AES-256 encrypted config file with machine-derived key.
-- During onboarding, the user will eventually be able to choose their preferred security tier.
-
-### 3.3. Cache-Optimized Context Engine
-
-- **Strict Prefix Ordering**: Separates static system instructions, tool definitions, project memory, and conversation history to align with LLM `cache_control` behaviors.
-- **`<system-reminder>` Injection**: Updates agent state natively via standard messages rather than system prompt overwrites, preserving the cache.
-- **Cache-Safe Compaction**: Forks and summarizes contexts seamlessly without full cache eviction.
-
-### 3.4. Hybrid Sandbox Execution
-
-- **Architecture**: The agent uses a **Hybrid** model — file operations (`write_file`, `read_file`) run on the **host machine** so the user sees changes live in their IDE, while all **code execution** (`bash`, tests, scripts) runs inside an [E2B](https://e2b.dev) cloud microVM sandbox.
-- **File Sync Mechanism**: Before each sandbox execution, changed files are synced from host → sandbox.
-  - _Tracking:_ Modifications are tracked via a "dirty paths" memory array. The `write_file` host tool explicitly marks paths as dirty upon successful write.
-  - _Concurrency:_ Concurrent modifications are prevented by the `ExecutionHarness`, which executes agent tool calls sequentially and blocks the LLM loop until file I/O and sandbox syncs are fully complete.
-  - _Conflict Resolution:_ The Host machine is the absolute source of truth. The sandbox filesystem is ephemeral and overwritten by the Host's dirty files before any command runs. Modifications made manually in the sandbox bypass the host and are lost upon destroy.
-- **Security**: The host machine is never exposed to agent-executed commands. Only file read/write touches the host.
-
-### 3.5. Middleware Harness
-
-- **Loop Detection (Anti-Doom Loop)**: Tracks agent action duplication and injects corrective context to break the loop.
-- **Pre-Completion Checklist**: Intercepts task submission to force a self-verification/testing phase.
-- **Guardrails for Scale**: Prevents loading oversized files (>1MB) entirely into memory; enforces chunked reads.
-
-### 3.6. Lazy & Interoperable Tooling
-
-- **Immutable Tool Definition**: Prevents mid-session tool swapping to preserve cache.
-- **Tool Search**: Implements "stub" tools, allowing dynamic loading of complex tools only when actively requested.
-
-### 3.7. Trace Analytics (V2)
-
-- Logs reasoning loops and tool execution traces to analyze points of failure.
-- Trace analyzer sub-agent that periodically reviews failures to suggest harness improvements.
-
-## 4. Non-Functional Requirements
-
-- **Latency:** High cache hit rates (>80% for long sessions) leading to sub-second Time-To-First-Token.
-- **Cost:** Minimize redundant prefix token generation.
-- **Extensibility:** Middleware pipeline should make it trivial to add new guardrails.
-- **Development Process:** Strict Red-Green-Refactor TDD for all new features.
-
-### 4.1 Error Handling & Degraded Modes
-
-- **Sandbox Failures:** The `SandboxManager` is architected with an `ISandboxWrapper` interface. If the primary cloud **E2B** sandbox fails to initialize (e.g., due to network drops, API outages, or invalid keys), the manager will automatically print a warning and gracefully degrade to a local **OpenSandbox** deployment (`localhost:8080`) to ensure execution continuity.
-- **LLM Failures (Planned):** If the remote LLM provider API goes down, the Model Factory should seamlessly fallback to a local Ollama instance if available.
-
-### 4.2 Rate Limiting & Cost Controls (Planned)
-
-- **Session Budgets:** Users can configure a maximum token budget (e.g., $5.00) per session. The `ExecutionHarness` tracks usage via the `SessionTracer` and will forcefully halt the agent if the threshold is reached.
-- **Loop Circuit Breakers:** The `LoopDetectionMiddleware` acts as a behavioral rate limit, preventing the agent from infinitely burning tokens on failed bash commands.
-
-### 4.3 Sandbox Authentication Management
-
-- **E2B:** Authentication is managed via the `E2B_API_KEY` environment variable or securely stored in `~/.joone/config.json`.
-- **OpenSandbox (Fallback):** Handled via `OPENSANDBOX_API_KEY` and `OPENSANDBOX_DOMAIN` properties in the config, falling back to a local Docker `localhost:8080` endpoint.
-
-### 4.4 Telemetry, Privacy & Data Retention
-
-- **Local-First Tracing:** All session reasoning and tool execution traces are logged to `~/.joone/traces/` as local JSON files for offline analysis using `TraceAnalyzer`.
-- **Data Retention:** Local traces are automatically rotated/deleted after 30 days to prevent infinite disk bloat.
-- **Privacy:** Project source code is _never_ sent to third-party telemetry providers unless the user intentionally enables the optional `LANGSMITH_API_KEY` for advanced dashboard debugging.

package/docs/06_user_stories.md

@@ -1,72 +0,0 @@
-# User Stories
-
-This document contains the foundational user stories for the Joone agent, organized by Epic. It does not include exhaustive acceptance criteria, but rather serves as a high-level requirements tracker for the core features.
-
-## Epic 1: CLI & Configuration
-
-- **US 1.1**: As a user, I want to install joone globally via `npm i -g joone` and run it with `joone` in any project directory.
-- **US 1.2**: As a user, I want to select my preferred LLM provider and model on first run or via `joone config`, choosing from at least 9 providers (Anthropic, OpenAI, Google, Mistral, Groq, DeepSeek, Fireworks, Together AI, Ollama).
-- **US 1.3**: As a user, I want my API key collected via masked interactive input during `joone config`, so I never have to manually create `.env` files.
-- **US 1.4**: As a user, I want my preferences stored at `~/.joone/config.json` with restrictive file permissions, so I don't re-enter them every session.
-- **US 1.5**: As a user, I want the CLI to tell me which provider package to install if it's missing (e.g., `Run: npm install @langchain/groq`).
-- **US 1.6** _(Planned)_: As a security-conscious user, I want to choose during onboarding whether to store my API key in a plain config file, OS Keychain, or encrypted config.
-
-## Epic 2: Streaming & Output
-
-- **US 2.1**: As a user, I want to see the agent's response stream token-by-token in my terminal, not wait for the entire response to finish.
-- **US 2.2**: As the system, I want to buffer tool call JSON during streaming until the full call is received, then execute it.
-- **US 2.3**: As a user, I want the option to disable streaming via `joone config` or a CLI flag (`--no-stream`).
-
-## Epic 3: The Context & Prompt Layer
-
-- **US 3.1**: As a developer, I want the system prompt to be strictly divided into static and dynamic sections, so that I maximize prompt caching and reduce costs.
-- **US 3.2**: As the system, I need to inject state updates (like time or file changes) into the conversation history as simulated messages (`<system-reminder>`), so I avoid invalidating the static prefix cache.
-- **US 3.3**: As the system, when the context window reaches 90% capacity, I want to execute a cache-safe compaction that summarizes early history while keeping the system prompt matching the parent thread.
-
-## Epic 4: Hybrid Sandbox Execution
-
-- **US 4.1**: As a user, I want `write_file` and `read_file` to operate on my host filesystem, so I can see the agent's code changes in my IDE in real-time.
-- **US 4.4**: As the system, I want to create a new E2B sandbox at the start of each agent session and destroy it when the session ends or times out, so that each session has a clean isolated environment and resources are properly released.
-- **US 4.5**: As a developer, I want the tool router to automatically determine whether a tool runs on the host or in the sandbox based on tool type.
-
-## Epic 5: Tooling & Lazy Loading
-
-- **US 5.1**: As an agent, I want access to core tools (`read_file`, `write_file`, `run_bash_command`) defined statically at the beginning of the session.
-- **US 5.2**: As an agent, I want to use a "Search Tools" endpoint to learn about complex or specific tools, rather than having all 50+ tool schemas loaded simultaneously into my context window.
-- **US 5.3**: As a developer, I want guardrails on `read_file` so the agent cannot accidentally load a 10MB file into the context window and blind itself.
-
-## Epic 6: Middleware Guards & Execution Loops
-
-- **US 6.1**: As a developer, I want a `LoopDetectionMiddleware` that counts how many consecutive times an agent has failed a specific action.
-- **US 6.2**: As an agent stuck in a loop, I want the system to interrupt me and tell me to reconsider my approach, so I don't waste tokens repeating a failure.
-- **US 6.3**: As an agent trying to finish a task, I want a `PreCompletionMiddleware` to ask me if I have run tests. If I haven't, it should block completion and ask me to run verifications.
-- **US 6.4**: As the system, I want to parse test exit codes; if a test fails (`exit 1`), I want to block the agent from declaring the task "Done" unless a max retry limit is reached.
-
-- **US 7.1**: As an operator, I want every agent decision, tool call, and token metric logged to a standard trace format so I can monitor cache hit rates.
-- **US 7.2**: As an operator, I want a script that can read failed traces and use an LLM to automatically summarize _why_ the agent failed tasks, allowing me to refine the harness.
-
-## Epic 8: TUI Slash Commands (M11)
-
-- **US 8.1**: As a user, I want to type `/help` or `/?` to see a list of all available commands without making an LLM call.
-- **US 8.2**: As a user, I want to switch models mid-session securely by typing `/model <name>`.
-- **US 8.3**: As a user with a bloated history context, I want to type `/compact` to manually force a context summarization.
-- **US 8.4**: As an error-prone user, if I type `/cls` instead of `/clear`, I want the UI to suggest `/clear` via Levenshtein distance grouping instead of sending garbage tokens to the API.
-
-## Epic 9: LLM-Powered Compaction (M12)
-
-- **US 9.1**: As an agent managing a huge conversation history, I want to delegate summarization of my older messages to an LLM, so the resulting summary is precise, preserving file paths and tool outcomes perfectly.
-- **US 9.2**: As the system, I want to automatically select a cheaper, faster LLM model (like `gpt-4o-mini` instead of `gpt-4o`) to perform the background compaction, saving the user money.
-- **US 9.3**: As a resumed agent, I want a seamless Handoff Prompt injected directly beneath the compaction summary, so I instantly understand my persona and context haven't broken.
-
-## Epic 10: Sub-Agent Orchestration (M13)
-
-- **US 10.1**: As the main reasoning agent, I want the ability to spawn named "sub-agents" to handle specialized tasks (e.g., executing scripts, analyzing directories) so I don't clutter my own context overhead.
-- **US 10.2**: As the main agent, I want to spawn certain sub-agents asynchronously, allowing me to continue reasoning or writing files while the sub-agent scans tests in the background.
-- **US 10.3**: As an orchestrator, I want hard limitations (a Depth-1 limit) that strictly prevent a sub-agent from accidentally spawning another sub-agent ad infinitum.
-
-## Epic 11: Stability & Reliability (M14)
-
-- **US 11.1**: As the core engine, I want a proactive `ContextGuard` that estimates API token payloads before sending the request to the provider, automatically triggering compaction at 80% usage.
-- **US 11.2**: As the core engine, I want an absolute Emergency Truncation trap door at 95% capacity to prevent immediate process death when compaction isn't fast enough.
-- **US 11.3**: As a user working on a long-running complex task, I want the `AutoSave` feature to quietly save my `.jsonl` session file atomically in the background every few turns.
-- **US 11.4**: As a user, when I hit `Ctrl+C` in my terminal, I want the CLI to intercept the shutdown signal, force a final instantaneous save, and clean up the sandbox before exiting.

package/docs/07_system_architecture.md

@@ -1,138 +0,0 @@
-# System Architecture
-
-## High-Level Architecture Overview
-
-The system operates as a CLI-based REPL (Read-Eval-Print Loop) Agent Wrapper. The user runs `joone` in their project directory. The LLM is nested within an "Execution Harness" that mediates all inputs, actions, and memory. Responses are **streamed** token-by-token.
-
-### Hybrid Sandbox Model
-
-Joone uses a **Hybrid** architecture for safety and developer experience:
-
-- **File operations** (`write_file`, `read_file`) run on the **host machine**, so the user sees changes in real-time in their IDE.
-- **Code execution** (`bash`, `npm test`, scripts) runs inside an **E2B sandboxed microVM**, protecting the host from destructive commands.
-- A **File Sync** layer mirrors changed files from host → sandbox before each execution.
-
-```
-┌─────────────────────────┐          ┌──────────────────────────┐
-│     HOST MACHINE        │   sync   │      E2B SANDBOX         │
-│                         │ ───────► │                          │
-│  write_file ──► disk    │          │  /workspace/ (mirror)    │
-│  read_file  ◄── disk    │          │                          │
-│                         │          │  bash, npm test, scripts │
-│  User sees changes      │          │  run here (isolated)     │
-│  live in their IDE      │          │                          │
-└─────────────────────────┘          └──────────────────────────┘
-```
-
-## System Diagram
-
-```mermaid
-graph TD
-    Client["User CLI (joone)"] -->|Task Input| Config
-    Config["Config Manager (~/.joone/config.json)"] -->|Provider + Key| Factory
-    Factory[Model Factory] -->|BaseChatModel| MainLoop
-
-    subgraph Agent Execution Harness
-        MainLoop[Execution Engine]
-        State[Conversation State Manager]
-        PromptBuilder[Cache-Oriented Prompt Builder]
-        StreamHandler[Stream Handler]
-
-        State --> PromptBuilder
-        MainLoop --> PromptBuilder
-        PromptBuilder --> LLM((LLM API))
-        LLM -->|Streamed Chunks| StreamHandler
-        StreamHandler -->|Complete Tool Call| Middlewares
-        StreamHandler -->|Text Tokens| Terminal[Terminal Output]
-    end
-
-    subgraph Middleware Pipeline
-        Middlewares{Middleware Orchestrator}
-        LoopDet[Loop Detection]
-        PreComp[Pre-Completion Check]
-        Guard[File Size Guardrails]
-
-        Middlewares --> LoopDet
-        Middlewares --> PreComp
-        Middlewares --> Guard
-    end
-
-    subgraph "Tool Routing (Hybrid)"
-        Middlewares -->|Approved Tool Call| Router{Tool Router}
-        Router -->|"write_file, read_file"| HostFS["Host Filesystem (Node.js fs)"]
-        Router -->|"bash, test, install"| Sync[File Sync Layer]
-        Sync -->|Upload changed files| Sandbox["E2B MicroVM (Ubuntu)"]
-        Sandbox -->|stdout/stderr| MainLoop
-        HostFS -->|File content| MainLoop
-    end
-```
-
-## Component Breakdown
-
-1. **CLI & Config Layer** (`src/cli/`):
-   - `index.ts`: Parses user commands (`joone`, `joone config`) via Commander.js.
-   - `config.ts`: Reads/writes `~/.joone/config.json`. Stores provider, model, API key (plain text + `chmod 600`), streaming preference, and temperature.
-   - `modelFactory.ts`: Factory that dynamically imports the correct LangChain provider package and returns a `BaseChatModel`. Supports 9+ providers.
-
-2. **State Manager & Prompt Builder** (`src/core/promptBuilder.ts`):
-   - Maintains the "Prefix Match". Compiles the static system prompt, appends project variables once, and exclusively appends subsequent messages.
-
-3. **Execution Engine** (`src/core/agentLoop.ts`):
-   - Polls the LLM via `.stream()` (default) or `.invoke()`.
-   - The **Stream Handler** prints text tokens to stdout in real-time and buffers tool call JSON chunks until complete.
-   - Routes completed tool calls to the Middleware pipeline.
-
-4. **Middleware Orchestrator** (`src/middleware/`):
-   - Implements the Observer pattern over the `on_tool_call` and `on_submit` events.
-   - Operates on a structured `ToolResult` interface (`{ content, metadata, isError }`) to robustly pass execution metadata (like process exit codes) through the pipeline without brittle string parsing.
-   - Can _intercept_ or _modify_ a tool request before it hits the tools.
-   - Can _inject_ `<system-reminder>` messages back to the Execution Engine.
-
-5. **Tool Router & Hybrid Execution**:
-   - **Host tools** (`write_file`, `read_file`): Execute directly on the host via Node.js `fs`. Changes appear instantly in the user's IDE.
-   - **Sandbox tools** (`bash`, `run_tests`, `install_deps`): Route through the File Sync layer → E2B sandbox.
-   - The split is determined by tool type, not configuration.
-
-6. **File Sync Layer** (`src/sandbox/sync.ts`):
-   - Tracks which files have changed on the host since the last sandbox sync.
-   - Before each sandbox execution, uploads only the changed files to the sandbox's `/workspace/` directory.
-   - Strategies: **upload-on-execute** (default) or **watch & mirror** (future).
-
-7. **E2B Sandbox** (`src/sandbox/`):
-   - Each agent session initializes an E2B cloud sandbox via the `e2b` TypeScript SDK.
-   - All bash commands and code execution run via `sandbox.commands.run()`.
-   - The sandbox is destroyed on session end or timeout.
-   - The host machine is **never** exposed to agent-executed commands.
-
-## Tool Routing Table
-
-| Tool                   | Runs On     | Why                                       |
-| ---------------------- | ----------- | ----------------------------------------- |
-| `write_file`           | **Host**    | User sees changes in IDE instantly        |
-| `read_file`            | **Host**    | Reads the real project files              |
-| `run_bash_command`     | **Sandbox** | Protects host from destructive commands   |
-| `run_tests`            | **Sandbox** | Tests may have side-effects               |
-| `install_dependencies` | **Sandbox** | npm install can execute arbitrary scripts |
-| `search_tools`         | **Host**    | Registry lookup, no execution             |
-
-## Supported LLM Providers
-
-| Provider       | Package                   | Dynamic Import             |
-| -------------- | ------------------------- | -------------------------- |
-| Anthropic      | `@langchain/anthropic`    | `ChatAnthropic`            |
-| OpenAI         | `@langchain/openai`       | `ChatOpenAI`               |
-| Google         | `@langchain/google-genai` | `ChatGoogleGenerativeAI`   |
-| Mistral        | `@langchain/mistralai`    | `ChatMistralAI`            |
-| Groq           | `@langchain/groq`         | `ChatGroq`                 |
-| DeepSeek       | OpenAI-compatible         | `ChatOpenAI` with base URL |
-| Fireworks      | `@langchain/community`    | `ChatFireworks`            |
-| Together AI    | `@langchain/community`    | `ChatTogetherAI`           |
-| Ollama (Local) | `@langchain/ollama`       | `ChatOllama`               |
-
-## Security Roadmap
-
-| Tier | Method                     | Status               |
-| ---- | -------------------------- | -------------------- |
-| 1    | Plain config + `chmod 600` | **Active (Default)** |
-| 2    | OS Keychain (`keytar`)     | Planned              |
-| 3    | AES-256 encrypted config   | Planned              |