@tachu/extensions 1.0.0-beta.1 → 1.0.0-rc.1

package/README.md

@@ -1,37 +1,79 @@
 # Tachu
 
-**Production-grade agentic engine — the Harness that turns any LLM into a reliable, observable, production-ready Agent.**
+**An agentic engine under active development — the *Harness* that aims to turn any LLM into a reliable, observable Agent.**
 
 [![npm version](https://img.shields.io/npm/v/@tachu/core?label=%40tachu%2Fcore)](https://www.npmjs.com/package/@tachu/core)
+[![status: rc](https://img.shields.io/badge/status-rc-blue)](#project-status)
 [![license: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue)](#license)
 [![bun](https://img.shields.io/badge/runtime-bun-orange)](https://bun.sh)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue)](https://www.typescriptlang.org)
 
+> **Project Status — Release Candidate.** The 9-phase pipeline, registry, prompt assembler, CLI, OpenAI / Anthropic / Qwen / Gemini adapters, MCP adapters, vector stores and observability emitters are wired up and tested. Phase 3 (Intent Analysis) is a real LLM call, Phase 5 routes complex tool-capable requests to the built-in `tool-use` loop, and Phase 8 runs deterministic validation rules with optional semantic judge. Runtime provider fallback and semantic judge production hardening remain post-rc work. Install via the `@rc` dist-tag.
+
 ---
 
 ## What is Tachu?
 
-Tachu is a **production-grade agentic engine** — not a toy demo, not a thin wrapper. It is the *Harness* in the equation **Agent = Model + Harness**: it provides the structural skeleton (protocol, lifecycle, safety, memory, orchestration) so that any LLM becomes a reliable, observable, production-capable Agent.
+Tachu aims to be an **agentic engine you can build a real product on** — not a toy demo, not a thin wrapper. It is the *Harness* in the equation **Agent = Model + Harness**: it provides the structural skeleton (protocol, lifecycle, safety, memory, orchestration) so that any LLM becomes a reliable, observable Agent.
 
-The engine is intentionally **domain-agnostic**: it knows nothing about your business logic, your users, or your domain vocabulary. Instead, it defines a small set of core abstractions (Rules, Skills, Tools, Agents) through which your business fills in all the intelligence. Tachu handles the hard parts — 9-phase execution pipeline, dual-plane semantic matching, context window management, token-precise prompt assembly, structured retry/fallback, cancellation propagation, and end-to-end observability.
+The engine is intentionally **domain-agnostic**: it knows nothing about your business logic, your users, or your domain vocabulary. Instead, it defines a small set of core abstractions (Rules, Skills, Tools, Agents) through which your business fills in all the intelligence. Tachu is designed to handle the hard parts — 9-phase execution pipeline, dual-plane semantic matching, context window management, token-precise prompt assembly, structured retry/fallback, cancellation propagation, and end-to-end observability.
 
-Tachu ships as a Bun-native TypeScript monorepo with three packages: the zero-dependency engine core (`@tachu/core`), a production-quality official extensions library (`@tachu/extensions`), and a fully-featured CLI program (`@tachu/cli`) that doubles as the reference implementation.
+Tachu ships as a Bun-native TypeScript monorepo with three published packages — the zero-dependency engine core (`@tachu/core`), an official extensions library (`@tachu/extensions`), and a fully-featured CLI program (`@tachu/cli`) that doubles as the reference implementation — plus `@tachu/host-defaults` for shared CLI/embedded host wiring, and an optional private sidecar package (`@tachu/web-fetch-server`) for remote browser-backed web tools.
+
+---
+
+## Project Status
+
+**Current release:** `1.0.0-rc.0` on the `rc` dist-tag.
+
+**Version terminology:** The product line is **Tachu v1**. Release candidates are stabilization builds for `1.0.0`, not a separate framework generation. HTTP paths like `/v1/extract` are API versions only. See [detailed-design § version glossary](docs/detailed-design.md#版本与发布术语必读).
+
+This is the **first release candidate**. The table below is a readability index only; runtime behavior, defaults, and edge cases are authoritative in the cited source files and tests.
+
+| Capability | Status | Notes |
+|-----------|--------|-------|
+| 9-phase pipeline skeleton (types, orchestrator, state machine, hooks) | ✅ Implemented | `packages/core/src/engine` |
+| Descriptor Registry (Rules / Skills / Tools / Agents) | ✅ Implemented | Markdown + YAML frontmatter loader, semantic indexing, startup validation |
+| Prompt assembler (tiktoken, KV-cache-friendly ordering) | ✅ Implemented | `packages/core/src/prompt` |
+| Task scheduler, DAG validator, turn/task retry bookkeeping | ✅ Implemented | `packages/core/src/engine/scheduler.ts`; **runtime provider fallback on LLM errors is not wired** (see [Providers guide](./docs/guides/providers-and-integrations.md)) |
+| Session / Memory / Runtime-state / Safety / Model-router / Provider / Observability / Hooks modules | ✅ Implemented | `packages/core/src/modules` |
+| OpenAI / Anthropic / Qwen / Mock Provider adapters | ✅ Implemented | CLI auto-wires via `@tachu/host-defaults`; streaming, function calling, tool schemas |
+| Gemini Provider adapter | ✅ Implemented (manual wiring) | `GeminiProviderAdapter` in `@tachu/extensions` with unit tests; **not** registered by default CLI / `buildProviderAdapter` — inject via `createEngine(..., { providers: [new GeminiProviderAdapter(...)] })` (see [Providers guide](./docs/guides/providers-and-integrations.md)) |
+| `apiKey` / `baseURL` / `organization` / `timeoutMs` configuration (env var / `tachu.config.ts` / CLI flags) | ✅ Implemented | Azure OpenAI / LiteLLM / OpenRouter / self-hosted gateways supported |
+| 22 built-in tools + Terminal / File / Web backends | ✅ Implemented | `packages/extensions/src/tools/index.ts` |
+| MCP stdio + SSE adapters | ✅ Implemented | `packages/extensions/src/mcp` |
+| `LocalFsVectorIndexAdapter` (file-backed) + `QdrantVectorIndexAdapter` (REST) | ✅ Implemented | |
+| OTel / JSONL emitters | ✅ Implemented | |
+| `tachu init` / `tachu run` / `tachu chat` CLI surface, streaming renderer, session persistence, Ctrl+C semantics | ✅ Implemented | |
+| **CLI terminal Markdown rendering** | ✅ **Implemented** | `marked` + `marked-terminal` + `cli-highlight` stack. Applied to the final assistant reply in `tachu chat` / `tachu run --output text` when stdout is a TTY; automatically disables under `NO_COLOR` / non-TTY / `--no-color`. Explicit control via `--markdown` / `--no-markdown` on `tachu run`. Dedicated `renderMarkdownToAnsi` wrapper (`packages/cli/src/renderer/markdown.ts`) with 12 unit tests in `markdown.test.ts`. |
+| **Phase 3 — Intent Analysis (LLM call, pure classification)** | ✅ **Implemented** | Pure classification only (`IntentResult`); final user reply is Phase 7 `direct-answer`. **Implementation:** `packages/core/src/engine/phases/intent.ts` (`INTENT_SYSTEM_PROMPT_BASE`, fast paths, JSON parse, heuristic fallback); tests: `intent.test.ts`. Hosts may **`config.intent.systemPromptBase`** to replace the base wholesale; optional extra few-shots: `config.intent.fewShotExamples` (Agent Context / explicit selections still appended by core). |
+| **Phase 5 — Task Planning (planning router)** | ✅ **Implemented** | Enforces `plans[0].tasks.length >= 1`. Rules: (1) `simple` intent → single `direct-answer` sub-flow task; (2) `complex` + visible tools → single `tool-use` sub-flow task; (3) `complex` + no visible tool → single `direct-answer` sub-flow task with `warn: true`; (4) defensive post-guard catches upstream regressions that leave `tasks` empty. Multi-step behavior lives inside `tool-use`; optional plan preview / human review may still evolve, but there is no separate default LLM pre-planner on the main path. |
+| **`direct-answer` built-in Sub-flow (Phase 7)** | ✅ **Implemented** | `packages/core/src/engine/subflows/direct-answer.ts`. Resolves `capabilityMapping.intent` (fallback to `fast-cheap`), composes system + ≤10 memory-history entries + user prompt, calls `ProviderAdapter.chat()` with a 60 s per-call timeout merged with the phase abort signal. System prompt mandates **natural-language Markdown**, forbids JSON wrappers / `"已识别请求：…"` templates / 4-space indented code blocks, and supports a `warn: true` flag for honest tool-missing disclaimers. Emits `llm_call_start` / `llm_call_end` observability events under `phase: "direct-answer"`. Non-overridable: `DescriptorRegistry` registers `direct-answer` as a reserved name and rejects business registration / unregistration with `RegistryError.reservedName`. |
+| **Phase 8 — Result Validation Outcome** | 🟡 **Partially wired** | `ValidationOutcome` union + `ValidationRuleRegistry` with **5 deterministic rules** via `buildDefaultValidationRuleRegistry()` (`packages/core/src/engine/phases/validation/index.ts`). Optional `ProviderSemanticJudgeAdapter` / `BudgetedSemanticJudgeAdapter`. Engine consumes `retry` (turn loop via `decideTurnRetry`), `degrade` / `handoff` (exit to Output). Gaps: no standalone `ExecutionPolicy` type; runtime provider fallback is not implemented and semantic judge is not production-complete. |
+| **Phase 9 — Output Assembly** | ✅ **Implemented** | Content selector: `taskResults['task-direct-answer']` → `{intent, taskResults}` structured JSON (tool-chain success path; semantic polish still depends on real Phase 8) → honest-fallback plain-language message with recognized intent + internal diagnosis + *"rephrase as simple"* suggestion (validation failed). Internal state JSON is never leaked to end users. Covered by `output.test.ts`. |
+| Real-world smoke tests against OpenAI / Anthropic / Azure | 🟡 **Manually verified; opt-in automated** | Mock unit tests cover adapters in CI. Maintainers have **hand-run** real LLM paths (custom gateways included). An **opt-in** scripted e2e exists — set `TACHU_REAL_E2E=1` plus `TACHU_E2E_API_KEY` / `TACHU_E2E_API_BASE` / `TACHU_E2E_PROVIDER` (see [Contributing](./CONTRIBUTING.md)) — but default CI does not publish signed recordings. |
+| Production hardening (SLO, error budgets, failure injection, signed provenance) | 🔴 Not yet | Target for `1.0.0` (Tachu v1). |
+
+Legend: ✅ implemented and tested · 🟡 stub / placeholder present, real implementation in progress · 🔴 not yet started.
 
 ---
 
 ## Key Features
 
-- **9-Phase Execution Pipeline** — session management → safety → intent analysis → pre-check → planning → DAG validation → execution → result validation → output normalization; each phase is a typed, hookable stage
+- **9-Phase Execution Pipeline** — session management → safety → intent analysis (pure classification) → pre-check → planning router → DAG validation → execution → result validation outcome → output normalization; each phase is a typed, hookable stage, and every request — simple or complex — flows through all nine phases with uniform Rules / Hooks / Observability / budget accounting
+- **Task Planning + Tool-use Loop** — Phase 5 does not precompute a ranked multi-step plan. It routes `simple` requests to `direct-answer`, routes `complex + visible tools` into the built-in `tool-use` Agentic Loop, and lets that loop perform iterative LLM tool selection → gated tool execution → tool-result feedback → final answer.
+- **`direct-answer` built-in Sub-flow** — answers to simple requests (and to complex requests with no matching tool) are produced by a first-class engine-internal Sub-flow running in Phase 7, not baked into the intent prompt.
 - **Dual-Plane Matching** — semantic discovery (vector similarity) + deterministic execution gate (scopes, whitelist, approval) for every Rule, Skill, Tool, and Agent
 - **Four Core Abstractions** — declare Rules, Skills, Tools, and Agents as Markdown + YAML frontmatter descriptors; the engine resolves, activates, and orchestrates them automatically
-- **Dual Provider Support** — production-grade OpenAI and Anthropic adapters; configurable fallback order for zero-downtime provider switching
+- **OpenAI & Anthropic Adapters** — streaming, function calling, configurable `baseURL` / `organization` / `timeoutMs`; works with Azure OpenAI, LiteLLM, OpenRouter, or any self-hosted gateway
 - **MCP Integration** — connect any MCP server (stdio or SSE) via `McpToolAdapter`; MCP tools become first-class engine Tools
 - **Token-Precise Prompt Assembly** — tiktoken-based exact token counting; KV-cache-friendly prompt layout; automatic context compression (Head-Middle-Tail strategy)
 - **Structured Memory** — session context window with configurable limits; archive-before-summarize guarantee; vector recall for long-term memory
 - **OpenTelemetry Observability** — every phase entry/exit, LLM call, tool call, retry, and fallback emits a structured `EngineEvent`; OTel and JSONL emitters included
-- **Production-Grade CLI** — `tachu chat` / `tachu run` / `tachu init` with full parameter sets, streaming render, session persistence, and Ctrl+C cancellation
+- **Interactive CLI** — `tachu chat` / `tachu run` / `tachu init` with full parameter sets, streaming render, session persistence, and Ctrl+C cancellation
+- **Terminal Markdown rendering** — final assistant replies are rendered via `marked` + `marked-terminal` + `cli-highlight`; headings, bold / italic, lists, block quotes, links, tables and fenced code blocks (with syntax highlighting) are all supported. Automatically disabled under `NO_COLOR` / non-TTY / `--no-color`; explicitly controllable with `--markdown` / `--no-markdown` on `tachu run`.
 - **Fail-Closed Safety Baseline** — loop protection, budget circuit-breaker, and basic input validation are hardwired into the engine and cannot be disabled
-- **Qdrant & LocalFs Vector Stores** — plug in Qdrant for production-grade vector storage or use the built-in in-memory store for development
+- **Qdrant & LocalFs Vector Stores** — plug in Qdrant for multi-process deployments or use the file-backed store for local/single-process setups
 
 ---
 
@@ -52,119 +94,17 @@ Tachu is built around three convictions:
 
 ## Core Abstractions
 
-Tachu's four core abstractions are **co-equal and orthogonal** — each independently registered, independently activated, and composable across all engine phases.
-
-| Abstraction | Nature | Activation Gate | Effect |
-|-------------|--------|-----------------|--------|
-| **Rules** | Constraints & guidance | Semantic discovery → direct activation | Injected into LLM System Prompt at each scoped phase |
-| **Skills** | Knowledge & instructions | Semantic discovery → direct activation | Injected into LLM context when activated |
-| **Tools** | Atomic executable operations | Semantic discovery → **mandatory gate** (scopes → whitelist → approval) | Executed with full side-effect tracking |
-| **Agents** | Natural-language-driven execution units | Semantic discovery → activatable | Recursively use engine capabilities; all Tool calls pass through the Tool gate |
-
-All four share a **common descriptor schema** (Markdown + YAML frontmatter):
+Rules, Skills, Tools, and Agents are Markdown + YAML descriptors. Semantic discovery proposes candidates; deterministic gates authorize execution (especially Tools).
 
-```yaml
-name: unique-name             # required, unique within type
-description: ...              # natural language (used for semantic discovery)
-tags: [tag1, tag2]            # for filtering and classification
-trigger: { type: always }     # activation condition
-requires:
-  - { kind: tool, name: read-file }  # explicit dependency references
-```
-
-### Dual-Plane Matching Model
-
-Every core abstraction is activated through a two-phase process:
-
-```mermaid
-graph LR
-    Input[Context Input] --> Discovery[Semantic Discovery Plane]
-    Discovery --> Index[(Vector Index)]
-    Index --> Candidates[Candidate Set]
-    Candidates --> Gate[Deterministic Execution Gate]
-    Gate -- scopes / whitelist / approval --> Execution[Execution Plane]
-```
-
-- **Semantic discovery plane**: `description` is vectorized on registration; at runtime, the current context is matched against the index to produce a candidate set
-- **Deterministic execution gate**: final activation requires passing a deterministic gate (explicit references, whitelist checks, permission scopes, approval checks)
-
-Rules and Skills pass through the gate freely (no side effects). Tools always pass through the full gate. Agents activate freely but all Tool calls they trigger still pass through the Tool gate.
+Details: [Overview · abstractions](./docs/overview-design.md#三四大核心抽象) · [Dual-plane matching](./docs/overview-design.md#共享特性).
 
 ---
 
-## Architecture Overview
-
-### Three-Layer Structure
-
-Tachu is published as three layers:
-
-```mermaid
-graph TD
-    subgraph "Business Layer"
-        A[Business Rules / Domain Tools / Custom Adapters / Domain Skills / Agents / Plan Templates]
-    end
-    subgraph "Extensions Library — @tachu/extensions"
-        B[OpenAI & Anthropic Adapters / 7 Common Tools / Terminal+File+Web Backends / Qdrant+LocalFs VectorStore / MCP Adapter / OTel+JSONL Emitters / 4 Common Rules]
-    end
-    subgraph "Engine Core — @tachu/core"
-        C[Protocol Definitions / 9-Phase Pipeline / Lifecycle Hooks / Session / Memory / Safety / Model Router / Runtime State]
-    end
-    A --> B
-    B --> C
-```
+## Architecture (summary)
 
-| Layer | Package | Responsibility |
-|-------|---------|----------------|
-| Engine Core | `@tachu/core` | Protocol interfaces, 9-phase pipeline skeleton, 8 core modules, Registry, Prompt assembler, VectorStore interface + built-in light implementation |
-| Extensions Library | `@tachu/extensions` | Official concrete implementations: Provider adapters, Tools, Backends, VectorStore adapters, OTel/JSONL emitters, common Rules |
-| Business / CLI | `@tachu/cli` or your code | Assembles core + extensions into a working Agent; provides domain Rules/Skills/Tools/Agents |
-
-### 9-Phase Execution Pipeline
-
-Every request processed by the engine flows through exactly 9 phases:
-
-```mermaid
-graph TD
-    Start([Business Request]) --> S1[Phase 1: Session Management]
-    S1 --> S2[Phase 2: Minimum Safety Check]
-    S2 --> S3[Phase 3: Intent Analysis — LLM]
-    S3 -- simple --> S9[Phase 9: Output Normalization]
-    S3 -- complex --> S4[Phase 4: Pre-Check]
-    S4 --> S5[Phase 5: Task Planning]
-    S5 -- Plan mode --> PlanLoop{Plan Review Loop}
-    S5 -- Template match --> S6
-    S5 -- Dynamic split --> S6[Phase 6: DAG Validation]
-    PlanLoop -- confirmed --> S6
-    S6 --> S7[Phase 7: Sub-task Execution]
-    S7 --> S8[Phase 8: Result Validation — LLM]
-    S8 -- pass --> S9
-    S8 -- fail --> Retry{Retry / Re-plan}
-    Retry -- within limits --> S5
-    Retry -- exhausted --> S9
-    S9 --> End([Output])
-
-    style S2 fill:#ffeaa7,stroke:#fdcb6e
-    style S7 fill:#dfe6e9,stroke:#b2bec3
-```
+Requests flow through a **9-phase pipeline**. Complex tool work enters the built-in `tool-use` loop in Phase 7.
 
-| # | Phase | LLM Call | Key Output |
-|---|-------|----------|------------|
-| 1 | Session Management | No | Session context loaded |
-| 2 | Minimum Safety Check | No | Pass / reject |
-| 3 | Intent Analysis | **Yes** | `IntentResult` (simple/complex, context relevance) |
-| 4 | Pre-Check | No | Resource availability, deep safety validation |
-| 5 | Task Planning | **Yes** | `PlanningResult` (ranked plans + DAG) |
-| 6 | DAG Validation | No | Cycle detection, node integrity (deterministic) |
-| 7 | Sub-task Execution | Per sub-task | `TaskResult[]` (parallel where possible) |
-| 8 | Result Validation | **Yes** | `ValidationResult` (pass / execution_issue / planning_issue) |
-| 9 | Output Normalization | No | `EngineOutput` (typed, with steps, metadata, artifacts) |
-
-**Key pipeline properties:**
-
-- **Full-path safety gating** — Phase 2 runs on every request, including fast-path simple responses
-- **Context guard** — Phase 3 decides whether session history is relevant; irrelevant history is not forwarded
-- **Three-strikes limit** — Task-level retries are bounded at 3 (configurable); system-level retries at 2
-- **Last-message-wins** — A new request on the same session cancels the current execution via `AbortController`
+Details: [Pipeline phases](./docs/architecture/pipeline-phases.md) · [Overview design](./docs/overview-design.md).
 
 ---
 
@@ -172,21 +112,23 @@ graph TD
 
 Tachu requires [Bun](https://bun.sh) as the runtime.
 
+> **Install via the `@rc` dist-tag** (or an exact version) until Tachu reaches stable.
+
 ```bash
 # Install the engine core
-bun add @tachu/core
+bun add @tachu/core@rc
 
 # Install the extensions library (providers, tools, backends, vector stores)
-bun add @tachu/extensions
+bun add @tachu/extensions@rc
 
 # Install and use the CLI globally
-bun add -g @tachu/cli
+bun add -g @tachu/cli@rc
 ```
 
 After global installation, verify with:
 
 ```bash
-tachu --version
+tachu --version   # expect 1.0.0-rc.0 or newer
 ```
 
 ---
@@ -212,608 +154,41 @@ tachu chat
 tachu chat --resume
 ```
 
-### Programmatic (TypeScript)
-
-```typescript
-import { Engine } from '@tachu/core';
-import { OpenAIProviderAdapter } from '@tachu/extensions/providers';
-import { FileBackend, TerminalBackend } from '@tachu/extensions/backends';
-import type { EngineConfig, InputEnvelope, ExecutionContext } from '@tachu/core';
-
-const config: EngineConfig = {
-  retry: { taskMaxRetries: 3, systemMaxRetries: 2 },
-  planning: { planCount: 1, enableValidation: true },
-  agent: { maxNestingDepth: 1 },
-  context: { maxTokens: 8000, compressionThreshold: 0.8 },
-  execution: { defaultTimeout: 120_000 },
-  models: {
-    capabilityMapping: {
-      'high-reasoning': 'gpt-4o',
-      'fast-cheap': 'gpt-4o-mini',
-    },
-    providerFallbackOrder: ['openai'],
-  },
-  safety: { maxInputSize: 1_000_000, maxRecursionDepth: 10 },
-  hooks: { writeHookTimeout: 5000, failureBehavior: 'continue' },
-  storage: {},
-};
-
-const engine = new Engine(config);
-
-// Register a provider
-engine.useProvider(new OpenAIProviderAdapter({ apiKey: process.env.OPENAI_API_KEY! }));
-
-// Stream results
-const input: InputEnvelope = {
-  content: 'Write a TypeScript function that debounces async operations',
-  metadata: { modality: 'text' },
-};
-
-const context: ExecutionContext = {
-  requestId: crypto.randomUUID(),
-  sessionId: 'session-001',
-  traceId: crypto.randomUUID(),
-  principal: { userId: 'user-001' },
-  budget: { maxTokens: 20_000, maxDurationMs: 60_000 },
-  scopes: ['read', 'write'],
-};
-
-for await (const chunk of engine.runStream(input, context)) {
-  if (chunk.type === 'delta') process.stdout.write(chunk.content);
-  if (chunk.type === 'done') console.log('\n\nCompleted:', chunk.output.status);
-}
-```
-
----
-
-## Package Layout
-
-### Package Summary
-
-| Package | Description | Key Exports |
-|---------|-------------|-------------|
-| `@tachu/core` | Zero-dependency engine core | `Engine`, `Registry`, `PromptAssembler`, all interfaces and types |
-| `@tachu/extensions` | Official implementations | `OpenAIProviderAdapter`, `AnthropicProviderAdapter`, `McpToolAdapter`, `QdrantVectorStore`, `OtelEmitter`, backends, tools, rules |
-| `@tachu/cli` | Production CLI program | `tachu chat`, `tachu run`, `tachu init` |
-
-### Dependency Relationship
-
-```mermaid
-graph LR
-    cli["@tachu/cli"]
-    extensions["@tachu/extensions"]
-    core["@tachu/core"]
-
-    cli --> extensions
-    cli --> core
-    extensions --> core
-
-    style core fill:#74b9ff,stroke:#0984e3
-    style extensions fill:#a29bfe,stroke:#6c5ce7
-    style cli fill:#fd79a8,stroke:#e84393
-```
-
-### Core Package Internal Structure
-
-```
-@tachu/core / src/
-├── types/          # All TypeScript interfaces: descriptors, context, I/O, config
-├── engine/         # Engine entry class, phase handlers, orchestrator, scheduler
-├── registry/       # Registry: register/lookup/startup validation for all 4 abstractions
-├── modules/        # 8 core modules (session, memory, runtime-state, model-router,
-│                   #   provider, safety, observability, hooks)
-├── prompt/         # PromptAssembler: token budgeting, KV-cache-friendly ordering
-└── vector/         # VectorStore interface + built-in lightweight implementation
-```
-
----
-
-## Providers & Integrations
-
-### LLM Providers
-
-| Provider | Package | Streaming | Function Calling | Notes |
-|----------|---------|-----------|-----------------|-------|
-| OpenAI | `@tachu/extensions/providers` | ✅ | ✅ | GPT-4o, GPT-4o-mini, and all listable models |
-| Anthropic | `@tachu/extensions/providers` | ✅ | ✅ | Claude 3.5 Sonnet and all listable models |
-| Mock | `@tachu/extensions/providers` | ✅ | ✅ | For testing; configurable responses |
-
-Provider fallback is configured via `models.providerFallbackOrder`. When a system-level error occurs (timeout, API error), the engine automatically switches to the next provider in the list without re-planning.
-
-### MCP (Model Context Protocol)
-
-```typescript
-import { McpToolAdapter } from '@tachu/extensions/mcp';
-
-const adapter = new McpToolAdapter();
-
-// Connect via stdio (local process)
-await adapter.connect('stdio://path/to/mcp-server');
-
-// Connect via SSE (remote server)
-await adapter.connect('http://localhost:3000/sse');
-
-// MCP tools are automatically registered in the engine Registry
-const tools = await adapter.listTools(); // returns ToolDescriptor[]
-engine.registry.registerAll(tools);
-```
-
-The `McpToolAdapter` handles MCP session lifecycle, capability negotiation, and propagates engine cancellation signals to the MCP server.
-
-### Vector Stores
-
-| Adapter | Package | Use Case |
-|---------|---------|----------|
-| `InMemoryVectorStore` | `@tachu/core` | Development / testing; built-in, zero dependencies |
-| `LocalFsVectorStore` | `@tachu/extensions/vector` | Single-process production; file-backed persistence |
-| `QdrantVectorStore` | `@tachu/extensions/vector` | Multi-process production; full Qdrant REST API support |
-
-```typescript
-import { QdrantVectorStore } from '@tachu/extensions/vector';
-
-const vectorStore = new QdrantVectorStore({
-  url: 'http://localhost:6333',
-  collectionName: 'tachu-descriptors',
-});
-
-engine.useVectorStore(vectorStore);
-```
-
-### Observability Emitters
-
-| Emitter | Package | Output |
-|---------|---------|--------|
-| `OtelEmitter` | `@tachu/extensions/emitters` | OpenTelemetry spans via `@opentelemetry/api` |
-| `JsonlEmitter` | `@tachu/extensions/emitters` | Append-only JSONL file |
-| `ConsoleEmitter` | `@tachu/extensions/emitters` | Structured console output (development) |
-
-### Execution Backends
-
-| Backend | Package | Description |
-|---------|---------|-------------|
-| `TerminalBackend` | `@tachu/extensions/backends` | Shell command execution in a sandboxed terminal |
-| `FileBackend` | `@tachu/extensions/backends` | File system read/write operations |
-| `WebBackend` | `@tachu/extensions/backends` | HTTP requests to external APIs / web resources |
-
----
-
-## Design Principles
-
-Tachu is built on seven core engineering principles drawn from its architecture:
-
-1. **Dual-Plane Matching** — All four core abstractions are discovered semantically (vector similarity) but activated deterministically (scopes, whitelist, approval). Semantic discovery is advisory; execution gates are authoritative.
-
-2. **Full-Path Safety Gating** — The minimum safety check (Phase 2) runs on *every* request path, including the fast path for simple questions. Safety is never traded for performance.
-
-3. **Three-Strikes Retry Limit** — Both the task-level retry loop and the system-level retry loop are strictly bounded. Unlimited retry is not allowed. When limits are exhausted, the engine outputs step-level completion status rather than a generic failure.
-
-4. **KV-Cache-Friendly Prompt Assembly** — The System Prompt is assembled in a stable order (hard rules → soft preferences → skills → tool definitions) so that the prefix remains unchanged across turns, maximizing KV cache reuse and reducing LLM cost.
-
-5. **Last-Message-Wins Cancellation** — When a new message arrives in the same session, the current execution is cancelled via `AbortController` and the new input is processed in the existing context. This guarantees coherent context while avoiding stale work.
-
-6. **Engine Agnostic of Business Permissions** — The engine only evaluates coarse-grained `scopes` from the execution context at the Tool gate. Fine-grained business authorization is the responsibility of Tool implementations or dedicated authorization Tools.
-
-7. **Fail-Closed Safety Baseline** — Loop protection, budget circuit-breaking, and basic input validation are hardwired into the engine core and *cannot* be disabled by configuration. Even with a completely empty business configuration, the engine does not run unconstrained.
-
----
-
-## Configuration
-
-The engine is configured via a `tachu.config.ts` file at the project root (generated by `tachu init`):
-
-```typescript
-import type { EngineConfig } from '@tachu/core';
-
-const config: EngineConfig = {
-  // Descriptor registry
-  registry: {
-    descriptorRoot: '.tachu',          // root directory for Rules/Skills/Tools/Agents
-  },
-
-  // Runtime behaviour
-  runtime: {
-    planMode: false,                   // enable Plan mode by default
-    maxConcurrency: 4,                 // max parallel sub-tasks
-  },
-
-  // Context window & memory
-  memory: {
-    maxTokens: 8000,                   // context window token limit
-    compressionThreshold: 0.8,         // trigger compression at 80% capacity
-    archivePath: '.tachu/archive.jsonl',
-    vectorIndexLimit: 10000,           // max entries in the built-in vector index
-  },
-
-  // Budget constraints
-  budget: {
-    maxTokens: 50000,                  // total token budget per execution
-    maxToolCalls: 50,                  // max tool calls per execution
-    maxWallTimeMs: 300000,             // 5 minutes wall-time limit
-  },
-
-  // Safety baseline (hardwired minimum; add business policies via hooks)
-  safety: {
-    maxInputSize: 1_000_000,           // bytes
-    maxRecursionDepth: 10,
-    enablePromptInjectionCheck: true,
-  },
-
-  // Model routing
-  models: {
-    capabilityMapping: {
-      'high-reasoning': { provider: 'openai', model: 'gpt-4o' },
-      'fast-cheap':     { provider: 'openai', model: 'gpt-4o-mini' },
-      'vision':         { provider: 'openai', model: 'gpt-4o' },
-    },
-    providerFallbackOrder: ['openai', 'anthropic'],
-  },
-
-  // Observability
-  observability: {
-    emitter: 'jsonl',
-    jsonlPath: '.tachu/events.jsonl',
-  },
-};
-
-export default config;
-```
-
-All fields have sensible defaults. `tachu init` generates this file pre-filled for your chosen provider.
-
----
-
-## CLI Reference
-
-### `tachu init`
 
-Initialize a new Tachu project workspace.
-
-```
-tachu init [options]
-
-Options:
-  --template <name>    Scaffold template: minimal | full  (default: minimal)
-  --force              Overwrite existing files without prompting
-  --path <dir>         Target directory                   (default: CWD)
-  --provider <name>    Default provider: openai | anthropic | mock  (default: mock)
-  --no-examples        Skip generating example rule/tool descriptors
-  -h, --help           Show help
-```
-
-Generates `.tachu/` directory skeleton + `tachu.config.ts` + `.gitignore` entries.
+Programmatic embedding: see [Configuration](./docs/guides/configuration.md) and `@tachu/host-defaults`.
 
 ---
 
-### `tachu run <prompt>`
-
-Execute a single prompt and stream the result to stdout.
-
-```
-tachu run <prompt> [options]
-
-Arguments:
-  <prompt>             The prompt text (or pipe via stdin)
-
-Options:
-  --session <id>       Use a specific session ID
-  --resume             Resume the most recent session
-  --model <name>       Override the high-reasoning model
-  --provider <name>    Override the default provider
-  --input <file>       Read prompt from a file
-  --json               Parse prompt as JSON (structured input)
-  --output <fmt>       Output format: text | json | markdown  (default: text)
-  --no-validation      Skip Phase 8 result validation
-  --plan-mode          Enable Plan mode (pause after Phase 5 for approval)
-  --verbose, -v        Verbose output (phase transitions, LLM call details)
-  --no-color           Disable ANSI color output (also respects NO_COLOR env var)
-  --timeout <ms>       Wall-time limit (overrides budget.maxWallTimeMs)
-  -h, --help           Show help
-```
-
----
-
-### `tachu chat`
-
-Start an interactive multi-turn chat session.
-
-```
-tachu chat [options]
-
-Options:
-  --session <id>       Use a specific session ID
-  --resume             Resume the most recent session
-  --history            List all sessions and exit (no interactive prompt)
-  --export <file>      Export a session to Markdown and exit
-  --model <name>       Override the high-reasoning model
-  --provider <name>    Override the default provider
-  --plan-mode          Enable Plan mode
-  --verbose, -v        Verbose output
-  --no-color           Disable color output
-  -h, --help           Show help
-```
-
-**Built-in interactive commands** (prefix with `/`):
-
-| Command | Description |
-|---------|-------------|
-| `/exit` | Save session and quit |
-| `/reset` | Clear the current session's memory |
-| `/new` | Start a new session |
-| `/list` | List all saved sessions |
-| `/load <id>` | Switch to a specific session |
-| `/save` | Manually persist the current session |
-| `/export <path>` | Export session to a Markdown file |
-| `/history` | Show this session's message history |
-| `/stats` | Show token usage, tool calls, and remaining budget |
-| `/help` | Show all commands |
-
-**Ctrl+C behaviour:**
-- First press: cancel the current LLM/Tool call (return to prompt, session intact)
-- Second press within 1 second: save session and exit gracefully
-- Third press: force exit
-
----
-
-## Extension Guide
-
-Tachu is extended by creating Markdown descriptor files in the `.tachu/` directory. No code changes are required for Rules, Skills, and Tools — only Agents need executable functions registered separately.
-
-### Custom Rule
-
-```markdown
-<!-- .tachu/rules/no-external-calls.md -->
----
-name: no-external-calls
-description: Prevent the agent from making external network calls without explicit approval
-type: rule
-scope: [execution]
-tags: [security, network]
----
-
-Do not make HTTP requests, DNS lookups, or any other external network calls unless
-the tool being invoked has `requiresApproval: true` and the user has confirmed.
-```
-
-### Custom Skill
-
-```markdown
-<!-- .tachu/skills/git-workflow/SKILL.md -->
----
-name: git-workflow
-description: Git branching, commit, and PR workflow knowledge for this repository
-tags: [development, git]
-requires:
-  - { kind: tool, name: run-command }
----
-
-## Git Workflow
-
-This repository follows trunk-based development with short-lived feature branches.
-
-### Branch Naming
-- Feature: `feat/<ticket>-<short-desc>`
-- Fix: `fix/<ticket>-<short-desc>`
-
-### Commit Convention
-Use Conventional Commits: `type(scope): subject`
-...
-```
-
-### Custom Tool
-
-```markdown
-<!-- .tachu/tools/query-db.md -->
----
-name: query-db
-description: Execute a read-only SQL query against the application database
-sideEffect: readonly
-idempotent: true
-requiresApproval: false
-timeout: 10000
-inputSchema:
-  type: object
-  properties:
-    sql:   { type: string, description: "SQL SELECT statement" }
-    limit: { type: number, description: "Max rows to return", default: 100 }
-  required: [sql]
-execute: queryDatabase
----
-
-Executes a parameterized read-only SQL query. Results are returned as a JSON array.
-```
-
-Register the execution function in your `engine-factory.ts`:
-
-```typescript
-engine.registry.registerExecutor('queryDatabase', async (input, ctx) => {
-  const { sql, limit = 100 } = input as { sql: string; limit?: number };
-  return db.query(sql).limit(limit).execute();
-});
-```
-
-### Custom Agent
-
-```markdown
-<!-- .tachu/agents/code-reviewer.md -->
----
-name: code-reviewer
-description: Reviews pull request diffs and produces structured code review feedback
-sideEffect: readonly
-idempotent: true
-requiresApproval: false
-timeout: 180000
-maxDepth: 1
-availableTools: [read-file, search-code, run-command]
----
-
-You are a careful code reviewer. When given a diff or a set of files:
-1. Understand the intent of the change
-2. Review for correctness, clarity, security, and performance
-3. Produce a structured review with severity levels: critical / major / minor / nit
-```
-
----
-
-## Observability & Safety
-
-### OpenTelemetry Integration
-
-Every engine event maps to an OTel span, enabling full distributed tracing:
-
-```typescript
-import { OtelEmitter } from '@tachu/extensions/emitters';
-import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
-import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
-import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
-
-const provider = new NodeTracerProvider();
-provider.addSpanProcessor(
-  new SimpleSpanProcessor(new OTLPTraceExporter({ url: 'http://localhost:4318/v1/traces' }))
-);
-provider.register();
-
-const engine = new Engine({
-  ...config,
-  // The OtelEmitter consumes EngineEvents and creates OTel spans
-});
-engine.useEmitter(new OtelEmitter());
-```
-
-**Events emitted for every request:**
-
-| Event Type | When |
-|-----------|------|
-| `phase_enter` / `phase_exit` | Every pipeline phase |
-| `llm_call_start` / `llm_call_end` | Every LLM invocation |
-| `tool_call_start` / `tool_call_end` | Every Tool execution |
-| `retry` | Task-level or system-level retry triggered |
-| `provider_fallback` | Provider downgrade initiated |
-| `budget_warning` | Budget at 80% of limit |
-| `budget_exhausted` | Budget circuit-breaker activated |
-| `error` | Any `EngineError` subclass |
-
-### Safety Module
-
-The Safety module operates in two independent layers:
-
-**Engine baseline (non-disableable):**
-- Input size enforcement (`maxInputSize` bytes)
-- Recursion depth limit (`maxRecursionDepth`)
-- Budget circuit-breaker (terminates immediately when token/time budget is exhausted)
-
-**Business-injectable policies** (via hooks or configuration):
-- Prompt injection detection (`enablePromptInjectionCheck: true`)
-- Sensitive operation interception (register via `engine.registerSafetyPolicy()`)
-- Output content compliance checks
-
-```typescript
-// Register a custom safety policy
-engine.registerSafetyPolicy(async (input, ctx) => {
-  if (containsPersonalData(input.content)) {
-    return { passed: false, violations: [{ type: 'pii', message: 'PII detected in input' }] };
-  }
-  return { passed: true, violations: [] };
-});
-```
-
----
-
-## Roadmap
-
-### v1.0.0 — Delivered
-
-- [x] `@tachu/core`: Engine, 9-phase pipeline, 8 modules, Registry, PromptAssembler, InMemoryVectorStore, full error taxonomy
-- [x] `@tachu/extensions`: OpenAI + Anthropic providers, 7 tools, Terminal/File/Web backends, LocalFs + Qdrant VectorStore, MCP stdio+SSE adapters, OTel+JSONL emitters, 4 common rules
-- [x] `@tachu/cli`: `tachu init` / `tachu run` / `tachu chat` with full parameter sets, session persistence, streaming renderer
-- [x] Test suite (≥80% line coverage) and performance benchmarks
-- [x] English and Chinese READMEs
-
-### v1.x — Planned
-
-- Additional VectorStore adapters (Pinecone, pgvector)
-- Additional Provider adapters (Google Gemini, Mistral)
-- Additional MCP tool integrations
-- More built-in compression strategies
-- Structured Plan template library
-
-### v2 — Vision
+## Documentation
 
-- Multi-agent collaboration (agent-to-agent communication protocol)
-- Persistent long-term memory across deployment restarts
-- Fine-grained budget allocation per sub-task
+| Document | Description |
+|----------|-------------|
+| [Overview Design](./docs/overview-design.md) | Vision, layers, abstractions, pipeline concepts |
+| [Detailed Design](./docs/detailed-design.md) | Types, modules, configuration schema |
+| [Technical Design](./docs/technical-design.md) | Engineering structure and implementation guide |
+| [Pipeline phases](./docs/architecture/pipeline-phases.md) | 9-phase pipeline and tool-use loop |
+| [Package layout](./docs/architecture/package-layout.md) | Monorepo packages and dependencies |
+| [Design principles](./docs/architecture/design-principles.md) | Core engineering principles |
+| [CLI reference](./docs/guides/cli.md) | All commands and flags |
+| [Configuration](./docs/guides/configuration.md) | `tachu.config.ts` / `EngineConfig` |
+| [Providers & integrations](./docs/guides/providers-and-integrations.md) | LLM, MCP, vector stores |
+| [Extension guide](./docs/guides/extension-guide.md) | Rules, Skills, Tools, Agents |
+| [Observability & safety](./docs/guides/observability-and-safety.md) | OTel, events, safety |
+| [CONTEXT.md](./CONTEXT.md) | Product vocabulary |
+| [CONTRIBUTING.md](./CONTRIBUTING.md) | Development workflow |
+| [Web Fetch Server](./packages/web-fetch-server/README.md) | Optional browser sidecar |
 
 ---
 
-## Contributing
-
-### Requirements
-
-- [Bun](https://bun.sh) >= 1.1.0
-- TypeScript 5.x (provided via dev dependencies)
+## Web Fetch Server (optional)
 
-### Development Workflow
+For browser-backed `web-fetch` / `web-search`, run the private sidecar — see [packages/web-fetch-server/README.md](./packages/web-fetch-server/README.md).
 
 ```bash
-# Clone and install
-git clone https://github.com/dangaogit/tachu
-cd tachu
-bun install
-
-# Run all tests
-bun test
-
-# Type check
-bun run typecheck
-
-# Build all packages
-bun run build
-
-# Run a specific package's tests
-bun test --filter packages/core
+bun run dev:server:install-browser
+bun run dev:server
 ```
 
-### Project Conventions
-
-- File names: `kebab-case`
-- Classes and types: `PascalCase`
-- Functions and variables: `camelCase`
-- Constants: `SCREAMING_SNAKE_CASE`
-- All public APIs must have TSDoc comments (`@param`, `@returns`, `@throws`, `@example`)
-- Test files co-located with source: `*.test.ts`
-- Integration tests under `__tests__/`
-
-Pull requests require:
-- All tests passing (`bun test`)
-- Zero TypeScript errors (`bun run typecheck`)
-- Coverage thresholds met (≥80% line, ≥70% branch)
-- TSDoc on any new public API
-
-See `CONTRIBUTING.md` for full guidelines.
-
----
-
-## Benchmarks
-
-Performance baselines are established in `packages/core/benchmarks/` and run with `bun test`:
-
-| Benchmark | Metric | Baseline |
-|-----------|--------|----------|
-| `scheduler.bench.ts` — 100 parallel tasks | Scheduling throughput | *Populated by verifier phase* |
-| `vector-store.bench.ts` — 10,000 entries, topK=10 | Search QPS | *Populated by verifier phase* |
-| `prompt-assembler.bench.ts` — 4K token window assembly | Assembly latency (p99) | *Populated by verifier phase* |
-
-Benchmarks serve as regression baselines; there are no minimum performance requirements for v1.
-
----
-
-## Documentation
-
-| Document | Description |
-|----------|-------------|
-| [Architecture Design](./docs/adr/architecture-design.md) | Vision, three-layer structure, four core abstractions, 9-phase pipeline design |
-| [Detailed Design](./docs/adr/detailed-design.md) | TypeScript interfaces, module specs, configuration schema |
-| [Technical Design](./docs/adr/technical-design.md) | Technology choices, engineering structure, implementation guide |
-
 ---
 
 ## License