soturail 0.4.1 → 0.7.0

package/README.md

@@ -6,7 +6,7 @@
 
 [![Node.js >=20](https://img.shields.io/badge/node-%3E%3D20-3c873a)](package.json)
 [![TypeScript](https://img.shields.io/badge/TypeScript-strict-3178c6)](tsconfig.json)
-[![MIT License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![License: Reserved Rights](https://img.shields.io/badge/license-reserved--rights-red)](LICENSE)
 [![CI](https://github.com/Soturine/soturail/actions/workflows/ci.yml/badge.svg)](https://github.com/Soturine/soturail/actions/workflows/ci.yml)
 [![npm version](https://img.shields.io/npm/v/soturail.svg)](https://www.npmjs.com/package/soturail)
 [![npm downloads](https://img.shields.io/npm/dm/soturail.svg)](https://www.npmjs.com/package/soturail)
@@ -18,100 +18,207 @@ SotuRail — trilhos locais de contexto para agentes de IA.
 
 ## 1. What Is SotuRail?
 
-SotuRail is a local-first Context OS for AI coding agents such as Claude Code, Codex CLI, Gemini CLI, Cursor and similar tools.
+SotuRail is a local-first Context OS for AI coding agents such as Claude Code, Codex CLI, Gemini CLI, Cursor, Antigravity-style hosts and similar tools.
 
-It wraps a repository and terminal session with reversible evidence rails: heuristic repo maps, progressive file reading, safe command execution, raw log recovery, reducers, prompt-cache-friendly blocks, Spec-Driven Development artifacts, local memory, hooks, benchmarks and rules extraction.
+It wraps a repository and terminal session with reversible evidence rails: heuristic repo maps, progressive file reading, safe command execution, raw log recovery, reducers, prompt-cache-friendly blocks, Spec-Driven Development artifacts, local memory, hooks, benchmarks, rules extraction, context packs, agent exports and workflow records.
 
-## Project Status
+SotuRail is not the agent, not a Claude-only harness, not a Mermaid-only workflow tool and not a heavy production gateway. It is the local rail layer that helps agents work with better context, safer logs, smaller prompts, approved memory, structured payloads, diagrams, policies and auditable workflows.
 
-v0.4.1 is early but functional. TypeScript mode is stable for local usage. Native Rust mode is optional and focused on hot paths. Skill Rail, MCP, context packs, agent exports and Workflow Rail are local-first and benchmarkable. External comparisons are optional and user-provided.
+## Project Status
 
-## Built With SotuRail
+v0.7.0 is the Workflow Rail 2.0, Harness Rail and Diagram Rail milestone. TypeScript mode is stable for local usage. Native Rust mode remains optional and focused on hot paths. Skill Rail, MCP, context packs, agent exports, Workflow Rail, Memory Rail, Context Intelligence, Policy Rail, evidence packs, the host-aware Agent Runtime Adapter, Diagram Rail and the local evaluation suite are local-first and benchmarkable. External comparisons are optional and user-provided.
 
-SotuRail now dogfoods itself for release-oriented development. `soturail self all` runs repository checks, indexing, build, tests, benchmarks and a local Markdown report through SotuRail's own rails.
+The next product direction is staged: v0.8.0 adds Knowledge Rail and Project Brain without turning SotuRail into an autonomous agent runtime. See [ROADMAP.md](ROADMAP.md) and [docs/future-rails-index.md](docs/future-rails-index.md).
 
-## Self-Dogfooding
+## v0.5.x MVP Rails
 
 ```bash
-soturail self doctor
-soturail self index
-soturail self build
-soturail self test
-soturail self bench
-soturail self report
-soturail self all
+soturail memory remember "Decision: keep MCP read-oriented by default" --tag architecture
+soturail memory recall "MCP safety" --limit 5
+soturail context select --query "prepare npm release"
+soturail context budget --target claude --explain
+soturail context pack --role reviewer
+soturail harness contract init
+soturail policy doctor
+soturail fs snapshot
+soturail mcp exposure
+soturail run workspace new "Try v0.5.x rails"
+soturail native doctor
+soturail validate json package.json --strict
+soturail format compare README.md
 ```
 
-Reports are written to `.soturail/reports/self-dogfood.md` with stable project context first and dynamic raw IDs, command status and benchmark data later.
+These rails write local JSON, JSONL and Markdown under `.soturail/`. They do not create cloud resources, background agents, global config writes or arbitrary MCP shell execution.
 
-## Release Workflow
-
-Release automation is local-first and conservative:
+## v0.6.0 Agent Runtime Adapter
 
 ```bash
-npm run release:check
-npm run release:publish -- X.Y.Z
-npm run release:github -- X.Y.Z
-npm run release:full -- X.Y.Z --publish-npm --github-release
+soturail agents capabilities
+soturail agents capabilities --json
+soturail agents status
+soturail agents status --json
+soturail agents doctor --verbose
+soturail agents install --agent claude --dry-run
+soturail agents install --agent cursor --dry-run
+soturail agents install --agent gemini --dry-run
+soturail agents export --agent deepagents
+soturail agents export --agent deepagents-js
 ```
 
-The release commands accept a positional version, `--target-version X.Y.Z`, or the backward-compatible `--version X.Y.Z` option. The release script never runs `npm audit fix --force`, never publishes when build/tests/runtime audit fail and never creates a GitHub release before npm publish succeeds. See [docs/release-workflow.md](docs/release-workflow.md).
+The adapter is host-aware but conservative. It reports real, experimental, prompt-only and planned surfaces for Claude Code, Codex, Gemini CLI, Cursor, Antigravity, Generic, OpenCode/Amp/Kiro-style hosts and Deep Agents-style targets. Dry-run installs show planned file writes, backups, context references, payload recommendations and policy warnings before anything changes.
 
-Release verification installs the packed `.tgz` into a clean temporary project and executes `node_modules/soturail/dist/cli.js` directly to avoid npm cache or global CLI false positives.
+## v0.6.1 Evaluation Suite
+
+```bash
+soturail eval list
+soturail eval run
+soturail eval report
+```
+
+The evaluation suite is deterministic and local. It checks memory recall, context selection, reducers, routing, role packs, agent-doc hygiene, offload/restore, payload formats, strict JSON validation, evidence packs, harness scenarios and Diagram Rail validation seeds. Reports are written to `.soturail/eval/latest.json` and `.soturail/eval/latest.md`.
 
-## Release Reliability
+Token savings alone are not treated as success. The suite checks whether important files, commands, errors, policy decisions and recovery pointers survive compression and selection. See [docs/evaluation-suite.md](docs/evaluation-suite.md).
 
-SotuRail includes a release preflight check to prevent stale package metadata, broken npm CLI binaries, missing release notes, and changelog/version mismatches.
+## v0.7.0 Workflow, Harness And Diagram Rails
 
 ```bash
-npm run build
-npm test
-npm run release:check
-npm pack --dry-run
+soturail workflow setup
+soturail workflow plan "Implement feature"
+soturail workflow work --note "Implemented the first slice"
+soturail workflow review --all
+soturail workflow verify
+soturail workflow evidence <id>
+soturail workflow diagram <id>
+
+soturail diagram init
+soturail diagram new "Feature flow"
+soturail diagram audit docs/diagrams/feature-flow.md
+soturail diagram validate
+soturail diagram from-workflow <id>
 ```
 
-## 2. Why SotuRail Exists
+Workflow Rail 2.0 writes local plan/work/review/verify artifacts under `.soturail/workflows/`. Harness Rail connects repeated failures and acceptance contracts to verification and evidence. Diagram Rail writes text-based Mermaid diagrams and `.spec.md` visual contracts under `docs/diagrams/` and `.soturail/diagrams/`.
+
+Release notes now live under `docs/releases/`, and release evidence points to `docs/releases/RELEASE_NOTES_vX.Y.Z.md`.
+
+## Why SotuRail Exists
 
 AI coding agents often receive too much unstable context: full files, noisy test logs, repeated terminal output and long conversational summaries. SotuRail is designed to unify those workflows into one independent local-first tool without sending telemetry or inventing provider metrics.
 
-## 3. Key Features
+The long-term product direction is simple:
+
+```txt
+SotuRail does not replace Claude, Codex, Gemini, Cursor or other agents.
+SotuRail prepares the local context, memory, logs, diagrams, policies, payload formats and reports those agents need.
+```
+
+## Where It Fits
+
+Useful mental model:
+
+```txt
+Hermes-like systems: agent brain and execution loop.
+Deep Agents-style systems: batteries-included harness with sub-agents, tools, filesystem, memory and approvals.
+Claude Code Harness-style systems: disciplined setup/plan/work/review/release loops with guardrails and evidence.
+MDDD-style systems: Mermaid/.spec.md visual contracts before implementation.
+Plano-like systems: gateway, router and production data plane.
+SotuRail: local Context OS for context, memory, reducers, policy, logs, workflows, diagrams, payload formats and reports.
+```
+
+A newer mental model from the v0.5 planning cycle:
+
+```txt
+Dense-agent setup: every task gets every instruction, file and rule.
+SotuRail setup: route the task to the right local context expert, memory, role pack, rule set, payload format, diagram and workflow evidence.
+```
+
+SotuRail absorbs patterns from the context-engineering ecosystem without vendoring or copying adjacent projects. It stays small, local-first, npm-friendly and safe-by-default. Research notes and product ideas live in [docs/ecosystem-influences.md](docs/ecosystem-influences.md), [docs/comparisons.md](docs/comparisons.md), [docs/deep-agents-patterns.md](docs/deep-agents-patterns.md), [docs/future-rails-index.md](docs/future-rails-index.md), [docs/harness-rail.md](docs/harness-rail.md), [docs/policy-rail.md](docs/policy-rail.md), [docs/diagram-rail.md](docs/diagram-rail.md), [docs/structured-payload-rail.md](docs/structured-payload-rail.md) and [docs/agent-docs-hygiene.md](docs/agent-docs-hygiene.md).
+
+## Built With SotuRail
+
+SotuRail dogfoods itself for release-oriented development. `soturail self all` runs repository checks, indexing, build, tests, benchmarks and a local Markdown report through SotuRail's own rails.
+
+```bash
+soturail self doctor
+soturail self index
+soturail self build
+soturail self test
+soturail self bench
+soturail self report
+soturail self all
+```
+
+Reports are written to `.soturail/reports/self-dogfood.md` with stable project context first and dynamic raw IDs, command status and benchmark data later.
+
+## Key Features
 
 - Heuristic Repo Map with cross-platform ignore handling.
 - Progressive reader for large files with reversible collapsed ranges.
 - Safe tee-stream runner with raw log preservation.
 - Git, test, npm, TypeScript, Docker, ESLint, Java/Maven/Gradle, build, JSON and generic terminal reducers.
-- Optional Rust native reducer and runner hot paths with TypeScript fallback.
 - Cross-call and block-level dedupe for repeated command output.
+- Optional Rust native reducer and runner hot paths with TypeScript fallback.
 - Reproducible local benchmark suite.
 - Agent response compression modes.
 - Knowledge-to-Rules ingestion and validators.
 - Prompt-only and hook-style agent integrations.
+- Agent exports for Claude, Codex, Gemini, Cursor, Antigravity and generic hosts.
+- MCP-compatible local stdio server and config helpers.
+- Workflow Rail for local task records, phase evidence and optional worktree planning.
+- Memory Rail with explicit local records, recall, capture and consolidation.
+- Context Intelligence with selection, pruning, budget reports, offload/restore and role packs.
+- Harness, Policy, Filesystem Evidence, Diagram and Run Workspace rails.
+- MCP exposure reports and skill routing seeds.
+- Agent docs hygiene checks for short root instruction files.
 - SDD specs, approved memory and cache block normalization.
 - Honest local metrics.
 
-## 4. How It Differs Conceptually
+## Planned Next Features
+
+The next roadmap stage moves toward Knowledge Rail and Project Brain:
+
+- architecture decisions and recurring bug patterns from local docs and workflow evidence;
+- stale evidence detection across richer project graphs;
+- safer knowledge-to-rules extraction;
+- project profiles for agents;
+- future local UI/report mode: HTML reports first, Mermaid rendering and MCP Apps/AG-UI-style event surfaces later.
 
-SotuRail is an independent implementation. It may be conceptually adjacent to terminal reducers, host hook systems, Spec-Driven Development kits and memory tools, but it does not depend on RTK, Squeez, Spec Kit, Caveman, MemPalace or proprietary workflows.
+## Future Rails Documentation
 
-SotuRail aims to unify these ideas into one local-first workflow: reversible raw logs, safety policy, benchmarks, prompt cache ordering, specs, memory, hooks and rules.
+Future planning is split across focused docs so the roadmap does not become the only source of truth:
 
-## 5. Installation
+- [Future Rails Index](docs/future-rails-index.md)
+- [Harness Rail](docs/harness-rail.md)
+- [Policy Rail](docs/policy-rail.md)
+- [Diagram Rail](docs/diagram-rail.md)
+- [Structured Payload Rail](docs/structured-payload-rail.md)
+- [Agent Docs Hygiene](docs/agent-docs-hygiene.md)
+- [Evaluation Suite](docs/evaluation-suite.md)
+- [Roadmap Addendum](docs/roadmap-harness-diagram-payload-addendum.md)
+
+## Installation
 
 Use directly with npx:
 
 ```bash
 npx soturail --help
-npx soturail@0.4.1 --help
+npx soturail@latest --help
 ```
 
 Install globally:
 
 ```bash
-npm install -g soturail@0.4.1
+npm install -g soturail
 soturail --help
 soturail --version
 ```
 
+When v0.6.1 is published, install that exact version with:
+
+```bash
+npm install -g soturail@0.6.1
+```
+
 For local development from source:
 
 ```bash
@@ -130,30 +237,19 @@ npm run build:all      # TypeScript + native, requires cargo
 
 npm package: https://www.npmjs.com/package/soturail
 
-## Native Performance Path
-
-TypeScript remains the public CLI, orchestration, docs and npm distribution layer. Rust handles optional hot paths where streaming, low overhead and binary execution matter:
-
-- native terminal reducers;
-- native JSON/tool payload reducer;
-- native tee-stream runner with raw log and summary sidecar support.
-
-SotuRail does not claim native speedups unless local benchmark results show them for your machine.
-
-## 6. Quick Start
+## Quick Start
 
 ```bash
 soturail init
 soturail index
 soturail read README.md --query "quick start"
-soturail context pack --target generic
+soturail context pack --target all
 soturail agents doctor
 soturail agents export --agent all
 soturail mcp smoke
 soturail workflow new "Try SotuRail"
 soturail skills init demo-skill
 soturail skills validate
-soturail mcp doctor
 soturail run npm test
 soturail expand <raw_id>
 soturail stats
@@ -161,10 +257,16 @@ soturail stats
 
 For an installed clean-folder walkthrough, see [docs/first-real-workflow.md](docs/first-real-workflow.md).
 
-## First clean project flow
+## First Clean Project Flow
 
 ```bash
 soturail init
+soturail memory doctor
+soturail context budget --explain
+soturail context pack --role planner
+soturail harness doctor
+soturail policy doctor
+soturail run workspace new "Try SotuRail"
 soturail context pack --target all
 soturail agents doctor
 soturail agents export --agent all
@@ -175,7 +277,19 @@ soturail workflow list
 
 `soturail init` scaffolds agent, MCP, context pack, hook, skill and workflow examples without overwriting user-edited files.
 
-## 7. Commands
+## Agent Setup Tutorials
+
+- [Claude Code](docs/tutorial-claude-code.md)
+- [Codex](docs/tutorial-codex.md)
+- [Gemini CLI](docs/tutorial-gemini-cli.md)
+- [Cursor](docs/tutorial-cursor.md)
+- [Antigravity prompt-only](docs/tutorial-antigravity.md)
+- [Deep Agents-style role packs](docs/tutorial-deep-agents-role-packs.md)
+- [Harness workflow](docs/tutorial-harness-workflow.md)
+- [Diagram specs](docs/tutorial-diagram-spec.md)
+- [Context formats](docs/tutorial-context-formats.md)
+
+## Commands
 
 ```bash
 soturail init
@@ -191,80 +305,56 @@ soturail bench prepare
 soturail bench run --engine ts
 soturail hooks list
 soturail agents list
+soturail agents doctor
 soturail agents export --agent all
 soturail mcp smoke
 soturail mcp config --agent generic
 soturail context pack --target all
+soturail context pack --role planner
+soturail context select --query "release checklist"
+soturail context budget --explain
 soturail workflow list
+soturail workflow show <id>
+soturail workflow close <id>
+soturail run workspace show <run-id>
 soturail format README.md --mode concise
+soturail format compare README.md
+soturail validate json package.json --strict
 soturail ingest README.md --type docs
 soturail rules check
 soturail skills init demo-skill
 soturail skills validate
 soturail skills export --target claude
-soturail context pack --target generic
-soturail mcp doctor
 soturail spec status
 soturail memory propose "decision"
 soturail doctor cache
 ```
 
-## 8. Benchmarks
-
-SotuRail includes deterministic local fixtures for terminal compression, agent response compression, JSON/tool payload compression, Knowledge-to-Rules structuring and native performance readiness.
-
-```bash
-npm run build
-soturail bench prepare
-soturail bench run --engine ts
-soturail bench report
-```
-
-The latest report is written to [benchmarks/reports/latest.md](benchmarks/reports/latest.md). External comparisons are optional and user-provided only:
-
-```bash
-soturail bench compare-optional --tool rtk
-soturail bench compare-optional --tool squeez
-```
-
-### Benchmark Interpretation
-
-- Terminal compression measures how reducers shrink logs while preserving errors, paths and recovery links.
-- Dedupe cases measure conservative block reuse and report `dedupe_tokens_saved`, metadata overhead and net savings.
-- Agent response compression measures deterministic formatting modes.
-- JSON/tool payload compression preserves relevant primitive values while collapsing repetitive structure.
-- Knowledge-to-Rules is reusable structuring, not pure compression; structured rules can be larger than a tiny source document because they add citations and validator metadata.
-- Native performance compares Rust and TypeScript only when `soturail-native` is built locally.
-
-## Honest Metrics
-
-Local token counts are deterministic estimates. SotuRail reports raw payload tokens, reduced payload tokens, metadata overhead and net estimated tokens. For tiny outputs, compression may be ineffective once recovery metadata is included; SotuRail says that directly while preserving raw recovery paths.
-
-## Reducers And Dedupe
-
-v0.3.2 adds stronger reducers for common developer commands including `npm install`, `npm test`, Vitest, `tsc`, `git diff`, `git status`, Docker logs, ESLint, Vite/Next build output and Java/Maven/Gradle failures. Reducers preserve errors, warnings, file paths, line/column references, stack traces, security warnings and the raw recovery hint.
+## Release Workflow
 
-Block-level dedupe is conservative by default. It can replace repeated safe blocks with references such as `[deduped block: ...]`, while preserving error blocks and current failure context. Similar-output dedupe is experimental and opt-in:
+Release automation is local-first and conservative:
 
 ```bash
-soturail run --similar-dedupe conservative npm test
-soturail dedupe stats
+npm run release:check
+npm run release:publish -- X.Y.Z
+npm run release:github -- X.Y.Z
+npm run release:full -- X.Y.Z --publish-npm --github-release
 ```
 
-## 9. Agent Hooks
+The release commands accept a positional version, `--target-version X.Y.Z`, or the backward-compatible `--version X.Y.Z` option. The release script never runs `npm audit fix --force`, never publishes when build/tests/runtime audit fail and never creates a GitHub release before npm publish succeeds. See [docs/release-workflow.md](docs/release-workflow.md).
 
-SotuRail provides cautious hook scaffolding and prompt-only fallbacks:
+Release verification installs the packed `.tgz` into a clean temporary project and executes `node_modules/soturail/dist/cli.js` directly to avoid npm cache or global CLI false positives.
 
 ```bash
-soturail hooks list
-soturail hooks doctor
-soturail hooks install claude --dry-run
-soturail hooks prompt-only codex
+npm run build
+npm test
+npm run release:check
+npm pack --dry-run
 ```
 
 ## Agent Integrations
 
-SotuRail adds reviewed agent integration profiles for Claude, Codex, Gemini, Cursor, Antigravity and generic agents.
+SotuRail provides reviewed agent integration profiles for Claude, Codex, Gemini, Cursor, Antigravity and generic agents.
 
 ```bash
 soturail agents list
@@ -276,33 +366,53 @@ soturail agents install --agent cursor --mode rules --dry-run
 
 Exports live under `.soturail/exports/agents/`. Install commands are dry-run-first, backup-first and project-local by default. SotuRail does not auto-enable arbitrary shell execution or unknown global host configs.
 
-## Workflow Rail
+## MCP And Context Packs
 
-Workflow Rail stores local task state under `.soturail/workflows/` and can optionally plan Git worktree isolation.
+SotuRail exposes local context through a minimal MCP-compatible stdio server and cache-friendly context packs.
 
 ```bash
-soturail workflow new "Implement feature"
-soturail workflow list
-soturail workflow start <id> --worktree --dry-run
-soturail workflow verify <id>
+soturail mcp doctor
+soturail mcp manifest
+soturail mcp serve --transport stdio
+soturail mcp smoke
+soturail context pack --target claude
+soturail context pack --target codex
+soturail context pack --target gemini
+soturail context pack --target cursor
+soturail context pack --target antigravity
+soturail context pack --target generic
+soturail context pack --target all
+soturail context explain
 ```
 
-SotuRail does not push, merge or delete worktrees automatically.
+MCP does not expose arbitrary shell execution. Raw log expansion redacts probable secrets unless raw output is explicitly requested.
+
+Context packs are written to `.soturail/context/<target>-context.md`. JSON-RPC examples live under [examples/mcp](examples/mcp).
+
+Role packs, structured payload validation and offload flows are tracked in [docs/context-packs.md](docs/context-packs.md), [docs/structured-payload-rail.md](docs/structured-payload-rail.md) and [docs/deep-agents-patterns.md](docs/deep-agents-patterns.md).
 
-## MCP Config Helpers
+## Workflow Rail
+
+Workflow Rail stores local task state under `.soturail/workflows/` and can optionally plan Git worktree isolation. v0.7.0 adds phase artifacts for setup, plan, work, review, verify and evidence.
 
 ```bash
-soturail mcp config --agent claude
-soturail mcp config --agent cursor
-soturail mcp config --agent generic
-soturail mcp smoke
+soturail workflow setup
+soturail workflow plan "Implement feature"
+soturail workflow work --note "Progress note"
+soturail workflow review --all
+soturail workflow verify
+soturail workflow evidence <id>
+soturail workflow diagram <id>
+soturail workflow new "Implement feature"
+soturail workflow list
+soturail workflow show <id>
+soturail workflow close <id>
+soturail workflow start <id> --worktree --dry-run
 ```
 
-MCP remains local stdio and does not expose arbitrary shell execution by default.
-
-Host APIs vary, so SotuRail never writes guessed config without showing the target and creating backups for existing files.
+SotuRail does not push, merge or delete worktrees automatically.
 
-Review generated hooks before enabling them. SotuRail should never auto-install unreviewed third-party skills, hooks or scripts.
+Harness-style phases and diagram-driven workflows are documented in [docs/workflow-rail.md](docs/workflow-rail.md), [docs/harness-rail.md](docs/harness-rail.md) and [docs/diagram-rail.md](docs/diagram-rail.md).
 
 ## Skill Rail
 
@@ -316,47 +426,39 @@ soturail skills export --target claude
 soturail skills pack --format markdown
 ```
 
-Exports are written under `.soturail/exports/skills/` and should be reviewed before enabling in an agent host.
+Exports are written under `.soturail/exports/skills/` and should be reviewed before enabling in an agent host. Examples live under [examples/skills](examples/skills).
 
-Examples live under [examples/skills](examples/skills).
+Future task/role-aware skill routing is tracked in [docs/skill-rail.md](docs/skill-rail.md).
 
-## MCP And Context Packs
+## Reducers And Dedupe
 
-SotuRail exposes local context through a minimal MCP-compatible stdio server and cache-friendly context packs.
+v0.3.2 added stronger reducers for common developer commands including `npm install`, `npm test`, Vitest, `tsc`, `git diff`, `git status`, Docker logs, ESLint, Vite/Next build output and Java/Maven/Gradle failures. Reducers preserve errors, warnings, file paths, line/column references, stack traces, security warnings and the raw recovery hint.
+
+Block-level dedupe is conservative by default. It can replace repeated safe blocks with references such as `[deduped block: ...]`, while preserving error blocks and current failure context. Similar-output dedupe is experimental and opt-in:
 
 ```bash
-soturail mcp doctor
-soturail mcp manifest
-soturail mcp serve --transport stdio
-soturail context pack --target claude
-soturail context pack --target codex
-soturail context pack --target gemini
-soturail context pack --target cursor
-soturail context pack --target antigravity
-soturail context pack --target generic
-soturail context pack --target all
-soturail context explain
+soturail run --similar-dedupe conservative npm test
+soturail dedupe stats
 ```
 
-MCP does not expose arbitrary shell execution. Raw log expansion redacts probable secrets unless raw output is explicitly requested.
+## Benchmarks And Honest Metrics
 
-Context packs are written to `.soturail/context/<target>-context.md`. JSON-RPC examples live under [examples/mcp](examples/mcp).
+SotuRail includes deterministic local fixtures for terminal compression, agent response compression, JSON/tool payload compression, Knowledge-to-Rules structuring, dedupe and native performance readiness.
 
-## 10. Agent Response Compression
+```bash
+npm run build
+soturail bench prepare
+soturail bench run --engine ts
+soturail bench report
+```
 
-SotuRail includes Caveman-like output compression as inspiration, implemented independently with professional modes:
+The latest report is written to [benchmarks/reports/latest.md](benchmarks/reports/latest.md). External comparisons are optional and user-provided only.
 
-- `normal`
-- `concise`
-- `ultra`
-- `review`
-- `commit`
-- `debug`
-- `docs`
+Local token counts are deterministic estimates. SotuRail reports raw payload tokens, reduced payload tokens, metadata overhead and net estimated tokens. For tiny outputs, compression may be ineffective once recovery metadata is included; SotuRail says that directly while preserving raw recovery paths.
 
-It preserves fenced code blocks, shell commands, file paths, line numbers, security warnings and failure details using deterministic reducers. See [docs/response-compression.md](docs/response-compression.md).
+Future benchmarks will also cover context quality, role-pack quality, diagram validation, evidence-pack completeness and format quality for Markdown vs tagged vs JSON vs compact payloads.
 
-## 11. Knowledge-to-Rules Engine
+## Knowledge-to-Rules Engine
 
 SotuRail can ingest Markdown, TXT, JSON and YAML into structured YAML/JSON rules, checklists, citations and simple validators.
 
@@ -367,9 +469,9 @@ soturail rules check
 soturail rules export --format yaml
 ```
 
-This makes heavy docs smaller, reusable and auditable without inventing rules that do not appear in the source. See [docs/knowledge-to-rules.md](docs/knowledge-to-rules.md).
+This makes heavy docs smaller, reusable and auditable without inventing rules that do not appear in the source. See [docs/knowledge-to-rules.md](docs/knowledge-to-rules.md), [docs/rules.md](docs/rules.md) and [docs/policy-rail.md](docs/policy-rail.md).
 
-## 12. Prompt Caching Design
+## Prompt Caching Design
 
 Stable blocks are ordered before dynamic data:
 
@@ -383,32 +485,65 @@ Stable blocks are ordered before dynamic data:
 
 SotuRail reports estimated cache stability only. It never claims real provider cache hits unless imported metadata exists.
 
-## 13. Security Model
+## Native Performance Path
+
+TypeScript remains the public CLI, orchestration, docs and npm distribution layer. Rust handles optional hot paths where streaming, low overhead and binary execution matter:
+
+- native terminal reducers;
+- native JSON/tool payload reducer;
+- native tee-stream runner with raw log and summary sidecar support.
+
+SotuRail does not claim native speedups unless local benchmark results show them for your machine.
+
+## Security Model
 
 `soturail run` blocks dangerous patterns by default, including `rm -rf`, `sudo`, `format`, `dd if=`, `curl | sh`, `del /s` and `git push`.
 
 Raw logs may contain secrets because they preserve real terminal output. Treat `.soturail/raw/` as local evidence, not public artifact material. `soturail expand <raw_id>` redacts probable secrets by default; use `--allow-raw --yes` only when you intentionally need exact raw output.
 
+Policy queue, auth guidance and MCP exposure reports are tracked in [docs/policy-rail.md](docs/policy-rail.md) and [docs/security-model.md](docs/security-model.md).
+
+## Migration
+
+Moving from v0.4.x to v0.5.x keeps the agent export, MCP and Workflow Rail commands, then adds local `.soturail/` folders for Memory, Context Intelligence, Harness, Policy, Filesystem Evidence and Run Workspace seeds. See [docs/migration-v0.5.md](docs/migration-v0.5.md).
+
 ## Windows Notes
 
 Windows users should see [docs/windows.md](docs/windows.md) for CMD vs PowerShell quoting, global install, `npx`, local tarball testing and common paste mistakes such as copying Markdown code-fence labels into CMD.
 
-## Road To Agent And Workflow Rails
+## Comparison Philosophy
 
-Skill Rail, agent integrations and Workflow Rail are documented in [docs/skill-rail.md](docs/skill-rail.md), [docs/agents.md](docs/agents.md) and [docs/workflow-rail.md](docs/workflow-rail.md).
+SotuRail is inspired by the broader context-engineering ecosystem, including terminal reducers, agent response compression, spec-driven workflows, local memory, rules extraction, hooks, benchmarks, skill registries, agent memory, harness workflows, Mermaid diagram-driven development, structured prompt payloads and gateway/observability ideas. SotuRail does not vendor or depend on those projects. It aims to unify similar ideas into one local-first workflow while keeping benchmarks honest. See [docs/comparisons.md](docs/comparisons.md), [docs/ecosystem-influences.md](docs/ecosystem-influences.md), [docs/deep-agents-patterns.md](docs/deep-agents-patterns.md) and [docs/future-rails-index.md](docs/future-rails-index.md).
 
-## Comparison Philosophy
+## Roadmap
 
-SotuRail is inspired by the broader context-engineering ecosystem, including terminal reducers, agent response compression, spec-driven workflows, local memory, rules extraction, hooks, benchmarks and skill registries. SotuRail does not vendor or depend on those projects. It aims to unify similar ideas into one local-first workflow while keeping benchmarks honest. See [docs/comparisons.md](docs/comparisons.md).
+See [ROADMAP.md](ROADMAP.md).
 
-## 14. Roadmap
+Near-term direction:
 
-See [ROADMAP.md](ROADMAP.md). v0.4.1 focuses on scaffold and UX polish; later releases target native performance, hardened knowledge ingestion, semantic memory and a stable API.
+```txt
+v0.5.0  Memory Rail + Context Intelligence + Role Packs + Harness/Policy seeds + reliability
+v0.5.1  Memory/context polish + Structured Payload Rail + Agent Docs Hygiene + Diagram docs
+v0.5.2  CI stabilization + lightweight quality fixtures + roadmap realignment
+v0.6.0  Real agent runtime integration + host capability matrix + host-aware payload/policy docs
+v0.6.1  Agent UX polish + full evaluation suite
+v0.7.0  Workflow Rail 2.0 + Harness Rail + Diagram Rail + .spec.md visual contracts
+v0.8.0  Knowledge Rail and Project Brain with specs, diagrams and recurring failures
+v0.9.0  Native Engine Real
+v0.10.0 Local reports, traces, Mermaid rendering and dashboard
+v1.0.0  Stable Context OS
+```
 
-## 15. Contributing
+## Contributing
 
 See [CONTRIBUTING.md](CONTRIBUTING.md). Reducers, hooks and rules should include tests, safety notes and benchmark impact when relevant.
 
-## 16. License
+## License Status
+
+SotuRail is not currently licensed as open source for the current repository state or future versions.
+
+All rights are reserved unless a later LICENSE file or written agreement explicitly grants permissions. This repository may be public for portfolio, planning, research, documentation and evaluation visibility, but broad copying, redistribution, sublicensing, commercial use or derivative works are not granted by default.
+
+Important: older SotuRail versions or copies that were explicitly released under MIT remain governed by the MIT terms that applied to those specific versions at the time of release.
 
-MIT © Rafael Ryan Ramos de Souza
+See [docs/licensing-strategy.md](docs/licensing-strategy.md).