soturail 0.3.0 → 0.3.2

package/docs/mcp.md

@@ -1,6 +1,6 @@
 # MCP Server
 
-SotuRail v0.3.0 includes a local MCP-compatible server over stdio using JSON-RPC 2.0 style messages.
+SotuRail includes a local MCP-compatible server over stdio using JSON-RPC 2.0 style messages. It is designed for local context access, not remote execution.
 
 ```bash
 soturail mcp doctor
@@ -10,9 +10,51 @@ soturail mcp serve --transport stdio
 
 The server exposes read-only resources such as the repo map, tree, rules, approved memory, self report, latest benchmarks and roadmap. It also exposes safe tools for indexing, progressive reads, formatting, rule checks, skill listing, context pack generation and raw log expansion.
 
+## Copyable JSON-RPC Examples
+
+Send one JSON object per line to `soturail mcp serve --transport stdio`.
+
+Initialize:
+
+```json
+{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"clientInfo":{"name":"manual-smoke","version":"0.1.0"}}}
+```
+
+List resources:
+
+```json
+{"jsonrpc":"2.0","id":2,"method":"resources/list"}
+```
+
+Read the repo map:
+
+```json
+{"jsonrpc":"2.0","id":3,"method":"resources/read","params":{"uri":"soturail://repo-map"}}
+```
+
+List tools:
+
+```json
+{"jsonrpc":"2.0","id":4,"method":"tools/list"}
+```
+
+The same payloads are available under `examples/mcp/`.
+
+## Safe Tools
+
+Default tools include:
+
+- `soturail.index`
+- `soturail.read`
+- `soturail.format`
+- `soturail.rules.check`
+- `soturail.skills.list`
+- `soturail.context.pack`
+- `soturail.expand`
+
 Security defaults:
 
 - no arbitrary shell execution;
-- no `soturail run` MCP tool in v0.3.0;
+- no `soturail.run` MCP tool by default;
 - raw log expansion redacts probable secrets unless `allow_raw=true`;
 - provider cache hits are never invented.

package/docs/metrics.md

@@ -14,6 +14,13 @@ SotuRail metrics are local, append-only and transparent.
 
 - estimated raw tokens;
 - estimated compressed tokens;
+- estimated reduced payload tokens;
+- terminal reducer estimated tokens saved;
+- dedupe estimated tokens saved;
+- metadata overhead tokens;
+- net estimated tokens saved;
+- dedupe blocks reused;
+- dedupe recent window;
 - compression ratio;
 - command count;
 - expansion count;
@@ -24,6 +31,8 @@ SotuRail metrics are local, append-only and transparent.
 - rules ingestion and validation counts;
 - benchmark fixture measurements.
 
+Small command outputs can be larger after SotuRail adds raw recovery metadata. When that happens, `compression_effective` is `false` and the CLI prints the small-output warning instead of hiding the overhead.
+
 ## Token Estimation
 
 v0.1.0 uses:

package/docs/reducers.md

@@ -6,6 +6,20 @@ Reducers compress terminal output while preserving the recovery path to raw logs
 
 Keeps first and last lines, error-looking lines, warnings, permission failures, timeouts, file paths and repeated-line summaries.
 
+## Developer Command Reducers
+
+v0.3.2 adds focused reducers for common developer commands:
+
+- `npm install`, package manager install noise and audit warnings;
+- `npm test`, Vitest/Jest/Mocha style failures;
+- `tsc` TypeScript diagnostics with file, line and column;
+- Docker logs with repeated heartbeat lines;
+- ESLint failures and rule names;
+- Vite and Next build output;
+- Java stack traces, Maven Surefire and Gradle failures.
+
+These reducers preserve error lines, warnings, security warnings, file paths, stack frames, failing test names, final summaries and recovery hints such as `Raw log: soturail expand <raw_id>`.
+
 ## Git
 
 Handles status, diff, log, show and branch output heuristically. It preserves changed files, rename/delete markers, conflict markers and hunk headers.
@@ -17,3 +31,20 @@ Handles Vitest, Jest, Mocha, Pytest, Maven Surefire and Gradle test output at a
 ## JSON TOON Lite
 
 Emits a compressed representation with relevant `path: value` lines. It preserves error messages, statuses, ids, file paths, booleans and nulls that affect debugging, while collapsing repetitive arrays and object structure. The reduced representation is clearly labeled when it is not valid JSON.
+
+## Block-Level Dedupe
+
+SotuRail tracks recent safe output blocks in `.soturail/dedupe/index.jsonl`. Repeated safe blocks can be replaced with a visible reference:
+
+```txt
+[deduped block: block_abc123, seen in raw_id cf42b9, 38 lines]
+```
+
+Error blocks, stack traces, security warnings and file paths tied to current failures are preserved in conservative mode.
+
+```bash
+soturail dedupe stats
+soturail run --similar-dedupe conservative npm test
+```
+
+Similar-output dedupe is deterministic and experimental. It normalizes whitespace, timestamps and temporary paths; it does not use AI or semantic matching.

package/docs/release-checklist.md

@@ -14,6 +14,7 @@ Before publishing:
 - [ ] `node dist/cli.js context --help`
 - [ ] `npm pack --dry-run`
 - [ ] `npm run release:check`
+- [ ] `node dist/cli.js release verify-package`
 - [ ] `node dist/cli.js --version` matches `package.json`.
 - [ ] Confirm docs mention limitations honestly.
 - [ ] Confirm no telemetry exists.
@@ -30,6 +31,15 @@ Check four separate release identifiers before publishing:
 
 The package version and CLI runtime version must match. `npm run build` regenerates the CLI version source from `package.json`, and `npm run release:check` verifies the built CLI before publication.
 
+v0.3.2 adds a packed-package verification gate. It runs `npm pack --json`, installs the generated `.tgz` into a temporary clean directory, and verifies both:
+
+```bash
+node node_modules/soturail/dist/cli.js --version
+npx --no-install soturail --version
+```
+
+This gate must print the same version as `package.json`.
+
 ## Audit Distinction
 
 `npm audit` checks all dependencies, including dev dependencies used for local tests and builds.

package/docs/release-workflow.md

@@ -15,9 +15,19 @@ The check also verifies:
 - `package.json` and `package-lock.json` version sync;
 - `node dist/cli.js --version` matches the package version;
 - npm pack dry-run emits the matching tarball name;
+- the packed tarball installs into a clean temp project and its installed CLI prints the package version;
+- expected package files are present and forbidden generated files are absent;
 - `CHANGELOG.md` and `RELEASE_NOTES_vX.Y.Z.md` exist for the local version;
 - README install instructions and `LICENSE` exist.
 
+You can run only the package verification gate with:
+
+```bash
+soturail release verify-package
+```
+
+This catches stale generated version files that local source checks might miss.
+
 ## Publish
 
 ```bash

package/docs/security-model.md

@@ -27,7 +27,19 @@ This is intentionally verbose so accidental bypasses are unlikely.
 
 Raw command logs remain on disk so compressed summaries can always be audited.
 
-Raw logs may contain secrets. Do not commit `.soturail/raw/`. MCP raw-log expansion redacts probable secrets by default unless `allow_raw=true` is explicitly passed.
+Raw logs may contain secrets. Do not commit `.soturail/raw/`. CLI expansion redacts probable secrets by default:
+
+```bash
+soturail expand <raw_id>
+```
+
+Exact raw output requires an explicit opt-in:
+
+```bash
+soturail expand <raw_id> --allow-raw --yes
+```
+
+MCP raw-log expansion also redacts probable secrets by default unless `allow_raw=true` is explicitly passed.
 
 ## MCP And Skills
 

package/docs/skill-rail.md

@@ -20,3 +20,7 @@ Skills live in `.soturail/skills/<skill-id>/` with `skill.yml`, `SKILL.md`, exam
 Validation checks required metadata, target names, duplicate IDs, deterministic content hashes, destructive shell patterns, prompt-injection style instructions and probable embedded secrets.
 
 Exports are written to `.soturail/exports/skills/<target>/`. Review every generated file before enabling it in Claude, Codex, Gemini, Cursor or another host.
+
+`soturail skills list` prints each skill ID, risk level, name, description, version, targets and local path. If there are no local skills, it prints the command to create one.
+
+The generated starter skill includes safe workflow steps, a verification checklist, example input/output, target metadata and human approval requirements for destructive commands, remote writes and dependency installation.

package/docs/usage.md

@@ -67,13 +67,21 @@ soturail bench compare-engines
 
 ```bash
 soturail skills init demo-skill
+soturail skills list
 soturail skills validate
 soturail skills export --target claude
+soturail context pack --target claude
+soturail context pack --target codex
+soturail context pack --target gemini
+soturail context pack --target cursor
 soturail context pack --target generic
 soturail mcp doctor
 soturail mcp manifest
+soturail mcp serve --transport stdio
 soturail hooks install --agent claude --mode safe-hooks --dry-run
 soturail release check
 ```
 
 MCP is local stdio JSON-RPC style transport and does not expose arbitrary shell execution in v0.3.0.
+
+For a first clean-folder walkthrough, see [first-real-workflow.md](first-real-workflow.md).

package/docs/windows.md

@@ -26,7 +26,7 @@ From a source checkout:
 npm run build
 npm pack --dry-run
 npm pack
-npm install -g .\soturail-0.3.0.tgz
+npm install -g .\soturail-0.3.2.tgz
 soturail --version
 ```
 
@@ -71,6 +71,26 @@ node app.js
 npx soturail --help
 ```
 
+## MCP And Context Packs
+
+PowerShell examples:
+
+```powershell
+soturail mcp doctor
+soturail mcp manifest
+soturail context pack --target generic
+```
+
+CMD examples:
+
+```bat
+soturail mcp doctor
+soturail mcp manifest
+soturail context pack --target generic
+```
+
+When testing `soturail mcp serve --transport stdio`, send one JSON object per line. See `examples\mcp\` for payloads.
+
 ## Safety
 
 SotuRail blocks destructive command shapes through `soturail run`, including `rm -rf`, `sudo`, `del /s`, downloaded script piping and automatic `git push`.
@@ -87,6 +107,8 @@ npm run release:check
 
 `package.json`, `package-lock.json`, `node dist/cli.js --version`, the npm tarball name, the changelog and the release notes must all agree on the same version.
 
+v0.3.2 also verifies the packed tarball by installing it in a temporary clean project and running the installed CLI. This helps catch stale generated version files before publish.
+
 Use browser-based npm login when needed:
 
 ```powershell

package/docs/workflow-rail.md

@@ -1,6 +1,6 @@
 # Workflow Rail
 
-Workflow Rail is planned for v0.4.0. It is not implemented in v0.3.0.
+Workflow Rail is planned for v0.4.0. It is not implemented in v0.3.x.
 
 The goal is to describe repeatable engineering workflows as local, auditable artifacts that can later export into Skill Rail or an MCP server.
 
@@ -24,3 +24,9 @@ soturail workflow export skill
 - Require human approval before enabling generated workflows or exported skills.
 
 Workflow Rail should complement, not replace, the existing SotuRail rails: `soturail run`, raw log recovery, Knowledge-to-Rules, benchmarks, cache-normalized payloads and self-dogfooding reports.
+
+## Current Road
+
+- v0.3.1: real usage polish, installed workflow examples and clean-folder smoke tests.
+- v0.3.2: stronger reducers and deduplication.
+- v0.4.0: real agent integrations and Workflow Rail.

package/examples/README.md

@@ -0,0 +1,22 @@
+# SotuRail Examples
+
+These examples are small starting points for real local usage:
+
+- `skills/`: reviewable Skill Rail YAML examples.
+- `context-packs/`: context pack workflows.
+- `mcp/`: JSON-RPC messages for the stdio MCP server.
+- `hooks/`: prompt-only and hook guidance.
+
+Run examples in a disposable folder first:
+
+```bash
+soturail init
+soturail index
+soturail context pack --target generic
+```
+
+Safety notes:
+
+- Review generated files before enabling them in an agent.
+- Do not paste raw logs into public issues without checking for secrets.
+- MCP does not expose arbitrary shell execution by default.

package/examples/context-packs/README.md

@@ -0,0 +1,17 @@
+# Context Pack Examples
+
+Context packs are Markdown payloads with stable project context before dynamic session data.
+
+```bash
+soturail init
+soturail index
+soturail context pack --target generic
+```
+
+Generated files:
+
+- `.soturail/context/claude-context.md`
+- `.soturail/context/codex-context.md`
+- `.soturail/context/gemini-context.md`
+- `.soturail/context/cursor-context.md`
+- `.soturail/context/generic-context.md`

package/examples/context-packs/generic-workflow.md

@@ -0,0 +1,9 @@
+# Generic Context Pack Workflow
+
+```bash
+soturail init
+soturail index
+soturail context pack --target generic
+```
+
+Open `.soturail/context/generic-context.md` and review it before sending it to an agent.

package/examples/hooks/README.md

@@ -0,0 +1,11 @@
+# Hook Examples
+
+Start with dry-run:
+
+```bash
+soturail hooks doctor
+soturail hooks install --agent claude --mode safe-hooks --dry-run
+soturail hooks install --agent codex --mode prompt-only --dry-run
+```
+
+Review every generated file before enabling hooks.

package/examples/hooks/prompt-only-codex.md

@@ -0,0 +1,7 @@
+# Prompt-only Codex Rules
+
+- Use `soturail index` before large repository changes.
+- Use `soturail read <file> --query "goal"` instead of reading giant files directly.
+- Use `soturail run` for tests, builds and logs so raw output is recoverable.
+- Use `soturail expand <raw_id>` only when the compressed summary lacks needed information.
+- Never use `soturail run` for `git push`.

package/examples/mcp/README.md

@@ -0,0 +1,17 @@
+# MCP Examples
+
+These files are one-message JSON-RPC examples for `soturail mcp serve --transport stdio`.
+
+Run:
+
+```bash
+soturail mcp serve --transport stdio
+```
+
+Then paste one JSON object per line. The server responds with one JSON-RPC response per line.
+
+Safety notes:
+
+- MCP does not expose arbitrary shell execution by default.
+- `soturail.run` is not listed as an MCP tool.
+- Raw log expansion redacts probable secrets unless `allow_raw=true`.

package/examples/mcp/initialize.json

@@ -0,0 +1 @@
+{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"clientInfo":{"name":"manual-smoke","version":"0.1.0"}}}

package/examples/mcp/resources-list.json

@@ -0,0 +1 @@
+{"jsonrpc":"2.0","id":2,"method":"resources/list"}

package/examples/mcp/resources-read-repo-map.json

@@ -0,0 +1 @@
+{"jsonrpc":"2.0","id":3,"method":"resources/read","params":{"uri":"soturail://repo-map"}}

package/examples/mcp/tools-list.json

@@ -0,0 +1 @@
+{"jsonrpc":"2.0","id":4,"method":"tools/list"}

package/examples/skills/README.md

@@ -0,0 +1,13 @@
+# Skill Examples
+
+Use these YAML files as references for `soturail skills init <name>` output.
+
+Recommended flow:
+
+```bash
+soturail skills init code-review
+soturail skills validate
+soturail skills export --target claude
+```
+
+The example YAML files are documentation templates. A real local skill also needs a matching `SKILL.md` and deterministic `content_hash`.

package/examples/skills/bug-triage-skill.yml

@@ -0,0 +1,26 @@
+id: bug-triage
+name: Bug Triage
+description: Turn a bug report into reproduction steps, likely causes, evidence and verification commands.
+version: 0.1.0
+author: Rafael Ryan Ramos de Souza
+risk_level: low
+targets:
+  - claude
+  - codex
+  - gemini
+  - cursor
+  - generic
+allowed_tools:
+  - read
+  - search
+  - summarize
+  - format
+forbidden_patterns:
+  - rm -rf
+  - curl | sh
+  - secret exfiltration
+requires_human_approval:
+  - destructive_command
+  - remote_write
+created_at: 2026-05-21T00:00:00.000Z
+content_hash: 0000000000000000000000000000000000000000000000000000000000000000

package/examples/skills/code-review-skill.yml

@@ -0,0 +1,28 @@
+id: code-review
+name: Code Review
+description: Review code changes for correctness, safety, regression risk and missing tests.
+version: 0.1.0
+author: Rafael Ryan Ramos de Souza
+risk_level: low
+targets:
+  - claude
+  - codex
+  - gemini
+  - cursor
+  - generic
+allowed_tools:
+  - read
+  - search
+  - summarize
+  - format
+forbidden_patterns:
+  - rm -rf
+  - curl | sh
+  - git push
+  - secret exfiltration
+requires_human_approval:
+  - destructive_command
+  - remote_write
+  - dependency_install
+created_at: 2026-05-21T00:00:00.000Z
+content_hash: 0000000000000000000000000000000000000000000000000000000000000000

package/examples/skills/java-student-helper-skill.yml

@@ -0,0 +1,25 @@
+id: java-student-helper
+name: Java Student Helper
+description: Help review Java exercises with explanations, compiler diagnostics and safe next practice steps.
+version: 0.1.0
+author: Rafael Ryan Ramos de Souza
+risk_level: low
+targets:
+  - claude
+  - codex
+  - gemini
+  - cursor
+  - generic
+allowed_tools:
+  - read
+  - search
+  - summarize
+  - format
+forbidden_patterns:
+  - delete student files without approval
+  - secret exfiltration
+requires_human_approval:
+  - destructive_command
+  - dependency_install
+created_at: 2026-05-21T00:00:00.000Z
+content_hash: 0000000000000000000000000000000000000000000000000000000000000000

package/examples/skills/php-web-reviewer-skill.yml

@@ -0,0 +1,27 @@
+id: php-web-reviewer
+name: PHP Web Reviewer
+description: Review PHP web changes for routing, validation, database safety and deployment risk.
+version: 0.1.0
+author: Rafael Ryan Ramos de Souza
+risk_level: medium
+targets:
+  - claude
+  - codex
+  - gemini
+  - cursor
+  - generic
+allowed_tools:
+  - read
+  - search
+  - summarize
+  - format
+forbidden_patterns:
+  - dumping environment variables
+  - committing credentials
+  - secret exfiltration
+requires_human_approval:
+  - destructive_command
+  - remote_write
+  - dependency_install
+created_at: 2026-05-21T00:00:00.000Z
+content_hash: 0000000000000000000000000000000000000000000000000000000000000000

package/examples/skills/release-manager-skill.yml

@@ -0,0 +1,26 @@
+id: release-manager
+name: Release Manager
+description: Prepare release validation, changelog notes, npm package checks and GitHub release evidence.
+version: 0.1.0
+author: Rafael Ryan Ramos de Souza
+risk_level: medium
+targets:
+  - claude
+  - codex
+  - gemini
+  - cursor
+  - generic
+allowed_tools:
+  - read
+  - search
+  - summarize
+  - format
+forbidden_patterns:
+  - npm publish without validation
+  - git push without explicit user request
+  - secret exfiltration
+requires_human_approval:
+  - remote_write
+  - dependency_install
+created_at: 2026-05-21T00:00:00.000Z
+content_hash: 0000000000000000000000000000000000000000000000000000000000000000

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "soturail",
-  "version": "0.3.0",
+  "version": "0.3.2",
   "description": "Local-first context rails for AI coding agents: reversible terminal compression, progressive repo reading, SDD workflows, hooks, benchmarks, memory and cache-friendly payloads.",
   "type": "module",
   "bin": {
@@ -13,7 +13,8 @@
     "dist",
     "README.md",
     "LICENSE",
-    "docs"
+    "docs",
+    "examples"
   ],
   "scripts": {
     "sync:version": "node scripts/sync-version.mjs",