@shadowclone/cli 0.0.4 → 0.0.6

package/.claude-plugin/hooks/hooks.json

@@ -11,6 +11,16 @@
         ]
       }
     ],
+    "SubagentStart": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "bun run \"${CLAUDE_PLUGIN_ROOT}/dist/shadowclone.js\" hook session-start"
+          }
+        ]
+      }
+    ],
     "SessionEnd": [
       {
         "hooks": [

package/README.md

@@ -1,191 +1,104 @@
 # shadowclone
 
-Memory and alignment compiler for AI coding agents.
+Shadowclone learns engineering preferences from coding-agent sessions you choose to share with it. It turns explicit guidance and corrections into an editable profile, and keeps a portable personal skill library in sync. Claude Code, Codex, Cursor, and Antigravity can receive that profile in ordinary sessions. Supported delegated runs use the same guidance.
 
-Shadowclone reads historical coding sessions already on your disk (Claude Code, Codex, Cursor), mines your steering habits without sending raw transcripts to any server, and compiles an editable profile into the agents you already use.
+The profile adds context to an existing model; it does not train a new one or guarantee that every instruction will be followed. With separate learning consent, useful sessions can update the guidance for later work. [Why I built it](docs/motivation.md).
 
-Developers who install the profile see a measurable action delta against unprofiled runs on their held-out corpus: the agent stops asking for confirmations you never gave, runs the tests you run, and avoids the tools you refuse.
+## Privacy comes first
 
-Verify the delta directly on your own machine:
+Every source has its own setting, defaults off, and appears by name before consent. The default setup asks one grouped question for detected session sources, Git remote names, and existing agent context. It asks separately about skills and background learning. `init --advanced` offers individual source choices. The paths that may be read are:
 
-```bash
-shadowclone eval --sessions 10
-```
-
-## Architectural scope: what shadowclone is and is not
-
-**Not an agent runtime.**
-Shadowclone does not provide an LLM chat loop, an autonomous worker daemon, or an IDE extension. It compiles behavioral profiles and subagents for the agent CLIs you already install, authenticate, and pay for.
-
-**A memory and alignment compiler.**
-Every turn where you interrupted an agent, refused a tool, corrected a proposal, or chose one implementation over another is an alignment signal. Shadowclone indexes those moments locally and distills them into plain markdown rules scoped by repository origin.
-
-## The pipeline
-
-```
-observe  ->  index  ->  signal  ->  distill  ->  profile  ->  dispatch / eval
-                          |                        |
-                     zero tokens          user's subscription
-```
-
-| Stage | Module | Function |
-| --- | --- | --- |
-| observe | `src/observe/` | Normalizes agent transcripts into one incremental event stream |
-| index | `src/index/` | Rebuildable SQLite cache of byte offsets and event kinds, never text |
-| signal | `src/signal/` | Detects interruptions, plan changes, and tool refusals in pure code |
-| distill | `src/distill/` | Distills high-signal moments into rules via your installed agent CLI |
-| profile | `src/profile/` | Plain markdown rules and subagents scoped to git origins |
-| dispatch | `src/dispatch/` | Executes unattended tasks on isolated worktrees with receipts |
-| eval | `src/eval/` | Replays historical prompts through baseline vs clone to score behavioral deltas |
-
-Model calls run through `claude`, `codex`, or `cursor-agent`. There is no shadowclone API key, no telemetry, and no hosted server.
-
-## Quickstart
-
-Install the global CLI:
-
-```bash
-npm i -g @shadowclone/cli
-```
-
-Verify your environment and supported provider CLIs:
-
-```bash
-shadowclone doctor
-```
-
-Grant consent for desired transcript sources:
-
-```bash
-shadowclone init
-```
-
-Index your historical sessions and build your profile:
+| Source | Path |
+| --- | --- |
+| Claude Code sessions | `~/.claude/projects/` |
+| Claude prompt history | `~/.claude/history.jsonl` |
+| Codex sessions | `~/.codex/sessions/` or `$CODEX_HOME/sessions/` when set |
+| Cursor chats | `~/.cursor/chats/` |
+| Antigravity logs | `~/.gemini/antigravity-cli/brain/` |
+| Shell history | `~/.zsh_history`, `~/.bash_history` |
+| Repository guidance | root `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, and direct `SKILL.md` files under `.claude/skills/` and `.agents/skills/` |
+| Agent context | `~/.claude/CLAUDE.md`, `~/.codex/AGENTS.md`, `~/.codex/AGENTS.override.md`, supported personal and repository skill roots, and `~/.claude/projects/<repo>/memory/` or `~/.codex/memories/`, only for consented evaluation |
+| Skill library | `~/.claude/skills/`, `~/.agents/skills/`, `~/.codex/skills/`, `~/.cursor/skills/`, `~/.gemini/config/skills/`, configured repository roots, and selected provider plugin caches |
+| Git metadata | repository Git remote names |
 
-```bash
-shadowclone learn
-```
+Source detection checks whether a configured root has content. A directory check reads at most one entry to establish that fact and retains no name before consent. Agent transcripts can contain private code, credentials, internal hosts, and customer data. Shadowclone never copies raw transcripts into its store. Its SQLite index holds pointers and event kinds. Eligible excerpts pass through `resolveRedacted`, the single secret-redaction gate, before a model receives them. Tool results, file-read contents, thinking blocks, and data-access results are excluded from learning by category.
 
-To preview without writing files or databases:
+Learning sends eligible redacted excerpts through your own authenticated agent CLI. Evaluation also gives that CLI access to a disposable repository snapshot and sends generated code to judges without redaction. Use only repositories you are authorized to send to that provider. A remote action from `shadowclone run` needs separate approval for that run. There is no Shadowclone service, API key, account, or telemetry. The profile is Markdown under `~/.shadowclone/profile/`; you can read, edit, or delete it. `shadowclone forget --all` removes Shadowclone's local state and recorded integrations, preserving unrelated content and stopping on conflicting edits. Your original transcripts remain where your agents wrote them. [Privacy design](docs/architecture/05-privacy.md).
 
-```bash
-shadowclone learn --dry-run
-```
-
-To enable deep distillation through your authenticated agent CLI:
+## Start from this checkout
 
 ```bash
-shadowclone learn --deep
+bun install
+bun run cli init
 ```
 
-Install the compiled profile into the current repository:
+The package release follows the evaluation and packaging gates. The commands below use `shadowclone`, the installed binary name; from this checkout, use `bun run cli` in its place.
 
-```bash
-shadowclone install
-```
-
-This writes `.claude/agents/shadowclone.md` and excludes it from git tracking.
-
-## Replay evaluation
-
-Shadowclone provides a reproducible fitness function to measure profile impact:
-
-```bash
-shadowclone eval --sessions 5 --max-budget-usd 0.50
-```
+`init` shows the detected agents and source paths, then asks three questions, each defaulting to yes:
 
-The evaluator replays historical user prompts through two isolated runs:
-1. **Baseline run:** unprofiled agent invocation without system prompt customization.
-2. **Clone run:** agent invocation with the compiled project profile injected.
+1. Learn how you work from these sessions?
+2. Keep your skills in sync across the detected agents?
+3. Keep improving in the background as you work?
 
-Replays are compared against historical developer actions across four dimensions:
-- **Tools:** Jaccard similarity of invoked tools.
-- **Verification:** Jaccard similarity of two-token bash verification commands (e.g. `bun test`, `cargo check`).
-- **Files:** Jaccard similarity of posix repository-relative edited paths.
-- **Planning:** Match on whether planning tools were invoked before the first file edit.
+Only detected transcript sources are enabled by the first answer. Setup then imports supported repository guidance if present, synchronizes consented skills, runs a first learning pass when background learning is enabled, and installs global native guidance for detected agents. The first pass selects recent unprocessed steering and limits model calls to 12 or 90 seconds; scanning local transcripts can take longer. Reaching a model budget does not block setup. Later sessions continue bounded learning when the agent marks a useful session. No extra `learn` command is required to finish setup.
 
-Evaluation receipts are written to `~/.shadowclone/eval/<evalId>.json`.
+Use `shadowclone init --advanced` for individual source choices and the seed guidance wizard. Run `shadowclone wizard` later to change seed preferences and starter skills. Run `shadowclone skills configure --global` or `--repo` to configure skill roots separately. Managed policy may limit any setting.
 
-## Unattended dispatch
+## How it works
 
-Execute tasks in an isolated git worktree without touching your working tree:
-
-```bash
-shadowclone run "fix the flaky test in src/auth.test.ts"
+```text
+enabled transcripts -> local index -> steering episodes -> redacted reconciliation
+                                                        -> editable profile
+profile + personal skills -> native agents and clones -> new transcripts
 ```
 
-The default dispatch mode creates a local worktree and branch, runs verification checks, and commits locally without pushing.
-
-Remote actions (push, open PR) require both a repository ceiling in `~/.shadowclone/config.toml` and an explicit per-run approval flag:
-
-```bash
-shadowclone run "prepare release notes" --approve push
-```
+Plain `shadowclone learn` updates the local index and prints a structural report without model calls or profile writes. `shadowclone learn --deep` works through the oldest unprocessed episodes in bounded batches, proposes changes, and asks before applying them. `--deep --apply` accepts that local write without a prompt. Explicit reusable guidance can become active from one session; inferred behavior needs three independent sessions. The processed ledger lets later runs continue through older history. `shadowclone remember --repo "Use complete variable names."` records a direct preference immediately.
 
-## Ground-truth privacy
+The first setup pass starts at the newest unprocessed episodes so the next session can benefit quickly. Reconciliation batches run concurrently, then apply results in their original order. Completed checkpoints let a later run reuse finished model work. All model calls use `claude`, `codex`, or `cursor-agent` under your existing authentication.
 
-Agent transcripts contain private code, environment variables, internal hosts, and customer data. Shadowclone protects data through structural guarantees:
+Repository imports preserve your edits and rejected rules. The compiler selects global guidance, the matching Git remote owner, and the exact repository. Without Git metadata consent, working directories stay isolated. It caps compiled guidance at 16 KiB and drops whole blocks when needed.
 
-**Pointers instead of text copies.**
-The SQLite index stores file offsets, timestamps, and event kinds. Raw transcripts are never duplicated to a secondary store.
+Native installation adds a small pointer and lifecycle hooks while preserving other instructions. Claude Code's `SessionStart` hook injects the current scoped profile into the main agent. Its `SubagentStart` hook gives spawned subagents that same profile without creating another learning request. Codex, Cursor, and Antigravity use their supported native delivery paths. A useful main-agent session can request bounded learning at session end. `shadowclone install --subagent` and `--auto-delegate` remain optional Claude repository modes.
 
-**Sliced secret redaction.**
-Distillation excerpts pass through a deterministic sliced replacer before reaching any model. Secrets keep identifying prefixes (such as `AKIA` or `sk_live_`) while stripping high-entropy characters, keeping code context intact without leaking credentials.
+Portable skills live in `~/.agents/skills` and sync to supported provider locations. A single edited copy becomes the source for the next sync; divergent edits are reported as conflicts. Existing user skills and plugin packages are preserved. Skill assessment reads only enabled roots, sends redacted `SKILL.md` content through the chosen agent CLI, and leaves technical or routing changes for review. [Skill maintenance](docs/skill-maintenance.md).
 
-**Shannon entropy layer.**
-Unstructured tokens exceeding 4.5 bits of entropy per character are scrubbed even if they do not match known vendor regex patterns.
+## Check the result
 
-**Third-party tool results are excluded.**
-Distillation inputs allowlist user prompts and developer steering corrections. Tool outputs from database queries, log dumps, and file reads are excluded by category rather than relying on regex filtering.
+`shadowclone doctor` reports installation and engine status. `shadowclone context` prints the compiled profile for the current scope. `shadowclone history` shows local revisions, and `shadowclone undo <revision-id>` restores one when no later edit conflicts.
 
-**Single-command wipe.**
-Wipe the entire local index, profile, checkpoints, and receipts:
+Transfer evaluation compares three matched setups: repository guidance only, that guidance plus the existing personal skill/context library, and both plus the Shadowclone profile. It freezes a source-backed coding-preference rubric before execution and saves three blinded votes per check. Correctness is graded separately. Missing grades are reported as ungraded, and a completed evaluation does not require Clone to win. [Evaluation design](docs/architecture/09-evaluation.md).
 
 ```bash
-shadowclone forget --all
+shadowclone eval --repo /path/to/repository --engine codex --model gpt-5.6-sol --reasoning-effort medium --tasks 3 --repeat 2
 ```
 
-## Enterprise governance
-
-Security teams can enforce policy ceilings fleet-wide via root-owned managed configuration:
-- **macOS:** `/Library/Application Support/shadowclone/managed.json`
-- **Linux:** `/etc/shadowclone/managed.json`
-
-```json
-{
-  "enabled": true,
-  "allowedSources": ["claude-code"],
-  "allowedEngines": ["claude-code"],
-  "distillation": "local-only",
-  "originScope": "strict",
-  "blockedOrigins": ["github.com/acme/security-*"],
-  "maxActionTier": "draft"
-}
-```
-
-Managed policies act as an absolute ceiling. Users cannot enable unapproved sources or engines, and `enabled: false` enforces an immediate stop across the machine.
+`shadowclone run "fix the flaky test"` runs a headless clone in a local worktree and records a receipt. The invocation approves one worktree, branch, and local commit. Remote actions also need a matching repository policy ceiling and `--approve` for that run. No action approval carries to the next run. [Acting policy](docs/architecture/04-acting.md).
 
-## CLI commands
+## Early evaluation results
 
-```bash
-shadowclone init                                 # Configure source consent and capabilities
-shadowclone learn [--deep] [--dry-run]           # Index sessions and synthesize rules
-shadowclone doctor                               # Inspect active paths, engines, and policies
-shadowclone install                              # Install profile as .claude/agents/shadowclone.md
-shadowclone run <task> [--approve <action>]      # Dispatch headless clone in a worktree
-shadowclone eval [--sessions N] [--json]         # Measure behavioral deltas against baseline
-shadowclone mcp                                  # Start stdio Model Context Protocol server
-shadowclone forget --all                         # Remove ~/.shadowclone/ completely
-```
+Across four small TypeScript tasks using GPT-5.6 Sol at medium reasoning effort, the recorded preference scores were:
 
-## Contributing
+| Repository only (Bare) | Personal skills/context (Skills) | Skills/context + profile (Clone) |
+| ---: | ---: | ---: |
+| 55/68 (80.9%) | 59/68 (86.8%) | 62/68 (91.2%) |
 
-Review `CONTRIBUTING.md` and `SECURITY.md`. All contributions must pass:
+These are guideline checks passed, not correctness or productivity scores. The skills were existing user-written or user-guided instructions, potentially outdated and not tuned for this comparison. This was one implementation per arm per task, with known judging limitations. Read [the tasks, setup, results, and caveats](evals.md) before drawing broader conclusions.
 
-```bash
-bun run check
-```
+## Commands
 
-## License
+| Command | Use |
+| --- | --- |
+| `shadowclone init [--advanced]` | Set consent and install detected integrations |
+| `shadowclone import`, `shadowclone wizard` | Refresh repository guidance or seed choices |
+| `shadowclone learn [--deep] [--dry-run] [--apply]` | Inspect history or reconcile guidance |
+| `shadowclone skills`, `shadowclone skills update` | List starter guidance or maintain consented skills |
+| `shadowclone skills pending`, `show`, `apply`, `reject` | Review skill proposals |
+| `shadowclone learning enable|disable|status` | Control session learning |
+| `shadowclone doctor`, `shadowclone context`, `shadowclone sync` | Inspect or refresh delivery |
+| `shadowclone install [--agent <agent>|all] [--global|--repo]` | Install native guidance manually |
+| `shadowclone uninstall [--global]` | Remove owned integrations |
+| `shadowclone remember`, `history`, `undo` | Manage direct rules and revisions |
+| `shadowclone eval`, `shadowclone run <task>` | Measure transfer or run a local clone |
+| `shadowclone forget --all` | Remove all recorded Shadowclone state |
 
-MIT. See `LICENSE`.
+Contributors can run `bun install` and `bun run check`. See [the architecture](docs/architecture/README.md), [design records](docs/design/README.md), [contribution guide](CONTRIBUTING.md), and [security policy](SECURITY.md). Shadowclone is MIT licensed.