@iowarp/clio-coder 0.3.0

package/docs/extensions-and-sharing.md

@@ -0,0 +1,306 @@
+# Extensions, Prompt Templates, Skills, and Share Archives
+
+> [!TIP]
+> **Interactive Spec Available:** An interactive dashboard is located at [docs/html/extensions_blueprint.html](html/extensions_blueprint.html) (Version: 0.3.0).
+
+Clio Coder has lightweight community-oriented resource packaging. Extensions are filesystem bundles that contribute prompts, skills, and future theme resources. Share archives are portable JSON files for moving project/user Clio resources between machines or collaborators.
+
+Source of truth: `src/domains/extensions/**`, `src/domains/resources/**`, `src/domains/share/**`, `src/cli/extensions.ts`, and `src/cli/share.ts`.
+
+---
+
+## Resource roots and precedence
+
+Prompts and skills are loaded from package, user, and project roots. Higher-ranked roots override lower-ranked resources with the same name.
+
+Prompts use the three-tier precedence:
+
+| Rank | Scope | Root |
+| --- | --- | --- |
+| 0 | package | enabled extension resource roots |
+| 1 | user | `<configDir>/prompts` |
+| 2 | project | `.clio-coder/prompts` |
+| 3 | cli | reserved for call-site injected resources |
+
+Skills add Agent Skills compatibility roots so that skills installed by other agents are usable without copying. The skill precedence, lowest to highest, is:
+
+| Precedence | Scope | Source | Root |
+| --- | --- | --- | --- |
+| 10 | package | extension | enabled extension resource roots |
+| 20 | user | agents / claude / codex / copilot / opencode | `~/.agents/skills`, `~/.claude/skills`, `~/.codex/skills`, `~/.copilot/skills`, `~/.config/opencode/skills` |
+| 30 | user | clio | `<configDir>/skills` |
+| 40 | project | agents / claude / codex / copilot / opencode | `.agents/skills`, `.claude/skills`, `.codex/skills`, `.github/skills`, `.opencode/skills` (untrusted by default) |
+| 50 | project | clio | `.clio-coder/skills` |
+| 60 | cli | path | reserved for call-site injected resources |
+
+Clio-native roots intentionally outrank shared compatibility roots at the same scope, so `.clio-coder/skills` overrides a project `.codex/skills` skill of the same name, and `<configDir>/skills` overrides `~/.agents/skills`. If multiple compatibility roots contain the same skill name at the same precedence, Clio resolves the collision deterministically by file path and records a diagnostic. If two roots resolve to the same canonical `SKILL.md` through a symlink, Clio keeps the higher-precedence entry and records a diagnostic.
+
+---
+
+## Prompt templates
+
+Prompt templates are Markdown files under a prompt root. Filename is the command name.
+
+Example `.clio-coder/prompts/bugfix.md`:
+
+```md
+---
+description: Focused bug-fix prompt
+argument-hint: "<file> <symptom>"
+---
+
+Investigate {{1}} for this symptom: {{2}}
+
+Return:
+1. likely root cause;
+2. minimal patch plan;
+3. validation commands.
+```
+
+Use in the TUI:
+
+```text
+/prompts
+/bugfix src/parser.ts empty input crashes
+```
+
+Templates without frontmatter are accepted; Clio derives a fallback description from the first non-empty line. Invalid frontmatter degrades to a warning for prompt templates rather than failing the whole load.
+
+---
+
+## Skills
+
+Skills follow the Agent Skills `SKILL.md` format. A skill is a directory containing `SKILL.md`, or a single Markdown file under a skill root. YAML frontmatter is required and must include a `description`. A missing description is the only hard rejection; every other validation issue degrades to a warning and the skill still loads.
+
+Example `.clio-coder/skills/hdf5-review/SKILL.md`:
+
+```md
+---
+name: hdf5-review
+description: Review HDF5/NetCDF validation logic and output assumptions.
+license: MIT
+allowed-tools:
+  - Read
+  - Grep
+---
+
+When asked to review scientific array output:
+- identify expected dimensions and attributes;
+- ask for validation data when absent;
+- prefer deterministic scripts over visual inspection;
+- cite files and commands used.
+```
+
+Use in the TUI:
+
+```text
+/skill
+/skill:hdf5-review review the output validation path
+```
+
+`/skill` opens the Skills Hub with discovered project skills, user skills, and marketplace entries. `/skill:name args` submits `args` with a pending skill request; the model must call `context` (scope="skills") for that skill before following the workflow. The same pending-request path runs in headless mode, so `clio-coder run "/skill:name args"` matches the interactive behavior.
+
+Every activation records a session ledger entry with the skill name, file path, hash, source, trigger (`slash-command` or `tool`), and turn id when one is available. The same ledger is mirrored into session metadata, prompt diagnostics, and run receipts. Compaction keeps the newest active skill turn in the retained suffix so a loaded skill is not silently summarized away.
+
+### Naming and validation
+
+The canonical invocation name is the frontmatter `name` when present, otherwise the directory or file subject. When `name` differs from the path subject Clio records a warning and keeps the frontmatter name, which lets shared cross-agent skill folders load without renaming. Names should use lowercase letters, numbers, and single hyphens; format violations warn but do not block loading.
+
+Recognized frontmatter fields:
+
+- `name`, `description`: core identity.
+- `disable-model-invocation: true`: hides the skill from the model-visible catalog while keeping it loadable by `/skill:name`.
+- `allowed-tools`, `disallowed-tools`: parsed as tool policy fields for the loaded skill workflow.
+- `license`, `version`, `compatibility`, and other non-core keys: captured as skill metadata and surfaced when the skill loads through `context`.
+- `source-url`, `registry-id`, `installed-at`, `updated-at`, `audit`: captured as install provenance when present.
+
+### Trust and compatibility roots
+
+Shared user roots are model-visible by default, like the Clio user root. Project-local compatibility roots are discovered but **untrusted by default**: they appear in `/skill` with an `untrusted` marker, but they are excluded from the model-visible catalog and cannot be loaded through `context`. This prevents an unreviewed project checkout from injecting skills the model will act on.
+
+Opt in to model-visible project compatibility roots by setting `skills.trustProjectCompatRoots: true` in `settings.yaml`. `CLIO_CODER_TRUST_PROJECT_SKILLS=1` remains an environment override. `.clio-coder/skills` is always trusted as the Clio-native project root.
+
+### Loading with context, writing directly
+
+`context(scope="skills")` lists model-visible skills when called with no `name`, or loads a pending skill body by `name`. It returns structured metadata (`name`, `description`, `path`, `base_dir`, `hash`, `source`, `scope`, `disable_model_invocation`, parsed tool policy fields, diagnostics, and frontmatter metadata) plus the body. Pass `include_tree: true` to list sibling files under the skill base directory, capped internally at 50 entries. The skills scope never executes bundled scripts and only resolves skills the model is allowed to see.
+
+Creating a skill is writing a `SKILL.md` file with the ordinary write tool: `.clio-coder/skills/<name>/SKILL.md` for project scope, or the Clio config skills directory for user scope. The loader validates frontmatter on load (`clio-coder skills validate` reports diagnostics), and the `skill-craft` shipped skill documents the frontmatter contract and craft rules.
+
+### Skills CLI
+
+```bash
+clio-coder skills list [--json] [--all]
+clio-coder skills search <query> [--json]
+clio-coder skills inspect <name> [--json]
+clio-coder skills validate [path] [--json]
+clio-coder skills install <name|path|github-url> [--user|--project] [--name <name>] [--force]
+clio-coder skills update <name> | --all [--force]
+clio-coder skills sync [--force]
+clio-coder skills eval <name|path> [--scenario <id>] [--target <id>] [--workspace <path>] [--timeout <seconds>] [--trust-fixtures] [--allow-network] [--json]
+```
+
+`eval` (experimental) executes a skill's `evals.md` RED-GREEN scenarios with
+baseline, treatment, and judge runs; see
+[skills-marketplace.md](skills-marketplace.md) for the catalog contract it
+verifies. Fixture commands in an `evals.md` are real shell and only run with
+`--trust-fixtures`.
+
+Every arm runs hermetic in a disposable workspace at autonomy `full-auto`: the network tool plane is stripped from child runs so a scenario measures the skill against its workspace and not against the open web. `--allow-network` keeps the web tools, and the run reports which network policy was in force. The per-arm execution timeout is set with `--timeout <seconds>`.
+
+Exit code is 1 when a treatment bullet fails. Exit code is 3 when a scenario goes unmeasured, such as when judge output is truncated, missing, or unparseable, or when a run dies at a permission wall. Permission-wall deaths and harness infrastructure failures are classified as unmeasured infrastructure errors rather than negative verdicts on the skill.
+
+
+Headless runs also accept `--no-skills` to disable discovery and repeatable `--skill <path>` to load one explicit `SKILL.md` file or skill directory for that run. Explicit `--skill` paths are honored even when `--no-skills` is set.
+
+### Agent Skills compatibility
+
+Clio is local-first. Skills run from disk and no chat turn depends on network access. Because the compatibility roots above use the standard `SKILL.md` shape, skills installed by the Skills.sh CLI for other agents are usable directly:
+
+```text
+npx skills add <skill> -a codex   # installs into ~/.codex/skills
+```
+
+Clio does not call Skills.sh during startup or prompt assembly, and does not emit its own telemetry. If you run `npx skills`, its telemetry follows that CLI and can be disabled with `DISABLE_TELEMETRY=1`. Skills.sh remote search and audit are not enabled in this release. Clio does support local marketplace search plus `clio-coder skills install <name|path|github-url>`: bare names resolve through the local marketplace, and explicit local paths or GitHub URLs install directly.
+
+### Prompt envelope and safety
+
+Skill bodies never enter the prompt uninvited. The model discovers skills only through `context(scope="skills")`: a call with no `name` returns a one-line listing (name, scope, description) of model-visible skills, and a body loads only when the pending-skill policy authorizes that name for the turn, which requires an explicit operator invocation such as `/skill:<name>`. Skills are prompt resources, not execution grants: any script a skill references still runs through normal Clio tools and safety gates, and a loaded skill's `allowed-tools` declaration narrows the tool surface at admission (reason code `skill_surface`) without ever granting anything the host would refuse.
+
+---
+
+## Extension package manifest
+
+An extension root contains `clio-coder-extension.yaml`, `clio-coder-extension.yml`, or `clio-coder-extension.json`.
+
+```yaml
+manifestVersion: 1
+id: lab-pack
+name: Lab Pack
+version: 1.0.0
+description: Prompts and skills for this lab
+resources:
+  prompts: prompts
+  skills: skills
+  themes: themes
+compatibility:
+  clio: ">=0.2.0"
+```
+
+Required fields are `manifestVersion: 1`, `id`, `version`, and `description`. `name` defaults to `id` when absent. The current resource kinds are `prompts`, `skills`, and `themes`; theme loading is reserved and currently returns an empty list in the resource loader.
+
+IDs must be lowercase and may include numbers, dots, underscores, and hyphens; they must start/end alphanumeric.
+
+---
+
+## Extension CLI
+
+```bash
+clio-coder extensions list [--all] [--json] [--user|--project]
+clio-coder extensions discover <path> [--json]
+clio-coder extensions install <path> [--user|--project] [--force] [--json]
+clio-coder extensions enable <id> [--user|--project] [--json]
+clio-coder extensions disable <id> [--user|--project] [--json]
+clio-coder extensions remove <id> [--user|--project] [--json]
+```
+
+Install locations:
+
+| Scope | Root |
+| --- | --- |
+| user | `<configDir>/extensions/<id>` |
+| project | `.clio-coder/extensions/<id>` |
+
+Project extensions shadow user extensions with the same ID. Use `--all` to list shadowed/disabled entries.
+
+### Skill pack distribution
+
+Clio Coder should not grow built-in skills in the harness. Distribute reusable Clio skills as extension packages instead. A future `iowarp/clio-kit` bundle can carry `clio-coder-extension.yaml` plus a `skills/` directory, and users can install it with `clio-coder extensions install <path> --user` or `--project`.
+
+Recommended layout:
+
+```text
+clio-kit/
+  clio-coder-extension.yaml
+  skills/
+    hpc-review/
+      SKILL.md
+      references/
+      scripts/
+    release-check/
+      SKILL.md
+```
+
+This keeps the runtime local-first and small. Clio Coder discovers enabled extension skill roots, records provenance as `source: extension`, and still requires normal tool safety gates for any script a skill asks the agent to run.
+
+---
+
+## Share archives
+
+Share archives are single JSON files:
+
+```json
+{
+  "kind": "clio-share-archive",
+  "formatVersion": 1,
+  "manifest": {
+    "format": "clio.share.v1",
+    "clioVersion": "0.3.0",
+    "createdAt": "...",
+    "files": []
+  },
+  "files": []
+}
+```
+
+Every file entry is base64 encoded and SHA-256 checked on import.
+
+### Export
+
+```bash
+clio-coder share export --out project.clio-coder-share.json --project
+clio-coder share export --out all.clio-coder-share.json --both --all
+```
+
+Options:
+
+| Flag | Meaning |
+| --- | --- |
+| `--project` | Export project resources only. Default scope. |
+| `--user` | Export user resources only. |
+| `--both` | Export both user and project resources. |
+| `--context` | Include project context files (`CLIO-CODER.md`, `AGENTS.md`, `CODEX.md`, `GEMINI.md`, `CLAUDE.md`). |
+| `--prompts` | Include prompt templates. |
+| `--skills` | Include skills. |
+| `--settings` | Include non-secret settings fragment. |
+| `--extensions` | Include extension bundle files, excluding extension `state.json`. |
+| `--all` | Include every supported resource class. |
+
+If no include flags are supplied, export includes all supported classes for the selected scope.
+
+Settings fragments include non-secret UI/runtime preferences such as `autonomy`, `scope`, `budget`, `theme`, `terminal`, `keybindings`, `compaction`, and `retry`. Targets and credentials are not included.
+
+### Import and inspect
+
+```bash
+clio-coder share inspect project.clio-coder-share.json
+clio-coder share import project.clio-coder-share.json --dry-run
+clio-coder share import project.clio-coder-share.json --force
+```
+
+Dry-run imports produce a plan and report conflicts without writing. Without `--force`, conflicting destination files block writes. With `--force`, conflicting files are overwritten and supported settings-fragment keys are merged into the current settings file.
+
+Aliases:
+
+```bash
+clio-coder export --out project.clio-coder-share.json
+clio-coder import project.clio-coder-share.json --dry-run
+```
+
+---
+
+## Community packaging guidance
+
+- Keep extension packages small and reviewable.
+- Treat prompts and skills as source code: document assumptions, expected evidence, and validation commands.
+- Do not put secrets in extension packages or share archives.
+- Prefer project-scoped resources for repository-specific instructions and user-scoped resources for personal workflow helpers.

package/docs/fleet-demo-runbook.md

@@ -0,0 +1,179 @@
+# Fleet Demo Runbook
+
+A repeatable multi-node demonstration: one orchestrator drives a real
+CMake/C++ fix through a reviewer-gated dispatch across SSH nodes, and every
+worker's receipt (including the remote ones) verifies afterward. The steps
+are executable in order; this document doubles as the recording script.
+Background and reference: [fleet-dispatch.md](fleet-dispatch.md).
+
+## Reference fabric
+
+| Role | Machine | Notes |
+| --- | --- | --- |
+| Orchestrator | `zbook` | Runs the interactive Clio session. |
+| SSH node | `blade` | General worker capacity. |
+| SSH node | `mini` | Serves the operator's resident models on its GPU; residency stays observe. |
+| SSH node | `dragon` | General worker capacity. |
+
+All four machines share the filesystem, so the project root resolves to the
+same absolute path everywhere. The demo project is any CMake/C++ repository
+with a known failing build or test; a one-line compile error in a `.cpp` file
+works well on camera.
+
+## 1. Declare the fleet (zbook)
+
+Add the nodes to `settings.yaml` (see `clio-coder paths` for its location):
+
+```yaml
+fleet:
+  nodes:
+    - id: blade
+      host: blade
+      maxWorkers: 2
+    - id: mini
+      host: mini
+      maxWorkers: 1
+      residency: observe
+    - id: dragon
+      host: dragon
+      maxWorkers: 2
+```
+
+The implicit `local` node (zbook itself) is never declared. `residency:
+observe` is the default and is written here only to make the demo point
+explicit: workers on mini must never evict its resident models.
+
+## 2. Preflight the nodes
+
+```
+clio-coder doctor
+```
+
+Doctor probes each node over its real SSH channel: reachability, a
+version-matched `clio-coder` on the remote path, path parity for the project root,
+and a writable remote state dir. Passing nodes become dispatch-eligible;
+failures are warnings that name the fix. Re-run doctor after any host,
+project, or version change; admission fails closed on a stale preflight.
+
+Confirm the durable view:
+
+```
+clio-coder fleet status
+```
+
+## 3. Open the session and the fleet views
+
+```
+clio-coder
+```
+
+In the session:
+
+- `/fleet` opens the fleet overlay; Tab cycles status, nodes, profiles, and
+  bindings. The nodes tab shows blade, mini, and dragon online with their
+  capacity.
+- Alt+W toggles the dispatch board, which will fill with per-run cards once
+  work starts.
+
+## 4. Dispatch the gated fix across nodes
+
+Give the orchestrator a concrete instruction that names the topology and the
+placement. Example prompt for the chat input:
+
+```
+Dispatch the fix for the failing CMake build as a reviewed task: builder on
+node blade, reviewer on node dragon, at most 2 review cycles. The task is:
+"Fix the compile error in src/mesh/loader.cpp so `cmake --build build` and
+`ctest --test-dir build` both pass. Keep the change minimal."
+```
+
+The model calls the dispatch tool with `review: {max_cycles: 2, node:
+"dragon"}` and `node: "blade"` on the task. Because a remote placement is
+plan-scale, supervised autonomy parks the call and shows the plan artifact
+(topology, per-task agent, model, node); one approval launches the whole
+plan. Full-auto skips the stop and seals the plan hash into the receipts
+instead.
+
+What to watch:
+
+- The board card for the builder shows `node blade`, live tool activity, and
+  the per-worker context meter.
+- The reviewer card shows `node dragon` and `gate reviewer c1`; the reviewer
+  runs read-only and ends with a `VERDICT:` line.
+- On a revise verdict, a second builder card appears with `gate builder c2`;
+  the reviewer's findings were threaded to it as input data.
+- `/fleet` status rows carry the node column for both runs.
+
+If a node dies mid-run (for the demo: stop sshd on blade), the run finalizes
+as stalled, the node is classified dead after consecutive channel failures,
+and the bounded retry reroutes to a survivor with the hop recorded on the new
+receipt.
+
+## 5. Collect the results
+
+The dispatch tool returns the gate verdict, the run ids, and the receipt
+paths. The monitor tool answers follow-ups inside the session (`mode=list`,
+`mode=status`, `mode=receipt`). For asynchronous work the same flow applies
+with `detach: true` plus `monitor mode="collect"`; the demo keeps the gate
+attached so the verdict lands in one message.
+
+Verify the fix like any local change:
+
+```
+cmake --build build && ctest --test-dir build
+```
+
+## 6. Verify every receipt, including the remote ones
+
+```
+clio-coder fleet status --json
+clio-coder evidence build --run <builderRunId>
+clio-coder evidence build --run <reviewerRunId>
+clio-coder evidence list
+clio-coder evidence inspect <evidenceId>
+```
+
+`clio-coder evidence build` recomputes the receipt's integrity digest against the
+run ledger; a tampered or mismatched receipt fails the build with the field
+that diverged. The receipts of the remote runs verify on zbook because the
+ledger and receipts live on the shared filesystem. Current receipts use strict
+v15 and authenticate every current receipt and reconstructed-ledger field.
+Every other receipt version is rejected rather than reported as partial; the
+current binary has no historical receipt reader.
+
+## Provenance walkthrough: what a PI can verify from receipts alone
+
+Each run's receipt is a JSON file under `<state>/receipts/<runId>.json`
+(`clio-coder paths` shows the state dir; the monitor tool prints the exact path per
+run). From the receipts alone, with no session transcript, a PI can
+reconstruct:
+
+1. What ran and where. `agentId`, `task`, `targetId`, `wireModelId`,
+   `runtimeKind`, and `node` name the agent, model, and machine. `identity`
+   anchors the host, user, and any HPC scheduler allocation. `reroutes` lists
+   every dead-node failover hop the run survived.
+2. Against which code. `reproducibility.git` records branch, commit, dirty
+   state, and a status hash of the working tree at run start; `cwd` is the
+   workspace.
+3. Under which authority. `autonomyEnforcement` seals the autonomy level and
+   how the runtime enforced it. `safety` counts allowed, blocked, and
+   permission-requested tool calls and lists blocked attempts.
+   `plan` proves the dispatch was operator-approved (or full-auto logged)
+   and hashes the exact plan artifact.
+4. Through which gate. `gate` on the reviewer receipt references the builder
+   run id and its receipt digest; a revise builder references the reviewer
+   that sent it back, with the verdict. Following `gate.subjects` digests
+   backward reconstructs the whole review chain, and any edit to an earlier
+   receipt breaks the digest the later one recorded.
+5. That nothing was altered. The `integrity` block is a sha256 over the
+   complete receipt schema and its stable ledger row. `clio-coder evidence build
+   --run <id>` recomputes and cross-checks it; `verifyReceiptIntegrity` in
+   `src/domains/dispatch/receipt-integrity.ts` is the reference
+	implementation. Current receipts use v15 and every other version fails
+	verification. Incompatible state must be archived or removed; it is never
+	read as evidence through a compatibility verifier.
+
+The walkthrough for an audience is three commands: `clio-coder evidence build
+--run <id>` (it verifies), open the receipt JSON (read `node`, `gate`,
+`plan`, `reproducibility.git`), and `clio-coder evidence build` again after
+hand-editing one byte of the receipt (it refuses, naming the mismatch).