dsh-evokit 0.0.0-stage → 0.1.5

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jiahao
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,68 @@
-# Temporary Holding Version
+# evokit for DeepSeek Harness
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The native DSH bundle adds evokit's stdio MCP tools and the `onboard`, `evolve`
+and `meta` skills. Install Python and the wheel first. Compatibility is pinned to
+DSH `0.2.2-rc.1`, Node 24 and DSH's pnpm prerequisite. Local runtime acceptance
+uses Windows. Other platforms remain unverified.
+
+## Install in an isolated profile
+
+Download and verify `dsh_evokit-0.1.5-py3-none-any.whl`,
+`dsh-evokit-0.1.5.tgz` and `SHA256SUMS` from the planned
+[release page](https://github.com/edsml-jz2321/dsh-evokit/releases).
+From PowerShell, with an existing scenario at `./my-system`:
+
+```powershell
+uv tool install ./dsh_evokit-0.1.5-py3-none-any.whl
+$env:EVOKIT_MCP_COMMAND = (Get-Command evokit).Source
+$env:EVOKIT_WORKSPACE = (Resolve-Path ./my-system).Path
+$env:DSH_HOME = Join-Path (Get-Location) 'private-dsh/home'
+$env:DSH_AGENTS_HOME = Join-Path (Get-Location) 'private-dsh/agents'
+$env:HOME = $env:DSH_HOME
+$env:USERPROFILE = $env:DSH_HOME
+$env:DSH_TELEMETRY_DISABLED = '1'
+$env:DSH_TELEMETRY_MODE = 'DISABLED'
+dsh plugin --profile evokit add ./dsh-evokit-0.1.5.tgz
+$env:EVOKIT_SKILL_ROOT = Join-Path $env:DSH_HOME 'profiles/evokit/node_modules/dsh-evokit/skills'
+dsh plugin --profile evokit list
+dsh --profile evokit --dump-config
+```
+
+Keep these variables for each launch. On other shells, set the same absolute
+paths using the shell's environment syntax. The bundle does not install Python.
+Package listing checks installation; `--dump-config` checks configuration without
+starting an agent or making model requests.
+
+## Configure your improver
+
+Use your own provider key in DSH's local credential settings or a process-local
+environment variable. Set the provider URL and model in your private DSH profile;
+do not commit credentials. No particular gateway is required. For optional
+file-key accounting, adapt [the relay example](../relay/README.md), set
+`EVOKIT_IMPROVER_BASE_URL` to its loopback `/v1` endpoint and
+`EVOKIT_RELAY_CLIENT_TOKEN` to its local client token, then apply
+`improver.patch.yml` with `--patch`. Its model policy is an example, not a
+universal provider default. Confirm model support and pricing before execution.
+
+Run the `onboard` skill with the scenario's config path. The `evolve` skill
+diagnoses and proposes; `meta` owns adoption, sealed confirmation and revert.
+The default MCP profile excludes operator decisions. Configure a separate
+`evokit mcp --profile operator` server for those decisions and protect its files
+from the improver's direct access.
+
+## Inspect tools without a model
+
+The source includes `list-tools.mjs` and `list-tools.patch.yml`. Copy the overlay
+to a private directory, replace `EVOKIT_TOOL_INSPECTOR_URI` with the module's file
+URI, and pass it with `--patch`. It disables model adapters and credential services
+and exits after comparing registered MCP tools against `evokit tools`.
+
+## Add the market after publication
+
+DSH ships no built-in plugin index. Enable **Custom plugin indexes**, open
+**Discover → Add index source**, and enter:
+`https://edsml-jz2321.github.io/dsh-evokit/index.json`.
+This URL becomes available only after the publisher deploys market/.
+The Git source pins the exported bundle subdirectory and a specific commit.
+
+MIT; see [LICENSE](LICENSE).

package/cordis.patch.yml

@@ -0,0 +1,21 @@
+- insert:
+    - id: evokit-mcp
+      name: '@deepseek-ai/dsh-mcp-client'
+      config:
+        serverName: evokit
+        transport: stdio
+        command: !!js process.env.EVOKIT_MCP_COMMAND
+        args: [mcp]
+        cwd: !!js process.env.EVOKIT_WORKSPACE
+        startupTimeoutMs: 30000
+        toolCallTimeoutMs: 300000
+        failOnStartupError: true
+        reconnect:
+          enabled: false
+    - id: evokit-skills
+      name: '@deepseek-ai/dsh-skill-filesystem'
+      config:
+        providerName: evokit
+        includeDefaultRoots: false
+        customSkillDirs: !!js "[process.env.EVOKIT_SKILL_ROOT]"
+        watch: false

package/improver.patch.yml

@@ -0,0 +1,26 @@
+# Apply alongside cordis.patch.yml. Point this provider at the accounting relay.
+# The deepseek-improver route injects thinking enabled and reasoning_effort high.
+- id: llm-pi-ai
+  disabled: false
+  config:
+    providers:
+      evokit-improver:
+        api: openai-completions
+        baseURL: !!js process.env.EVOKIT_IMPROVER_BASE_URL
+        apiKeyEnv: EVOKIT_RELAY_CLIENT_TOKEN
+        retryPolicy: {mode: normal, maxRetries: 0}
+        reasoning: high
+        compat:
+          supportsDeveloperRole: false
+          maxTokensField: max_tokens
+          thinkingFormat: deepseek
+        models:
+          - id: deepseek-improver
+            contextWindow: 128000
+            maxTokens: 32768
+            input: [text]
+            reasoningEfforts: {high: high}
+- id: agent-default-model
+  config:
+    provider: evokit-improver
+    model: deepseek-improver

package/mcp.json

@@ -0,0 +1,10 @@
+{
+  "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json",
+  "mcpServers": {
+    "evokit": {
+      "type": "stdio",
+      "command": "evokit",
+      "args": ["mcp"]
+    }
+  }
+}

package/package.json

@@ -1,6 +1,35 @@
 {
   "name": "dsh-evokit",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.5",
+  "type": "module",
+  "files": [
+    "cordis.patch.yml",
+    "improver.patch.yml",
+    "plugin.json",
+    "mcp.json",
+    "skills"
+  ],
+  "engines": {
+    "dsh": "0.2.2-rc.1"
+  },
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
+  "license": "MIT",
+  "author": "Jiahao",
+  "description": "Measured self-improvement: paired experiments, noise-aware gates and sealed confirmation.",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/edsml-jz2321/dsh-evokit.git",
+    "directory": "integrations/dsh"
+  },
+  "homepage": "https://github.com/edsml-jz2321/dsh-evokit#readme",
+  "keywords": [
+    "dsh-plugin",
+    "self-evolution",
+    "evaluation",
+    "mcp"
+  ]
+}

package/plugin.json

@@ -0,0 +1,11 @@
+{
+  "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
+  "name": "dsh-evokit",
+  "version": "0.1.5",
+  "description": "Domain-neutral self-evolution experiments, paired gates and lineage",
+  "keywords": [
+    "evolution",
+    "evaluation",
+    "experiments"
+  ]
+}

package/skills/evolve/SKILL.md

@@ -0,0 +1,101 @@
+---
+name: evolve
+description: Improve editable system artifacts through evokit proposals, paired experiments, noise-aware gates and append-only lineage. Use with an onboarded evokit scenario.
+metadata:
+  version: 0.1.4
+---
+
+# Evolve a system
+
+META decides; the improver analyzes and proposes. Use the `meta` skill for the
+epoch contract, generation plans, adoption, confirmation reverts, budget, stopping
+and the final measurement. Delegated improvers never make those decisions.
+
+Use the scenario's `config` path with every tool. `scenario_describe` explains
+editable scope, task counts, budgets and the domain primer. `version_current`
+and `ledger_history` establish the current base and earlier attempts. The host
+supplies the improver; these instructions do not require a particular agent.
+Inspect the available tools (`evokit tools --config PATH`). For an HB workspace,
+use `records_query` to compare tasks, groups, outcomes, proposal versions and
+adoption generations, including `flipped_between: [parent, child]`. Use
+`failures_similar` to find related earlier mechanisms before proposing. These
+capabilities are absent for files storage; the generation loop below works on both.
+Neither tool can query private confirmation or final records.
+
+## Generation loop
+
+1. Read the full failure index with `failure_cards` on a diagnosis run. Do not
+   reason from a few attractive examples. Cluster failures by mechanism, then
+   read original traces through their archived pointers. Keep task identities
+   and observed answers out of every editable artifact, including derived files.
+2. Read the measured selection noise threshold before proposing. State the
+   predicted net score-sum effect and threshold in the same units, and whether
+   the effect is detectable on this selection set. If noise is unmeasured, request
+   incumbent repeats first. If the effect does not exceed the threshold,
+   propose a stronger mechanism or state that more tasks or repeats are needed.
+   Write distinct, falsifiable hypotheses. Cite prior attempts only from
+   `proposal_list` / `ledger_history` records with their ids, or use null when
+   no related record exists. Each proposal names target files, a generic mechanism,
+   a falsification criterion and what differs from its cited prior. Verify that
+   the failing tasks actually load the artifact, including its language and stage.
+   Name the specific public failing tasks expected to flip and the trace changes
+   that would demonstrate the mechanism in the proposal record. Edit in a candidate checkout,
+   inspect the full diff for task, fixture, repository and expected-output literals,
+   and remove them before claiming genericity. Keep task evidence in records.
+   then use `version_create` and `proposal_create`. Each proposal pins this
+   playbook's content version. Improve the playbook as a separate level-1 artifact
+   using the same evidence discipline, preserving system/scorer anchors.
+3. Smoke each candidate with `run_launch(smoke=true)` (two tasks when available).
+   Fix execution errors before buying a full experiment. A completed result is
+   never rerun on resume: pass the original run id and exact arm parameters.
+4. Repeat the incumbent on selection with `repeats >= 2`; use `noise_estimate`.
+   Measure noise before setting thresholds. Small apparent gains can be noise.
+   Run a same-window incumbent control because providers drift over time.
+5. Launch candidate arms in parallel using distinct run ids; use task workers
+   appropriate to the host and budget. A host can pin one arm per machine when
+   orchestration supports it. Keep a launched arm on its original machine and
+   inputs. Reuse stage replay only when the scenario declares complete cache keys;
+   a changed dependency must invalidate its stage. Cached upstream cost is zero.
+6. Wait for completion via `run_status`; inspect `run_results` and original traces.
+   Use `compare_paired` on identical selection tasks and repeats. Choose `majority`
+   for binary votes (ties fail), or `mean` for scores, explicitly. Review fixes,
+   breaks, group tables, costs and latency, including failures of the hypothesis.
+   Check the predicted task flips against original traces for the stated mechanism.
+7. `gate_check` enforces anchors, public instance guards, holdout isolation, budgets and
+   a measured threshold. Attest `generic=true` only after reviewing the mechanism.
+   `adopt-no-harm` also requires a concrete rationale. Report a passing verdict to META for
+   `ledger_adopt` in the operator profile. Recommend promising runner-ups for META
+   to queue with `ledger_adopt(queue=true)` and retest them against the next base
+   through a new proposal. Earlier scores do not establish performance on a changed base.
+
+## Constraints and lessons
+
+- Diagnosis supports analysis; selection decides adoption. Confirmation (the
+  backward-compatible `holdout` role) is operator-only and decides keep-or-revert
+  of the already chosen adoption under the anchored rule. It never chooses
+  proposals. Improvers see only the not-confirmed bit, never private task ids,
+  scores, outputs or traces. META owns its wear budget (default 4 decisions).
+- Final is a separate optional split opened once, with human approval, by META
+  at the end. It decides nothing. Without final tasks, use a fresh-task epoch.
+  Split assignments, scorer and contract stay anchored within the epoch.
+- Prefer a generic mechanism over an instance patch even when scores tie. Leak
+  checks cover derived examples, memories, lookup tables and other artifacts.
+  Literal scans are a backstop, not a proof against encoded or semantic leaks.
+- Separate bug fixes from behavior changes. A defect in implementation can merge
+  after review and focused tests; a change to system behavior needs paired
+  evidence. If it changes scoring or frozen support, begin a new epoch.
+- Keep records minimal and sufficient: proposals, arms, outcomes, verdicts and
+  rationale. Do not add reservations or elaborate approval machinery.
+- Replay, caching and cheap counterfactuals multiply iteration capacity. Verify
+  cache identity and cost attribution; do not count stale results as an improvement.
+- Use `meta_metrics` to examine acceptance rate, prediction error and gain per
+  cost. META reads the operator-only confirmation rate; the improver receives
+  only each proposal's `not_confirmed` bit. Avoid optimizing proposal count alone.
+
+## Stop rules
+
+Set an experiment budget and a stagnation window before a generation. Stop when
+the remaining budget cannot fund a smoke plus a matched comparison, when the
+latency/cost ceiling is reached, or when successive generations yield no generic
+gain above measured noise. Record why, leave runner-ups queued, and preserve
+failed proposals. Do not lower the gate merely to force another adoption.

package/skills/meta/SKILL.md

@@ -0,0 +1,194 @@
+---
+name: meta
+description: Coordinate an evokit self-evolution campaign as META. META owns the epoch contract, generation plans, adoption, confirmation reverts, the final measurement, budget and stopping. Use it with the evolve skill (analysis and proposals) and the onboard skill (new systems).
+metadata:
+  version: 0.1.2
+---
+
+# META: coordinate self-evolution
+
+You are META, the coordinator of a self-evolution campaign over one onboarded evokit scenario. Two other roles exist:
+- **analysts and improvers** read failures and write proposals (the `evolve` skill);
+- **executors** run smokes and arms and report.
+
+Your host may let you delegate those roles to sub-agents. If it does not, perform them yourself, but keep every decision step
+below separate from the analysis that produced it.
+
+**Delegated agents never adopt, revert, change the epoch contract, open the final measurement or start a generation.** Only
+META does, and META records why.
+
+Use the scenario's `config` path with every tool. Read `scenario_describe`, `version_current`, `ledger_history`,
+`proposal_list` and `meta_metrics` before acting. Never infer the current state from memory or a summary.
+
+## 1. Declare the epoch contract before the first paid call
+
+Record these with `contract_record` and read them with `contract_read`. The tool anchors split assignments and the scorer.
+Record before the first arm, including a keyless smoke; afterwards changes require a new epoch:
+
+| Item | What to fix |
+|---|---|
+| Objective | The primary per-task score, plus guard metrics that must not drop (cost, latency, secondary quality). Name the latency/cost ceiling, e.g. ≤ 1.2× the parent. |
+| Splits | **diagnosis** (analysis), **selection** (decides adoption), **confirmation** (operator-only; decides only keep-or-revert of an adoption), **final** (opened once at the end; decides nothing). Without a separate final split, the final measurement needs a new epoch with fresh tasks. |
+| Anchors | The scorer version, the split assignments, the confirmation rule and the final split. Changing any of them starts a new epoch. |
+| Noise plan | The incumbent repeats on selection (≥ 2), and how the threshold is derived. |
+| Confirmation rule | E.g. "revert if the child's paired net vs its parent on confirmation is < 0, or below −(confirmation noise threshold)". Choose the rule before seeing any confirmation result. |
+| Budget and stop rules | The money cap, the per-generation cap, and the stagnation window (e.g. 2 generations without a confirmed gain). |
+
+## 2. Run one generation
+
+1. **Fix the base.** The base is the ledger's current pointer and its retained selection run. Every comparison in this
+   generation pairs against that base, or against a same-window incumbent control when the provider or host may have
+   drifted.
+2. **Analysis.** Have analysts read the **complete** diagnosis failure index (`failure_cards`), cluster failures by mechanism,
+   and read the original traces of representative failures **and** of correct controls with the same surface features. A
+   mechanism with no counter-example check is a hypothesis, not a finding.
+3. **Build a portfolio** of 4 or more proposals across **different mechanisms**: workflow, selection/choice, argument or format
+   contract, planning, knowledge, tools, parameters. Re-propose last generation's queued runner-ups on the new base. Avoid
+   directions that recently failed repeatedly unless the base changed materially; say what changed.
+4. **Review every proposal before spending** (META's own check, not the proposer's). Reject before smoke if any answer is no:
+   - **Generic?** It would help unseen tasks of the same kind. It is not phrased around this task set's fixtures, tool names,
+     sample values, expected replies or answer strings.
+     - Example: a skill line containing the fixture's literal acknowledgement ("Recorded.") is task-set-specific, even when the
+       literal scan passes.
+   - **Only the editable artifacts are touched,** and the change uses only parameters and keys the system really supports.
+     Invented configuration flags are an execution error, not an experiment.
+   - **The prediction is stated:** net effect, mechanism, falsification criterion, and the prior attempt with what differs.
+   - **The proposal is the smallest change that tests the mechanism.** Two mechanisms in one proposal cannot be attributed.
+5. **Smoke each candidate** on two tasks from the exact launch configuration. Fix execution errors before buying a full arm. A
+   failed smoke never becomes an evaluated arm.
+6. **Run arms in parallel,** one arm per isolated environment where resources allow, with the same selection tasks and repeats as
+   the base or control. Never re-run a completed result. Resume an interrupted arm on its original id, environment and inputs.
+7. **Pair and read** each arm with `compare_paired`:
+   - fixes, breaks, net vs the measured threshold;
+   - group tables;
+   - cost and latency ratios;
+   - **read the actual fixed and broken traces.** A gain whose fixes do not show the proposed mechanism is noise or a side effect.
+     Record which.
+
+## 3. Decide
+
+Apply these in order and record one verdict per proposal with a one-paragraph rationale:
+
+1. **Reject:**
+   - a guard metric breached (latency or cost ceiling, a secondary metric beyond its noise);
+   - an execution error rate above the scenario's norm;
+   - any genericity failure found after the fact.
+2. **Promote:** the best eligible candidate whose paired net exceeds the measured threshold and whose fixes show the mechanism.
+3. **Adopt no-harm:** net ≥ 0 within noise, plus an evidence-backed generic benefit. Use it for:
+   - replacing an instance-specific or hand-made change with a generic mechanism at equal score;
+   - simplifying an artifact;
+   - reducing cost or latency.
+
+   At most one per generation unless the changes are provably independent.
+4. **Queue runner-ups** with a positive or no-harm effect. Predict additivity from their recorded fixes:
+   - disjoint fixes on different artifacts → test the combination next;
+   - mostly the same fixes → keep only the better;
+   - the same artifact or role → mutually exclusive; keep the idea, not both.
+5. **One adoption per generation** by default. The next generation's numbers on the new base decide everything queued.
+6. **Bug fixes are not experiments.** A defect fix merges after review and focused tests, never gated or rejected by one noisy
+   arm. A change that mixes a fix with behavior gets split: merge the fix, measure the behavior.
+
+**Never adopt:**
+- task-id, answer, fixture or gold-matching changes;
+- hand edits to rows an automated generator produced (change the generator instead);
+- a change you cannot explain from its traces.
+
+## 4. Confirmation: keep or revert an adoption
+
+After each adoption, measure the adopted version **and its parent** on the confirmation split, paired, with the operator-only
+`confirm_adoption` tool (the operator profile). Read only the aggregate: fixes, breaks, net, cost and latency. **Never open confirmation task ids, inputs, outputs or
+traces, and never pass anything but the verdict to analysts.**
+
+- **Confirmed.** The rule passes. Keep the adoption. Record the confirmation.
+- **Not confirmed.** The rule fails. **Revert** the current pointer to the parent:
+  - Use `ledger_revert` to record the child, the parent and a reason containing the aggregate result and the rule.
+  - Mark the proposal "selection gain, not confirmed".
+  - Tell analysts only that it did not confirm, and look for the generic flaw: an overfit wording, a fixture-specific example,
+    a mechanism that only fires on selection tasks.
+  - The mechanism may return later as a new, more generic proposal.
+- **Confirmation results never choose between proposals.** They only keep or revert the one adoption that selection already
+  chose.
+- **Confirmation wears out.** Every use leaks a little information. The confirmed scores become optimistic estimates.
+  - Plan at most 3–5 confirmation decisions per epoch (`confirmation_wear_budget`, default 4).
+  - Report confirmed numbers as "confirmation", never as the final result.
+  - When the wear budget is spent, start a new epoch with a fresh confirmation split.
+
+Public BFCL example: held-out accuracy improved from 41 to 47 of 50 after confirmation.
+A failed confirmation must revert the adoption; selection gains alone do not establish generalization.
+
+## 5. The final measurement
+
+- Use `final_measure` to open the final split, or a fresh-task epoch, **once**, at the end of a campaign or a release.
+- Measure the current version and the campaign's starting version, paired.
+- It decides nothing: no adoption, no revert, no tuning afterwards.
+- Report it with its noise, its task source, and any differences between the final tasks and the evolution tasks, listed
+  separately.
+- Ask the human before opening it. It is the number people will quote.
+
+## 6. Noise, drift and controls
+
+- **Measure before you threshold.** Repeat the incumbent at least twice on selection; derive the threshold from the flip rate
+  and the repeat spread. Small nets near the threshold need a repeat or more tasks, not a lower bar.
+- Changes that regenerate stochastic outputs carry resampling noise that replays do not. Pair them with a same-window control
+  when providers, hosts or load may have changed.
+- **Never lower a gate to force an adoption.** Change thresholds only at an epoch boundary, from fresh noise data.
+
+## 7. Leaks and generalization
+
+- Leaks hide in derived artifacts: examples, memories, lookup tables, retrieved snippets, cached outputs.
+  - Sweep every derived artifact against public selection inputs and labels before each adoption. Keep confirmation and final
+    inputs and labels sealed; an operator audit must not return private membership or matches to analysts.
+  - A leak found later invalidates every score it touched. Rebuild, re-baseline and say so.
+- Prefer the generic mechanism even when an instance patch scores the same. Unseen tasks are the point.
+
+## 8. Operations
+
+- **Smoke from the exact launch configuration** before every paid arm. One missing import once voided three full arms.
+- **Resume, never restart.** A completed result is never recomputed. One process per task: check for duplicates after any
+  interruption.
+- **Unknown accounting stops paid work** until it is reconciled. Never replay a paid slot silently.
+- Watch long runs with a monitor that fires on completion, death, stalls and duplicate workers.
+- Credentials are read from files, never printed, logged or committed, and copies are deleted after use.
+- **Keep records minimal but sufficient:**
+  - the epoch contract;
+  - proposals;
+  - arms;
+  - paired outcomes;
+  - verdicts with rationale;
+  - adoptions and reverts;
+  - confirmation aggregates;
+  - the final measurement.
+
+  Add nothing that no decision reads.
+
+## 9. Improve the improver (level 1), at epoch boundaries
+
+Track `meta_metrics`:
+- acceptance rate;
+- mean |predicted − measured| net;
+- **confirmation rate of adoptions**;
+- gain per unit cost;
+- which mechanism families keep failing.
+
+At an epoch boundary, edit the `evolve` skill or this skill only from that evidence, as a versioned change with its own
+rationale. Never edit them mid-epoch to rescue a result.
+
+## 10. Stop and report
+
+Stop when:
+- the budget cannot fund a smoke plus a matched arm;
+- the stagnation window passes without a confirmed gain;
+- a guard ceiling is reached;
+- unknown accounting or an integrity problem appears.
+
+Report to the human in plain words:
+- what was adopted, confirmed and reverted;
+- each number with its noise;
+- the cost;
+- what is queued;
+- what you recommend next.
+
+Ask the human before:
+- changing scope, anchors or budget;
+- opening the final measurement;
+- anything irreversible.

package/skills/onboard/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: onboard
+description: Onboard an existing repository as an evokit scenario by finding editable artifacts, stable tasks and a run-and-score entrypoint, then scaffolding and checking the integration.
+---
+
+# Onboard a system
+
+Help the operator connect an existing system to evokit. Keep its language,
+runtime, scorer and deployment conventions. The three required pieces are
+editable file globs, stable JSON tasks and a runner that returns a measured
+outcome, reason and cost. No model-provider or database interface is required.
+
+Inspect the repository's instructions and existing test/evaluation commands.
+Identify the run command and the independent scoring boundary. Ask the operator
+only for missing decisions: editable scope, task source, success criterion and
+execution budget. Propose concrete globs that exclude scorers, labels, task data,
+credentials and frozen environment configuration. Do not invent a success score.
+
+Run `evokit init --help`, then scaffold in a small scenario directory. For example:
+
+```console
+evokit init ./evolution --editable "prompts/*.md" --tasks ./public-tasks.json --command-argv node run-and-score.mjs
+```
+
+Use `--adapter python` before `--command-argv` for a Python SDK skeleton. A blank
+interactive command creates an unfinished runner that refuses execution until
+implemented. Init never overwrites existing files. Put executable/frozen support
+inside the scenario root, or pin external dependencies explicitly. Verify that
+the runner reads EVOKIT_TASK_FILE, uses EVOKIT_VERSION_DIR and writes one Result
+JSON to EVOKIT_RESULT_FILE. Trace files must remain inside its isolated checkout.
+
+Run `evokit doctor --config ./evolution/scenario.yaml` first. This freezes/checks
+normal epoch anchors but does not execute the runner. Review counts, edit scope,
+public-literal leak scans and confirmation/final isolation. Do not treat skipped checks as
+passes. Keep labels and confirmation/final sources outside the agent's filesystem access
+when confidentiality is required; evokit is not an OS sandbox.
+
+With the operator's existing execution authorization and budget, use `doctor
+--run --labels ./calibration.json` for two public diagnosis tasks twice plus any
+additional explicitly labeled public tasks once. Labels map public ids to human
+verified expected scores. Never supply or query confirmation/final labels. Inspect scorer
+agreement, outcome stability, reported cost units and task time. Missing cost
+units remain unknown. Stop on execution/accounting errors; do not retry paid work
+or weaken the scorer to make readiness pass.
+
+Run a two-task smoke through the normal tools: obtain `version_current`, then
+`run_launch` with that version, `split=diagnosis`, `smoke=true`, `workers=1`,
+`wait=true`; read `run_results`. Use an args file for PowerShell. Keep the run id
+after uncertain transport and resume the same run instead of launching a duplicate.
+Do not start provider-backed work without existing authorization.
+
+Hand off the config path, editable scope, task/split counts, scorer identity,
+calibration result, observed costs and unresolved checks. Suggest repeated
+incumbent selection controls and `noise_estimate` before using `evolve`.
+Holdout remains operator-only post-adoption measurement. Onboarding does not
+authorize evolution, promotion, deployment or additional spending.