dsh-harbor-evolution 0.6.0 → 0.7.0

package/README.md

@@ -2,7 +2,7 @@
 
 Installable DeepSeek Harness Plugin + Skill for running stable Harbor evaluation and controlled Agent evolution loops, with a native DSH Web dashboard.
 
-The package gives DSH twelve strict Harbor tools, dedicated Tool cards, a nine-stage Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill clarifies and initializes the Evaluation Stack, validates Dataset identity, checks Trial Lifecycle and Score Validity, governs independent Ground Truth meta-evaluation, diagnoses evidence provenance, limits each iteration to one controlled Candidate change, and invokes the Promotion Gate only as an explicit action.
+The package gives DSH twelve strict Harbor tools, dedicated Tool cards, a nine-stage Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill starts with four user-facing concepts—Dataset (what to test), Generator (who answers), Evaluator plus criteria (what good means), and Optimizer (who improves it)—then compiles accepted choices into the strict Evaluation Stack. It validates Dataset identity, checks Trial Lifecycle and Score Validity, governs independent Ground Truth meta-evaluation, diagnoses evidence provenance, limits each iteration to one controlled Candidate change, and invokes the Promotion Gate only as an explicit action.
 
 ## Install
 
@@ -14,8 +14,8 @@ npx --yes dsh-harbor-evolution@latest setup --project-root "$PWD"
 
 The setup command installs both required runtimes:
 
-- `harbor-dsh-evolution==0.6.0` in a managed Python environment.
-- `dsh-harbor-evolution@0.6.0` in the selected DSH profile.
+- `harbor-dsh-evolution==0.7.0` in a managed Python environment.
+- `dsh-harbor-evolution@0.7.0` in the selected DSH profile.
 
 It then stores the absolute Harbor executable paths and `projectRoot` in the profile's `harbor-evolution` block and verifies the integration. Existing unrelated profile entries are preserved, and rerunning setup updates the same block.
 
@@ -32,6 +32,8 @@ Stop any old DSH process and run the exact restart command printed by setup. The
 Inspect this workspace and help me clarify and initialize a stable Harbor self-evolution loop.
 ```
 
+Users may provide a single Query or Dataset path, a Generator curl or local Agent path, an Evaluator curl/path or natural-language criteria, and an optional Optimizer such as Codex or Claude Code. The Skill inspects the workspace first, defaults the Optimizer to the current Agent, and shows one confirmation card before writing files. Evaluation Stack roles, ids, versions, Judge configuration, Contract, and Policy stay behind advanced configuration unless they materially affect a decision.
+
 The Plugin registers:
 
 - `harbor_candidate_snapshot`

package/index.js

@@ -74,7 +74,7 @@ export function apply(ctx, config) {
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evolution_init',
-    description: 'Initialize a strict, non-overwriting Evaluation Stack project after the Skill has clarified identities, primary metric, judge, and promotion threshold.',
+    description: 'Compile an accepted Dataset, Generator, Evaluator/criteria, and Optimizer onboarding card into a strict, non-overwriting Evaluation Stack project. Detailed identity fields are internal tool inputs, not a user questionnaire.',
     parameters: {
       datasetPath: { type: 'string', required: true },
       stackId: { type: 'string', required: true },

package/lib/setup.js

@@ -267,7 +267,11 @@ export async function setupIntegration(raw = {}, dependencies = {}) {
   try {
     await run('pnpm', [
       '--silent', 'dlx', `@deepseek-ai/dsh@${DSH_VERSION}`,
-      'plugin', '--profile', config.profile, 'add', '-w', '--save-exact', config.pluginSpec,
+      // Registry packages ship their built client and need no lifecycle
+      // scripts. Skipping scripts also prevents pnpm 11 from reclassifying
+      // unrelated DSH native dependencies as newly unapproved builds while
+      // adding this plugin to an existing profile.
+      'plugin', '--profile', config.profile, 'add', '-w', '--save-exact', '--ignore-scripts', config.pluginSpec,
     ], { env: { ...env, DSH_HOME: config.dshHome } })
   } catch (error) {
     throw new Error(processFailure(error))

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-harbor-evolution",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "DeepSeek Harness plugin and bundled Skill for safely evolving Cordis Candidates with Harbor.",
   "type": "module",
   "main": "index.js",

package/skills/evolve-agent-with-harbor/SKILL.md

@@ -5,19 +5,22 @@ description: Architect, initialize, run, diagnose, compare, and safely improve a
 
 # Evolve Agent With Harbor
 
-Build a reproducible improvement loop around three boundaries:
+Build a reproducible improvement loop around four concepts that users can describe in business language:
 
-- **Generator/Candidate**: the immutable DSH Agent composition being improved.
-- **Evaluator/Evaluation Stack**: Integration, Renderer, Evaluator, Rubric, Diagnoser, Optimizer, Runner, Reporter, and Judge identities.
-- **Optimizer**: proposes one evidence-linked Candidate change; it never controls the final Gate.
+- **评测集 (Dataset)** — what should be tested: one Query, a file, a directory of instructions, or an existing Harbor Dataset.
+- **生成器 (Generator)** — who produces the answer or artifact: a curl request, a local Agent entry, or an Agent already found in the workspace.
+- **评测器（含评测标准） (Evaluator)** — what “good” means and who scores it: an evaluator endpoint, a local evaluator, or a versioned evaluator drafted from natural-language criteria.
+- **优化器 (Optimizer)** — who uses badcases and evidence to propose the next controlled change: this Agent by default, Codex/Claude Code, or a local command/Agent.
+
+Keep these four names visible during onboarding and confirmation; they establish the product's concept space. Treat Integration, Renderer, Rubric, Diagnoser, Runner, Reporter, Judge identity, Contract, Policy, Context, and Gate as the compiled evaluation architecture. Explain them only when they affect a decision or the user opens advanced configuration.
 
 Treat Harbor as the experiment boundary. Deployment, CI/CD, and Champion replacement remain external actions requiring separate authority.
 
 ## Select the narrowest mode
 
-- **Clarify**: define progress, identities, constraints, and promotion ownership.
+- **Clarify**: identify the Dataset, Generator, Evaluator/criteria, and Optimizer with the least user effort.
 - **Architecture**: inspect role boundaries and run `harbor_evolution_doctor`.
-- **Initialize**: read `references/initialization.md`, obtain explicit values, then call `harbor_evolution_init`.
+- **Initialize**: read `references/initialization.md`, compile the accepted four-concept card, then call `harbor_evolution_init`.
 - **Diagnostic**: investigate failures without making a promotion claim.
 - **Promotion**: run a `promotion-eligible` Job and apply the deterministic Gate.
 - **Evolve**: baseline → diagnose → one controlled change → regression Job → Gate.
@@ -26,20 +29,52 @@ Treat Harbor as the experiment boundary. Deployment, CI/CD, and Champion replace
 
 Do not turn an inspection or diagnostic request into Agent mutation or deployment.
 
-## Clarify before initializing
+## Start with four clear concepts
+
+Inspect the current workspace before asking questions. Look for Agent entry files, package metadata, curl examples, Dataset instructions, existing Harbor configuration/Jobs, tests, and available Codex or Claude Code commands. Reuse reliable findings and say what was inferred; do not ask the user to transcribe information already present in files.
+
+When no Harbor workspace exists, propose `./harbor-evolution/` under the current working directory as the managed evaluation workspace. Keep imported snapshots and generated evaluation files there. If the installed Plugin is configured to a different `projectRoot`, explain the mismatch and propose the exact configuration change before writing anything.
+
+Ask only for missing parts of the four-concept intake, using the user's language and short examples:
+
+1. **评测集：测什么？** Accept one Query, a file path, a directory containing multiple instructions, or an existing Dataset path.
+2. **生成器：谁来回答？** Accept a curl request or a local Agent file/directory. Offer an Agent entry discovered in the workspace instead of asking again.
+3. **评测器（评测标准）：怎样算好？** Accept an evaluator curl request, a local evaluator path, or “请你生成”. If no evaluator exists, ask only for natural-language criteria and draft a versioned evaluator plus Rubric for confirmation.
+4. **优化器：谁根据结果改进？** Default to the current Agent. If Codex CLI or Claude Code is available, present it as an optional alternative; also accept a local command or Agent path.
+
+One compact message may contain all unresolved concepts, but do not turn it into a form full of architecture terms. Never ask a first-time user to enumerate Evaluation Stack roles, identities, versions, Judge parameters, Contract fields, Policy fields, holdout rules, or CI/CD details.
+
+Interpret common inputs helpfully:
+
+- A single Query becomes a one-task **diagnostic** Dataset. It is useful for checking wiring, but never counts as promotion evidence by itself.
+- A file or directory may become a multi-task Dataset after instructions and safe paths are validated.
+- For curl input, infer method, endpoint, headers, body shape, and response protocol. Never persist Authorization values or other credentials.
+- “请你生成评测器” means the current Agent may author a pinned evaluator implementation; it does not permit ad-hoc, unversioned scoring by the current chat model.
+- “你来优化” selects the current Agent as Optimizer; it does not authorize mutation before baseline evidence and an accepted hypothesis exist.
+
+Before creating files, show one confirmation card:
+
+```text
+开始前确认
+- 工作空间：<path>
+- 评测集：<source and task count; diagnostic or regression>
+- 生成器：<curl/local Agent/detected Agent>
+- 评测器（评测标准）：<implementation or draft criteria>
+- 优化器：<current Agent/Codex/Claude Code/local>
+- 我将自动补全：版本标识、适配器、产物呈现、诊断、报告和运行配置
+- 暂不启用：<holdout / formal promotion Gate / deployment, when unresolved>
+```
+
+Offer three next actions in natural language: **开始初始化**, **修改以上内容**, or **查看高级配置**. Call `harbor_evolution_init` only after the user accepts the card. Generate internal ids and initial versions from the workspace/project identity, use `reward`/`maximize` as a visible draft when the criteria imply quality scoring, and do not use the generated Policy for a `promotion-eligible` Job until real business thresholds are accepted.
 
-Inspect the workspace first. Resolve only material gaps, preferably in no more than three grouped questions. Obtain:
+Ask advanced questions just in time:
 
-1. Business behavior, failure pattern, and Candidate path/product identity.
-2. Dataset path/id/version, task population, holdout boundary, and side-effect sandbox.
-3. Evaluation Stack id/version and one entry for every required role.
-4. Judge provider/model/version/parameters without credentials.
-5. Evaluation Contract id/version, primary metric and direction, diagnostic metrics, groups, and hard requirements.
-6. Promotion Policy id/version, delta, minimums, maximums, non-regression metrics, and metric directions.
-7. Baseline Job/Candidate, repeat policy, run budget, stopping rule, allowed mutation surface, and forbidden files.
-8. Promotion owner and external CI/CD handoff.
+- Ask for holdout boundaries and side-effect constraints before they can affect a real run.
+- Ask for metric thresholds, non-regression limits, repeat policy, and promotion owner before the first `promotion-eligible` Job.
+- Ask for GT identity, provenance, and independence only when calibrating or replacing the Evaluator.
+- Ask for deployment/CI authority only after Gate recommends promotion.
 
-Never invent GT, a Judge model, reward definitions, thresholds, credentials, or deployment authority. Offer draft values only when clearly labeled and accepted.
+Never invent GT labels, business thresholds, credentials, production side-effect permission, or deployment authority. A Generator and Evaluator may share a provider only after disclosing the coupling; do not treat that as independent GT.
 
 ## Enforce the strict architecture
 
@@ -68,7 +103,7 @@ Keep Runner orchestration-only. Treat these as architecture errors:
 
 ## Initialize without overwriting
 
-Read `references/initialization.md` when required files are missing. After the user accepts all required identities and metric semantics, call `harbor_evolution_init`. It preserves existing files and creates explicit placeholders that still require business implementation.
+Read `references/initialization.md` when required files are missing. Translate the accepted four-concept card into strict internal identities and call `harbor_evolution_init`; do not send the user back a second architecture questionnaire. It preserves existing files and creates explicit placeholders that still require business implementation.
 
 After initialization:
 

package/skills/evolve-agent-with-harbor/evals/evals.json

@@ -0,0 +1,38 @@
+{
+  "skill_name": "evolve-agent-with-harbor",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "帮我给当前目录里的业务 Agent 建一个 Harbor 自进化流程，我还没有准备任何评测配置。",
+      "expected_output": "先检查工作区，再用评测集、生成器、评测器（评测标准）和优化器四个业务概念提出少量短问题；不得要求用户枚举 Evaluation Stack 八个角色或版本号。",
+      "files": [],
+      "assertions": [
+        "The response visibly uses the four concepts Dataset, Generator, Evaluator with criteria, and Optimizer.",
+        "The response does not ask for eight Evaluation Stack roles, Judge version parameters, Contract ids, or Policy ids.",
+        "The response proposes a managed workspace and states that inferred values will be confirmed before files are created."
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "评测问题先用‘什么是重力？’，生成器就是 ./agent.ts。没有现成评测器，你帮我做；我觉得答到问题、表达有趣、引用可靠就算好。",
+      "expected_output": "识别为单任务诊断评测，复用本地生成器，依据三条自然语言标准起草版本化评测器，并默认由当前 Agent 作为优化器；先展示确认卡，不把单条 Query 当作晋级依据。",
+      "files": [],
+      "assertions": [
+        "The single query is explicitly classified as diagnostic rather than promotion evidence.",
+        "The evaluator is described as a pinned draft derived from the three supplied criteria, not ad-hoc chat scoring.",
+        "The output includes a confirmation card before initialization."
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "评测集在 ./evals，生成器调用这个 curl：curl -X POST http://127.0.0.1:9000/run -H 'Authorization: Bearer secret' -d '{\"input\":\"hi\"}'。评测器在 ./judge.py，优化交给 codex。帮我初始化。",
+      "expected_output": "自动解析四个概念并展示确认卡；不回显或持久化 secret；底层身份与适配器由 Skill 推断，不再追问专业字段；未确认前不写文件或运行 Job。",
+      "files": [],
+      "assertions": [
+        "The Authorization credential is redacted and explicitly excluded from persisted configuration.",
+        "The response maps the supplied paths and curl into the four-concept confirmation card.",
+        "The response does not run initialization or evaluation before confirmation."
+      ]
+    }
+  ]
+}

package/skills/evolve-agent-with-harbor/references/initialization.md

@@ -1,10 +1,25 @@
-# Strict Project Initialization
+# Progressive Project Initialization
 
 Load this reference only when the project is missing the Evaluation Stack structure or the user asks to initialize it.
 
-## Readiness worksheet
+## User-facing concept card
 
-Resolve every value before calling `harbor_evolution_init`:
+Keep onboarding anchored on four visible concepts:
+
+| User concept | Plain-language prompt | Accepted input | Compiled architecture |
+| --- | --- | --- | --- |
+| 评测集 (Dataset) | 测什么？ | One Query, file, instruction directory, or Dataset | Dataset tasks, manifest, population, diagnostic/regression mode |
+| 生成器 (Generator) | 谁来回答？ | curl or local/detected Agent | Candidate snapshot, Integration, Renderer, runtime identity |
+| 评测器（评测标准） (Evaluator) | 怎样算好？ | evaluator curl/path, or natural-language criteria | Evaluator, Rubric, Judge identity, Evaluation Contract |
+| 优化器 (Optimizer) | 谁根据结果改进？ | current Agent, Codex, Claude Code, or local command | Optimizer identity, mutation surface, rollback workflow |
+
+Inspect first and prefill everything reliable. Ask only about missing rows. Use `./harbor-evolution/` as the proposed managed workspace when no project exists, but do not write outside the Plugin's configured `projectRoot`; surface a configuration mismatch before initialization.
+
+Before any write, show the four rows plus inferred workspace, diagnostic/promotion scope, and deferred capabilities. The user may start initialization, modify the card, or open advanced configuration.
+
+## Internal compilation worksheet
+
+The Agent, not the user, compiles the accepted concept card into the strict `harbor_evolution_init` arguments:
 
 | Field | Required meaning |
 | --- | --- |
@@ -13,11 +28,13 @@ Resolve every value before calling `harbor_evolution_init`:
 | `datasetId` / `datasetVersion` | Stable identity of task population and GT boundary |
 | `contractId` / `contractVersion` | Stable metric semantics |
 | `primaryMetric` / `primaryDirection` | Exact reward key and `maximize` or `minimize` |
-| Judge provider/model/version | Reproducible Judge identity, never credentials |
+| Judge provider/model/version | Infer from the accepted Evaluator implementation; never persist credentials |
 | Policy id/version | Stable Gate identity |
-| `minImprovement` | Accepted primary-metric delta |
+| `minImprovement` | Use only an explicitly accepted business delta for promotion; a diagnostic scaffold may use `0` and remain ineligible for promotion |
+
+Derive ids from a stable project slug and start newly generated identities at `1.0.0`. Use `reward` with `maximize` only as a visible draft when the user's criteria describe answer quality. The initializer creates a minimal Policy; do not pass it to a `promotion-eligible` Job until the user accepts real thresholds, non-regression metrics, repeat policy, mutation boundaries, and promotion ownership.
 
-Also establish diagnostic metrics, min/max thresholds, non-regression metrics, mutation surface, repeat policy, and promotion owner. The initializer creates a minimal Policy; update its explicit placeholders before a formal Gate.
+Do not ask the user to name Integration, Renderer, Diagnoser, Runner, or Reporter during ordinary onboarding. Generate their identities and explain them only in advanced configuration or when Doctor finds a boundary problem.
 
 If the Evaluator itself will be optimized, separately establish Ground Truth id/version, source kind (`human`, `programmatic`, `consensus`, `model`, or `external`), provenance, owner, Criteria, case population, and adjudication rule. Do not hide these semantics inside the Agent Dataset identity. Use `harbor_ground_truth_init` only after they are accepted.
 
@@ -122,11 +139,12 @@ Use business-accepted thresholds. Do not assume all metrics are `/10`, maximized
 
 Show the user:
 
-1. Resolved Candidate, Dataset, Stack, Policy, and Jobs paths.
-2. Role identities and which ones affect reward comparability.
-3. Metric directions, thresholds, groups, and hard requirements.
-4. Holdout, mutation, side-effect, repeat, and deployment boundaries.
-5. Doctor findings and Context preview.
+1. The accepted four-concept card and resolved paths.
+2. Whether the run is a quick diagnostic or a promotion-eligible regression.
+3. Any remaining choice that materially affects cost, safety, score meaning, or comparability.
+4. Doctor findings and Context preview in plain language.
+
+Put role identities, digests, metric directions, hard requirements, and generated defaults under an advanced/audit section. Do not make the user reconfirm unchanged internal fields.
 
 Start with a baseline. Do not create Candidate v2 until baseline evidence supports one controlled hypothesis.