dsh-harbor-evolution 0.5.0 → 0.6.0

package/README.md

@@ -2,7 +2,7 @@
 
 Installable DeepSeek Harness Plugin + Skill for running stable Harbor evaluation and controlled Agent evolution loops, with a native DSH Web dashboard.
 
-The package gives DSH eight strict Harbor tools, dedicated Tool cards, a paginated Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill clarifies and initializes the Evaluation Stack, validates Dataset identity, runs Architecture Doctor and Context v2 preview, establishes a baseline, diagnoses evidence, limits each iteration to one controlled Candidate change, and applies the Promotion Gate.
+The package gives DSH twelve strict Harbor tools, dedicated Tool cards, a nine-stage Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill clarifies and initializes the Evaluation Stack, validates Dataset identity, checks Trial Lifecycle and Score Validity, governs independent Ground Truth meta-evaluation, diagnoses evidence provenance, limits each iteration to one controlled Candidate change, and invokes the Promotion Gate only as an explicit action.
 
 ## Install
 
@@ -14,8 +14,8 @@ npx --yes dsh-harbor-evolution@latest setup --project-root "$PWD"
 
 The setup command installs both required runtimes:
 
-- `harbor-dsh-evolution==0.5.0` in a managed Python environment.
-- `dsh-harbor-evolution@0.5.0` in the selected DSH profile.
+- `harbor-dsh-evolution==0.6.0` in a managed Python environment.
+- `dsh-harbor-evolution@0.6.0` in the selected DSH profile.
 
 It then stores the absolute Harbor executable paths and `projectRoot` in the profile's `harbor-evolution` block and verifies the integration. Existing unrelated profile entries are preserved, and rerunning setup updates the same block.
 
@@ -45,7 +45,9 @@ The Plugin registers:
 
 In the `web` profile, the same package also registers:
 
-- a lightweight Harbor Job overview and on-demand Evaluation Workbench for Contract, Stack, Dataset, Doctor, paginated Trials, evidence, optimization, promotion, and audit artifacts;
+- a localized nine-stage Workbench that directly exposes fixed experiment identities, Agent-visible Dataset queries/instructions, safe business-artifact previews, Ground Truth meta-evaluation, paginated per-Trial evidence and recommendations, Population validity/coverage, controlled optimization hypotheses, and Baseline/Gate deltas; raw JSON remains in the audit drawer;
+- descriptor-authorized Evaluator/Rubric source editing for `script` and `llm-as-judge` implementations, with optimistic concurrency and mandatory new identities;
+- a `harbor-dsh-evaluator/v1` interface shared by deterministic scripts and LLM-as-Judge implementations;
 - compact result cards for all Harbor Tool calls;
 - a `Harbor Evolution` Settings section that checks the configured project, Evaluation Stack, Jobs directory, and CLI paths.
 
@@ -53,6 +55,8 @@ The Web UI is intentionally read-only. Starting an evaluation or deciding promot
 
 A direct evaluation requires `candidatePath`, `datasetPath`, `stackPath`, and explicit `mode`; `promotion-eligible` additionally requires `policyPath`. Prefer the Skill because it will not run or compare Jobs until the material identities and evaluation contract are resolved.
 
+`harbor_eval_result` defaults to the stable Summary. Use `view=job`, `view=dataset`, `view=progress`, `view=trial` plus a returned `trialId`, or `view=governance` to inspect sanitized instructions, generated output, evidence, and evaluator source without coupling the Agent to artifact file paths.
+
 ## What setup writes
 
 The selected profile receives one id-targeted override:

package/index.js

@@ -144,12 +144,61 @@ export function apply(ctx, config) {
 
   ctx.tools.register(jsonTool({
     name: 'harbor_eval_result',
-    description: 'Read the stable evaluation summary produced for a Harbor Job.',
+    description: 'Read a stable Job summary or a sanitized Workbench, Dataset instruction, Trial output/evidence, progress, or Evaluator governance view. Invalid scores remain distinct from raw verifier rewards.',
     parameters: {
       jobPath: { type: 'string', required: true },
+      view: { type: 'string', description: 'summary (default), job, dataset, progress, trial, or governance' },
+      trialId: { type: 'string', description: 'Required only for view=trial; use an id returned by the Job/Progress view' },
+      compareJob: { type: 'string', description: 'Optional previous Job for view=governance impact analysis' },
+      since: { type: 'string', description: 'Optional ISO timestamp for incremental progress changes' },
     },
   }, args => service.result(args)))
 
+  ctx.tools.register(jsonTool({
+    name: 'harbor_evaluator_inspect',
+    description: 'Inspect the active harbor-dsh-evaluator/v1 descriptor, implementation kind, ternary Criteria, and safely editable source files.',
+    parameters: {
+      stackPath: { type: 'string', description: 'Defaults to .harbor/evaluation-stack.yml' },
+    },
+  }, args => service.evaluatorInspect(args)))
+
+  ctx.tools.register(jsonTool({
+    name: 'harbor_evaluator_update',
+    description: 'Update one descriptor-authorized Evaluator source file with optimistic concurrency. Requires new Evaluator and Stack identities and never runs evaluation or Gate automatically.',
+    parameters: {
+      stackPath: { type: 'string', description: 'Defaults to .harbor/evaluation-stack.yml' },
+      filePath: { type: 'string', required: true },
+      content: { type: 'string', required: true },
+      expectedDigest: { type: 'string', required: true },
+      newEvaluatorVersion: { type: 'string', required: true },
+      newStackVersion: { type: 'string', required: true },
+    },
+  }, args => service.evaluator(args)))
+
+  ctx.tools.register(jsonTool({
+    name: 'harbor_ground_truth_init',
+    description: 'Create a non-overwriting Ground Truth draft for evaluator meta-evaluation. GT may be human, programmatic, consensus, model, or external, but must have explicit provenance and remain independent of the Candidate evaluator.',
+    parameters: {
+      outputPath: { type: 'string', description: 'Defaults to .harbor/ground-truth.json' },
+      groundTruthId: { type: 'string', required: true },
+      version: { type: 'string', required: true },
+      sourceKind: { type: 'string', required: true, description: 'human, programmatic, consensus, model, or external' },
+      sourceDescription: { type: 'string', required: true },
+      provenance: { type: 'string', required: true },
+      criteria: { type: 'string', required: true, description: 'Comma-separated criterion ids' },
+    },
+  }, args => service.groundTruthInitialize(args)))
+
+  ctx.tools.register(jsonTool({
+    name: 'harbor_evaluator_meta_evaluate',
+    description: 'Compare repeated evaluator-observations/v1 with independent ground-truth/v1 and write an ESF, SCE, and RCR meta-evaluation report.',
+    parameters: {
+      groundTruthPath: { type: 'string', description: 'Defaults to .harbor/ground-truth.json' },
+      observationsPath: { type: 'string', required: true },
+      outputPath: { type: 'string', description: 'Defaults to .harbor/meta-evaluation-report.json' },
+    },
+  }, args => service.evaluatorMetaEvaluate(args)))
+
   ctx.tools.register(jsonTool({
     name: 'harbor_candidate_compare',
     description: 'Apply the deterministic Promotion Gate to a baseline Job and a Candidate Job.',