dsh-harbor-evolution 0.7.1 → 0.7.3

package/README.md

@@ -2,7 +2,7 @@
 
 Installable DeepSeek Harness Plugin + Skill for running stable Harbor evaluation and controlled Agent evolution loops, with a native DSH Web dashboard.
 
-The package gives DSH twelve strict Harbor tools, dedicated Tool cards, a nine-stage Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill starts with four user-facing concepts—Dataset (what to test), Generator (who answers), Evaluator plus criteria (what good means), and Optimizer (who improves it)—then compiles accepted choices into the strict Evaluation Stack. It validates Dataset identity, checks Trial Lifecycle and Score Validity, governs independent Ground Truth meta-evaluation, diagnoses evidence provenance, limits each iteration to one controlled Candidate change, and invokes the Promotion Gate only as an explicit action.
+The package gives DSH fourteen strict Harbor tools, dedicated Tool cards, a nine-stage Evaluation Workbench, an installation Doctor, and the model- and user-invocable `evolve-agent-with-harbor` Skill. The Skill starts with four user-facing concepts—Dataset (what to test), Generator (who answers), Evaluator plus criteria (what good means), and Optimizer (who improves it)—then compiles accepted choices into the strict Evaluation Stack. A DSH Generator may explicitly pin the current default model as a non-secret Candidate identity while retaining the per-Job Host Broker credential boundary. The Plugin validates Dataset identity, checks Trial Lifecycle and Score Validity, governs independent Ground Truth meta-evaluation, diagnoses evidence provenance, limits each iteration to one controlled Candidate change, and invokes the Promotion Gate only as an explicit action.
 
 ## Install
 
@@ -14,10 +14,10 @@ npx --yes dsh-harbor-evolution@latest setup --project-root "$PWD"
 
 The setup command installs both required runtimes:
 
-- `harbor-dsh-evolution==0.7.1` in a managed Python environment.
-- `dsh-harbor-evolution@0.7.1` in the selected DSH profile.
+- `harbor-dsh-evolution==0.7.3` in a managed Python environment.
+- `dsh-harbor-evolution@0.7.3` in the selected DSH profile.
 
-It then stores the absolute Harbor executable paths and `projectRoot` in the profile's `harbor-evolution` block and verifies the integration. Existing unrelated profile entries are preserved, and rerunning setup updates the same block.
+It then stores the absolute Harbor executable paths and a fallback `projectRoot` in the profile's `harbor-evolution` block and verifies the integration. Agent Tool calls always use the calling session's absolute working directory as their project root; the configured value remains the Web Workbench and non-Agent fallback. Existing unrelated profile entries are preserved, and rerunning setup updates the same block.
 
 The default profile is `web`. Use `--profile headless` only when that is the profile you actually run. See all options with:
 
@@ -37,6 +37,7 @@ Users may provide a single Query or Dataset path, a Generator curl or local Agen
 The Plugin registers:
 
 - `harbor_candidate_snapshot`
+- `harbor_model_binding`
 - `harbor_evolution_init`
 - `harbor_evolution_doctor`
 - `harbor_dataset_validate`
@@ -51,7 +52,7 @@ In the `web` profile, the same package also registers:
 - descriptor-authorized Evaluator/Rubric source editing for `script` and `llm-as-judge` implementations, with optimistic concurrency and mandatory new identities;
 - a `harbor-dsh-evaluator/v1` interface shared by deterministic scripts and LLM-as-Judge implementations;
 - compact result cards for all Harbor Tool calls;
-- a `Harbor Evolution` Settings section that checks the configured project, Evaluation Stack, Jobs directory, and CLI paths.
+- a `Harbor Evolution` Settings section that checks the configured project, Evaluation Stack, Jobs directory, and CLI paths, supports process-local `projectRoot` reload, and checks npm for a newer formal release without silently installing it.
 
 The Web UI is intentionally read-only. Starting an evaluation or deciding promotion remains an explicit Agent + Skill workflow, so a page refresh can never launch an expensive Job.
 
@@ -63,6 +64,10 @@ Before each Job, the Plugin snapshots the current DSH Agent selection—provider
 
 `harbor_eval_run`, `harbor_context_preview`, and `harbor_evolution_doctor` inherit that selection by default. Advanced callers can override `candidateProvider` and `candidateModel` only as a pair, plus an optional `candidateReasoningEffort`. `openai-codex` performs a GPT Auth sign-in check before Harbor starts. The resulting model binding is part of Context v2 comparison identity, so any provider/model/reasoning change requires a new baseline.
 
+`harbor_model_binding` returns the current default selection as a credential-free `model-binding.json` draft. Once included before Candidate snapshot, it enters the Candidate digest and becomes the required Job model identity. Conflicting Job or Plugin overrides fail before Harbor starts. Even for `openai-codex`, the Candidate receives only the short-lived Broker capability—never the Host OAuth file or an upstream API key.
+
+When Settings opens, the Host performs a bounded npm registry check and caches successful results. An available release is shown with its exact installer command and release link. The browser never installs, rewrites a DSH profile, or restarts DSH; registry failures are non-blocking.
+
 `harbor_eval_result` defaults to the stable Summary. Use `view=job`, `view=dataset`, `view=progress`, `view=trial` plus a returned `trialId`, or `view=governance` to inspect sanitized instructions, generated output, evidence, and evaluator source without coupling the Agent to artifact file paths.
 
 ## What setup writes
@@ -79,7 +84,7 @@ The selected profile receives one id-targeted override:
     pythonPath: ""
 ```
 
-Keep `pythonPath` empty for the published Python package. `candidatePath`, `datasetPath`, `jobPath`, and `policyPath` are constrained to `projectRoot`.
+Keep `pythonPath` empty for the published Python package. For Agent Tool calls, `projectRoot` is replaced by the calling session's working directory for that call. `candidatePath`, `datasetPath`, `jobPath`, and `policyPath` remain constrained to that request-local root, so concurrent sessions cannot redirect each other's Harbor operations.
 
 For source development from the repository:
 

package/index.js

@@ -6,6 +6,7 @@ import { fileURLToPath } from 'node:url'
 
 import { loadBundledSkill } from './lib/official-skill.js'
 import { CandidateModelRuntime } from './lib/model-runtime.js'
+import { RUNTIME_POLICY } from './lib/runtime-identity.js'
 import { EvolutionService } from './lib/service.js'
 import { installDashboardWeb } from './lib/web.js'
 
@@ -26,7 +27,6 @@ export const Config = Schema.object({
   jobsDir: Schema.string().default('jobs'),
   harborBin: Schema.string().default(''),
   harborDshBin: Schema.string().default(''),
-  dshVersion: Schema.string().default('0.1.0-rc.6'),
   agentImportPath: Schema.string().default('harbor_dsh_evolution.agent:DshCandidateAgent'),
   pluginImportPath: Schema.string().default('dsh-evolution'),
   pythonPath: Schema.string().default(''),
@@ -47,15 +47,30 @@ function jsonTool(definition, execute) {
       schema: { type: 'string' },
       render: (_args, value) => [{ type: 'text', text: value }],
     },
-    async execute(args) {
-      return JSON.stringify(await execute(args), null, 2)
+    async execute(args, exec) {
+      return JSON.stringify(await execute(args, exec), null, 2)
     },
   })
 }
 
+function toolProjectRoot(exec) {
+  const cwd = exec?.agent?.session?.header?.cwd
+  if (typeof cwd !== 'string' || !path.isAbsolute(cwd)) {
+    throw new Error('Harbor tools require an Agent session with an absolute working directory')
+  }
+  return path.resolve(cwd)
+}
+
+export function synchronizeWorkbenchProjectRoot(service, exec) {
+  const projectRoot = toolProjectRoot(exec)
+  service.activateProjectRoot(projectRoot, 'agent-session')
+  return projectRoot
+}
+
 export function apply(ctx, config) {
   const resolved = {
     ...config,
+    runtimePolicy: RUNTIME_POLICY,
     projectRoot: path.resolve(config.projectRoot),
     harborBin: config.harborBin || process.env.HARBOR_BIN || checkoutExecutable('harbor'),
     harborDshBin: config.harborDshBin || process.env.HARBOR_DSH_BIN || checkoutExecutable('harbor-dsh'),
@@ -66,7 +81,12 @@ export function apply(ctx, config) {
     ),
   }
   const modelRuntime = new CandidateModelRuntime(ctx, resolved)
-  const service = new EvolutionService(resolved, { pluginVersion: packageJson.version }, modelRuntime)
+  const metadata = { pluginVersion: packageJson.version, projectRootSource: 'configured' }
+  const service = new EvolutionService(resolved, metadata, modelRuntime)
+  const serviceForTool = exec => {
+    const projectRoot = synchronizeWorkbenchProjectRoot(service, exec)
+    return new EvolutionService({ ...resolved, projectRoot }, metadata, modelRuntime)
+  }
 
   ctx.skills.register(loadBundledSkill())
   installDashboardWeb(ctx, service)
@@ -79,13 +99,20 @@ export function apply(ctx, config) {
       candidateId: { type: 'string' },
       version: { type: 'string' },
     },
-  }, args => service.snapshot(args)))
+  }, (args, exec) => serviceForTool(exec).snapshot(args)))
+
+  ctx.tools.register(jsonTool({
+    name: 'harbor_model_binding',
+    description: 'Freeze the current DSH default provider, model, and reasoning identity into a non-secret model-binding.json draft. Runtime access still uses the short-lived Host Model Broker capability.',
+    parameters: {},
+  }, (_args, exec) => serviceForTool(exec).modelBinding()))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evolution_init',
     description: 'Compile an accepted Dataset, Generator, Evaluator/criteria, and Optimizer onboarding card into a strict, non-overwriting Evaluation Stack project. Detailed identity fields are internal tool inputs, not a user questionnaire.',
     parameters: {
       datasetPath: { type: 'string', required: true },
+      workspaceSubdir: { type: 'string', description: 'Optional namespace under the current project root. Defaults to the project root; use it to host multiple independent Harbor projects.' },
       stackId: { type: 'string', required: true },
       stackVersion: { type: 'string', required: true },
       datasetId: { type: 'string', required: true },
@@ -101,7 +128,7 @@ export function apply(ctx, config) {
       policyVersion: { type: 'string', required: true },
       minImprovement: { type: 'number', required: true },
     },
-  }, args => service.initialize(args)))
+  }, (args, exec) => serviceForTool(exec).initialize(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evolution_doctor',
@@ -116,7 +143,17 @@ export function apply(ctx, config) {
       candidateModel: { type: 'string' },
       candidateReasoningEffort: { type: 'string' },
     },
-  }, args => service.doctor(args)))
+  }, (args, exec) => serviceForTool(exec).doctor(args)))
+
+  ctx.tools.register(jsonTool({
+    name: 'harbor_quick_diagnostic_init',
+    description: 'Create a non-overwriting Harbor 1.4 wiring diagnostic with one Query, a minimal Host-model Candidate, a runnable Task, and an explicit non-promotion Evaluator. The supplied Rubric is recorded as a draft but is not treated as executed.',
+    parameters: {
+      query: { type: 'string', required: true },
+      rubric: { type: 'string', required: true },
+      workspaceSubdir: { type: 'string', description: 'Defaults to harbor-diagnostic under the current Agent session directory.' },
+    },
+  }, (args, exec) => serviceForTool(exec).quickDiagnostic(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_dataset_validate',
@@ -124,7 +161,7 @@ export function apply(ctx, config) {
     parameters: {
       datasetPath: { type: 'string', required: true },
     },
-  }, args => service.validateDataset(args)))
+  }, (args, exec) => serviceForTool(exec).validateDataset(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_context_preview',
@@ -140,7 +177,7 @@ export function apply(ctx, config) {
       candidateModel: { type: 'string' },
       candidateReasoningEffort: { type: 'string' },
     },
-  }, args => service.previewContext(args)))
+  }, (args, exec) => serviceForTool(exec).previewContext(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_eval_run',
@@ -158,7 +195,7 @@ export function apply(ctx, config) {
       candidateModel: { type: 'string' },
       candidateReasoningEffort: { type: 'string' },
     },
-  }, args => service.run(args)))
+  }, (args, exec) => serviceForTool(exec).run(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_eval_result',
@@ -170,7 +207,7 @@ export function apply(ctx, config) {
       compareJob: { type: 'string', description: 'Optional previous Job for view=governance impact analysis' },
       since: { type: 'string', description: 'Optional ISO timestamp for incremental progress changes' },
     },
-  }, args => service.result(args)))
+  }, (args, exec) => serviceForTool(exec).result(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evaluator_inspect',
@@ -178,7 +215,7 @@ export function apply(ctx, config) {
     parameters: {
       stackPath: { type: 'string', description: 'Defaults to .harbor/evaluation-stack.yml' },
     },
-  }, args => service.evaluatorInspect(args)))
+  }, (args, exec) => serviceForTool(exec).evaluatorInspect(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evaluator_update',
@@ -191,13 +228,14 @@ export function apply(ctx, config) {
       newEvaluatorVersion: { type: 'string', required: true },
       newStackVersion: { type: 'string', required: true },
     },
-  }, args => service.evaluator(args)))
+  }, (args, exec) => serviceForTool(exec).evaluator(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_ground_truth_init',
     description: 'Create a non-overwriting Ground Truth draft for evaluator meta-evaluation. GT may be human, programmatic, consensus, model, or external, but must have explicit provenance and remain independent of the Candidate evaluator.',
     parameters: {
       outputPath: { type: 'string', description: 'Defaults to .harbor/ground-truth.json' },
+      evaluationRoot: { type: 'string', description: 'Optional evaluation workspace root used to register custom Ground Truth paths.' },
       groundTruthId: { type: 'string', required: true },
       version: { type: 'string', required: true },
       sourceKind: { type: 'string', required: true, description: 'human, programmatic, consensus, model, or external' },
@@ -205,7 +243,7 @@ export function apply(ctx, config) {
       provenance: { type: 'string', required: true },
       criteria: { type: 'string', required: true, description: 'Comma-separated criterion ids' },
     },
-  }, args => service.groundTruthInitialize(args)))
+  }, (args, exec) => serviceForTool(exec).groundTruthInitialize(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_evaluator_meta_evaluate',
@@ -214,8 +252,9 @@ export function apply(ctx, config) {
       groundTruthPath: { type: 'string', description: 'Defaults to .harbor/ground-truth.json' },
       observationsPath: { type: 'string', required: true },
       outputPath: { type: 'string', description: 'Defaults to .harbor/meta-evaluation-report.json' },
+      evaluationRoot: { type: 'string', description: 'Optional evaluation workspace root used to register custom report paths.' },
     },
-  }, args => service.evaluatorMetaEvaluate(args)))
+  }, (args, exec) => serviceForTool(exec).evaluatorMetaEvaluate(args)))
 
   ctx.tools.register(jsonTool({
     name: 'harbor_candidate_compare',
@@ -225,5 +264,5 @@ export function apply(ctx, config) {
       candidateJob: { type: 'string', required: true },
       policyPath: { type: 'string', required: true },
     },
-  }, args => service.compare(args)))
+  }, (args, exec) => serviceForTool(exec).compare(args)))
 }

package/lib/candidate.js

@@ -2,7 +2,10 @@ import { createHash } from 'node:crypto'
 import { mkdir, readFile, readdir, stat, writeFile } from 'node:fs/promises'
 import path from 'node:path'
 
+import { CANDIDATE_ACP_PACKAGE, DSH_RUNTIME_VERSION, RUNTIME_POLICY } from './runtime-identity.js'
+
 export const MANIFEST_NAME = 'candidate-manifest.json'
+export const MODEL_BINDING_NAME = 'model-binding.json'
 const DIGEST_PREFIX = Buffer.from('harbor-dsh-candidate-v1\0')
 const EXCLUDED_DIRS = new Set(['.git', 'node_modules', '__pycache__', '.harbor-runtime'])
 const EXCLUDED_FILES = new Set([MANIFEST_NAME, '.DS_Store'])
@@ -16,6 +19,7 @@ const CREDENTIAL_FILES = new Set([
   'id_rsa',
   'id_ed25519',
 ])
+const MODEL_BINDING_KEYS = new Set(['schema_version', 'source', 'provider', 'model', 'reasoning_effort'])
 
 async function walk(root, current = root) {
   const entries = await readdir(current, { withFileTypes: true })
@@ -33,6 +37,45 @@ async function walk(root, current = root) {
   return files.sort((a, b) => Buffer.compare(Buffer.from(a.relative), Buffer.from(b.relative)))
 }
 
+export async function loadModelBinding(candidateDir) {
+  const pathname = path.join(path.resolve(candidateDir), MODEL_BINDING_NAME)
+  let value
+  try {
+    value = JSON.parse(await readFile(pathname, 'utf8'))
+  } catch (error) {
+    if (error.code === 'ENOENT') return undefined
+    if (error instanceof SyntaxError) throw new Error(`${MODEL_BINDING_NAME} is not valid JSON`)
+    throw error
+  }
+  if (value === null || typeof value !== 'object' || Array.isArray(value)) {
+    throw new Error(`${MODEL_BINDING_NAME} must be an object`)
+  }
+  const unknown = Object.keys(value).filter(key => !MODEL_BINDING_KEYS.has(key)).sort()
+  if (unknown.length) {
+    throw new Error(`${MODEL_BINDING_NAME} contains unsupported or secret-bearing fields: ${unknown.join(', ')}`)
+  }
+  if (value.schema_version !== 1) throw new Error(`${MODEL_BINDING_NAME} requires schema_version=1`)
+  const source = typeof value.source === 'string' ? value.source.trim() : ''
+  const provider = typeof value.provider === 'string' ? value.provider.trim() : ''
+  const model = typeof value.model === 'string' ? value.model.trim() : ''
+  if (!source || !provider || !model) {
+    throw new Error(`${MODEL_BINDING_NAME} requires non-empty source, provider, and model`)
+  }
+  const reasoningEffort = value.reasoning_effort === undefined
+    ? undefined
+    : typeof value.reasoning_effort === 'string' ? value.reasoning_effort.trim() : ''
+  if (value.reasoning_effort !== undefined && !reasoningEffort) {
+    throw new Error(`${MODEL_BINDING_NAME} reasoning_effort must be a non-empty string when present`)
+  }
+  return {
+    schema_version: 1,
+    source,
+    provider,
+    model,
+    ...(reasoningEffort ? { reasoning_effort: reasoningEffort } : {}),
+  }
+}
+
 export async function computeCandidate(candidateDir) {
   const root = path.resolve(candidateDir)
   if (!(await stat(root)).isDirectory()) throw new Error(`Candidate path is not a directory: ${root}`)
@@ -83,6 +126,7 @@ async function validateCandidateContract(root) {
   if (credentialPaths.length > 0) {
     throw new Error(`Candidate contains credential-bearing files: ${credentialPaths.join(', ')}; inject credentials at runtime instead`)
   }
+  await loadModelBinding(root)
 }
 
 export async function snapshotCandidate(candidateDir, options = {}) {
@@ -96,11 +140,18 @@ export async function snapshotCandidate(candidateDir, options = {}) {
   }
   const candidateId = options.candidateId ?? packageJson.name
   const version = options.version ?? packageJson.version
-  const runtimeVersion = options.runtimeVersion ?? '0.1.0-rc.6'
-  if (!candidateId || !version || !runtimeVersion) {
-    throw new Error('Candidate id, version, and runtime version must not be empty; set package.json name/version or pass explicit values')
+  if (!candidateId || !version) {
+    throw new Error('Candidate id and version must not be empty; set package.json name/version or pass explicit values')
   }
   const computed = await computeCandidate(root)
+  const metadata = { ...(options.metadata ?? {}) }
+  const modelBinding = await loadModelBinding(root)
+  if (modelBinding) {
+    if (metadata.model_binding !== undefined && JSON.stringify(metadata.model_binding) !== JSON.stringify(modelBinding)) {
+      throw new Error('Candidate metadata model_binding must match model-binding.json')
+    }
+    metadata.model_binding = modelBinding
+  }
   const manifest = {
     schema_version: 1,
     candidate_id: String(candidateId),
@@ -109,11 +160,13 @@ export async function snapshotCandidate(candidateDir, options = {}) {
     created_at: new Date().toISOString(),
     runtime: {
       kind: 'deepseek-harness',
-      version: String(runtimeVersion),
+      policy: RUNTIME_POLICY,
+      version: DSH_RUNTIME_VERSION,
+      package: CANDIDATE_ACP_PACKAGE,
       transport: 'acp',
     },
     files: computed.files,
-    metadata: options.metadata ?? {},
+    metadata,
   }
   await mkdir(root, { recursive: true })
   await writeFile(path.join(root, MANIFEST_NAME), `${JSON.stringify(manifest, null, 2)}\n`)