harness-everything 0.29.0 → 0.30.1

package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-sonnet.md

@@ -42,10 +42,10 @@ hasn't been confirmed present. Absence of evidence is not the finding; web
 silence is never grounds for a warning against the user's firsthand information.
 Confirm, then flag.
 
-**Warning threshold.** Keep a running count of minor concerns. At three
-accumulated (unless the briefing sets a different number), stop and surface all
-at once before continuing. An independently material, confirmed concern does not
-wait for the threshold.
+**Warning batching.** Collect minor concerns and list them together in the next
+natural report or handoff; their count alone MUST NOT stop, pause, or return the
+worker/stage. An independently material, confirmed concern may stop the current
+stage and be surfaced immediately.
 
 **Find-and-replace safety.** Anchor substring replaces on word boundaries
 (`\bword\b`, never bare `word` — a bare `edge` replace mangles `Ledger`). Prefer

package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md

@@ -7,7 +7,7 @@ description: >-
 license: Apache-2.0
 metadata:
   author: Miya Daniel
-  version: 0.16.3
+  version: 0.30.1
 ---
 
 # Execution Guardrails
@@ -17,7 +17,7 @@ metadata:
 | Component | Spec |
 | :--- | :--- |
 | **Trigger / Input** | Turn raising a warning or performing search-and-replace edits. |
-| **Expected Output** | Verified warnings; batched caveats (3-item threshold); corruption-checked edits. |
+| **Expected Output** | Verified warnings; batched caveats without count-based stops; corruption-checked edits. |
 | **State Mutations** | None — governs execution quality for file-editing turns. |
 | **Enforcement Gate** | Verify before flagging; validate edits post-write. |
 
@@ -35,9 +35,11 @@ metadata:
 ## Core Rules
 
 1. **Verify before flag** — Confirm a problem exists before reporting it. Never convert absence of evidence into a warning.
-2. **Warning threshold** — Batch minor concerns; surface them together at three (default). Material concerns do not wait.
+2. **Warning batching.** Collect minor concerns and list them together in the next natural report or handoff; their count alone MUST NOT stop, pause, or return the worker/stage. An independently material, confirmed concern may stop the current stage and be surfaced immediately.
 3. **Find-and-replace safety** — Prefer structured edit tools over shell `sed`; anchor with unique context or `\bword\b` boundaries; verify file integrity post-edit. Never replace-all blindly.
 
+Repeated matching execution failures remain governed by Rule-of-3 / `zoom-out`; warning batching never substitutes for that recovery path.
+
 These rules are always-on, even when `fable-mode`'s staged loop is not running.
 
 Deep dive: <this-skill-dir>/references/guardrail-rules.md

package/plugins/harness-everything/skills/fable-mode/execution-guardrails/references/guardrail-rules.md

@@ -26,17 +26,21 @@ attempted and where it failed — not a vague appeal to difficulty.
 
 ## 2. Warning threshold
 
-Across any run, minor concerns accumulate that aren't worth halting on individually.
-Keep a running count. At the threshold — **default three, tunable if the user sets a
-different number** — stop and surface all of them to the user at once before continuing.
+Across any run, minor concerns accumulate that are not worth interrupting execution
+for individually. Keep a running count. At the threshold — **default three, tunable if
+the user sets a different number** — batch them for the next natural report or handoff.
+The threshold changes presentation only: it MUST NOT stop, pause, return, or otherwise
+end a worker/stage by count alone.
 
-Rationale: three small things pointing the same direction usually mean one real problem
-worth a decision. Below threshold, keep working; a drip of trivial caveats is noise.
-At threshold, batch them — one interruption with full context beats three fragmentary
-ones.
+Rationale: three small things pointing the same direction are easier to evaluate as one
+batch. Below threshold, keep working; a drip of trivial caveats is noise. At threshold,
+keep working and preserve the batch for the report instead of creating a second circuit
+breaker.
 
 A concern that independently meets the verify-before-flag bar and is material on its own
-does not wait for the threshold. The threshold governs minor concerns only.
+may stop the current stage and be surfaced immediately. Repeated matching execution
+failures are governed separately by Rule-of-3 -> zoom-out -> RESUME/ESCALATE. The
+warning threshold governs minor-concern reporting only.
 
 ## 3. Find-and-replace safety
 

package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md

@@ -4,7 +4,7 @@ description: "Run staged fable work on Haiku for bulk mechanical tasks with expl
 license: Apache-2.0
 metadata:
   author: Miya Daniel
-  version: 0.16.3
+  version: 0.30.1
 ---
 
 # Fable Mode — Haiku

package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md

@@ -4,7 +4,7 @@ description: "Run staged fable orchestration on Opus for cross-stage synthesis,
 license: Apache-2.0
 metadata:
   author: Miya Daniel
-  version: 0.16.3
+  version: 0.30.1
 ---
 
 # Fable Mode — Opus

package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md

@@ -4,7 +4,7 @@ description: "Run staged fable work on Sonnet for non-trivial implementation, re
 license: Apache-2.0
 metadata:
   author: Miya Daniel
-  version: 0.16.3
+  version: 0.30.1
 ---
 
 # Fable Mode — Sonnet

package/plugins/harness-everything/skills/harness-everything/SKILL.md

@@ -4,7 +4,7 @@ description: "Route software/project work through Harness: classify tier/topolog
 license: Apache-2.0
 metadata:
   author: Miya Daniel
-  version: 0.29.0
+  version: 0.30.0
 ---
 
 # Harness Everything

package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js

@@ -105,6 +105,15 @@ function persistWorkflow(contract, payload, prompt) {
   return { ...context, hooksRoot, retained: false, memoryCapability };
 }
 
+// Printed last so it is the nearest instruction when the turn ends (docs/system-one-observations.md#label-line).
+function printLabelContract() {
+  console.log('\n=> TURN LABEL LINE (MUST, EVERY TURN — short answers, lookups and trivial edits included):');
+  console.log('   - The last line of your final message MUST be one hidden label line with your own post-work judgement, not a copy of the router tier above:');
+  console.log('     <!-- harness-label {"v":1,"validity":"actionable|invalid","contextDependent":true|false,"tier":"tier1|tier2|tier3|null","intents":["up to 3 of explain,discuss,git,fix,edit,feature,refactor,review,test,docs,plan,investigate"],"workflow":"strategy id or null","skills":["skills used"]} -->');
+  console.log('   - tier1: answer, lookup, status or git operation with no code change; tier2: bounded change; tier3: new feature, refactor, redefinition or cross-component work. Use null only when validity is invalid.');
+  console.log('   - It is invisible in rendered Markdown and records local training data; it never gates routing.');
+}
+
 function run(raw) {
   const promptArg = process.argv.slice(2).join(' ');
   let payload = null;
@@ -162,6 +171,7 @@ function run(raw) {
   } else {
     console.log('   - Runtime state was not persisted; reminders may be less precise, but execution remains available.');
   }
+  printLabelContract();
   if (failure) console.error('[Workflow Reminder] routing state was not persisted: ' + failure.message);
   if (result && result.status !== 0) console.error('[Workflow Reminder] router returned status ' + result.status + '; contract observation is degraded, not blocking.');
 }

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/catalogs.js

@@ -0,0 +1,35 @@
+'use strict';
+// Fixed System One catalogs, one per stage (docs/system-one-routing.md#staged-classification).
+// Option order is the scorer's output order; changing it invalidates trained weights.
+const { TIER_OPTIONS } = require('./router');
+const INTENT_OPTIONS = Object.freeze([
+  { id: 'explain', text: 'Answer a question or explain how something works.' },
+  { id: 'discuss', text: 'Weigh options, give an opinion or settle a decision.' },
+  { id: 'git', text: 'Git or GitHub housekeeping: commit, push, branch, pull request.' },
+  { id: 'fix', text: 'Repair wrong behavior: a bug, crash, failing test or build.' },
+  { id: 'edit', text: 'Change an existing value, setting, text or behavior on request.' },
+  { id: 'feature', text: 'Add new behavior: a command, option, format or script.' },
+  { id: 'refactor', text: 'Restructure or unify existing code without new behavior.' },
+  { id: 'review', text: 'Evaluate an existing artifact against a standard.' },
+  { id: 'test', text: 'Run tests or builds, or write tests as the main deliverable.' },
+  { id: 'docs', text: 'Write or edit documentation as the main deliverable.' },
+  { id: 'plan', text: 'Produce a plan, specification or breakdown before implementation.' },
+  { id: 'investigate', text: 'Find facts or a cause and report back without changing anything.' },
+  { id: 'unclassified', text: 'Unclear intent, insufficient context or outside these categories.' },
+].map(Object.freeze));
+const CATALOGS = Object.freeze({ tier: TIER_OPTIONS, intent: INTENT_OPTIONS });
+// Gold labels of a stage: its catalog ids, with the abstaining option recorded as null.
+function goldLabels(task) {
+  if (!Object.hasOwn(CATALOGS, task)) throw new TypeError('unknown-task');
+  return CATALOGS[task].map(o => (o.id === 'unclassified' ? null : o.id));
+}
+// Stages whose labels carry secondary intents beside the primary choice (docs/system-one-intent.md).
+const MULTI = Object.freeze(['intent']);
+// Secondary labels: distinct catalog ids other than the primary; none when the primary abstains.
+function validSecondary(task, primary, secondary) {
+  if (!MULTI.includes(task)) return secondary === undefined;
+  const ids = goldLabels(task).filter(g => g !== null);
+  return Array.isArray(secondary) && new Set(secondary).size === secondary.length
+    && secondary.every(s => ids.includes(s) && s !== primary) && (primary !== null || secondary.length === 0);
+}
+module.exports = { INTENT_OPTIONS, CATALOGS, MULTI, goldLabels, validSecondary };

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/contract.js

@@ -42,4 +42,48 @@ function decide(req, res, policy = {}) {
   return { status: reason === 'scored' ? 'accepted' : 'abstain', reason,
     selectedId: reason === 'scored' ? ranked[0].id : null, confidence, margin, model: { ...res.model } };
 }
-module.exports = { createRequest, validateRequest, decide };
+// Relevance-native decisions (#233): every option except unclassified is an
+// independent Bernoulli against its own threshold. No simplex rule applies:
+// the raw probabilities must each stand in 0..1 but need not sum to one.
+function validThresholds(req, thresholds) {
+  const ids = req.options.map(o => o.id).filter(id => id !== 'unclassified');
+  if (!thresholds || typeof thresholds !== 'object' || Array.isArray(thresholds)) return false;
+  const keys = Object.keys(thresholds);
+  return keys.length === ids.length && ids.every(id => probability(thresholds[id]))
+    && keys.every(k => ids.includes(k));
+}
+function decideIntents(req, res, policy = {}) {
+  validateRequest(req);
+  if (!policy || typeof policy !== 'object' || Array.isArray(policy)
+    || Object.keys(policy).some(k => !['domain', 'thresholds'].includes(k))
+    || !validThresholds(req, policy.thresholds)) fail('invalid-policy');
+  const { domain = 'harness-routing-v1', thresholds } = policy;
+  if (!text(domain)) fail('invalid-policy');
+  const empty = { status: 'invalid-output', reason: 'invalid-response', selectedId: null,
+    confidence: null, margin: null, model: null, intents: null };
+  if (!exact(res, ['schemaVersion', 'requestHash', 'catalogHash', 'model', 'scores'])
+    || res.schemaVersion !== 1 || res.requestHash !== req.requestHash || res.catalogHash !== req.catalogHash
+    || !exact(res.model, ['id', 'revision', 'domain']) || !Object.values(res.model).every(v => text(v))
+    || !Array.isArray(res.scores) || res.scores.length !== req.options.length
+    || res.scores.some((s, i) => !exact(s, ['id', 'probability']) || s.id !== req.options[i].id || !probability(s.probability))) return empty;
+  const model = { ...res.model };
+  if (model.domain !== domain) {
+    return { status: 'abstain', reason: 'domain-mismatch', selectedId: null,
+      confidence: null, margin: null, model, intents: null };
+  }
+  const intents = {};
+  for (const s of res.scores) {
+    if (s.id === 'unclassified') continue;
+    intents[s.id] = { probability: s.probability, fired: s.probability >= thresholds[s.id] };
+  }
+  const fired = Object.entries(intents).filter(([, v]) => v.fired).sort((a, b) => b[1].probability - a[1].probability);
+  if (!fired.length) {
+    return { status: 'abstain', reason: 'no-intent-fired', selectedId: null,
+      confidence: null, margin: null, model, intents };
+  }
+  const rest = Object.values(intents).filter(v => v !== fired[0][1]).map(v => v.probability);
+  const confidence = fired[0][1].probability;
+  return { status: 'accepted', reason: 'scored', selectedId: fired[0][0], confidence,
+    margin: confidence - Math.max(...rest, 0), model, intents };
+}
+module.exports = { createRequest, validateRequest, decide, decideIntents };

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/evaluate.js

@@ -1,26 +1,32 @@
 'use strict';
 const { validateRequest } = require('./contract');
-const { TIER_OPTIONS } = require('./router');
+const { CATALOGS, MULTI, goldLabels, validSecondary } = require('./catalogs');
+const { gradedAgreement, familyConsistency } = require('./intent');
 const exact = (obj, keys) => obj && typeof obj === 'object' && !Array.isArray(obj) && Object.keys(obj).length === keys.length && keys.every(k => Object.hasOwn(obj, k));
 const nonempty = v => typeof v === 'string' && v.trim().length > 0;
-const labels = ['tier1', 'tier2', 'tier3', null];
 const finite = v => typeof v === 'number' && Number.isFinite(v);
 const fail = () => { throw new TypeError('invalid-evaluation-input'); };
-function validateCorpus(corpus) {
+// A corpus holds one stage: every request uses that stage's fixed catalog. Returns the stage's gold labels.
+function corpusLabels(corpus) {
   if (!exact(corpus, ['schemaVersion', 'cases']) || corpus.schemaVersion !== 1 || !Array.isArray(corpus.cases) || !corpus.cases.length) fail();
+  const task = corpus.cases[0]?.request?.task;
+  if (!Object.hasOwn(CATALOGS, task)) fail();
+  const labels = goldLabels(task); const catalog = JSON.stringify(CATALOGS[task]);
   const ids = new Set(); const hashes = new Set(); const families = new Map();
   for (const c of corpus.cases) {
-    if (!exact(c, ['id', 'family', 'split', 'language', 'source', 'reviewed', 'request', 'gold'])
+    if (!exact(c, ['id', 'family', 'split', 'language', 'source', 'reviewed', 'request', 'gold', ...(MULTI.includes(task) ? ['secondary'] : [])])
       || ![c.id, c.family, c.source].every(nonempty) || !['train', 'validation', 'holdout'].includes(c.split)
-      || !['en', 'zh-TW'].includes(c.language) || typeof c.reviewed !== 'boolean' || !labels.includes(c.gold)) fail();
+      || !['en', 'zh-TW'].includes(c.language) || typeof c.reviewed !== 'boolean' || !labels.includes(c.gold)
+      || !validSecondary(task, c.gold, c.secondary)) fail();
     validateRequest(c.request);
-    if (c.request.task !== 'tier' || JSON.stringify(c.request.options) !== JSON.stringify(TIER_OPTIONS)
+    if (c.request.task !== task || JSON.stringify(c.request.options) !== catalog
       || ids.has(c.id) || hashes.has(c.request.requestHash) || (families.has(c.family) && families.get(c.family) !== c.split)) fail();
     ids.add(c.id); hashes.add(c.request.requestHash); families.set(c.family, c.split);
   }
-  return true;
+  return labels;
 }
-function prediction(decision) {
+function validateCorpus(corpus) { corpusLabels(corpus); return true; }
+function prediction(decision, labels) {
   if (exact(decision, ['status', 'reason']) && decision.status === 'unavailable' && nonempty(decision.reason)) return null;
   if (!exact(decision, ['status', 'reason', 'selectedId', 'confidence', 'margin', 'model'])
     || !['accepted', 'abstain', 'invalid-output'].includes(decision.status) || !nonempty(decision.reason)
@@ -31,15 +37,15 @@ function prediction(decision) {
   return decision.selectedId;
 }
 // Scored decisions carry the full catalog-order probability vector; unavailable/invalid outputs carry null.
-function checkScores(decision, scores) {
+function checkScores(decision, scores, labels) {
   const scored = ['accepted', 'abstain'].includes(decision.status);
   if (!scored) { if (scores !== null) fail(); return; }
-  if (!Array.isArray(scores) || scores.length !== TIER_OPTIONS.length
+  if (!Array.isArray(scores) || scores.length !== labels.length
     || scores.some(p => !finite(p) || p < 0 || p > 1) || Math.abs(scores.reduce((s, p) => s + p, 0) - 1) > 1e-6) fail();
   const ranked = [...scores].sort((a, b) => b - a);
   if (decision.confidence !== ranked[0] || decision.margin !== ranked[0] - ranked[1]) fail();
 }
-function metrics(gold, predicted) {
+function metrics(gold, predicted, labels) {
   const count = gold.length;
   const accepted = predicted.filter(x => x !== null).length;
   const correct = predicted.filter((x, i) => x === gold[i]).length;
@@ -55,9 +61,95 @@ function metrics(gold, predicted) {
     acceptedErrorRate: accepted ? (accepted - acceptedCorrect) / accepted : null };
 }
 function p95(values) { return values.length ? [...values].sort((a, b) => a - b)[Math.ceil(values.length * 0.95) - 1] : null; }
+// Intent corpora: accepted predictions are graded against { gold, secondary }; predicted secondaries are the
+// other intents whose probability reaches the checkpoint's secondary threshold (docs/system-one-intent.md#scoring).
+function gradeIntents(cases, byId, labels, threshold) {
+  const ids = labels.map(l => (l === null ? 'unclassified' : l));
+  let accepted = 0; let points = 0; let exact = 0;
+  for (const c of cases) {
+    const run = byId.get(c.id).runs[0]; const selected = prediction(run.decision, labels);
+    if (selected === null) continue;
+    const secondary = ids.filter((id, i) => id !== selected && id !== 'unclassified' && run.scores[i] >= threshold);
+    accepted++; points += gradedAgreement({ gold: c.gold, secondary: c.secondary }, { gold: selected, secondary }); if (selected === c.gold) exact++;
+  }
+  return { acceptedPrecision: accepted ? points / accepted : null, exactPrecision: accepted ? exact / accepted : null };
+}
+// Relevance-native grading (#233): every label except unclassified is an
+// independent Bernoulli against its own threshold. No simplex rule applies
+// to the score vector. Gates live in docs/system-one-intent-gates.md;
+// this function reports metrics only.
+function gradeRelevance(cases, byId, labels, thresholds) {
+  const ids = labels.map(l => (l === null ? 'unclassified' : l));
+  const per = new Map(ids.filter(id => id !== 'unclassified').map(id => [id, { tp: 0, fp: 0, fn: 0, support: 0 }]));
+  let abstain = 0; let firesTotal = 0; let truthTotal = 0; let exact = 0;
+  for (const c of cases) {
+    const scores = byId.get(c.id).scores;
+    const truth = new Set(c.gold === null ? [] : [c.gold, ...(c.secondary || [])]);
+    const fired = new Set(ids.filter((id, i) => id !== 'unclassified' && scores[i] >= thresholds[id]));
+    firesTotal += fired.size; truthTotal += truth.size;
+    abstain += fired.size === 0 ? 1 : 0;
+    exact += fired.size === truth.size && [...fired].every(id => truth.has(id)) ? 1 : 0;
+    for (const [id, row] of per) {
+      if (fired.has(id) && truth.has(id)) row.tp++;
+      else if (fired.has(id)) row.fp++;
+      else if (truth.has(id)) row.fn++;
+      if (truth.has(id)) row.support++;
+    }
+  }
+  const table = {};
+  for (const [id, row] of per) {
+    const p = row.tp + row.fp ? row.tp / (row.tp + row.fp) : 0;
+    const r = row.tp + row.fn ? row.tp / (row.tp + row.fn) : 0;
+    table[id] = { ...row, tau: thresholds[id], precision: p, recall: r, f1: p + r ? 2 * p * r / (p + r) : 0 };
+  }
+  const tp = [...per.values()].reduce((s, r) => s + r.tp, 0);
+  const fp = [...per.values()].reduce((s, r) => s + r.fp, 0);
+  const fn = [...per.values()].reduce((s, r) => s + r.fn, 0);
+  const precision = tp + fp ? tp / (tp + fp) : 0;
+  const recall = tp + fn ? tp / (tp + fn) : 0;
+  const tableEntries = [...per.entries()].map(([id, row]) => [id, table[id]]);
+  const scored = tableEntries.filter(([, t]) => t.support > 0 || t.tp + t.fp > 0);
+  const supported = tableEntries.filter(([, t]) => t.support > 0);
+  const n = cases.length;
+  return {
+    micro: { precision, recall, f1: precision + recall ? 2 * precision * recall / (precision + recall) : 0 },
+    macroF1: scored.length ? scored.reduce((s, [, t]) => s + t.f1, 0) / scored.length : 0,
+    perIntent: table,
+    cardinality: { predicted: n ? firesTotal / n : 0, gold: n ? truthTotal / n : 0 },
+    abstainRate: n ? abstain / n : 0,
+    exactSetMatch: n ? exact / n : 0,
+    weakPrecision: supported.length ? Math.min(...supported.map(([, t]) => t.precision)) : 0,
+  };
+}
+// Relevance-native promotion gates. Thresholds are the reviewed numbers in
+// docs/system-one-intent-gates.md; policy/live-host evidence stays reported
+// (false) until independently produced, so rolloutReady is red by
+// construction pre-production.
+function relevanceGates(multi, evidence) {
+  const supported = Object.values(multi.perIntent).filter(t => t.support >= 10);
+  const gates = {
+    microF1: multi.micro.f1 >= 0.55,
+    macroF1: multi.macroF1 >= 0.50,
+    perIntentFloors: supported.every(t => t.f1 >= 0.30),
+    weakPrecision: multi.weakPrecision >= 0.30,
+    cardinality: Math.abs(multi.cardinality.predicted - multi.cardinality.gold) <= 1.0,
+    abstainRate: multi.abstainRate <= 0.20,
+    familyConsistency: evidence.family.model.rate >= evidence.family.gold.rate,
+    repeatability: evidence.agreement === 1,
+    warmLatency: evidence.warmP95Ms <= 250,
+    structuralControls: !!(evidence.structural && evidence.structural.unchanged && evidence.structural.rollbackLive),
+    policyEvidence: false,
+    liveHostEvidence: false,
+  };
+  // rolloutReady stays false until policy/live-host evidence exists; the
+  // gates above are necessary but not sufficient pre-production.
+  return { ...gates, rolloutReady: false };
+}
 // sourceEvidence is the provider provenance result; only the pinned cua_s1 revision satisfies the gate.
-function evaluate(corpus, records, sourceEvidence = null) {
-  validateCorpus(corpus);
+function evaluate(corpus, records, sourceEvidence = null, { secondaryThreshold } = {}) {
+  const labels = corpusLabels(corpus);
+  const multi = MULTI.includes(corpus.cases[0].request.task);
+  if (multi && !(finite(secondaryThreshold) && secondaryThreshold >= 0.05 && secondaryThreshold <= 0.5)) fail();
   const cases = corpus.cases.filter(c => c.split === 'holdout');
   if (!cases.length || !Array.isArray(records) || records.length !== cases.length) fail();
   const byId = new Map(); const models = new Set(); const cold = []; const warm = [];
@@ -66,8 +158,8 @@ function evaluate(corpus, records, sourceEvidence = null) {
       || !labels.includes(record.baseline) || !Array.isArray(record.runs) || record.runs.length !== 2) fail();
     for (const run of record.runs) {
       if (!exact(run, ['decision', 'scores', 'latencyMs', 'coldStart']) || !finite(run.latencyMs) || run.latencyMs < 0 || typeof run.coldStart !== 'boolean') fail();
-      prediction(run.decision);
-      checkScores(run.decision, run.scores);
+      prediction(run.decision, labels);
+      checkScores(run.decision, run.scores, labels);
       if (run.decision.model) models.add(JSON.stringify([run.decision.model.id, run.decision.model.revision, run.decision.model.domain]));
       (run.coldStart ? cold : warm).push(run.latencyMs);
     }
@@ -75,8 +167,12 @@ function evaluate(corpus, records, sourceEvidence = null) {
   }
   if (models.size > 1) fail();
   const gold = cases.map(c => c.gold);
-  const baseline = metrics(gold, cases.map(c => byId.get(c.id).baseline));
-  const model = metrics(gold, cases.map(c => prediction(byId.get(c.id).runs[0].decision)));
+  const baseline = metrics(gold, cases.map(c => byId.get(c.id).baseline), labels);
+  const predicted = cases.map(c => prediction(byId.get(c.id).runs[0].decision, labels));
+  const model = { ...metrics(gold, predicted, labels), ...(multi ? gradeIntents(cases, byId, labels, secondaryThreshold) : {}) };
+  const families = multi ? { familyConsistency: {
+    gold: familyConsistency(cases.map(c => ({ family: c.family, primary: c.gold }))),
+    model: familyConsistency(cases.map((c, i) => ({ family: c.family, primary: predicted[i] }))) } } : {};
   const canonicalRun = ({ decision: d, scores }) => JSON.stringify([d.status, d.reason, d.selectedId ?? null, d.confidence ?? null, d.margin ?? null,
     d.model ? [d.model.id, d.model.revision, d.model.domain] : null, scores]);
   const agreement = records.filter(r => canonicalRun(r.runs[0]) === canonicalRun(r.runs[1])).length / cases.length;
@@ -94,6 +190,6 @@ function evaluate(corpus, records, sourceEvidence = null) {
     policyEvidence: false,
     liveHostEvidence: false,
   };
-  return { schemaVersion: 1, cases: cases.length, baseline, model, agreement, latency, gates, rolloutReady: Object.values(gates).every(Boolean) };
+  return { schemaVersion: 1, cases: cases.length, baseline, model, ...families, agreement, latency, gates, rolloutReady: Object.values(gates).every(Boolean) };
 }
-module.exports = { validateCorpus, evaluate };
+module.exports = { validateCorpus, evaluate, gradeRelevance, relevanceGates };

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/intent.js

@@ -0,0 +1,22 @@
+'use strict';
+// Intent-stage comparisons (docs/system-one-intent.md#scoring). A label is { gold, secondary }.
+const full = l => new Set([l.gold, ...(l.secondary || [])]);
+// The owner's grades: same primary 1.0; swapped order 0.6; one-sided, or swapped with a
+// secondary-count gap of 3 or more, 0.3; otherwise 0.
+function gradedAgreement(a, b) {
+  if (a.gold === b.gold) return 1;
+  if (a.gold === null || b.gold === null) return 0;
+  const aInB = full(b).has(a.gold); const bInA = full(a).has(b.gold);
+  if (aInB && bInA) return Math.abs((a.secondary || []).length - (b.secondary || []).length) >= 3 ? 0.3 : 0.6;
+  return aInB || bInA ? 0.3 : 0;
+}
+// Share of families with two or more members whose members share one primary intent.
+function familyConsistency(rows) {
+  const byFamily = new Map();
+  for (const r of rows) { if (!byFamily.has(r.family)) byFamily.set(r.family, new Set()); byFamily.get(r.family).add(r.primary); }
+  const sizes = new Map(); for (const r of rows) sizes.set(r.family, (sizes.get(r.family) || 0) + 1);
+  const multi = [...byFamily].filter(([f]) => sizes.get(f) >= 2);
+  const consistent = multi.filter(([, set]) => set.size === 1).length;
+  return { families: multi.length, consistent, rate: multi.length ? consistent / multi.length : null };
+}
+module.exports = { gradedAgreement, familyConsistency };

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/laya_adapter.py

@@ -0,0 +1,141 @@
+"""Fine-tuned Laya intent provider bridge (#233 shadow, research use).
+
+Speaks the same adapter protocol as cua_adapter.py (oneshot stdin/stdout,
+resident --serve, --provenance). It emits RAW independent relevance scores
+(one per intent plus the unclassified ramp): they deliberately do NOT sum to
+one, so the single-winner contract.decide fails closed on them while
+contract.decideIntents consumes them directly. Do not re-normalize into a
+simplex anywhere in this path.
+"""
+import hashlib
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+_HERE = Path(__file__).resolve()
+if str(_HERE.parent) not in sys.path:
+    sys.path.insert(0, str(_HERE.parent))
+import cua_adapter  # noqa: E402  (resident serve machinery: socket server, token, phases)
+
+_label_spec = importlib.util.spec_from_file_location(
+    'laya_labeler', _HERE.parents[3] / 'scripts' / 'system-one-label-laya.py')
+_label = importlib.util.module_from_spec(_label_spec)
+_label_spec.loader.exec_module(_label)
+INTENTS = _label.INTENTS
+QUESTIONS_VERSION = _label.QUESTIONS_VERSION
+build_questions = _label.build_questions
+
+# B1 evidence (full-model validation sweep): cutoff 0.45 maximizes graded
+# agreement with null rate near the teacher's 14%. Pinned, not tuned per run.
+NULL_CUTOFF = 0.45
+# Byte limits: holdout contexts peak at 368 B, validation at 1005 B; the
+# multilingual window is 1024 tokens (~4 KB). Rejection is fail-safe lexical.
+LIMITS = {'context_tokens': 4096, 'option_tokens': 1024}
+
+
+def verify_artifacts(manifest):
+    """Pin weights plus the training config beside them (rl_agent_config.json)."""
+    weights = Path(manifest['checkpoint'])
+    if not weights.is_absolute() or weights.suffix != '.safetensors':
+        raise ValueError('checkpoint-format')
+    config = weights.parent / 'rl_agent_config.json'
+    for file, key in [(weights, 'weightsSha256'), (config, 'configSha256')]:
+        with file.open('rb') as handle:
+            digest = hashlib.file_digest(handle, 'sha256').hexdigest()
+        if digest != manifest[key]:
+            raise ValueError('artifact-hash')
+    return weights
+
+
+def check_limits(request, config):
+    if len(request['context'].encode('utf-8')) > config['context_tokens'] or any(
+        len(option['text'].encode('utf-8')) > config['option_tokens'] for option in request['options']
+    ):
+        raise ValueError('input-too-long')
+
+
+def make_scorer(predict_fn, cutoff=NULL_CUTOFF):
+    """Build score(request) -> 13 raw probabilities in request-option order.
+
+    Raw means independent: 12 noul answers plus the unclassified ramp, each
+    in 0..1, with no sum-to-one normalization.
+    """
+    questions = build_questions()
+
+    def score(request):
+        if request.get('task') != 'intent':
+            raise ValueError('task: laya intent bridge only')
+        ids = [o['id'] for o in request['options']]
+        if set(INTENTS) - set(ids):
+            raise ValueError('options: all 12 intents required')
+        raw = predict_fn(request['context'], questions)
+        top = max(float(raw[i]) for i in INTENTS)
+        ramp = max(0.0, min(1.0, (cutoff - top) / cutoff)) if cutoff > 0 else 0.0
+        by_id = {i: float(raw[i]) for i in INTENTS}
+        by_id['unclassified'] = ramp
+        return [by_id[i] for i in ids]
+
+    return score, dict(LIMITS)
+
+
+def load_scorer(weights, manifest):
+    """Load the fine-tuned checkpoint once; return (score, limits)."""
+    import laya  # noqa: PLC0415
+    agent = laya.load(str(weights.parent))
+    questions = build_questions()
+
+    def predict_fn(state, _questions):
+        answers = agent.predict(state, questions)['answers']
+        return {intent: float(answers[intent]['noul']) for intent in INTENTS}
+
+    return make_scorer(predict_fn)
+
+
+def provenance():
+    """Installed laya facts only; never loads a checkpoint."""
+    import importlib.metadata as metadata  # noqa: PLC0415
+    import platform  # noqa: PLC0415
+
+    def version(name):
+        try:
+            return metadata.version(name)
+        except metadata.PackageNotFoundError:
+            return None
+
+    return {'schemaVersion': 1, 'python': {'implementation': platform.python_implementation(),
+                                           'version': platform.python_version()},
+            'laya': {'distribution': 'laya', 'version': version('laya'),
+                     'checkpoint': 'convaiinnovations/laya:multilingual',
+                     'questions': QUESTIONS_VERSION, 'nullCutoff': NULL_CUTOFF},
+            'torch': version('torch')}
+
+
+def infer(request, manifest):
+    weights = verify_artifacts(manifest)
+    score, config = load_scorer(weights, manifest)
+    check_limits(request, config)
+    return cua_adapter.build_response(request, manifest, score(request))
+
+
+def main():
+    try:
+        if sys.argv[1:] == ['--provenance']:
+            print(json.dumps(provenance(), allow_nan=False))
+            return 0
+        if sys.argv[1:2] == ['--serve']:
+            return cua_adapter.serve_main(sys.argv[2:], scorer_factory=load_scorer)
+        manifest = json.loads(sys.argv[1])
+        raw = sys.stdin.buffer.read(2 * 1024 * 1024 + 1)
+        if len(raw) > 2 * 1024 * 1024:
+            raise ValueError('request-limit')
+        request = json.loads(raw)
+        print(json.dumps(infer(request, manifest), allow_nan=False))
+    except Exception:
+        print('laya-provider-unavailable', file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

package/plugins/harness-everything/skills/harness-everything/scripts/system-one/provider.js

@@ -10,9 +10,15 @@ const unavailable = reason => ({ status: 'unavailable', reason });
 const exact = (value, keys) => value && typeof value === 'object' && !Array.isArray(value)
   && Object.keys(value).length === keys.length && keys.every(k => Object.hasOwn(value, k));
 const label = value => typeof value === 'string' && value.length > 0 && value.length <= 128 && !/[\u0000-\u001f]/.test(value);
+// Resident adapter selection: a basename inside the system-one scripts dir, never a path.
+const adapterName = value => typeof value === 'string' && /^[a-z][a-z0-9_-]{0,63}\.py$/.test(value);
 // Thresholds calibrated on validation for this checkpoint (docs/system-one-training.md).
 const inRange = (v, lo, hi) => typeof v === 'number' && Number.isFinite(v) && v >= lo && v <= hi;
-const validAcceptance = a => exact(a, ['minConfidence', 'minMargin']) && inRange(a.minConfidence, 0.5, 0.99) && inRange(a.minMargin, 0, 0.9);
+// secondaryThreshold is optional and belongs to intent checkpoints (docs/system-one-intent.md#scoring).
+const validAcceptance = a => (exact(a, ['minConfidence', 'minMargin']) || (exact(a, ['minConfidence', 'minMargin', 'secondaryThreshold']) && inRange(a.secondaryThreshold, 0.05, 0.5)))
+  && inRange(a.minConfidence, 0.5, 0.99) && inRange(a.minMargin, 0, 0.9);
+// The part of the acceptance thresholds that contract.decide takes.
+const decisionPolicy = acceptance => (acceptance ? { minConfidence: acceptance.minConfidence, minMargin: acceptance.minMargin } : {});
 // A scored result carries the manifest's thresholds so every caller decides with them.
 const withAcceptance = (result, manifest) => (result && result.status === 'scored' && manifest.acceptance
   ? { ...result, acceptance: { ...manifest.acceptance } } : result);
@@ -22,8 +28,9 @@ function validateManifest(m) {
   const required = ['schemaVersion', ...(ngram ? [] : ['python']), 'checkpoint', 'weightsSha256', 'configSha256', 'modelId', 'revision', 'domain'];
   const strings = ['modelId', 'revision', 'domain', ...(ngram && !Object.hasOwn(m, 'python') ? [] : ['python'])];
   if (!m || typeof m !== 'object' || Array.isArray(m) || required.some(k => !Object.hasOwn(m, k))
-    || Object.keys(m).some(k => ![...required, 'python', 'timeoutMs', 'transport', 'idleTimeoutMs', 'acceptance'].includes(k)) || m.schemaVersion !== 1
+    || Object.keys(m).some(k => ![...required, 'python', 'timeoutMs', 'transport', 'idleTimeoutMs', 'acceptance', 'adapter'].includes(k)) || m.schemaVersion !== 1
     || (m.acceptance !== undefined && !validAcceptance(m.acceptance))
+    || (m.adapter !== undefined && !adapterName(m.adapter))
     || strings.some(k => typeof m[k] !== 'string' || !m[k].trim() || m[k].length > 4096 || m[k].includes('\0'))
     || typeof m.checkpoint !== 'string' || !path.isAbsolute(m.checkpoint) || path.extname(m.checkpoint) !== (ngram ? '.bin' : '.safetensors')
     || ['weightsSha256', 'configSha256'].some(k => typeof m[k] !== 'string' || !/^[a-f0-9]{64}$/.test(m[k]))
@@ -105,4 +112,4 @@ function provenance(manifestPath) {
   try { return { status: 'recorded', provenance: validateProvenance(result.value) }; }
   catch (_) { return unavailable('provider-json'); }
 }
-module.exports = { score, scoreAsync, runProvider, validateManifest, readManifest, provenance, validateProvenance, PINNED_CUA_S1_REVISION };
+module.exports = { score, scoreAsync, runProvider, validateManifest, readManifest, provenance, validateProvenance, decisionPolicy, PINNED_CUA_S1_REVISION };