@useshifu/coding-harness 0.3.31 → 0.3.32

package/README.md

@@ -44,6 +44,8 @@ To disconnect this device, run `npx --yes @useshifu/coding-harness disconnect --
 
 ## Operate
 
+Version `0.3.32` preserves failed and pending source status during preparation, distinguishes people by their original source occurrences, and keeps the two-attempt review limit across metadata changes. Existing queues, V5 payloads and receipts remain compatible.
+
 Version `0.3.30` adds separately versioned episode and entity continuity without changing the V5 sync body or older queued payloads. Preparation uses bounded prior-turn references, source coverage checks and durable two-attempt review holds. Base acceptance and attachment completion have separate status. See [V5 continuity](#continuity-beside-version-five) below for privacy, recovery and mixed-version operation.
 
 Version `0.3.29` verifies native v5 sync across Codex, Claude Code, OpenCode 1 and OpenCode 2. Codex now loads a private allowlisted configuration and home instead of an ignored profile, with executable tools disabled. OpenCode 2 uses its standalone CLI, native provider/agent configuration, typed session database and isolated account storage. Native CI tests exercise original-source discovery, extraction, redaction, frozen retries and matching receipt checkpoints against local synthetic endpoints, without live credentials or paid models.
@@ -128,7 +130,7 @@ The AI Profile page shows detected categories, model and thinking labels, and Je
 
 The connection endpoint returns `recommendedPackageVersion`, pinned to a published release. Before `sync`, `sync-files`, `approve`, or `scheduled-sync`, the runner compares its embedded release version with that property. An older runner installs the exact recommended release through npm and resumes the original command with the updated runner. Equal or newer runners continue unchanged. Explicitly recommended major releases follow the same comparison rule. Missing or invalid recommendations, connection failures, and npm failures do not block sync. The connection, pending work, checkpoints, and schedule are preserved.
 
-This update mechanism first ships in `0.3.10`. Runners from earlier releases need a one-time installation of a release containing it (`npx --yes @useshifu/coding-harness@0.3.30 install --harness codex`). Raise the server recommendation only after the target release is published and its backend requirements are deployed.
+This update mechanism first ships in `0.3.10`. Runners from earlier releases need a one-time installation of a release containing it (`npx --yes @useshifu/coding-harness@0.3.32 install --harness codex`). Raise the server recommendation only after the target release is published and its backend requirements are deployed.
 
 ## Force sync local work
 
@@ -163,7 +165,7 @@ Setup preserves other instruction text and MCP settings. Package installation re
 The context branch adds a remote MCP connection using the same saved Codex key:
 
 ```sh
-npx --yes @useshifu/coding-harness@0.3.30 mcp-setup --harness codex --mcp-url https://YOUR-BRANCH-HOST/mcp
+npx --yes @useshifu/coding-harness@0.3.32 mcp-setup --harness codex --mcp-url https://YOUR-BRANCH-HOST/mcp
 ```
 
 For development, install dependencies and run from the checkout root:
@@ -194,7 +196,7 @@ Official configuration reference: https://learn.chatgpt.com/docs/config-file/con
 OpenCode uses the same `/mcp` endpoint and three tools, authenticated with its own saved OpenCode connection key:
 
 ```sh
-npx --yes @useshifu/coding-harness@0.3.30 mcp-setup --harness opencode --mcp-url https://YOUR-SHIFU-HOST/mcp
+npx --yes @useshifu/coding-harness@0.3.32 mcp-setup --harness opencode --mcp-url https://YOUR-SHIFU-HOST/mcp
 ```
 
 For checkout usage, install dependencies with `npm ci --prefix packages/shifu-coding-harness --ignore-scripts` from the checkout root, then run the checkout CLI with the same arguments. Setup preserves existing sync settings and other MCP servers, checks authenticated initialization and tool discovery, and installs the `shifu-context` usage instructions. A missing OpenCode connection prompts for its key in the terminal. A Codex key cannot authorize an OpenCode connection.
@@ -214,7 +216,7 @@ For the optional native integration gate, set `SHIFU_TEST_OPENCODE_BIN` to a tem
 Claude Code connects to the same `/mcp` endpoint and three tools with its own saved connection key:
 
 ```sh
-npx --yes @useshifu/coding-harness@0.3.30 mcp-setup --harness claude_code --mcp-url https://YOUR-SHIFU-HOST/mcp
+npx --yes @useshifu/coding-harness@0.3.32 mcp-setup --harness claude_code --mcp-url https://YOUR-SHIFU-HOST/mcp
 ```
 
 For checkout usage, install dependencies as above and run the checkout CLI with these arguments. Setup checks the existing Claude Code key, authenticated initialization and tool discovery, and installs the `shifu-context` skill. It preserves the sync endpoint, schedule, checkpoints and other MCP servers. A missing connection prompts for a Claude Code key in the terminal; do not substitute a Codex or OpenCode key.
@@ -234,11 +236,11 @@ Cursor support ships in `0.3.31` and requires backend migration `071_cursor_harn
 Install [Cursor Agent CLI](https://cursor.com/docs/cli/installation) and sign in with `agent login`. The desktop app's login does not necessarily authenticate the CLI. Then create a dedicated Cursor key in Shifu and run:
 
 ```sh
-npx --yes @useshifu/coding-harness@0.3.31 connect --harness cursor
-npx --yes @useshifu/coding-harness@0.3.31 mcp-setup --harness cursor
+npx --yes @useshifu/coding-harness@0.3.32 connect --harness cursor
+npx --yes @useshifu/coding-harness@0.3.32 mcp-setup --harness cursor
 ```
 
-For a checkout before publication, run `npm ci --prefix packages/shifu-coding-harness --ignore-scripts` and replace `npx --yes @useshifu/coding-harness@0.3.31` with `node packages/shifu-coding-harness/bin/shifu-harness.js`. `connect` installs the runner, skills and work-capture hooks, verifies the Cursor-specific key, saves history and feedback choices, and configures the existing sync schedule. MCP setup verifies authenticated initialization and discovery, preserves those choices and other MCP entries, and adds a local stdio bridge in `~/.cursor/mcp.json`. The bridge reads the protected Shifu connection file; no key is copied into Cursor configuration or process arguments. A conflicting existing `shifu` entry is preserved with a rename instruction. Other harness keys cannot authorize Cursor.
+For a checkout before publication, run `npm ci --prefix packages/shifu-coding-harness --ignore-scripts` and replace `npx --yes @useshifu/coding-harness@0.3.32` with `node packages/shifu-coding-harness/bin/shifu-harness.js`. `connect` installs the runner, skills and work-capture hooks, verifies the Cursor-specific key, saves history and feedback choices, and configures the existing sync schedule. MCP setup verifies authenticated initialization and discovery, preserves those choices and other MCP entries, and adds a local stdio bridge in `~/.cursor/mcp.json`. The bridge reads the protected Shifu connection file; no key is copied into Cursor configuration or process arguments. A conflicting existing `shifu` entry is preserved with a rename instruction. Other harness keys cannot authorize Cursor.
 
 Reload Cursor and start a new chat. Enable the Shifu MCP server if Cursor requests approval; CLI users can run `agent mcp enable shifu` and `agent mcp list-tools shifu`. Normal tool approvals still apply. Setup's HTTP probe is separate from native-client discovery and a model-driven tool call.
 

package/bin/continuity-preparation.js

@@ -15,6 +15,7 @@ schema.properties.evidence.items.properties.episode = { anyOf: [{ type: "null" }
 schema.properties.evidence.items.required.push("contextRefs", "entityRefs", "episode");
 schema.properties.entities = { type: "array", maxItems: 32, items: exactSchema({ ref: opaque, kind: { type: "string", enum: KINDS },
   visibility: { type: "string", enum: ["public", "private"] }, sourceRef: opaque, mention: { type: "string", minLength: 1, maxLength: 160 },
+  mentionIndex: { type: "integer", minimum: 0, maximum: 262144 },
   bindingRef: { type: "string", maxLength: 80 }, publicKey: { type: "string", enum: ["", ...Object.keys(CATALOG)] } }) };
 schema.properties.assessment = exactSchema({ disposition: { type: "string", enum: ["useful", "no_safe_context", "unresolved"] },
   emptyReason: { type: "string", enum: ["none", "no_reusable_material", "privacy_withheld", "non_work_noise"] },
@@ -29,7 +30,7 @@ Return the additional entities and assessment fields and per-item contextRefs, e
 
 Every emitted fact must have current sourceRefs. contextRefs may contain only preceding referenceSources locators. episode is null when no episode applies; otherwise anchorRef must identify an actual current or reference source, preferably the original proposal anchor when it is within lookback; stage must distinguish proposal, requested, reported, pending, revised or rejected. Repeated topic alone is not an episode.
 
-Preserve relevant technology names only through code-permitted public bindings: ${JSON.stringify(CATALOG)}. Every material selected named entity gets an entities entry with a short unique ref, kind, visibility, sourceRef and its exact source mention. publicKey must be an allowed key for public technology and bindingRef empty; otherwise publicKey is empty and visibility private. For private identity, select bindingRef only from relevant packet.entityBindings when the source supports the same identity, or use empty bindingRef for a new distinct entity. Equal names alone do not establish identity; ambiguous identities are unresolved. Confidential employer/customer relationships remain private even when the same word names public technology. The code, not you, assigns public labels and private aliases. Use {{ref}} placeholders for every entity mention in title, summary, decisions, verification, statement and workContext; never output raw private names there. Private placeholders expand to about fifty characters; keep enough room under all text limits after binding. Each evidence item lists pertinent entityRefs only. Entity source mention and raw identity are local-only and must not appear anywhere else in the result. Source role is authorship, never personal ownership.
+Preserve relevant technology names only through code-permitted public bindings: ${JSON.stringify(CATALOG)}. Every material selected named entity gets an entities entry with a short unique ref, kind, visibility, sourceRef, its exact source mention and mentionIndex (zero-based occurrence of that exact whole-word mention in that source). publicKey must be an allowed key for public technology and bindingRef empty; otherwise publicKey is empty and visibility private. For private identity, match packet.entityBindings sourceAnchors against the original sourceRef and mentionIndex, using the original surrounding words to distinguish roles and people. Reuse that explicit bindingRef for a known original occurrence; an empty bindingRef is only for a genuinely new occurrence. Two people called Alex in one message have different mentionIndex values and must keep their original bindings. Equal names alone do not establish identity; ambiguous identities are unresolved. When the current message refers to their earlier roles, identify the person through that preceding original source occurrence rather than creating a new binding from the repeated name. Confidential employer/customer relationships remain private even when the same word names public technology. The code, not you, assigns public labels and private aliases. Use {{ref}} placeholders for every entity mention in title, summary, decisions, verification, statement and workContext; never output raw private names there. Private placeholders expand to about fifty characters; keep enough room under all text limits after binding. Each evidence item lists pertinent entityRefs only. Entity source mention and raw identity are local-only and must not appear anywhere else in the result. Source role is authorship, never personal ownership.
 
 Assessment coverage accounts for every current source locator exactly once. represented entries identify distinct evidence indexes actually citing that source; irrelevant and privacy_withheld entries have no evidence indexes. A material omitted decision, pending check, constraint or ambiguity is unresolved. For useful output every decision and verification entry must have a pertinent evidence reference. Empty evidence cannot coexist with decisions or verification. no_safe_context requires an explicit reason and a neutral nonfactual title and summary; do not erase meaningful work merely to produce an empty result. unresolved is valid and keeps this range pending. Never create filler or require one claim per message.\n`;
 }
@@ -58,14 +59,41 @@ function metadata(source) {
   return { role: source.role, turn: source.turn, locator: source.locator, ...(source.eventAt ? { eventAt: source.eventAt } : {}), excerptMode: "synthesized" };
 }
 
-function sourceHasMention(content, mention) {
+function sourceMentionOffsets(content, mention) {
   const characters = [...mention];
   const word = /[\p{L}\p{N}\p{M}_]/u;
   const boundary = "[\\p{L}\\p{N}\\p{M}_]";
   const escaped = mention.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
   const before = word.test(characters[0]) ? `(?<!${boundary})` : "";
   const after = word.test(characters.at(-1)) ? `(?!${boundary})` : "";
-  return new RegExp(before + escaped + after, "u").test(content);
+  return [...content.matchAll(new RegExp(before + escaped + after, "gu"))].map((match) => match.index);
+}
+
+function sameSourceAnchor(first, second) {
+  return first.sessionRef === second.sessionRef && first.sourceRef === second.sourceRef && first.sourceDigest === second.sourceDigest && first.offset === second.offset;
+}
+
+function availableSourceAnchors(packet, entry) {
+  const sources = [...packet.sources, ...(packet.referenceSources || [])];
+  // Legacy createdBy records the preparation range start, not the entity's original source.
+  return (entry.sourceAnchors || []).filter((anchor) => sources.some((source) =>
+    source.locator === anchor.sourceRef && anchor.sessionRef === packet.sessionRef && anchor.sourceDigest === digest(source.content) &&
+    sourceMentionOffsets(source.content, entry.mention).includes(anchor.offset)));
+}
+
+function relevantEntityBindings(packet, scope) {
+  const sources = new Map([...packet.sources, ...(packet.referenceSources || [])].map((source) => [source.locator, source]));
+  const entities = Object.values(scope.entities).map((entry) => ({ entry, anchors: availableSourceAnchors(packet, entry) }));
+  return entities.filter(({ entry }) => entry.createdBy !== `${packet.sessionRef}:${packet.fromTurn}`).flatMap(({ entry, anchors }) => {
+    const sourceAnchors = anchors.flatMap((anchor) => {
+      if (entities.filter((other) => other.anchors.some((candidate) => sameSourceAnchor(candidate, anchor))).length !== 1) return [];
+      const source = sources.get(anchor.sourceRef);
+      if (!source || anchor.sessionRef !== packet.sessionRef || anchor.sourceDigest !== digest(source.content)) return [];
+      const mentionIndex = sourceMentionOffsets(source.content, entry.mention).indexOf(anchor.offset);
+      return mentionIndex < 0 ? [] : [{ sourceRef: source.locator, mentionIndex, role: source.role, turn: source.turn }];
+    });
+    return sourceAnchors.length ? [{ bindingRef: entry.key, mention: entry.mention, kind: entry.kind, sourceAnchors }] : [];
+  });
 }
 
 function bindContinuity(packet, output, dictionary, workspace) {
@@ -80,12 +108,17 @@ function bindContinuity(packet, output, dictionary, workspace) {
   dictionary.scopes ||= {};
   const scope = dictionary.scopes[workspace] ||= { scopeRef: randomKey("s_"), entities: {} };
   for (const entity of array(output.entities, 32)) {
-    exact(entity, ["ref", "kind", "visibility", "sourceRef", "mention", "bindingRef", "publicKey"]);
+    const fields = ["ref", "kind", "visibility", "sourceRef", "mention", "bindingRef", "publicKey"];
+    exact(entity, Object.hasOwn(entity, "mentionIndex") ? [...fields, "mentionIndex"] : fields);
     const source = all.get(entity.sourceRef);
+    const offsets = source && typeof entity.mention === "string" && entity.mention.trim() ? sourceMentionOffsets(source.content, entity.mention) : [];
     if (typeof entity.ref !== "string" || !/^[A-Za-z0-9_-]{1,80}$/.test(entity.ref) || selected.has(entity.ref) || !KINDS.includes(entity.kind) ||
-        !source || typeof entity.mention !== "string" || !entity.mention.trim() || [...entity.mention].length > 160 || !sourceHasMention(source.content, entity.mention)) {
+        !source || typeof entity.mention !== "string" || !entity.mention.trim() || [...entity.mention].length > 160 || !offsets.length) {
       throw new Error("Entity binding is not grounded in a supplied source mention.");
     }
+    if (Object.hasOwn(entity, "mentionIndex") && (!Number.isSafeInteger(entity.mentionIndex) || entity.mentionIndex < 0 || entity.mentionIndex >= offsets.length)) {
+      throw new Error("Entity binding selected an invalid source mention occurrence.");
+    }
     let binding;
     if (entity.visibility === "public") {
       if (entity.bindingRef !== "" || entity.kind !== "technology" || !Object.hasOwn(CATALOG, entity.publicKey) || entity.mention.toLocaleLowerCase() !== CATALOG[entity.publicKey].toLocaleLowerCase()) {
@@ -93,14 +126,24 @@ function bindContinuity(packet, output, dictionary, workspace) {
       }
       binding = { key: entity.publicKey, scopeRef: "public", kind: "technology", visibility: "public", label: CATALOG[entity.publicKey] };
     } else if (entity.visibility === "private" && entity.publicKey === "") {
+      const mentionIndex = entity.mentionIndex ?? (offsets.length === 1 ? 0 : undefined);
+      if (!Number.isSafeInteger(mentionIndex) || mentionIndex < 0 || mentionIndex >= offsets.length) throw new Error("Private identity needs an unambiguous source mention occurrence.");
+      const anchor = { sessionRef: packet.sessionRef, sourceRef: source.locator, sourceDigest: digest(source.content), offset: offsets[mentionIndex] };
       let key = entity.bindingRef;
       if (typeof key !== "string") throw new Error("Private identity needs an explicit local binding selection.");
       if (key) {
         const known = scope.entities[key];
-        if (!known || known.kind !== entity.kind || known.mention.normalize("NFC").toLocaleLowerCase() !== entity.mention.normalize("NFC").toLocaleLowerCase() || !(packet.entityBindings || []).some((entry) => entry.bindingRef === key)) throw new Error("Private identity selected an unavailable binding.");
+        if (!known || known.kind !== entity.kind || known.mention.normalize("NFC").toLocaleLowerCase() !== entity.mention.normalize("NFC").toLocaleLowerCase() ||
+            !(packet.entityBindings || []).some((entry) => entry.bindingRef === key) || !relevantEntityBindings(packet, scope).some((entry) => entry.bindingRef === key) ||
+            !availableSourceAnchors(packet, known).some((knownAnchor) => sameSourceAnchor(knownAnchor, anchor))) {
+          throw new Error("Private identity selected an unavailable or conflicting source binding.");
+        }
       } else {
+        if (Object.values(scope.entities).some((known) => availableSourceAnchors(packet, known).some((knownAnchor) => sameSourceAnchor(knownAnchor, anchor)))) {
+          throw new Error("This private source occurrence already has a binding; select it explicitly or leave the identity unresolved.");
+        }
         key = randomKey("e_");
-        scope.entities[key] = { key, mention: entity.mention, kind: entity.kind, createdBy: `${packet.sessionRef}:${packet.fromTurn}` };
+        scope.entities[key] = { key, mention: entity.mention, kind: entity.kind, createdBy: `${packet.sessionRef}:${packet.fromTurn}`, sourceAnchors: [anchor] };
       }
       binding = { key, scopeRef: scope.scopeRef, kind: entity.kind, visibility: "private", label: `${entity.kind}_${key}` };
     } else throw new Error("Entity binding requested an invalid visibility.");
@@ -183,4 +226,4 @@ function bindContinuity(packet, output, dictionary, workspace) {
     assessment: { disposition: assessment.disposition, emptyReason: assessment.emptyReason, currentSources: current.size, referenceSources: references.size } };
 }
 
-module.exports = { CONTINUITY_SCHEMA: schema, POLICY_VERSION, CATALOG, continuityPrompt, bindContinuity, digest };
+module.exports = { CONTINUITY_SCHEMA: schema, POLICY_VERSION, CATALOG, continuityPrompt, bindContinuity, relevantEntityBindings, digest };

package/bin/continuity-state.js

@@ -92,13 +92,12 @@ function continuityStore(root, harness, config) {
     const prior = state.attempts[identity];
     if (prior?.payloadKey) return { prepared: state.outbox.find((entry) => entry.key === prior.payloadKey) };
     if (prior && prior.toTurn !== packet.toTurn) throw new Error("The unresolved preparation range remains pinned.");
-    const attempt = prior?.fingerprint === fingerprint ? prior : { fingerprint, fromTurn: packet.fromTurn, toTurn: packet.toTurn, attempts: 0 };
-    if (attempt.attempts >= 2) {
-      attempt.status = "needs_review";
-      state.attempts[identity] = attempt;
+    if (prior?.attempts >= 2) {
+      prior.status = "needs_review";
       save(state);
-      throw new Error("Continuity preparation needs review; its unchanged input exhausted two native attempts.");
+      throw new Error("Continuity preparation needs review; this range exhausted two native attempts. Retry it explicitly after review.");
     }
+    const attempt = prior?.fingerprint === fingerprint ? prior : { fingerprint, fromTurn: packet.fromTurn, toTurn: packet.toTurn, attempts: prior?.attempts || 0 };
     attempt.attempts += 1;
     attempt.status = "preparing";
     state.attempts[identity] = attempt;

package/bin/shifu-harness.js

@@ -47,7 +47,7 @@ const MAX_TURNS_PER_SEGMENT = 12;
 const MAX_TITLE_LENGTH = 160;
 const MAX_WORK_DETAIL_LENGTH = 600;
 const CONFIG_VERSION = 2;
-const RUNNER_VERSION = "0.3.31";
+const RUNNER_VERSION = "0.3.32";
 const REDACTION_VERSION = 5;
 const LEGACY_REDACTION_VERSION = 4;
 const MAX_PREPARATIONS_PER_RUN = 4;
@@ -2647,9 +2647,7 @@ async function prepareV5Payload(harness, segment, prepare = require("./native-pr
   dictionary.scopes[workspace] ||= { scopeRef: `s_${randomUUID().replaceAll("-", "")}`, entities: {} };
   store.saveDictionary(dictionary);
   const relevantBindings = () => {
-    const content = [...packet.sources, ...packet.referenceSources].map((source) => source.content).join("\n").toLocaleLowerCase();
-    return Object.values(dictionary.scopes[workspace].entities).filter((entry) => entry.createdBy !== `${packet.sessionRef}:${packet.fromTurn}` && content.includes(entry.mention.toLocaleLowerCase()))
-      .map(({ key, mention, kind }) => ({ bindingRef: key, mention, kind }));
+    return require("./continuity-preparation.js").relevantEntityBindings(packet, dictionary.scopes[workspace]);
   };
   packet.entityBindings = relevantBindings();
   while (packet.referenceSources.length && Buffer.byteLength(JSON.stringify(packet)) + 1 > 256 * 1024) {
@@ -2712,6 +2710,7 @@ async function retryPreparation(harness) {
 
 function preparationValidationFeedback(error, output) {
   const fields = [["title", output?.title, 160], ["summary", output?.summary, 1100]];
+  const issues = [];
   for (const name of ["decisions", "verification"]) {
     if (Array.isArray(output?.[name])) output[name].forEach((value, index) => fields.push([`${name}[${index}]`, value, 240]));
   }
@@ -2734,9 +2733,9 @@ function preparationValidationFeedback(error, output) {
       if (fragment) reason = `the forbidden literal fragment ${JSON.stringify(fragment)}`;
     }
     // Field names and rule literals are code-owned; never echo rejected text.
-    return `${error.message} Field ${field} fails the privacy or length rule for ${reason}. Rephrase that field without copying the rejected content.`;
+    issues.push(`Field ${field} fails the privacy or length rule for ${reason}.`);
   }
-  return error.message;
+  return `${error.message}${issues.length ? ` ${issues.join(" ")} Rephrase those fields without copying the rejected content.` : ""} Recheck every text field before returning. Describe a user's go-ahead as approval, permission or adoption; the literal word authorization is rejected by the privacy filter. Preserve supported results, failed or pending checks, unresolved problems and release status while correcting the validation error.`;
 }
 
 function preparationState(harness, queue) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@useshifu/coding-harness",
-  "version": "0.3.31",
+  "version": "0.3.32",
   "description": "Scheduled, redacted coding-harness activity sync for Shifu",
   "bin": {
     "coding-harness": "bin/shifu-harness.js",

package/skills/shifu-context/SKILL.md

@@ -12,7 +12,7 @@ Before first connection or MCP personalization, ask for any choices that are not
 When asked to connect Shifu context, run:
 
 ```sh
-npx --yes @useshifu/coding-harness@0.3.30 mcp-setup --harness codex
+npx --yes @useshifu/coding-harness@0.3.32 mcp-setup --harness codex
 ```
 
 MCP setup verifies the connection and tools, saves the choices and installs instructions; it does not sync sessions or upload the selected history. History selection governs eligible work for subsequent sync. Do not start sync as part of this setup unless the user separately asks for it. If a saved connection exists, reuse its key and do not run `connect` again.