@useshifu/coding-harness 0.3.9 → 0.3.10

package/README.md

@@ -70,3 +70,9 @@ AI Profile observations are created automatically with each coding-session activ
 The category is one of `system_design`, `detailed_design`, `implementation`, `debugging_testing`, `production_support`, `research_analysis`, `product_planning`, `writing_communication`, `marketing_growth`, `learning`, or `other`. Evidence dimensions are `task_selection`, `goal_clarity`, `context_sharing`, `constraint_setting`, `work_decomposition`, `iterative_steering`, `critical_judgment`, `verification`, `application`, and `responsible_use`. Automatic observations cite the relevant turn and never infer a limiting behavior from silence. Unsupported dimensions remain unscored.
 
 The AI Profile page shows detected categories, model and thinking labels, and Jev scores supported by these limited observations. Local signals do not prove work quality or establish that another model setting would have performed better. Identical retries are safe.
+
+## Recommended package updates
+
+The connection endpoint returns `recommendedPackageVersion`, currently pinned to the latest published release, `0.3.9`. Before `sync`, `sync-files`, `approve`, or `scheduled-sync`, the runner compares its embedded release version with that property. An older runner installs the exact recommended release through npm and resumes the original command with the updated runner. Equal or newer runners continue unchanged. Explicitly recommended major releases follow the same comparison rule. Missing or invalid recommendations, connection failures, and npm failures do not block sync. The connection, pending work, checkpoints, and schedule are preserved.
+
+This update mechanism first ships in `0.3.10`. Runners from earlier releases need a one-time installation of a release containing it (`npx --yes @useshifu/coding-harness@0.3.10 install --harness codex`, after publication). Raise the server recommendation only after the target release is published.

package/bin/shifu-harness.js

@@ -4,7 +4,9 @@ const fs = require("node:fs");
 const { createHash } = require("node:crypto");
 const os = require("node:os");
 const path = require("node:path");
-const { execFileSync } = require("node:child_process");
+const { execFile, execFileSync, spawn } = require("node:child_process");
+const { promisify } = require("node:util");
+const execFileAsync = promisify(execFile);
 const readline = require("node:readline/promises");
 
 const DEFAULT_API_URL = "https://api.useshifu.com";
@@ -37,7 +39,7 @@ const MAX_TURNS_PER_SEGMENT = 12;
 const MAX_TITLE_LENGTH = 160;
 const MAX_WORK_DETAIL_LENGTH = 600;
 const CONFIG_VERSION = 2;
-const RUNNER_VERSION = "0.3.9";
+const RUNNER_VERSION = "0.3.10";
 const REDACTION_VERSION = 4;
 const DEFAULT_INTERVAL_HOURS = 1;
 const LEGACY_INTERVAL_HOURS = 4;
@@ -562,6 +564,7 @@ const WORK_ACTIONS = [
   { key: "reviewed", title: "Reviewed", pattern: /^(?:I\s+|We\s+)?(?:reviewed|audited)\b/i },
   { key: "investigated", title: "Investigated", pattern: /^(?:I\s+|We\s+)?(?:investigated|researched|evaluated|compared)\b/i },
   { key: "deployed", title: "Deployed", pattern: /^(?:I\s+|We\s+)?(?:deployed|published)\b/i },
+  { key: "decided", title: "Decided", pattern: /^(?:I\s+|We\s+)?(?:decided|chose|selected)\b/i },
 ];
 
 const WORK_OBJECTS = [
@@ -572,6 +575,11 @@ const WORK_OBJECTS = [
   { key: "claim_extraction", title: "claim extraction", pattern: /\b(?:claim extraction|claim generation|claim matching)\b/i },
   { key: "retry", title: "retry handling", pattern: /\b(?:retries|retry)\b/i },
   { key: "cache", title: "cache behavior", pattern: /\b(?:cache|caching|ttl)\b/i },
+  { key: "outbox", title: "transactional outbox", pattern: /\b(?:transactional outbox|outbox pattern)\b/i },
+  { key: "token_bucket", title: "token bucket limiter", pattern: /\btoken bucket limiter\b/i },
+  { key: "actor_scheduler", title: "actor scheduler", pattern: /\bactor scheduler\b/i },
+  { key: "quota_balancing", title: "quota balancing", pattern: /\bquota balancing\b/i },
+  { key: "shard_rebalancing", title: "shard rebalancing", pattern: /\bshard rebalancing\b/i },
   { key: "navigation", title: "navigation", pattern: /\b(?:navigation|sidebar|sub.?nav)\b/i },
   { key: "sync", title: "sync behavior", pattern: /\b(?:sync|synchronization|checkpoint)\b/i },
   { key: "auth", title: "authentication", pattern: /\b(?:authentication|sign.?in|login)\b/i },
@@ -626,6 +634,39 @@ const WORK_DETAILS = [
   { title: "mobile layout", pattern: /\bmobile layout\b/i },
 ];
 
+// These terms are safe to retain without copying surrounding private prose.
+const TECHNICAL_DETAILS = [
+  { title: "Redis", pattern: /\bRedis\b/i },
+  { title: "Kafka", pattern: /\bKafka\b/i },
+  { title: "Postgres", pattern: /\bPostgres(?:ql)?\b/i },
+  { title: "MongoDB", pattern: /\bMongoDB\b/i },
+  { title: "React", pattern: /\bReact\b/i },
+  { title: "Node.js", pattern: /\bNode\.js\b/i },
+  { title: "TypeScript", pattern: /\bTypeScript\b/i },
+  { title: "Python", pattern: /\bPython\b/i },
+  { title: "Spring Boot", pattern: /\bSpring Boot\b/i },
+  { title: "Kubernetes", pattern: /\bKubernetes\b/i },
+  { title: "Docker", pattern: /\bDocker\b/i },
+  { title: "GraphQL", pattern: /\bGraphQL\b/i },
+  { title: "gRPC", pattern: /\bgRPC\b/i },
+  { title: "cache-aside", pattern: /\bcache.aside\b/i },
+  { title: "transactional outbox", pattern: /\btransactional outbox\b/i },
+  { title: "circuit breaker", pattern: /\bcircuit breaker\b/i },
+  { title: "Builder Pattern", pattern: /\bBuilder Pattern\b/i },
+  { title: "event sourcing", pattern: /\bevent sourcing\b/i },
+  { title: "CQRS", pattern: /\bCQRS\b/i },
+  { title: "Saga Pattern", pattern: /\bSaga Pattern\b/i },
+  { title: "Repository Pattern", pattern: /\bRepository Pattern\b/i },
+  { title: "Dependency Injection", pattern: /\bDependency Injection\b/i },
+  { title: "OpenTelemetry", pattern: /\bOpenTelemetry\b/i },
+  { title: "Temporal", pattern: /\bTemporal\b/i },
+  { title: "Rust", pattern: /\bRust\b/i },
+  { title: "Tokio", pattern: /\bTokio\b/i },
+  { title: "consistent hashing", pattern: /\bconsistent hashing\b/i },
+  { title: "virtual nodes", pattern: /\bvirtual nodes\b/i },
+  { title: "adaptive jitter", pattern: /\badaptive jitter\b/i },
+];
+
 function controlledContexts(text) {
   const matches = WORK_CONTEXTS.flatMap((item) => {
     const match = item.pattern.exec(text);
@@ -640,15 +681,27 @@ function controlledContexts(text) {
 }
 
 function controlledDetails(text) {
-  return WORK_DETAILS.map((item) => ({ item, index: item.pattern.exec(text)?.index }))
+  const known = [...WORK_DETAILS, ...TECHNICAL_DETAILS];
+  const ttl = /\b(\d{1,3})[ -]hour TTL\b/i.exec(text);
+  const latency = /\bp(?:50|90|95|99) latency from (\d{1,4}) ms to (\d{1,4}) ms\b/i.exec(text);
+  const reductionPattern = /\b(?:cutting|cut|reducing|reduced|decreasing|decreased|lowering|lowered)\s+(queue time|memory(?: usage)?)\s+by\s*(\d{1,3}(?:\.\d{1,2})?)%(?![\d.])/i;
+  const reduction = reductionPattern.exec(text);
+  if (ttl) known.push({ title: `${ttl[1]}-hour TTL`, pattern: /\b\d{1,3}[ -]hour TTL\b/i });
+  if (latency) known.push({ title: latency[0], pattern: /\bp(?:50|90|95|99) latency from \d{1,4} ms to \d{1,4} ms\b/i });
+  if (reduction && Number(reduction[2]) <= 100) {
+    const metric = reduction[1].toLowerCase().startsWith("memory") ? "memory usage" : "queue time";
+    known.push({ title: `${metric} reduced by ${reduction[2]}%`, pattern: reductionPattern });
+  }
+  return known.map((item) => ({ item, index: item.pattern.exec(text)?.index }))
     .filter(({ index }) => index !== undefined)
     .sort((left, right) => left.index - right.index)
-    .slice(0, 3)
-    .map(({ item }) => item.title);
+    .map(({ item }) => item.title)
+    .filter((detail, index, details) => details.indexOf(detail) === index)
+    .slice(0, 6);
 }
 
 function withoutNamedLabels(text) {
-  return text.replace(/\b(?:Project|Client|Customer|Company)\s+[A-Z][A-Za-z0-9_-]*/g, "");
+  return text.replace(/\b(?:Project|Client|Customer|Company)\s+[A-Z][A-Za-z0-9_-]*/gi, "");
 }
 
 function safeActivityEvidence(notes) {
@@ -666,7 +719,7 @@ function safeActivityEvidence(notes) {
         sectionContexts = /\b(?:project|feature|flow|service|system|product)\b/i.test(label) ? controlledContexts(label) || [] : [];
         continue;
       }
-      const line = rawLine.trim().replace(/^[-*]\s+/, "").split(/[.;—]/, 1)[0];
+      const line = rawLine.trim().replace(/^[-*]\s+/, "").split(/[;—]|[.!?](?=\s|$)/, 1)[0];
       const clauses = line.split(/\s+(?:and|but|while)\s+/i);
       const nextAction = clauses.findIndex((clause, index) => index > 0 && WORK_ACTIONS.some((item) => item.pattern.test(clause)));
       const workClause = withoutNamedLabels(nextAction > 0 ? clauses.slice(0, nextAction).join(" and ") : line);
@@ -710,7 +763,7 @@ function payloadForSegment(harness, segment) {
     summary,
     evidence,
     verification: [],
-    decisions: [],
+    decisions: evidence.filter((item) => item.title.startsWith("Decided ")).slice(0, MAX_SYNC_ITEMS).map((item) => item.statement),
     redactionVersion: REDACTION_VERSION,
   };
 }
@@ -940,8 +993,14 @@ async function readSecret(input = process.stdin, output = process.stderr) {
 
 function copyRunner() {
   fs.mkdirSync(configRoot(), { recursive: true, mode: 0o700 });
-  fs.copyFileSync(__filename, installedRunnerPath());
-  fs.chmodSync(installedRunnerPath(), 0o700);
+  const temporary = `${installedRunnerPath()}.${process.pid}.tmp`;
+  try {
+    fs.copyFileSync(__filename, temporary);
+    fs.chmodSync(temporary, 0o700);
+    fs.renameSync(temporary, installedRunnerPath());
+  } finally {
+    fs.rmSync(temporary, { force: true });
+  }
 }
 
 function copyDirectory(source, destination) {
@@ -1160,6 +1219,7 @@ async function validateConnection(apiUrl, token, harness) {
   if (!response.ok) throw new Error(`Shifu could not verify the connection key (HTTP ${response.status}). Nothing was changed.`);
   const result = await response.json().catch(() => null);
   if (result?.data?.connected !== true) throw new Error("Shifu returned an invalid connection check. Nothing was changed.");
+  return result.data;
 }
 
 async function connect(harness, readKey = readSecret) {
@@ -1838,11 +1898,54 @@ function review(harness) {
   }, null, 2));
 }
 
+function olderPackageVersion(current, recommended) {
+  const pattern = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)$/;
+  if (!pattern.test(current) || !pattern.test(recommended)) return false;
+  const left = current.split(".").map(BigInt);
+  const right = recommended.split(".").map(BigInt);
+  for (let index = 0; index < 3; index += 1) {
+    if (left[index] !== right[index]) return left[index] < right[index];
+  }
+  return false;
+}
+
+async function installPackageVersion(version, harness) {
+  await execFileAsync("npm", ["exec", "--yes", `--package=@useshifu/coding-harness@${version}`, "--", "shifu-harness", "install", "--harness", harness, "--config-root", configRoot()], {
+    timeout: 60_000,
+    env: { ...process.env, PATH: `${process.env.PATH || ""}${path.delimiter}${path.dirname(process.execPath)}`, SHIFU_UPDATE_CHECKED: "1" },
+  });
+}
+
+async function updateRecommendedPackage(harness, { currentVersion = RUNNER_VERSION, fetchConnection = validateConnection, installVersion = installPackageVersion } = {}) {
+  try {
+    const config = readConfig(harness);
+    const connection = await fetchConnection(config.apiUrl, config.token, harness);
+    const recommended = connection?.recommendedPackageVersion;
+    if (!olderPackageVersion(currentVersion, recommended)) return false;
+    await installVersion(recommended, harness);
+    console.error(`Updated Shifu coding harness to recommended version ${recommended}.`);
+    return true;
+  } catch {
+    console.error("Could not check or install the recommended Shifu package. Continuing with the current runner.");
+    return false;
+  }
+}
+
 async function main() {
   if (option("--config-root")) process.env.SHIFU_CONFIG_ROOT = path.resolve(option("--config-root"));
   const command = process.argv[2];
   const subCommand = process.argv[3];
   const harness = requireHarness(option("--harness"));
+  if (["sync", "sync-files", "scheduled-sync", "approve"].includes(command) && process.env.SHIFU_UPDATE_CHECKED !== "1" && await updateRecommendedPackage(harness)) {
+    const child = spawn(process.execPath, [installedRunnerPath(), ...process.argv.slice(2)], {
+      env: { ...process.env, SHIFU_UPDATE_CHECKED: "1" }, stdio: "inherit",
+    });
+    await new Promise((resolve, reject) => {
+      child.once("error", reject);
+      child.once("exit", (code) => code === 0 ? resolve() : reject(new Error("The updated Shifu sync failed; pending work is preserved.")));
+    });
+    return;
+  }
   if (command === "install") return install(harness);
   if (command === "connect") return connect(harness);
   if (command === "disconnect") return disconnect(harness);
@@ -1867,4 +1970,4 @@ if (require.main === module) {
     });
 }
 
-module.exports = { approvalNotificationInvocation, claudeReview, claudeSession, codexReview, codexSession, configPath, configuredSyncPolicy, connect, defaultInputFolder, disconnect, finalAssistantNote, install, installDestination, localSessions, normalizedPolicy, notificationSessionRef, opencodeQuery, opencodeReview, opencodeSessions, payloadForSegment, pendingSegments, prepareInputFolder, profileForSegment, readSecret, removeHooks, requireSafeApiUrl, reviewCandidates, schedulerArtifact, statePath, syncWorkFiles, textIsSafe, unsyncedCodexSessions, validateAIProfileObservation, validateConnection, validateInputFolder, validSyncReceipt, validateApprovalMode, validateIntervalHours, validateSync, workFiles };
+module.exports = { olderPackageVersion, updateRecommendedPackage, installPackageVersion, approvalNotificationInvocation, claudeReview, claudeSession, codexReview, codexSession, configPath, configuredSyncPolicy, connect, defaultInputFolder, disconnect, finalAssistantNote, install, installDestination, localSessions, normalizedPolicy, notificationSessionRef, opencodeQuery, opencodeReview, opencodeSessions, payloadForSegment, pendingSegments, prepareInputFolder, profileForSegment, readSecret, removeHooks, requireSafeApiUrl, reviewCandidates, schedulerArtifact, statePath, syncWorkFiles, textIsSafe, unsyncedCodexSessions, validateAIProfileObservation, validateConnection, validateInputFolder, validSyncReceipt, validateApprovalMode, validateIntervalHours, validateSync, workFiles };

package/commands/shifu-sync.md

@@ -2,6 +2,8 @@
 description: Inspect or sync redacted OpenCode activity using the saved Shifu policy.
 ---
 
+Before sync, the runner installs Shifu's exact `recommendedPackageVersion` only if its current version is older. Equal or newer versions are left alone. Update failures allow sync to continue; do not independently install npm latest.
+
 Run `npx --yes @useshifu/coding-harness status --harness opencode` first.
 
 For an explicit Shifu input-folder sync, run `sync-files --harness opencode`. It polls unfinished jobs before submitting new or changed `.txt`, `.md`, `.rtf`, and `.docx` files up to 20 MB. Upload starts asynchronous server processing; check status on the next manual or scheduled sync. Only a final `processed`, `no_claims`, or `duplicate` result completes the local checkpoint; a `partial` result remains retryable. Completed file checkpoints from the previous runner are replayed once; existing in-flight jobs are polled before replay. Report published claims and needs-evidence suggestions separately. `no_claims` may still include suggestions, which are not verified user contributions. Unsupported files are counted but not uploaded. No claim-review step is needed. Do not treat another speaker's work or a meeting action item as the user's contribution. The server parses and sanitizes files; the local runner does not redact file contents, and file text or excerpts must not appear in logs.
@@ -12,7 +14,7 @@ For an explicit sync-now request:
 
 1. For a time-window request, run `sessions --harness opencode --hours 24`, using 48 or 72 only when requested. For a request by number of recent sessions, run `sessions --harness opencode --count <count>`; this ignores the hours window and returns the most recently updated sessions first.
 2. Review each returned segment with `review --harness opencode --session-ref <sessionRef> --hours <hours>`. For count-based selection, use `review --harness opencode --session-ref <sessionRef> --all` so older sessions are allowed.
-3. Prepare a redaction-version-four payload from final assistant notes only. Use `activity` evidence for writing, feedback, research, planning, product work, and any work that is not clearly an engineering implementation or decision. Activity evidence has no technical scope. Preserve the supported action, feature or project purpose, specific problem, approach, and verified result. Synthesize a descriptive project label (such as "payment reconciliation service") from its purpose; never copy a raw label that could identify a company or person. Keep distinct contributions separate, including different fixes within one feature. An empty evidence list advances the checkpoint without creating a claim.
+3. Prepare a redaction-version-four payload from final assistant notes only. Use `activity` evidence for writing, feedback, research, planning, product work, and any work that is not clearly an engineering implementation or decision. Activity evidence has no technical scope. Preserve the supported action, feature or project purpose, specific problem, approach, technologies and design patterns used, and verified result. Synthesize a descriptive project label (such as "payment reconciliation service") from its purpose; never copy a raw label that could identify a company or person. Keep distinct contributions separate, including different fixes within one feature. An empty evidence list advances the checkpoint without creating a claim.
 4. In automatic mode, pipe the payload to `sync --harness opencode` without asking again. In manual mode, show the exact payload, ask once, and then use `sync --approved --harness opencode`.
 5. Keep the returned session reference and turn range unchanged. Never upload prompts, raw transcripts, tool output, code, paths, commands, URLs, credentials, company or person names, customer details, or proprietary identifiers.
 6. For a count-based sync, keep `selectedSessionRefs` from the first discovery and review only pending `sessions`. `skippedSessions` reports already-synced selections; do not replace them with older sessions. If one has remaining turns, request its next segment with `sessions --harness opencode --session-ref <sessionRef> --all`. Stop after those selected sessions are exhausted. Report completed, already-synced, and failed/pending sessions separately. Manual sync runs immediately, independently of the hourly schedule.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@useshifu/coding-harness",
-  "version": "0.3.9",
+  "version": "0.3.10",
   "description": "Scheduled, redacted coding-harness activity sync for Shifu",
   "bin": {
     "coding-harness": "bin/shifu-harness.js",

package/skills/shifu-sync/SKILL.md

@@ -5,6 +5,8 @@ description: Connect, configure, inspect, or sync coding-harness activity, AI Pr
 
 # Shifu sync
 
+Before manual or scheduled sync, the runner checks Shifu's `recommendedPackageVersion`. It automatically installs that exact release only when the installed version is older; equal or newer versions are left alone. Do not independently install npm latest. If the check or update fails, continue syncing with the current runner. Connection settings, pending work, checkpoints, and the schedule are preserved.
+
 Shifu records useful work completed through the harness. Work may be engineering, product discovery, writing, interview feedback, research, planning, or another user-directed activity. Do not force non-engineering work into engineering categories.
 
 Never include user prompts, raw transcripts, tool output, source code, commands, credentials, URLs, file paths, company or person names, customer details, or proprietary identifiers in a coding-session payload. Synthesize descriptive project labels from their purpose rather than copying a raw name. The local coding runner reads final assistant notes for career activity and scans user turns locally for fixed AI Profile signals; it sends neither raw source. Files placed in the user's chosen Shifu input folder are a separate path: the runner sends supported files over TLS for server-side parsing and sanitization. Do not describe folder files as locally redacted.
@@ -64,7 +66,7 @@ For an explicit sync request, first read `status` to determine the saved approva
 
 Use `kind: "activity"` for general harness work; it does not require a technical scope. Use `implementation` or `decision` only when the note clearly supports that engineering classification, and then use the smallest reviewed scope among `unit`, `module`, `service`, `system`, and `product`. Evidence may be empty when the safe final note supports only a session-level title and summary. Never invent impact, ownership, verification, or outcomes.
 
-Make each supported item useful for later recall: retain the user's action, the feature or project purpose, the specific problem, the approach, and a verified result when the final note supports each part. A descriptive project label such as "payment reconciliation service" is useful; synthesize it from the work's purpose instead of copying a raw project label that might contain a company or person's name. Keep distinct contributions separate, including two fixes in one feature that addressed different problems. Omit names of companies and people, customer details, proprietary identifiers, paths, code snippets, and unsupported impact. If a detail cannot be safely described, keep the supported parts. Prefer a summary that names the main distinct work items over a generic count alone.
+Make each supported item useful for later recall: retain the user's action, the feature or project purpose, the specific problem, the approach, and a verified result when the final note supports each part. Preserve supported technologies, design patterns, configuration choices, and measured results; these are useful work details, not identifiers to redact. A descriptive project label such as "payment reconciliation service" is useful; synthesize it from the work's purpose instead of copying a raw project label that might contain a company or person's name. Keep distinct contributions separate, including two fixes in one feature that addressed different problems. Omit names of companies and people, customer details, proprietary identifiers, paths, code snippets, credentials, and unsupported impact. If a detail cannot be safely described, keep the supported parts. Prefer a summary that names the main distinct work items over a generic count alone.
 
 ```json
 {