software-defence-factory 0.15.4 → 0.15.5

package/docs/npm.md

@@ -282,3 +282,13 @@ reject tampered selections/images/policies and unsupported future writers, and
 verify that readback does not rewrite retained evidence. Image adoption is a
 separate idle operator action; existing attempt images are not migrated by a CLI
 update. See [the migration/rollback recipe](setup.md#local-context-budget-and-worker-image-0152-100).
+
+
+The 0.15.5 compatibility audit adds that exact writer to the explicit v1/v2/v3
+allowlists. The examined change is confined to local Pi projection, proactive
+recovery and completion validation. Evidence schemas, frozen commands, model and
+image selections, policy hashes, candidate reconstruction and acceptance guards
+remain unchanged from 0.15.4. Profile tests exercise all phases and inherited,
+cloud, hybrid and local definitions, reject changed selections and future writers,
+and verify immutable readback. This does not upgrade or qualify historical local
+attempts; installed image adoption and actual inference remain separate gates.

package/docs/recovery.md

@@ -44,6 +44,22 @@ retry around a retained writer or reuse earlier story evidence for a changed
 candidate or policy. Browser traces and screenshots are private evidence and
 are presented through the shared interfaces. See [the web contract](web-verification.md).
 
+## Local Pi history exhaustion
+
+The 0.15.5 candidate adds bounded proactive recovery for short histories, including
+productive writes whose immutable arguments require fewer retained groups;
+see [limits and image adoption](setup.md#local-context-budget-and-worker-image-0152-100).
+A failed/empty summary, exhausted recovery limit, cancellation or irreducible
+request remains a failed attempt even if a tool already wrote a report. Keep its
+checkout, raw events and private diagnostic evidence. Reconcile the stopped
+worker before admitting a smaller task or explicitly selecting another qualified
+profile as a separate attempt. Do not enlarge the allocation, rewrite a failed
+history or label an explicit cloud retry as local success. Successful transport
+recovery still needs semantic checks and independent Review. The reviewed first
+checkpoint failed real local qualification; the revised artifact needs a new
+lead-owned installed and real-task pass before acceptance. See the
+[rejected trial and source regression](proof.md#short-local-pi-history-recovery-0155-candidate-105).
+
 ## Review feedback or phase retry
 
 Use **Request changes** on a failed software review only when its validated

package/docs/setup.md

@@ -509,6 +509,40 @@ guessed or silently rewritten. Do not add project Pi settings to bypass
 the admitted binding or rely on custom tuning that has not been qualified.
 See [pinned compaction behavior](https://github.com/earendil-works/pi/blob/v0.87.1/packages/coding-agent/docs/compaction.md).
 
+Starting with the **0.15.5 candidate (#105)**, the local adapter also checks
+history pressure at Pi's completed-tool-turn boundary. This conservative byte
+signal counts the system/task/tool envelope once, call arguments, non-tool text
+and capped tool excerpts, separately from provider-reported usage. At 65% of the
+input budget it considers Pi's supported summary generator and append-only
+compaction entry. The newest complete tool group is retained; up to three older
+complete groups can remain as additional evidence when the actual summary and
+retained envelope fit. The original task is restored exactly on every generation;
+raw events and existing session entries are not rewritten. Large-result bursts
+still use upstream compaction and the final request projection. Projection gives
+recent short results (up to 2 KiB each, 8 KiB total and at most a quarter of the
+input budget) priority when they fit.
+
+Recovery allows at most 16 proactive summaries per process and two retries per
+summary. It must reduce pressure by at least 4 KiB and leave headroom of 8 KiB
+(or 15% of the input budget when smaller). Its preferred reduction target adapts
+to the immutable latest-group/envelope floor and actual summary size; it cannot
+require that floor to fit an arbitrary fraction of the trigger. The next trigger
+also leaves room for new work. If the latest write/group alone makes recovery
+unproductive, the adapter defers to a later complete turn under the unchanged
+wire guard, when that group can become summary input. This does not permit an
+oversized request through transport. Empty, failed, cancelled or ineffective
+summaries fail the attempt. Oversized instructions, arguments or retained groups
+can still fail closed. These are bounded recovery limits, not automatic
+context/output tuning or a guarantee of useful model work.
+
+The selected context, generation output and model remain unchanged; summary
+output is separately capped (6,553 for proactive summaries with 8,192 selected
+output on pinned Pi). `factory_context_recovery` JSONL events record only
+pressure, envelope/floor sizes, group counts and recovery progress, without
+prompts/content. Pi token accounting stays partial; this summary path is not
+added to ordinary assistant usage totals. See
+[source measurements, rejected real trial and pending qualification](proof.md#short-local-pi-history-recovery-0155-candidate-105).
+
 Before adoption, stop and reconcile all work, retain the old immutable image ID
 and a stopped private-state backup. Build/qualify the exact candidate image, then
 explicitly select it with `factory install --image LOCAL_IMAGE_REF --state PRIVATE_STATE`

package/factory/execution-profile.mjs

@@ -29,16 +29,18 @@ import { expectedWebStories, webPolicyHash } from './web-verification.mjs';
 // compatibility does not qualify an old local startup or accept a failed attempt.
 // 0.15.4 adds Pi usage metadata only; protected evidence schemas, frozen
 // execution selection, isolation and acceptance policy remain unchanged.
+// 0.15.5 changes only local context recovery/projection; v1/v2/v3 schemas,
+// frozen selection hashes, provenance, checks and acceptance guards are unchanged.
 // Older writers still require exact patch/tree reconstruction before publication.
 // Deliberately independent of VERSION: a release bump is not
 // evidence compatibility. Re-audit this list for every trust-relevant writer,
 // isolation or validation change; remove versions whose guarantees no longer
 // satisfy current policy. See docs/npm.md. This predicate alone grants no trust.
-const SUPPORTED_EXECUTION_RUNTIMES_V1 = new Set(['0.8.0', '0.9.0', '0.9.1', '0.10.0', '0.11.0', '0.11.1', '0.11.2', '0.12.0', '0.13.0', '0.13.1', '0.14.0', '0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4']);
+const SUPPORTED_EXECUTION_RUNTIMES_V1 = new Set(['0.8.0', '0.9.0', '0.9.1', '0.10.0', '0.11.0', '0.11.1', '0.11.2', '0.12.0', '0.13.0', '0.13.1', '0.14.0', '0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4', '0.15.5']);
 
 export function isSupportedExecutionProfile(profile) {
-  if ([2, 3].includes(profile?.version)) return (profile.version === 3 ? ['0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4'].includes(profile.runtimeVersion) && Object.hasOwn(profile, 'localBinding')
-    : ['0.14.0', '0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4'].includes(profile.runtimeVersion) && !Object.hasOwn(profile, 'localBinding') && profile.inferenceProvider !== 'factory-local')
+  if ([2, 3].includes(profile?.version)) return (profile.version === 3 ? ['0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4', '0.15.5'].includes(profile.runtimeVersion) && Object.hasOwn(profile, 'localBinding')
+    : ['0.14.0', '0.15.0', '0.15.1', '0.15.2', '0.15.3', '0.15.4', '0.15.5'].includes(profile.runtimeVersion) && !Object.hasOwn(profile, 'localBinding') && profile.inferenceProvider !== 'factory-local')
     && profile.role === (Object.keys(ROLE_PHASES).find(role => ROLE_PHASES[role] === profile.phase) || null)
     && (profile.reasoningEffort === null || ['low', 'medium', 'high'].includes(profile.reasoningEffort))
     && /^[a-f0-9]{64}$/.test(profile.selectionHash || '');

package/factory/pi-context-budget.mjs

@@ -35,6 +35,16 @@ function excerpt(text, limit) {
   return Buffer.byteLength(shortened) < Buffer.byteLength(text) ? shortened : text;
 }
 
+// A pressure signal, not a token estimate: cap large tool bodies so upstream's
+// existing large-result recovery still owns bursts, but count every call/argument
+// and a useful short excerpt. Provider usage cannot measure this growing floor.
+export function historyPressure(messages, envelope) {
+  const copy = structuredClone(messages);
+  for (const message of copy) if (message.role === 'toolResult')
+    for (const part of message.content || []) if (part.type === 'text') part.text = excerpt(part.text, 1024);
+  return envelope + bytes(copy) + 64 * copy.length;
+}
+
 // Uniform cap shares space among large results without penalizing small ones.
 // Calls, arguments, identities, ordering and all non-tool text are immutable.
 export function fitToolText(value, budget, measure = bytes, native = false) {
@@ -49,8 +59,27 @@ export function fitToolText(value, budget, measure = bytes, native = false) {
       for (const part of message.content) if (part.type === 'text') slots.push([part, 'text', part.text]);
     }
   }
-  const cap = limit => { for (const [object, key, text] of slots) object[key] = excerpt(text, limit); };
+  // Prefer recent short discoveries within a bounded allowance; large batches
+  // still share the remaining space. Prefix stability is measured separately.
+  const recent = new Set();
+  let recentBytes = 0;
+  for (const slot of [...slots].reverse()) {
+    const size = bytes(slot[2]);
+    if (size <= 2048 && recentBytes + size <= Math.min(8192, budget / 4)) {
+      recent.add(slot); recentBytes += size;
+    }
+  }
+  const cap = limit => { for (const slot of slots) {
+    const [object, key, text] = slot;
+    object[key] = recent.has(slot) ? text : excerpt(text, limit);
+  } };
   cap(0);
+  // Release older preferences first if the complete preferred set cannot fit.
+  // Even one useful recent result is better than uniformly shortening them all.
+  for (const slot of [...recent].reverse()) {
+    if (measure(original) <= budget) break;
+    recent.delete(slot); cap(0);
+  }
   if (measure(original) > budget) throw new Error('Local Pi context budget cannot fit required instructions, tool pairs and truncation guidance; use smaller task/input or qualify the configured allocation. No fallback.');
   let low = 0, high = Math.max(0, ...slots.map(([, , text]) => text.length));
   while (low < high) {

package/factory/pi-context-extension.mjs

@@ -1,19 +1,87 @@
 // Explicitly mounted Factory adapter, never discovered from the host/repository.
 import { readFileSync, writeSync } from 'node:fs';
 import { openAICompletionsApi } from '@earendil-works/pi-ai';
-import { bytes, fitToolText, projectedTokens, restoreOutputBudget, validatePairs } from './pi-context-budget.mjs';
+import { generateSummaryWithUsage } from '@earendil-works/pi-coding-agent';
+import { historyPressure, bytes, fitToolText, projectedTokens, restoreOutputBudget, validatePairs } from './pi-context-budget.mjs';
 
 export default function factoryLocalContext(pi) {
   const registry = JSON.parse(readFileSync(new URL('./models.json', import.meta.url), 'utf8'));
   const configured = registry.providers['factory-local'].models[0];
   const window = configured.contextWindow, output = configured.maxTokens;
-  let task;
+  let task, taskEntryId, envelope = 0, recoveries = 0, nextRecovery = 0;
   // Pi catches extension exceptions and can continue. Budget violations must
   // stop before transport, even during summarization, with a durable diagnostic.
   const fatal = error => {
     writeSync(2, `Factory local Pi context protection failed: ${error.message}\n`);
     process.exit(1);
   };
+  // Pi 0.87.1's public manual compact() aborts the active run. Use its
+  // supported completed-turn compaction draft instead: all tools have finished,
+  // and the next request rebuilds the canonical projection from this entry.
+  pi.on('turn_end', async (event, ctx) => {
+    if (event.outcome !== 'completed' || event.message.stopReason !== 'toolUse') return;
+    try {
+      const entries = event.context.contextEntries;
+      taskEntryId ??= entries.find(entry => entry.messages.some(m => m.role === 'user'))?.sourceEntry.id;
+      // Pi includes the system message in its boundary projection. The wire
+      // envelope already counts it and the restored task: count each once and
+      // use the same basis before/after, so removal cannot fake progress.
+      const history = list => list.filter(entry => entry.sourceEntry.id !== taskEntryId)
+        .flatMap(entry => entry.messages).filter(message => message.role !== 'system');
+      const budget = window - output;
+      const threshold = Math.max(Math.floor(budget * 0.65), nextRecovery);
+      const pressure = historyPressure(history(entries), envelope);
+      if (pressure < threshold) return;
+      const starts = entries.flatMap((entry, index) => entry.messages.some(m => m.role === 'assistant'
+        && m.content?.some(p => p.type === 'toolCall')) ? [index] : []);
+      // The newest complete group is the retention floor, including immutable
+      // write arguments. Summarize the older prefix once; after seeing its real
+      // size we can retain up to four groups as additional recent evidence.
+      // Never split a group or summarize only part of a pending tool batch.
+      if (starts.length < 2) return;
+      const newest = starts.at(-1);
+      const floor = historyPressure(history(entries.slice(newest)), envelope);
+      const headroom = Math.min(8192, Math.floor(budget * 0.15));
+      // A large current call may leave no useful recovery yet. Let it reach the
+      // next ordinary turn under the unchanged wire guard, then it can enter
+      // the summarized prefix. Avoid paying for a predictably futile summary.
+      if (floor + 1024 > budget - headroom || pressure - floor < 4096) return;
+      if (++recoveries > 16) throw new Error('Local Pi proactive recovery limit reached; split the task. No fallback.');
+      if (ctx.signal?.aborted) throw new Error('Local Pi proactive recovery cancelled');
+      // Pi's summary serializer ignores system messages. Do not charge their
+      // bytes against the summary input projection; generation still keeps the
+      // exact system envelope. Keep the task and all summarized tool pairs.
+      const older = entries.slice(0, newest).flatMap(entry => entry.messages).filter(m => m.role !== 'system');
+      writeSync(1, `${JSON.stringify({ type: 'factory_context_recovery', phase: 'start', pressure, threshold, floor, envelope, groups: starts.length, recovery: recoveries })}\n`);
+      const input = fitToolText(older, window - output - 8192, bytes, true);
+      const result = await generateSummaryWithUsage(input, ctx.model, Math.min(16384, output),
+        'factory-local-keyless', undefined, ctx.signal,
+        'Preserve the task, verified edits, failed attempts, discoveries and remaining work. Do not claim completion without evidence.',
+        undefined, ctx.thinkingLevel, streamLocal, undefined, { enabled: true, maxRetries: 2, baseDelayMs: 250 });
+      if (ctx.signal?.aborted) throw new Error('Local Pi proactive recovery cancelled');
+      if (!result.text?.trim()) throw new Error('Local Pi proactive recovery returned an empty summary');
+      // Include framing for Pi's summary message. A fixed percentage cannot be
+      // required below the immutable envelope/newest-group floor. Bound both
+      // remaining headroom and actual progress instead, using the real summary.
+      const summaryBytes = bytes(result.text) + 1024;
+      const target = Math.min(budget - headroom, pressure - 4096,
+        Math.max(Math.floor(budget * 0.65 * 0.75), floor + summaryBytes));
+      let cut, after;
+      for (const candidate of starts.slice(-4)) {
+        const candidatePressure = historyPressure(history(entries.slice(candidate)), envelope + summaryBytes);
+        if (candidatePressure <= target) { cut = candidate; after = candidatePressure; break; }
+      }
+      if (cut === undefined) throw new Error('Local Pi proactive recovery did not free enough budget; split the task. No fallback.');
+      // Re-arm above the measured retained floor, so a productive large write
+      // does not cause an immediate repeat summary with nothing new to reclaim.
+      nextRecovery = Math.min(budget - Math.floor(headroom / 2), after + 4096);
+      writeSync(1, `${JSON.stringify({ type: 'factory_context_recovery', phase: 'complete', pressure: after, target,
+        summaryBytes, keptGroups: starts.filter(index => index >= cut).length, recovery: recoveries })}\n`);
+      return { entries: [{ type: 'compaction', summary: result.text,
+        firstKeptEntryId: entries[cut].sourceEntry.id, usage: result.usage,
+        details: { factoryLocalRecovery: 1 } }] };
+    } catch (error) { fatal(error); }
+  });
   pi.on('session_before_compact', event => {
     try {
       // These are request-local copies. Upstream still selects cut points and
@@ -25,39 +93,39 @@ export default function factoryLocalContext(pi) {
         preparation[key] = fitToolText(preparation[key], budget, bytes, true);
     } catch (error) { fatal(error); }
   });
-  pi.registerProvider('factory-local', {
-    api: 'openai-completions',
-    streamSimple(model, context, options) {
-      if (model.id !== configured.id || model.provider !== 'factory-local') fatal(new Error('Selected local model changed'));
-      // Capture before streamSimple/buildBaseOptions clamps against unprojected
-      // history. Generation defaults to the binding; summaries supply their cap.
-      const requestedOutput = options?.maxTokens ?? output;
-      return openAICompletionsApi().streamSimple(model, context, {
-        ...options,
-        onPayload: async (payload, requestModel) => {
-          try {
-            payload = await options?.onPayload?.(payload, requestModel) ?? payload;
-            payload = structuredClone(payload);
-            if (payload.model !== configured.id
-              || !payload.messages?.some(m => m.role === 'system')) throw new Error('Unsupported local request structure or output budget');
-            restoreOutputBudget(payload, configured, requestedOutput, options?.reasoning);
-            if (payload.tools?.length) {
-              task ??= structuredClone(payload.messages.find(m => m.role === 'user'));
-              if (!task) throw new Error('Missing required task');
-              if (!payload.messages.some(m => m.role === 'user' && JSON.stringify(m) === JSON.stringify(task)))
-                payload.messages.splice(1, 0, structuredClone(task));
-              validatePairs(payload.messages);
-            }
-            if (!payload.messages.some(m => m.role === 'user')) throw new Error('Missing required user message');
-            // Restore BEFORE measuring so the final output field is counted.
-            // No request is sent unless projection fits with the full configured
-            // reserve, even when a summary requests a smaller bounded output.
-            return fitToolText(payload, window - output, projectedTokens);
-          } catch (error) { fatal(error); }
-        },
-      });
-    },
-  });
+  function streamLocal(model, context, options) {
+    if (model.id !== configured.id || model.provider !== 'factory-local') fatal(new Error('Selected local model changed'));
+    // Capture before streamSimple/buildBaseOptions clamps against unprojected
+    // history. Generation defaults to the binding; summaries supply their cap.
+    const requestedOutput = options?.maxTokens ?? output;
+    return openAICompletionsApi().streamSimple(model, context, {
+      ...options,
+      onPayload: async (payload, requestModel) => {
+        try {
+          payload = await options?.onPayload?.(payload, requestModel) ?? payload;
+          payload = structuredClone(payload);
+          if (payload.model !== configured.id
+            || !payload.messages?.some(m => m.role === 'system')) throw new Error('Unsupported local request structure or output budget');
+          restoreOutputBudget(payload, configured, requestedOutput, options?.reasoning);
+          if (payload.tools?.length) {
+            task ??= structuredClone(payload.messages.find(m => m.role === 'user'));
+            if (!task) throw new Error('Missing required task');
+            if (!payload.messages.some(m => m.role === 'user' && JSON.stringify(m) === JSON.stringify(task)))
+              payload.messages.splice(1, 0, structuredClone(task));
+            validatePairs(payload.messages);
+            envelope = bytes(payload.tools) + bytes(payload.messages.filter(m => m.role === 'system'))
+              + bytes(task) + 4096;
+          }
+          if (!payload.messages.some(m => m.role === 'user')) throw new Error('Missing required user message');
+          // Restore BEFORE measuring so the final output field is counted.
+          // No request is sent unless projection fits with the full configured
+          // reserve, even when a summary requests a smaller bounded output.
+          return fitToolText(payload, window - output, projectedTokens);
+        } catch (error) { fatal(error); }
+      },
+    });
+  }
+  pi.registerProvider('factory-local', { api: 'openai-completions', streamSimple: streamLocal });
   // Launcher requires this handshake; an extension load failure cannot bypass it.
   writeSync(1, `${JSON.stringify({ type: 'factory_context_ready', version: 1 })}\n`);
 }

package/factory/pi-local-launch.mjs

@@ -24,6 +24,8 @@ export class PiCompletion {
   settled = false;
   lastAssistant = null;
   contextReady = false;
+  proactiveRecovery = false;
+  proactiveCount = 0;
 
   write(chunk) {
     this.pending += chunk;
@@ -42,6 +44,17 @@ export class PiCompletion {
   event(value) {
     if (!value || typeof value.type !== 'string') { this.failed = true; return; }
     if (value.type === 'factory_context_ready' && value.version === 1) this.contextReady = true;
+    if (value.type === 'factory_context_recovery') {
+      if (value.phase === 'start') {
+        if (this.proactiveRecovery || value.recovery !== this.proactiveCount + 1 || value.recovery > 16) this.failed = true;
+        this.proactiveCount = value.recovery;
+        this.proactiveRecovery = true;
+        this.settled = false;
+      } else if (value.phase === 'complete') {
+        if (!this.proactiveRecovery || value.recovery !== this.proactiveCount) this.failed = true;
+        this.proactiveRecovery = false;
+      } else this.failed = true;
+    }
     if (value.type === 'agent_start' || value.type === 'message_start') {
       this.completed = false; this.settled = false;
     }
@@ -95,7 +108,7 @@ export class PiCompletion {
 
   finish() {
     return !this.pending && !this.failed && !this.error && !this.retry && !this.compacting && !this.recovery
-      && this.completed && (!this.modern || this.settled);
+      && !this.proactiveRecovery && this.completed && (!this.modern || this.settled);
   }
 }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "software-defence-factory",
-  "version": "0.15.4",
+  "version": "0.15.5",
   "description": "Factory: scoped software delivery and defence investigations with isolated jobs, evidence and review",
   "type": "module",
   "license": "MIT",
@@ -58,7 +58,7 @@
     "check": "node scripts/check.mjs && node --test tests/*.test.mjs && npm run test:dashboard",
     "build:dashboard": "npm --prefix dashboard ci --ignore-scripts && npm --prefix dashboard run build",
     "test:dashboard": "npm --prefix dashboard test",
-    "qualify:pi": "FACTORY_REQUIRE_PI=1 node --test tests/pi-local-adapter.test.mjs tests/pi-trust.test.mjs tests/pi-context.test.mjs tests/pi-context-batch.test.mjs"
+    "qualify:pi": "FACTORY_REQUIRE_PI=1 node --test tests/pi-local-adapter.test.mjs tests/pi-trust.test.mjs tests/pi-context.test.mjs tests/pi-context-batch.test.mjs tests/pi-context-history.test.mjs tests/pi-context-productive.test.mjs"
   },
   "dependencies": {
     "yaml": "2.9.1"