mixdog 0.9.150 → 0.9.151

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (53) hide show
  1. package/README.md +28 -11
  2. package/package.json +1 -1
  3. package/src/headless-exec.mjs +16 -3
  4. package/src/headless-exec.test.mjs +44 -1
  5. package/src/rules/shared/30-exploration.md +2 -1
  6. package/src/runtime/agent/orchestrator/session/compact/handoff.mjs +2 -4
  7. package/src/runtime/agent/orchestrator/session/compact/runner.mjs +18 -33
  8. package/src/runtime/agent/orchestrator/session/loop/recall-fasttrack.mjs +21 -42
  9. package/src/runtime/agent/orchestrator/session/manager/compaction-runner.mjs +20 -125
  10. package/src/runtime/agent/orchestrator/session/manager/session-lifecycle.mjs +8 -20
  11. package/src/runtime/agent/orchestrator/tools/builtin/absolute-glob-expand.test.mjs +63 -0
  12. package/src/runtime/agent/orchestrator/tools/builtin/atomic-write.mjs +29 -10
  13. package/src/runtime/agent/orchestrator/tools/builtin/builtin-tools.mjs +1 -1
  14. package/src/runtime/agent/orchestrator/tools/builtin/enoent-outside-project.test.mjs +56 -0
  15. package/src/runtime/agent/orchestrator/tools/builtin/external-tool-adapters.mjs +6 -2
  16. package/src/runtime/agent/orchestrator/tools/builtin/git-command-tool.mjs +32 -16
  17. package/src/runtime/agent/orchestrator/tools/builtin/git-command-tool.test.mjs +28 -0
  18. package/src/runtime/agent/orchestrator/tools/builtin/grep-single-file-rescue.test.mjs +106 -0
  19. package/src/runtime/agent/orchestrator/tools/builtin/lib/absolute-glob-expand.mjs +129 -0
  20. package/src/runtime/agent/orchestrator/tools/builtin/lib/grep-single-file-rescue.mjs +218 -0
  21. package/src/runtime/agent/orchestrator/tools/builtin/list-tool.mjs +33 -0
  22. package/src/runtime/agent/orchestrator/tools/builtin/noise-dir-visibility.test.mjs +69 -0
  23. package/src/runtime/agent/orchestrator/tools/builtin/path-diagnostics.mjs +33 -6
  24. package/src/runtime/agent/orchestrator/tools/builtin/read-tool.mjs +26 -6
  25. package/src/runtime/agent/orchestrator/tools/builtin/runtime-capabilities.mjs +7 -66
  26. package/src/runtime/agent/orchestrator/tools/builtin/search-builders.mjs +11 -0
  27. package/src/runtime/agent/orchestrator/tools/builtin/search-glob-tool.mjs +25 -1
  28. package/src/runtime/agent/orchestrator/tools/builtin/search-grep-tool.mjs +60 -5
  29. package/src/runtime/agent/orchestrator/tools/builtin/search-path-diagnostics.mjs +54 -1
  30. package/src/runtime/agent/orchestrator/tools/builtin/write-symlink.test.mjs +153 -0
  31. package/src/runtime/agent/orchestrator/tools/code-graph/aggregate-anchor-relocation.test.mjs +55 -0
  32. package/src/runtime/agent/orchestrator/tools/code-graph/dispatch.mjs +54 -3
  33. package/src/runtime/agent/orchestrator/tools/graph-manifest.json +11 -11
  34. package/src/runtime/agent/orchestrator/tools/patch/dispatch.mjs +44 -10
  35. package/src/runtime/agent/orchestrator/tools/patch/orchestrator.mjs +57 -5
  36. package/src/runtime/agent/orchestrator/tools/patch/patch-symlink.test.mjs +141 -0
  37. package/src/runtime/agent/orchestrator/tools/patch/v4a-convert.mjs +7 -3
  38. package/src/runtime/agent/orchestrator/tools/patch/v4a-pure-move.test.mjs +57 -0
  39. package/src/runtime/agent/orchestrator/tools/patch/v4a-section-coalesce.test.mjs +81 -0
  40. package/src/runtime/memory/index.mjs +2 -4
  41. package/src/runtime/memory/lib/pg/process.mjs +29 -2
  42. package/src/runtime/memory/lib/pg/process.test.mjs +36 -0
  43. package/src/runtime/memory/lib/pg/supervisor.mjs +5 -1
  44. package/src/runtime/memory/lib/query-handlers.mjs +7 -6
  45. package/src/runtime/memory/lib/recall-format.mjs +13 -31
  46. package/src/runtime/shared/pristine-execution-contract.json +3 -0
  47. package/src/runtime/shared/pristine-execution.mjs +5 -2
  48. package/src/session-runtime/lifecycle-api.mjs +1 -0
  49. package/src/standalone/channel-worker-heartbeat.test.mjs +30 -0
  50. package/src/standalone/channel-worker.mjs +31 -1
  51. package/src/standalone/memory-runtime-proxy.mjs +5 -2
  52. package/src/standalone/session-client.mjs +23 -3
  53. package/src/tui/session/turn.mjs +12 -0
package/README.md CHANGED
@@ -150,17 +150,34 @@ npm run dev
150
150
 
151
151
  ## Terminal-Bench 2.1
152
152
 
153
- Controlled single-model runs on the same 89 tasks produced:
154
-
155
- - **82/89** with Claude Opus 5 vs Claude Code's **77/89**
156
- - **79/89** with GPT-5.6 Sol xhigh vs Codex CLI's **75/89**
157
- - **1.21×** faster vs Claude Code and **1.15×** faster vs Codex CLI
158
- - **31–47%** smaller median final context
159
- - **16%** lower priced cost vs Claude Code and **41%** lower vs Codex CLI
160
-
161
- These are self-reported single runs (`k=1`, 2026-08-23), not leaderboard
162
- submissions. Raw artifacts, commands, comparison charts, and metric scripts
163
- live under [`benchmarks/terminal-bench-2.1/`](benchmarks/terminal-bench-2.1/).
153
+ Head-to-head runs against other agent CLIs on the same 89 tasks, holding the
154
+ model fixed and changing only the harness. Every run uses the official Harbor
155
+ verifier, fast mode off, a 272k context window, and zero retries.
156
+
157
+ **GPT-5.6 Sol xhigh Mixdog vs Codex CLI**
158
+
159
+ - Score: **86.5%** (385/445) vs Codex CLI's **84.3%** (75/89)
160
+ - **1.11×** faster per trial (339s vs 378s)
161
+ - **42%** lower priced cost ($0.641 vs $1.096 per trial)
162
+ - **45%** smaller median final context (18.5k vs 33.5k tokens)
163
+
164
+ **Claude Opus 5 — Mixdog vs Claude Code**
165
+
166
+ - Score: **79/89** vs Claude Code's **77/89**
167
+ - **1.15×** faster and **19%** lower priced cost ($104.29 vs $129.21 per run)
168
+ - **28%** smaller median final context (27.6k vs 38.2k tokens)
169
+
170
+ Measurement: the Mixdog Sol run follows the same protocol the official
171
+ Terminal-Bench leaderboard requires — all 89 tasks repeated five times
172
+ (`k=5`, 445 trials), scored by the official Harbor verifier. The Codex CLI
173
+ baseline and both Opus-side runs are single passes (`k=1`, 89 trials each).
174
+
175
+ The official leaderboard is not accepting community submissions, so every run
176
+ here ships its raw artifacts instead — Harbor verdicts, official verifier
177
+ output, pinned task checksums, and the usage snapshots behind every cost
178
+ figure — alongside the harness, presets, and metric scripts that recompute
179
+ each number above:
180
+ [`benchmarks/terminal-bench-2.1/`](benchmarks/terminal-bench-2.1/).
164
181
 
165
182
  ## Data and configuration
166
183
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mixdog",
3
- "version": "0.9.150",
3
+ "version": "0.9.151",
4
4
  "private": false,
5
5
  "type": "module",
6
6
  "description": "Standalone mixdog coding-agent CLI/TUI workspace.",
@@ -14,6 +14,7 @@ import {
14
14
  import { hasActiveBackgroundTasks } from './runtime/shared/background-tasks.mjs';
15
15
  import { installProcessSignalCleanup } from './runtime/shared/process-shutdown.mjs';
16
16
  import { stopStandaloneMemoryRuntimesForProcess } from './standalone/memory-runtime-proxy.mjs';
17
+ import { shutdownDaemonForRuntimeRoot } from './standalone/session-client.mjs';
17
18
  import { applyUsageDelta, createSessionStats } from './ui/session-stats.mjs';
18
19
 
19
20
  function clean(value) {
@@ -558,6 +559,7 @@ export async function runHeadlessExec({
558
559
  boundaryFactory = createPristineExecutionBoundary,
559
560
  runtimeFactory = null,
560
561
  memoryRuntimeCleanup = stopStandaloneMemoryRuntimesForProcess,
562
+ daemonRuntimeCleanup = shutdownDaemonForRuntimeRoot,
561
563
  hasActiveTasks = hasActiveBackgroundTasks,
562
564
  installSignalCleanupFn = installProcessSignalCleanup,
563
565
  } = {}) {
@@ -616,17 +618,28 @@ export async function runHeadlessExec({
616
618
  } catch (error) {
617
619
  errors.push(error);
618
620
  }
619
- let memoryCleanupFailed = false;
621
+ let resourceCleanupFailed = false;
622
+ if (boundary?.runtimeRoot) {
623
+ try {
624
+ await daemonRuntimeCleanup(boundary.runtimeRoot, {
625
+ waitForExit: true,
626
+ timeoutMs: 8_000,
627
+ });
628
+ } catch (error) {
629
+ resourceCleanupFailed = true;
630
+ errors.push(error);
631
+ }
632
+ }
620
633
  if (boundary) {
621
634
  try {
622
635
  await memoryRuntimeCleanup({ waitForExit: true, timeoutMs: 10_000 });
623
636
  } catch (error) {
624
- memoryCleanupFailed = true;
637
+ resourceCleanupFailed = true;
625
638
  errors.push(error);
626
639
  }
627
640
  }
628
641
  try {
629
- const cleanupResult = boundary?.cleanup(memoryCleanupFailed
642
+ const cleanupResult = boundary?.cleanup(resourceCleanupFailed
630
643
  ? { preserveRoot: true }
631
644
  : { tolerateRootRemovalFailure: true });
632
645
  if (cleanupResult?.rootRemovalError) {
@@ -62,6 +62,7 @@ test('headless exec runs one implicit-approval session and waits for tracked tas
62
62
  let activeChecks = 0;
63
63
  let boundaryCleaned = false;
64
64
  let runtimeClosed = false;
65
+ const daemonCleanupCalls = [];
65
66
  const cleanupOrder = [];
66
67
  try {
67
68
  const code = await runHeadlessExec({
@@ -75,6 +76,7 @@ test('headless exec runs one implicit-approval session and waits for tracked tas
75
76
  write: (text) => output.push(text),
76
77
  writeErr: (text) => errors.push(text),
77
78
  boundaryFactory: () => ({
79
+ runtimeRoot: join(root, 'runtime-root'),
78
80
  loadConfig: () => ({ providers: { 'openai-oauth': { enabled: true } } }),
79
81
  cleanup: () => {
80
82
  boundaryCleaned = true;
@@ -106,6 +108,10 @@ test('headless exec runs one implicit-approval session and waits for tracked tas
106
108
  memoryRuntimeCleanup: async () => {
107
109
  cleanupOrder.push('memory');
108
110
  },
111
+ daemonRuntimeCleanup: async (runtimeRoot, options) => {
112
+ daemonCleanupCalls.push({ runtimeRoot, options });
113
+ cleanupOrder.push('daemon');
114
+ },
109
115
  hasActiveTasks: (scope) => {
110
116
  activeScopes.push(scope);
111
117
  activeChecks += 1;
@@ -128,7 +134,11 @@ test('headless exec runs one implicit-approval session and waits for tracked tas
128
134
  });
129
135
  assert.equal(boundaryCleaned, true);
130
136
  assert.equal(runtimeClosed, true);
131
- assert.deepEqual(cleanupOrder, ['runtime', 'memory', 'boundary']);
137
+ assert.deepEqual(cleanupOrder, ['runtime', 'daemon', 'memory', 'boundary']);
138
+ assert.deepEqual(daemonCleanupCalls, [{
139
+ runtimeRoot: join(root, 'runtime-root'),
140
+ options: { waitForExit: true, timeoutMs: 8_000 },
141
+ }]);
132
142
  const usage = JSON.parse(readFileSync(usageLogPath, 'utf8'));
133
143
  assert.deepEqual(usage.sessions[0].models, ['gpt-test', 'gpt-fallback']);
134
144
  assert.deepEqual(usage.totals, {
@@ -143,6 +153,39 @@ test('headless exec runs one implicit-approval session and waits for tracked tas
143
153
  }
144
154
  });
145
155
 
156
+ test('headless exec preserves the pristine root when isolated daemon shutdown fails', async () => {
157
+ const errors = [];
158
+ let cleanupOptions = null;
159
+ const code = await runHeadlessExec({
160
+ message: 'done',
161
+ provider: 'openai-oauth',
162
+ model: 'gpt-test',
163
+ usageLogPath: '',
164
+ write() {},
165
+ writeErr: (text) => errors.push(text),
166
+ boundaryFactory: () => ({
167
+ runtimeRoot: '/isolated/runtime',
168
+ loadConfig: () => ({ providers: { 'openai-oauth': { enabled: true } } }),
169
+ cleanup: (options) => { cleanupOptions = options; },
170
+ }),
171
+ runtimeFactory: async () => ({
172
+ id: 'sess_cleanup_failure',
173
+ model: 'gpt-test',
174
+ clientHostPid: 123,
175
+ async ask() { return { result: { content: 'done' } }; },
176
+ async close() {},
177
+ }),
178
+ daemonRuntimeCleanup: async () => { throw new Error('daemon stuck'); },
179
+ memoryRuntimeCleanup: async () => {},
180
+ hasActiveTasks: () => false,
181
+ installSignalCleanupFn: () => ({ uninstall() {} }),
182
+ });
183
+
184
+ assert.equal(code, 1);
185
+ assert.deepEqual(cleanupOptions, { preserveRoot: true });
186
+ assert.ok(errors.some((line) => line.includes('shutdown failed: daemon stuck')));
187
+ });
188
+
146
189
  test('headless exec flushes the usage snapshot mid-session, before any exit path', async () => {
147
190
  const root = mkdtempSync(join(tmpdir(), 'mixdog-headless-usage-flush-test-'));
148
191
  const usageLogPath = join(root, 'usage.json');
@@ -27,7 +27,8 @@
27
27
  locator searches. Within the current project, pass project-relative paths and
28
28
  omit optional scopes equal to its root; explicit paths may be outside cwd
29
29
  only for targets outside the project.
30
- - Inspect source content only when its format is required and unknown.
30
+ - Before deciding how to parse, count, transform, or summarize files whose
31
+ format has not been inspected, inspect the original content itself.
31
32
  - Returned declarations, bodies, usages, relations, and contextual spans from
32
33
  any tool — not only `read` — are source context; `read` covers only omitted
33
34
  lines or missing anchored ranges.
@@ -1,6 +1,5 @@
1
1
  import { estimateTokens } from '../context-utils.mjs';
2
2
 
3
- const CONVERSATION_LINE_CHARS = 800;
4
3
  const TOOL_OUTCOME_CHARS = 80;
5
4
 
6
5
  function textOf(m) {
@@ -339,7 +338,7 @@ export function conversationLinesFromMemoryText(text) {
339
338
  .replace(/\s+#\d+\s*$/, '')
340
339
  .trim();
341
340
  if (body && body !== '.' && body !== '…') {
342
- rows.push(`${current.role}: ${body.slice(0, CONVERSATION_LINE_CHARS)}`);
341
+ rows.push(`${current.role}: ${body}`);
343
342
  }
344
343
  current = null;
345
344
  };
@@ -372,8 +371,7 @@ export function excludeTailFromConversation(lines, tailMessages) {
372
371
  if (m?.role !== 'user' && m?.role !== 'assistant') continue;
373
372
  const body = String(typeof m.content === 'string' ? m.content : '')
374
373
  .replace(/\s+/g, ' ')
375
- .trim()
376
- .slice(0, CONVERSATION_LINE_CHARS);
374
+ .trim();
377
375
  if (body) tails.push(body);
378
376
  }
379
377
  if (!tails.length) return Array.isArray(lines) ? lines : [];
@@ -45,7 +45,6 @@ import {
45
45
  enforceSemanticSummarySchema,
46
46
  fitSemanticSummaryMessage,
47
47
  fitRecallFastTrackSummaryMessage,
48
- stripNestedSummaryHeaderLines,
49
48
  RECALL_TAIL_TRUNCATION_MARKER,
50
49
  RECALL_TAIL_SHORT_TRUNCATION_MARKER,
51
50
  } from './summary.mjs';
@@ -57,7 +56,6 @@ import {
57
56
  fitRecallHandoffText,
58
57
  conversationLinesFromMemoryText,
59
58
  excludeTailFromConversation,
60
- stripWorkingFileSections,
61
59
  } from './handoff.mjs';
62
60
 
63
61
  // Post-compact file re-attachment: re-inject fresh reads
@@ -555,12 +553,12 @@ export function recallFastTrackCompactMessages(messages, budgetTokens, opts = {}
555
553
  //
556
554
  // Turns are anchored on user-role boundaries: each turn = a user message plus
557
555
  // the assistant/tool/system/developer messages that follow it. Recall-fasttrack
558
- // keeps the newest live turn verbatim. Older conversation comes from memory.
556
+ // keeps the newest five live turns verbatim. Older conversation comes from memory.
559
557
  //
560
558
  // Partial tool_call/tool_result pairs that truncation might leave behind are
561
559
  // repaired by sanitizeToolPairs/reconcileDedupStubs in the caller, so pairing
562
560
  // stays valid even after trimming.
563
- const RECALL_TAIL_USER_MAX = 1;
561
+ const RECALL_TAIL_USER_MAX = 5;
564
562
  const RECALL_TAIL_TOKEN_CAP = DEFAULT_COMPACTION_KEEP_TOKENS; // 8k
565
563
  // A caller may request a cap smaller than one structurally valid message. Keep
566
564
  // the cap strict above this unavoidable floor while retaining a real anchor.
@@ -695,10 +693,10 @@ function truncateTailToCap(messages, cap) {
695
693
  return estimateMessagesTokens(sanitized) <= cap ? sanitized : [];
696
694
  }
697
695
 
698
- function splitRecallFitInputs(recallText, previousSummary) {
696
+ function splitRecallFitInputs(recallText) {
699
697
  return {
700
698
  recall: String(recallText || '').trim(),
701
- prior: previousSummary ? stripNestedSummaryHeaderLines(previousSummary) : '',
699
+ prior: '',
702
700
  };
703
701
  }
704
702
 
@@ -799,7 +797,7 @@ function _recallFastTrackCompactMessages(messages, budgetTokens, opts = {}) {
799
797
  }
800
798
  const sanitized = redactToolCallSecretsInMessages(baseSanitized);
801
799
 
802
- const { system: safeSystem, live, previousSummary } = splitLiveCompactionContext(sanitized);
800
+ const { system: safeSystem, live } = splitLiveCompactionContext(sanitized);
803
801
  const recallTailOpts = {
804
802
  maxUsers: opts.recallTailMaxUsers ?? opts.tailTurns ?? RECALL_TAIL_USER_MAX,
805
803
  tokenCap: (() => {
@@ -815,8 +813,8 @@ function _recallFastTrackCompactMessages(messages, budgetTokens, opts = {}) {
815
813
  head: recallHead,
816
814
  lastTurnStartIdx = recallHead.length,
817
815
  } = selectRecallPreservedTail(live, recallTailOpts);
818
- const recallFit = splitRecallFitInputs(opts.recallText, previousSummary);
819
- if (recallHead.length === 0 && !previousSummary
816
+ const recallFit = splitRecallFitInputs(opts.recallText);
817
+ if (recallHead.length === 0
820
818
  && !(recallFit.recall || recallFit.prior || opts.allowEmptyRecall === true)) {
821
819
  throw new Error('recallFastTrackCompactMessages: no compactable prior history before preserved tail');
822
820
  }
@@ -848,44 +846,31 @@ function _recallFastTrackCompactMessages(messages, budgetTokens, opts = {}) {
848
846
  const recallRoom = (Number.isFinite(recallTokenCap) && recallTokenCap > 0)
849
847
  ? Math.min(recallRoomUncapped, Math.max(512, recallTokenCap - tailTokens))
850
848
  : recallRoomUncapped;
851
- const priorWithoutWorkingFiles = stripWorkingFileSections(recallFit.prior);
852
- const priorLines = new Set(
853
- priorWithoutWorkingFiles
854
- .split('\n')
855
- .map((line) => line.trim())
856
- .filter(Boolean),
857
- );
858
- const toolLines = collectToolOutcomeLines(live)
859
- .filter((line) => !priorLines.has(String(line || '').trim()));
860
- const workingFiles = collectWorkingFileGroups(live, undefined, {
849
+ const toolLines = collectToolOutcomeLines(recallHead)
850
+ .filter((line) => !recallFit.recall.includes(String(line || '').trim()));
851
+ const workingFiles = collectWorkingFileGroups(recallHead, undefined, {
861
852
  cwd: opts.cwd,
862
- previousSummary: recallFit.prior,
853
+ previousSummary: '',
863
854
  now: Date.now(),
864
855
  });
865
856
  const conversationLines = excludeTailFromConversation(
866
857
  conversationLinesFromMemoryText(recallFit.recall),
867
858
  recallTail,
868
- ).filter((line) => !priorLines.has(String(line || '').trim()));
859
+ );
869
860
  const composedRecall = composeRecallHandoff({
870
861
  sessionId: opts.sessionId || '',
871
862
  conversationLines,
872
863
  toolLines,
873
864
  workingFiles,
874
865
  });
875
- const priorRoom = priorWithoutWorkingFiles
876
- ? Math.max(256, Math.floor((recallRoom - 200) * 0.55))
877
- : 0;
878
- const currentRoom = Math.max(256, recallRoom - priorRoom - 200);
866
+ const currentRoom = Math.max(256, recallRoom - 200);
879
867
  const fittedRecall = fitRecallHandoffText(composedRecall, currentRoom);
880
- const priorPart = priorWithoutWorkingFiles
881
- ? fitRecallHandoffText(priorWithoutWorkingFiles, priorRoom)
882
- : '';
883
868
  const summaryMessage = fitRecallFastTrackSummaryMessage(
884
869
  oldHistory,
885
870
  fittedRecall,
886
871
  recallRoom,
887
872
  recallMeta,
888
- priorPart,
873
+ '',
889
874
  );
890
875
  if (!summaryMessage) {
891
876
  throw new Error(`recallFastTrackCompactMessages: summary cannot fit remaining budget=${recallRoom}`);
@@ -929,18 +914,18 @@ function _recallFastTrackCompactMessages(messages, budgetTokens, opts = {}) {
929
914
  recallRoom,
930
915
  recallChars: recallFit.recall.length,
931
916
  recallBytes: textByteLength(recallFit.recall),
932
- priorChars: recallFit.prior.length,
933
- priorBytes: textByteLength(recallFit.prior),
917
+ priorChars: 0,
918
+ priorBytes: 0,
934
919
  summaryMessageChars: summaryContent.length,
935
920
  summaryMessageBytes: textByteLength(summaryContent),
936
921
  recallEmpty: !recallFit.recall,
937
922
  priorEmpty: !recallFit.prior,
938
923
  recallTruncatedInSummary: !!recallFit.recall && !summaryContent.includes(recallFit.recall),
939
- priorTruncatedInSummary: !!priorPart && !summaryContent.includes(priorPart),
924
+ priorTruncatedInSummary: false,
940
925
  tailTruncated: recallTail.some((m) => messageContentHasMarker(m, RECALL_TAIL_TRUNCATION_MARKER) || messageContentHasMarker(m, RECALL_TAIL_SHORT_TRUNCATION_MARKER)),
941
926
  fileReattached: reattach.reattached,
942
927
  tailOptions: recallTailOpts,
943
- previousSummary: !!previousSummary,
928
+ previousSummary: false,
944
929
  durationMs: Date.now() - startedAt,
945
930
  };
946
931
  compactDebugLog('recall-fasttrack result', diagnostics);
@@ -19,43 +19,11 @@ import {
19
19
  compactDebugLog,
20
20
  } from './compact-debug.mjs';
21
21
  import { positiveTokenInt } from './env.mjs';
22
- import { TOOL_OUTPUT_MAX_BYTES } from '../../tools/builtin/tool-output-limit.mjs';
23
22
 
24
23
  // ── Digest injection ────────────────────────────────────────────────────────
25
- // Inject a small newest-first digest plus an instruction telling the model to
26
- // pull details lazily via recall(sessionId/query/period). The memory DB holds
27
- // the full session (ingest_session below), and raw rows are embedded
28
- // synchronously at ingest, so recall serves everything the old full-dump
29
- // injection used to carry.
30
- // Default digest cap = the SHARED tool-output limit (TOOL_OUTPUT_MAX_BYTES,
31
- // 50KB default, env MIXDOG_TOOL_OUTPUT_MAX_BYTES) — the digest injection is
32
- // budgeted like any other tool result, not a special context share.
33
- // compaction.recallDigestMaxKb still overrides per-session.
34
- export const DIGEST_DEFAULT_MAX_KB = Math.max(1, Math.floor(TOOL_OUTPUT_MAX_BYTES / 1024));
35
-
36
- // Byte-capped line-boundary truncation. Digest source is newest-first, so
37
- // keeping the HEAD keeps the newest turns.
38
- // Exported for manager/compaction-runner.mjs (manual//clear digest path) so
39
- // both digest producers share one cap implementation.
40
- export function truncateToKb(text, maxKb) {
41
- const maxBytes = Math.max(1, maxKb) * 1024;
42
- const s = String(text || '');
43
- if (Buffer.byteLength(s, 'utf8') <= maxBytes) return s;
44
- const lines = s.split('\n');
45
- const marker = '[digest truncated at ' + maxKb + 'KB]';
46
- const contentBudget = maxBytes - Buffer.byteLength(marker, 'utf8');
47
- const out = [];
48
- let used = 0;
49
- for (const line of lines) {
50
- const cost = Buffer.byteLength(line, 'utf8') + 1;
51
- if (used + cost > contentBudget) break;
52
- out.push(line);
53
- used += cost;
54
- }
55
- return out.length ? out.join('\n') + '\n' + marker : marker;
56
- }
57
-
58
- function buildRecallDigestText(sessionId, digestBody, maxKb) {
24
+ // Memory provides every available summary/raw row. Only the final compaction
25
+ // budget is allowed to reduce the handoff.
26
+ function buildRecallDigestText(sessionId, digestBody) {
59
27
  // No recall-usage instruction block here: the recall tool description
60
28
  // already carries the usage-pattern cheatsheet (tool-defs.mjs), so
61
29
  // repeating it per-compaction would be redundant injected tokens. The
@@ -64,7 +32,7 @@ function buildRecallDigestText(sessionId, digestBody, maxKb) {
64
32
  return [
65
33
  `[context compacted — session ${sessionId}]`,
66
34
  `Recent digest (newest first):`,
67
- truncateToKb(digestBody, maxKb),
35
+ String(digestBody || '').trim(),
68
36
  ].join('\n');
69
37
  }
70
38
 
@@ -88,7 +56,17 @@ export function isUsableRecallDigestText(value) {
88
56
  return !!trimmed && !/^\((?:no results|no current session)\)$/i.test(trimmed);
89
57
  }
90
58
 
91
- export async function runRecallFastTrackCompact({ sessionRef, messages, compactBudgetTokens, compactPolicy, sessionId, signal }) {
59
+ export const RECALL_FAST_TRACK_TAIL_TURNS = 5;
60
+
61
+ export async function runRecallFastTrackCompact({
62
+ sessionRef,
63
+ messages,
64
+ compactBudgetTokens,
65
+ compactPolicy,
66
+ sessionId,
67
+ signal,
68
+ executeMemorySearch,
69
+ }) {
92
70
  if (!sessionId) throw new Error('recall-fasttrack requires a session id');
93
71
  const startedAt = Date.now();
94
72
  const diagnostics = {
@@ -126,14 +104,15 @@ export async function runRecallFastTrackCompact({ sessionRef, messages, compactB
126
104
  // watcher already persisted. Do not retransmit the live transcript through
127
105
  // the memory RPC: that duplicates canonical history and can exceed the
128
106
  // bounded HTTP request body before compaction gets a chance to run.
129
- const digestMaxKb = positiveTokenInt(sessionRef?.compaction?.recallDigestMaxKb) || DIGEST_DEFAULT_MAX_KB;
130
107
  let digestBody = '';
131
108
  t0 = Date.now();
132
109
  try {
133
- const browsed = await executeInternalTool('memory', {
110
+ const searchMemory = typeof executeMemorySearch === 'function'
111
+ ? executeMemorySearch
112
+ : (args, ctx) => executeInternalTool('memory', args, ctx);
113
+ const browsed = await searchMemory({
134
114
  action: 'search',
135
115
  sessionId,
136
- limit: positiveTokenInt(sessionRef?.compaction?.recallDigestLimit) || 100,
137
116
  includeMembers: true,
138
117
  includeRaw: true,
139
118
  compactHandoff: true,
@@ -171,7 +150,7 @@ export async function runRecallFastTrackCompact({ sessionRef, messages, compactB
171
150
  try { process.stderr.write(`[loop] recall-fasttrack fail-safe abort (sess=${sessionId || 'unknown'}): stored session unavailable — keeping full history, no recall notice injected\n`); } catch {}
172
151
  throw new Error(`recall-fasttrack aborted: stored session memory unavailable; head preserved`);
173
152
  }
174
- const digestText = buildRecallDigestText(sessionId, digestBody, digestMaxKb);
153
+ const digestText = buildRecallDigestText(sessionId, digestBody);
175
154
  diagnostics.finalRecallChars = digestText.length;
176
155
  diagnostics.finalRecallBytes = compactByteLength(digestText);
177
156
  const contextWindow = positiveTokenInt(compactPolicy?.contextWindow)
@@ -192,7 +171,7 @@ export async function runRecallFastTrackCompact({ sessionRef, messages, compactB
192
171
  // Empty/sentinel browse output was rejected above, so the handoff always
193
172
  // contains real stored session context.
194
173
  allowEmptyRecall: false,
195
- tailTurns: compactPolicy.tailTurns,
174
+ tailTurns: RECALL_FAST_TRACK_TAIL_TURNS,
196
175
  keepTokens: compactPolicy.keepTokens,
197
176
  preserveRecentTokens: compactPolicy.preserveRecentTokens,
198
177
  recallTokenCap,
@@ -6,22 +6,17 @@
6
6
  import { createHash } from 'crypto';
7
7
  import { getProvider } from '../../providers/registry.mjs';
8
8
  import {
9
- recallFastTrackCompactMessages,
10
9
  semanticCompactMessages,
11
10
  pruneToolOutputsUnanchored,
12
11
  effectiveBudget as compactEffectiveBudget,
13
12
  compactTypeIsRecallFastTrack,
14
13
  compactTypeIsSemantic,
15
14
  normalizeCompactType,
16
- CONTEXT_SHARE_RATIO,
17
- RECALL_TOKEN_CAP_FLOOR_TOKENS,
18
15
  } from '../compact.mjs';
19
16
  import { estimateMessagesTokens, estimateRequestReserveTokens, estimateTranscriptContextUsage, resolveCompactBufferRatio } from '../context-utils.mjs';
20
17
  import { executeInternalTool } from '../../internal-tools.mjs';
21
18
  import {
22
- truncateToKb,
23
- DIGEST_DEFAULT_MAX_KB,
24
- isUsableRecallDigestText,
19
+ runRecallFastTrackCompact,
25
20
  } from '../loop/recall-fasttrack.mjs';
26
21
  import {
27
22
  positiveContextWindow,
@@ -159,58 +154,6 @@ async function callMemoryBounded(args, callerCtx, timeoutMs, executeMemory = exe
159
154
  try { outer?.removeEventListener?.('abort', onOuterAbort); } catch {}
160
155
  }
161
156
  }
162
- async function runRecallFastTrackForSession(session, sessionId, opts = {}) {
163
- if (!sessionId) throw new Error('recall-fasttrack requires a session id');
164
- const query = `session:${sessionId}:all-chunks`;
165
- const querySha = createHash('sha256').update(query).digest('hex').slice(0, 16);
166
- const callerCtx = {
167
- callerSessionId: sessionId,
168
- callerCwd: session?.cwd || undefined,
169
- routingSessionId: sessionId,
170
- clientHostPid: session?.clientHostPid,
171
- signal: opts.signal || null,
172
- };
173
- const memoryTimeoutMs = recallMemoryTimeoutMs(session);
174
- const executeMemory = typeof opts.executeInternalToolFn === 'function'
175
- ? opts.executeInternalToolFn
176
- : executeInternalTool;
177
- // The transcript watcher already persists this session incrementally.
178
- // Manual/clear compaction reads those rows directly instead of sending the
179
- // entire live message array through the memory RPC again.
180
- let recallText = '';
181
- try {
182
- const browsed = await callMemoryColdStart({
183
- action: 'search',
184
- sessionId,
185
- limit: positiveContextWindow(session?.compaction?.recallDigestLimit) || 100,
186
- includeMembers: true,
187
- includeRaw: true,
188
- compactHandoff: true,
189
- }, callerCtx, memoryTimeoutMs, executeMemory);
190
- recallText = typeof browsed === 'string' ? browsed : String(browsed?.text ?? browsed ?? '');
191
- if (!isUsableRecallDigestText(recallText)) {
192
- throw new Error('memory has no stored history for this session');
193
- }
194
- } catch (err) {
195
- // Without real stored context we cannot safely replace the live head.
196
- // Bail to the semantic fallback rather than injecting a false recall
197
- // handoff.
198
- try { process.stderr.write(`[session] recall-digest browse failed — bailing (sess=${sessionId}): ${err?.message || err}\n`); } catch {}
199
- throw new Error(`recall-fasttrack search failed: ${err?.message || err}`);
200
- }
201
- return {
202
- query,
203
- querySha,
204
- recallText: [
205
- `session_id=${sessionId}`,
206
- // Same byte cap as the loop digest path (recallDigestMaxKb,
207
- // default = shared tool-output limit) — without it the memory
208
- // renderer bounds the browse at ~200 rows × 1000 chars, letting a
209
- // manual//clear compact process a far larger digest than loop's.
210
- truncateToKb(recallText, positiveContextWindow(session?.compaction?.recallDigestMaxKb) || DIGEST_DEFAULT_MAX_KB),
211
- ].map(v => String(v || '').trim()).filter(Boolean).join('\n\n'),
212
- };
213
- }
214
157
  // Element-identity change detection (same approach as loop.mjs messagesArrayChanged): two
215
158
  // arrays are "unchanged" only when same length AND every slot is the same object
216
159
  // reference. Used to reject a no-op prune (which returns a fresh array whose
@@ -307,27 +250,27 @@ export async function runSessionCompaction(session, opts = {}) {
307
250
  let recallFastTrackError = null;
308
251
  if (compactTypeIsRecallFastTrack(compactType)) {
309
252
  try {
310
- const recallPayload = await runRecallFastTrackForSession(session, resolvedSessionId, opts);
311
253
  const contextWindow = positiveContextWindow(session.contextWindow) || boundary;
312
- const recallTokenCap = Math.max(
313
- RECALL_TOKEN_CAP_FLOOR_TOKENS,
314
- Math.floor(contextWindow * CONTEXT_SHARE_RATIO),
315
- );
316
- recallFastTrackResult = recallFastTrackCompactMessages(messages, budget, {
317
- reserveTokens,
318
- force: true,
319
- recallText: recallPayload.recallText,
320
- query: recallPayload.query,
321
- querySha: recallPayload.querySha,
322
- cwd: session.cwd,
323
- // The stored-memory browse above returned real session context;
324
- // empty/sentinel output already threw into semantic fallback.
325
- allowEmptyRecall: false,
326
- tailTurns: positiveContextWindow(session.compaction?.tailTurns) || 1,
327
- keepTokens: positiveContextWindow(session.compaction?.keepTokens ?? session.compaction?.keep?.tokens),
328
- preserveRecentTokens: positiveContextWindow(session.compaction?.preserveRecentTokens),
329
- recallTokenCap,
254
+ const memoryTimeoutMs = recallMemoryTimeoutMs(session);
255
+ const executeMemory = typeof opts.executeInternalToolFn === 'function'
256
+ ? opts.executeInternalToolFn
257
+ : executeInternalTool;
258
+ recallFastTrackResult = await runRecallFastTrackCompact({
259
+ sessionRef: session,
260
+ messages,
261
+ compactBudgetTokens: budget,
262
+ compactPolicy: {
263
+ reserveTokens,
264
+ contextWindow,
265
+ boundaryTokens: boundary,
266
+ keepTokens: positiveContextWindow(session.compaction?.keepTokens ?? session.compaction?.keep?.tokens),
267
+ preserveRecentTokens: positiveContextWindow(session.compaction?.preserveRecentTokens),
268
+ },
330
269
  sessionId: resolvedSessionId,
270
+ signal: opts.signal || null,
271
+ executeMemorySearch: (args, callerCtx) => (
272
+ callMemoryColdStart(args, callerCtx, memoryTimeoutMs, executeMemory)
273
+ ),
331
274
  });
332
275
  if (Array.isArray(recallFastTrackResult?.messages)) {
333
276
  compacted = recallFastTrackResult.messages;
@@ -338,54 +281,6 @@ export async function runSessionCompaction(session, opts = {}) {
338
281
  try {
339
282
  process.stderr.write(`[session] recall-fasttrack ${mode} compact failed (sess=${session.id || 'unknown'}): ${err?.message || err}\n`);
340
283
  } catch { /* best-effort */ }
341
- // Degraded-compact fallback: recall-fasttrack failed (empty recall,
342
- // ingest error, fit failure). Before recording a hard failure, try
343
- // the semantic path once so auto-clear/manual compaction still makes
344
- // progress WITHOUT shipping an empty-recall summary. History is only
345
- // replaced when the semantic summary actually succeeds.
346
- if (semanticCompactionEnabledForSession(session)
347
- && provider && typeof provider.send === 'function') {
348
- try {
349
- semanticCompactResult = await semanticCompactMessages(
350
- provider,
351
- messages,
352
- opts.model || resolveSemanticSummaryModel(session, { budgetTokens: budget }) || session.model,
353
- budget,
354
- {
355
- reserveTokens,
356
- providerName: session.provider || provider?.name || null,
357
- sessionId: resolvedSessionId,
358
- cwd: session.cwd,
359
- signal: opts.signal || null,
360
- // Carries the session's wire identity so the
361
- // summary request stays on the session's own
362
- // prefix-cache slot instead of opening a second one.
363
- sendOpts: { session },
364
- promptCacheKey: session.promptCacheKey || null,
365
- providerCacheKey: session.promptCacheKey || null,
366
- timeoutMs: semanticCompactTimeoutMs(session, beforeMessageTokens),
367
- tailTurns: positiveContextWindow(session.compaction?.tailTurns) || 2,
368
- keepTokens: positiveContextWindow(session.compaction?.keepTokens ?? session.compaction?.keep?.tokens),
369
- preserveRecentTokens: positiveContextWindow(session.compaction?.preserveRecentTokens),
370
- filterOldHistoryForIngest: opts.filterOldHistoryForIngest === true,
371
- force: true,
372
- },
373
- );
374
- if (Array.isArray(semanticCompactResult?.messages)) {
375
- compacted = semanticCompactResult.messages;
376
- compactError = null;
377
- addCompactUsageToSession(session, semanticCompactResult.usage);
378
- try {
379
- process.stderr.write(`[session] degraded compact: recall-fasttrack failed, semantic fallback succeeded (sess=${session.id || 'unknown'}, mode=${mode})\n`);
380
- } catch { /* best-effort */ }
381
- }
382
- } catch (fallbackErr) {
383
- semanticCompactError = fallbackErr;
384
- try {
385
- process.stderr.write(`[session] degraded compact: semantic fallback also failed (sess=${session.id || 'unknown'}): ${fallbackErr?.message || fallbackErr}\n`);
386
- } catch { /* best-effort */ }
387
- }
388
- }
389
284
  }
390
285
  } else if (compactTypeIsSemantic(compactType)) {
391
286
  try {