mixdog 0.9.3 → 0.9.5

package/scripts/bench/round-r11-codexalign.json

@@ -0,0 +1,257 @@
+{
+  "round": "11",
+  "runner": "lead",
+  "opts": {
+    "provider": "openai-oauth",
+    "model": "gpt-5.5",
+    "effort": null,
+    "fast": false,
+    "env": {
+      "MIXDOG_DATA_DIR": "C:\\Users\\tempe\\.mixdog\\bench-data"
+    }
+  },
+  "tasks": 3,
+  "completed": 3,
+  "completion_rate": 100,
+  "sessions": [
+    "sess_47100_1783055745198_1783055745435_dcf2093a36ef942ce3a48b2cc5b2544f",
+    "sess_37868_1783055745212_1783055745449_8ddb3a100a3e63e79471cd301c4b4e9d",
+    "sess_24344_1783055745227_1783055745464_1171ce7c8fbe4ebac0323e11cd6150ac"
+  ],
+  "results": [
+    {
+      "id": "r4-debug-forwarding-gap",
+      "agent": "reviewer",
+      "ok": true,
+      "ms": 110161,
+      "sessionId": "sess_47100_1783055745198_1783055745435_dcf2093a36ef942ce3a48b2cc5b2544f",
+      "output": "sessionId: sess_47100_1783055745198_1783055745435_dcf2093a36ef942ce3a48b2cc5b2544f\n재영님, 조사 결과 활성화 직후 출력은 “지연”은 대체로 복구되지만, 이미 존재하는 새 transcript에 대한 첫 바인딩은 실제 유실 경로입니다.\n\n**경로 요약**: `bridge-state.json`은 `activate_channel_bridge`에서 먼저 쓰이고(`src/runtime/channels/lib/tool-dispatch.mjs:45-59`), worker start도 동일하게 먼저 활성화합니다(`src/runtime/channels/index.mjs:1735-1745`). 이후 ownership/connect, persisted transcript bind, `fs.watch`, `forwardNewText()`가 이어집니다.\n\n**1. 최대 영향 — 기존 transcript를 첫 바인딩하면 EOF로 점프해 이전 출력 유실**: `catchUpFromPersisted`는 같은 persisted path일 때만 이전 cursor를 쓰고, 아니면 `currentSize`로 시작합니다(`src/runtime/channels/lib/output-forwarder.mjs:117-126`). 활성화 bind는 이 상태로 `forwardNewText()`를 호출하므로(`src/runtime/channels/index.mjs:625-629`, `src/runtime/channels/index.mjs:801-812`) 바인딩 전 이미 파일에 쓰인 assistant 출력은 backlog로 복구되지 않습니다. 단, 같은 transcript의 persisted `lastFileSize`가 남아 있으면 복구됩니다.\n\n**2. 채널 바인딩이 없으면 나중의 첫 inbound에서 기존 출력이 EOF 처리될 수 있음**: 활성화 시 status/main channel이 없으면 bind가 아예 중단됩니다(`src/runtime/channels/index.mjs:610-624`), `forwardNewText()`도 `channelId` 없이 no-op입니다(`src/runtime/channels/lib/output-forwarder.mjs:351-354`). 나중에 inbound가 기존 transcript를 `applyTranscriptBinding()`하면 catch-up 옵션 없이 EOF로 잡힐 수 있어(`src/runtime/channels/index.mjs:1597-1603`) backlog는 기존 파일에 대해 복구하지 못합니다.\n\n**3. not-yet-created transcript의 `fs.watch` 실패는 설계상 지연 복구됨**: `startWatch()`는 없는 파일에서 catch 후 watcher를 닫습니다(`src/runtime/channels/lib/output-forwarder.mjs:647-670`), 이를 `schedulePendingTranscriptRearm()`이 250ms polling으로 재설치하고 즉시 `forwardNewText()` 합니다(`src/runtime/channels/index.mjs:339-346`, `src/runtime/channels/index.mjs:376-381`, `src/runtime/channels/index.mjs:493-498`). 60초 내 생성되고 ownership이 유지되면 backlog 복구됩니다; timeout/ownership loss 후 재bind가 없으면 복구되지 않습니다.\n\n**4. bridge-state write 후 ownership/backend connect 전 출력은 지연됨**: forwarder는 `bridgeRuntimeConnected && currentOwnerState().owned`가 false면 읽지 않습니다(`src/runtime/channels/index.mjs:316`, `src/runtime/channels/lib/output-forwarder.mjs:351-353`). connect 완료 후 `bridgeRuntimeConnected = true`가 되고 post-connect `forwardNewText()`가 실행됩니다(`src/runtime/channels/index.mjs:769-781`, `src/runtime/channels/index.mjs:801-812`), 그래서 cursor가 보존된 경우 backlog 복구됩니다.\n\n**5. watch 설치 직전/직후의 짧은 race는 복구됨**: `setContext()`의 stat 이후 watch 설치 사이에 쓰인 bytes는 `startWatch()`가 즉시 `scheduleWatchFlush()`를 호출해 읽습니다(`src/runtime/channels/lib/output-forwarder.mjs:665-666`, `src/runtime/channels/lib/output-forwarder.mjs:703-731`). 단, `setContext()` stat 이전 bytes는 1번 유실 조건에 속합니다.\n\n**Verdict**: 활성화-연결-미생성 파일 구간의 지연은 대부분 catch-up 되지만, “새로 발견된 이미 존재하는 transcript”를 EOF로 바인딩하는 경로는 첫 assistant 출력을 조용히 유실할 수 있습니다.",
+      "card": {
+        "session": "sess_47100_1783055745198_1783055745435_dcf2093a36ef942ce3a48b2cc5b2544f",
+        "provider": "openai-oauth",
+        "model": "gpt-5.5",
+        "wall_ms": 99667,
+        "turns": 6,
+        "tool_calls": 7,
+        "tools_per_turn": 1.2,
+        "total_tool_ms": 654,
+        "llm_stream_ms": 98986,
+        "cache_ratio": 0.4810080500736896,
+        "cached_tokens": 81920,
+        "prompt_tokens": 170309,
+        "uncached_tokens": 88389,
+        "output_tokens": 10473,
+        "cache_weighted_input_10": 96581,
+        "cache_weighted_input_25": 108869,
+        "prompt_growth": 49597,
+        "antipatterns": 1,
+        "issues": 5,
+        "issue_types": [
+          "cache_miss",
+          "cache",
+          "broad_tool_result",
+          "service_tier",
+          "tool_churn_cluster"
+        ],
+        "read_fragmentation_paths": [],
+        "grep_sweep_count": 0
+      }
+    },
+    {
+      "id": "r4-impl-usage-summary",
+      "agent": "worker",
+      "ok": true,
+      "ms": 76472,
+      "sessionId": "sess_37868_1783055745212_1783055745449_8ddb3a100a3e63e79471cd301c4b4e9d",
+      "output": "sessionId: sess_37868_1783055745212_1783055745449_8ddb3a100a3e63e79471cd301c4b4e9d\n구현 상태를 확인하고 요청하신 검증을 바로 실행하겠습니다.요청한 기본 경로에 맞도록 `MIXDOG_DATA_DIR`가 없을 때는 `~/.mixdog/data`만 사용하게 조정하겠습니다.패치 도구가 실패해 PowerShell로 동일 변경을 적용하고 검증까지 다시 실행하겠습니다.완료했습니다: `usage:today` npm script와 새 `scripts/usage-today.mjs`를 추가했습니다.\n\n- **변경 요약**: `package.json:58`에 `usage:today`를 추가했고, 새 스크립트가 `MIXDOG_DATA_DIR` 또는 `~/.mixdog/data/history/agent-trace.jsonl`을 읽어 오늘 사용량을 모델별/총합으로 출력합니다.\n\n- **출력 항목**: 모델별 `calls`, `input`, `cached`, `output`, `costUsd`와 `GRAND TOTAL`을 표로 출력하며 `--trace`, `--json`도 지원합니다.\n\n- **검증 완료**: `node --check scripts/usage-today.mjs` 통과, `npm run usage:today` 실행 완료; 현재 환경에서는 총 `1157` calls, `76.2205` costUsd가 출력됐습니다.\n\n- **참고**: 기존 작업트리의 무관한 수정 `src/runtime/channels/lib/event-queue.mjs`, `src/runtime/channels/lib/output-forwarder.mjs`는 건드리지 않았습니다.",
+      "card": {
+        "session": "sess_37868_1783055745212_1783055745449_8ddb3a100a3e63e79471cd301c4b4e9d",
+        "provider": "openai-oauth",
+        "model": "gpt-5.5",
+        "wall_ms": 65981,
+        "turns": 9,
+        "tool_calls": 9,
+        "tools_per_turn": 1,
+        "total_tool_ms": 3173,
+        "llm_stream_ms": 62795,
+        "cache_ratio": 0.6369198951943337,
+        "cached_tokens": 81920,
+        "prompt_tokens": 128619,
+        "uncached_tokens": 46699,
+        "output_tokens": 5005,
+        "cache_weighted_input_10": 54891,
+        "cache_weighted_input_25": 67179,
+        "prompt_growth": 16779,
+        "antipatterns": 1,
+        "issues": 7,
+        "issue_types": [
+          "cache_miss",
+          "cache",
+          "tool_error",
+          "service_tier",
+          "missed_parallelism",
+          "tool_churn_cluster"
+        ],
+        "read_fragmentation_paths": [],
+        "grep_sweep_count": 0
+      }
+    },
+    {
+      "id": "r4-explain-compact-engine",
+      "agent": "explore",
+      "ok": true,
+      "ms": 63493,
+      "sessionId": "sess_24344_1783055745227_1783055745464_1171ce7c8fbe4ebac0323e11cd6150ac",
+      "output": "sessionId: sess_24344_1783055745227_1783055745464_1171ce7c8fbe4ebac0323e11cd6150ac\nI’ll trace the code paths and line anchors directly from the repository, then condense the map.Architecture brief:\n\n- **Public surface:** `compact.mjs` is the facade and re-exports `semanticCompactMessages` / `recallFastTrackCompactMessages` from `engine.mjs` (`src/runtime/agent/orchestrator/session/compact.mjs:1-5`, `:52-55`; `engine.mjs:278`, `:445-447`).\n\n- **Main callers:** active turns enter through `askSession → agentLoop` (`manager.mjs:1379-1380`), then pre-send auto compaction calls recall-fasttrack or semantic paths (`loop.mjs:414-428`, `:439-465`).\n\n- **Manual/clear callers:** `compactSessionMessages` and clear-before-compact call `runSessionCompaction` (`manager.mjs:1841-1850`, `:1943-1957`), which dispatches recall-fasttrack or semantic (`compaction-runner.mjs:207-224`, `:280-310`).\n\n- **Fire decision:** loop drains pending steering before sizing so the next provider payload is measured (`loop.mjs:324-329`), resolves policy (`loop.mjs:329`; `compact-policy.mjs:89-148`), and compacts when pressure crosses trigger or reactive overflow forces it (`loop.mjs:351-364`; `compact-policy.mjs:170-174`).\n\n- **Policy math:** boundary comes from explicit/session context window, trigger is boundary minus buffer, reserve includes request/tool schema reserve, and target budget is capped to compact target plus reserve (`compact-policy.mjs:94-117`, `:121-145`, `:163-168`).\n\n- **Reactive path:** if provider send returns context overflow, one retry marks `reactiveOverflowRetryPending`, loops back through normal pre-send compaction, then surfaces overflow if it fails again (`loop.mjs:981-1010`, `:1012-1026`).\n\n- **Manual threshold:** `runSessionCompaction` treats manual as forced, skips auto if disabled, returns “below threshold” when non-forced pressure is under trigger (`compaction-runner.mjs:143-176`).\n\n- **Semantic window:** `splitLiveCompactionContext` sanitizes/repairs tool pairs, separates protected system/reminder prefix, extracts previous summary, and filters summary messages out of live history (`engine.mjs:100-111`; `messages.mjs:56-81`).\n\n- **Semantic tail choice:** if live has user turns, keep up to `tailTurns` recent turns within preserved budget and split newest turn suffix if needed; otherwise choose a valid non-`tool` cut point so tool results never start the tail (`engine.mjs:46`, `:134-185`, `:121-127`, `:193-232`, `:235-242`).\n\n- **Semantic assembly:** head is summarized, tail is preserved, tail is re-sanitized/reconciled for provider-valid tool pairs, and empty live windows or no prior head without summary throw (`engine.mjs:244-255`, `:314-317`, `:319-337`, `:367-397`).\n\n- **Recall tail choice:** recall fast-track keeps newest structured turns, default max two user turns / 8k cap, drops oldest kept turns first, truncates the oldest survivor if still too large, and permits no-user assistant/tool tails (`engine.mjs:449-468`, `:593-619`).\n\n- **Recall assembly:** recall path splits protected/live context, selects preserved tail, requires compactable head/previous/recall unless explicitly allowed, caps recall injection room, then emits `system + recall summary + preserved tail` (`engine.mjs:644-668`, `:679-699`).\n\n- **Redaction hook:** no-op fast paths return sanitized messages unchanged, but real semantic/recall compaction redacts tool-call secret arguments before window selection so preserved system/tail are measured and emitted redacted (`engine.mjs:286-312`, `:389-392`, `:622-645`).\n\n- **Redaction semantics:** secret-key and raw `key:value`/`key=value` values are replaced while preserving tool-call argument shape for verbatim tail messages (`text-utils.mjs:12-17`, `:40-80`, `:219-229`, `:230-288`).\n\n- **Fallbacks:** semantic auto failures can recover by unanchored tool-output prune; recall-fasttrack auto failures do not silently prune and escalate as overflow (`loop.mjs:500-537`, `:546-626`).\n\n- **Session fallback:** manual/clear recall-fasttrack failure tries semantic once when enabled/provider exists, then non-recall failures may use the same unanchored prune safety net; otherwise failure metadata is recorded without replacing history (`compaction-runner.mjs:231-278`, `:322-347`, `:348-397`).\n\n- **State after success:** changed loop compaction replaces in-memory messages, clears provider state, resets iteration/usage epoch, and emits telemetry/events; session compaction persists compacted messages and clears provider state (`loop.mjs:630-648`, `:651-716`; `compaction-runner.mjs:399-447`).",
+      "card": {
+        "session": "sess_24344_1783055745227_1783055745464_1171ce7c8fbe4ebac0323e11cd6150ac",
+        "provider": "openai-oauth",
+        "model": "gpt-5.5",
+        "wall_ms": 52999,
+        "turns": 8,
+        "tool_calls": 7,
+        "tools_per_turn": 0.9,
+        "total_tool_ms": 484,
+        "llm_stream_ms": 52522,
+        "cache_ratio": 0.27245049171494007,
+        "cached_tokens": 40448,
+        "prompt_tokens": 148460,
+        "uncached_tokens": 108012,
+        "output_tokens": 3624,
+        "cache_weighted_input_10": 112056.8,
+        "cache_weighted_input_25": 118124,
+        "prompt_growth": 39153,
+        "antipatterns": 1,
+        "issues": 9,
+        "issue_types": [
+          "cache_miss",
+          "cache",
+          "broad_tool_result",
+          "service_tier",
+          "missed_parallelism",
+          "tool_churn_cluster"
+        ],
+        "read_fragmentation_paths": [],
+        "grep_sweep_count": 0
+      }
+    }
+  ],
+  "task_complete": true,
+  "task_errors": [],
+  "score_complete": true,
+  "score_errors": [],
+  "group": {
+    "n": 3,
+    "wall_ms": 72882.3,
+    "turns": 7.7,
+    "tool_calls": 7.7,
+    "tools_per_turn": 1,
+    "total_tool_ms": 1437,
+    "llm_stream_ms": 71434.3,
+    "cache_ratio": 0.5,
+    "cached_tokens": 68096,
+    "prompt_tokens": 149129.3,
+    "uncached_tokens": 81033.3,
+    "output_tokens": 6367.3,
+    "cache_weighted_input_10": 87842.9,
+    "cache_weighted_input_25": 98057.3,
+    "prompt_growth": 35176.3,
+    "antipatterns": 1,
+    "issues": 7,
+    "grep_sweep_count": 0
+  },
+  "cards": [
+    {
+      "session": "sess_47100_1783055745198_1783055745435_dcf2093a36ef942ce3a48b2cc5b2544f",
+      "provider": "openai-oauth",
+      "model": "gpt-5.5",
+      "wall_ms": 99667,
+      "turns": 6,
+      "tool_calls": 7,
+      "tools_per_turn": 1.2,
+      "total_tool_ms": 654,
+      "llm_stream_ms": 98986,
+      "cache_ratio": 0.4810080500736896,
+      "cached_tokens": 81920,
+      "prompt_tokens": 170309,
+      "uncached_tokens": 88389,
+      "output_tokens": 10473,
+      "cache_weighted_input_10": 96581,
+      "cache_weighted_input_25": 108869,
+      "prompt_growth": 49597,
+      "antipatterns": 1,
+      "issues": 5,
+      "issue_types": [
+        "cache_miss",
+        "cache",
+        "broad_tool_result",
+        "service_tier",
+        "tool_churn_cluster"
+      ],
+      "read_fragmentation_paths": [],
+      "grep_sweep_count": 0
+    },
+    {
+      "session": "sess_37868_1783055745212_1783055745449_8ddb3a100a3e63e79471cd301c4b4e9d",
+      "provider": "openai-oauth",
+      "model": "gpt-5.5",
+      "wall_ms": 65981,
+      "turns": 9,
+      "tool_calls": 9,
+      "tools_per_turn": 1,
+      "total_tool_ms": 3173,
+      "llm_stream_ms": 62795,
+      "cache_ratio": 0.6369198951943337,
+      "cached_tokens": 81920,
+      "prompt_tokens": 128619,
+      "uncached_tokens": 46699,
+      "output_tokens": 5005,
+      "cache_weighted_input_10": 54891,
+      "cache_weighted_input_25": 67179,
+      "prompt_growth": 16779,
+      "antipatterns": 1,
+      "issues": 7,
+      "issue_types": [
+        "cache_miss",
+        "cache",
+        "tool_error",
+        "service_tier",
+        "missed_parallelism",
+        "tool_churn_cluster"
+      ],
+      "read_fragmentation_paths": [],
+      "grep_sweep_count": 0
+    },
+    {
+      "session": "sess_24344_1783055745227_1783055745464_1171ce7c8fbe4ebac0323e11cd6150ac",
+      "provider": "openai-oauth",
+      "model": "gpt-5.5",
+      "wall_ms": 52999,
+      "turns": 8,
+      "tool_calls": 7,
+      "tools_per_turn": 0.9,
+      "total_tool_ms": 484,
+      "llm_stream_ms": 52522,
+      "cache_ratio": 0.27245049171494007,
+      "cached_tokens": 40448,
+      "prompt_tokens": 148460,
+      "uncached_tokens": 108012,
+      "output_tokens": 3624,
+      "cache_weighted_input_10": 112056.8,
+      "cache_weighted_input_25": 118124,
+      "prompt_growth": 39153,
+      "antipatterns": 1,
+      "issues": 9,
+      "issue_types": [
+        "cache_miss",
+        "cache",
+        "broad_tool_result",
+        "service_tier",
+        "missed_parallelism",
+        "tool_churn_cluster"
+      ],
+      "read_fragmentation_paths": [],
+      "grep_sweep_count": 0
+    }
+  ]
+}