@turmind/halo 1.0.2 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/admin-out/404.html +1 -1
- package/admin-out/_next/static/chunks/212-394c4fd40216e633.js +1 -0
- package/admin-out/_next/static/chunks/220.e5a2532bda77e816.js +1 -0
- package/admin-out/_next/static/chunks/4-e7ba0b1ced30190b.js +1 -0
- package/admin-out/_next/static/chunks/442.68ed26e27eb1d84f.js +1 -0
- package/admin-out/_next/static/chunks/515.4d98fae302278913.js +1 -0
- package/admin-out/_next/static/chunks/896.ed21ec1e0846d20d.js +1 -0
- package/admin-out/_next/static/chunks/935.543aeec8456b6061.js +1 -0
- package/admin-out/_next/static/chunks/app/page-86cb00582129ae76.js +1 -0
- package/admin-out/_next/static/chunks/webpack-fc1574554e7b1108.js +1 -0
- package/admin-out/_next/static/css/0d0ca7b3c8cc5562.css +3 -0
- package/admin-out/index.html +1 -1
- package/admin-out/index.txt +5 -5
- package/bundled-docs/design/agent.md +13 -1
- package/bundled-docs/design/agentcore.md +10 -0
- package/bundled-docs/design/architecture.md +28 -29
- package/bundled-docs/design/background-dispatch.md +30 -43
- package/bundled-docs/design/cron.md +19 -4
- package/bundled-docs/design/evolution.md +18 -2
- package/bundled-docs/design/feishu.md +19 -6
- package/bundled-docs/design/goal-mode.md +2 -2
- package/bundled-docs/design/halo-city.md +7 -0
- package/bundled-docs/design/session.md +53 -3
- package/bundled-docs/design/slack.md +20 -6
- package/bundled-docs/design/storage.md +39 -2
- package/bundled-docs/design/telegram.md +20 -8
- package/bundled-docs/design/web.md +1 -0
- package/bundled-docs/design/wechat.md +13 -17
- package/bundled-docs/design/ws.md +56 -13
- package/bundled-docs/dev/add-channel.md +49 -37
- package/bundled-docs/dev/api.md +158 -38
- package/bundled-docs/dev/deploy.md +48 -0
- package/bundled-docs/dev/desktop-packaging.md +41 -7
- package/bundled-docs/dev/env.md +3 -0
- package/bundled-docs/dev/previews.md +58 -3
- package/bundled-docs/dev/tools.md +13 -4
- package/bundled-docs/guide/channels/wechat.md +3 -3
- package/bundled-docs/guide/chat.md +1 -1
- package/bundled-docs/guide/cli.md +24 -5
- package/bundled-docs/guide/workspace.md +2 -2
- package/bundled-docs/requirements/chat.md +11 -2
- package/bundled-docs/requirements/editor.md +5 -2
- package/bundled-docs/requirements/explorer.md +1 -0
- package/bundled-docs/requirements/session.md +4 -3
- package/bundled-docs/requirements/settings.md +8 -0
- package/bundled-docs/requirements/source-control.md +1 -0
- package/dist/index.js +14092 -12842
- package/package.json +4 -1
- package/templates/agents/__apply_agent__/agent.yaml +1 -1
- package/templates/agents/__evo_agent__/agent.yaml +1 -1
- package/templates/agents/__score__/agent.yaml +1 -1
- package/templates/agents/deep-executor/agent.yaml +1 -1
- package/templates/agents/default/agent.yaml +1 -1
- package/templates/agents/goal/agent.yaml +1 -1
- package/templates/schema.sql +8 -10
- package/templates/skills/agent/SKILL.md +2 -2
- package/admin-out/_next/static/chunks/212-e217e00ce6cc2bdc.js +0 -1
- package/admin-out/_next/static/chunks/220.3dc41cd6269b88e6.js +0 -1
- package/admin-out/_next/static/chunks/397-fc5a406d80d46ca7.js +0 -1
- package/admin-out/_next/static/chunks/442.f4302087bddb1ec1.js +0 -1
- package/admin-out/_next/static/chunks/app/page-f8208e0c7f315faa.js +0 -1
- package/admin-out/_next/static/chunks/webpack-47520cdc55be63e7.js +0 -1
- package/admin-out/_next/static/css/80eca65cb72e5c69.css +0 -3
- /package/admin-out/_next/static/{WvYu_osMZVpohYN4ntkpS → lOrGs0dR-jmXz9PgabRBc}/_buildManifest.js +0 -0
- /package/admin-out/_next/static/{WvYu_osMZVpohYN4ntkpS → lOrGs0dR-jmXz9PgabRBc}/_ssgManifest.js +0 -0
|
@@ -29,14 +29,16 @@ SessionManager.emitEvent(sessionId, event)
|
|
|
29
29
|
| State | Listener | Events go to |
|
|
30
30
|
|---|---|---|
|
|
31
31
|
| **Connected** | `registerEventListener(rootId, handler)` | `sendWsNotification(event, state, turnId, ctx)` → WS JSON |
|
|
32
|
-
| **Background** (after `/session new`) |
|
|
33
|
-
| **Detached** (WS disconnect) |
|
|
32
|
+
| **Background** (after `/session new`) | **none** — the listener is released | Nowhere. SessionUIStore keeps folding + persisting on its own |
|
|
33
|
+
| **Detached** (WS disconnect) | Inline `bufferDetachedNotification` closure | `pendingEvents[]` on `DetachedSession` |
|
|
34
34
|
|
|
35
35
|
State is NOT duplicated — all handlers read from `SessionManager.getUIState(rootId)`.
|
|
36
36
|
|
|
37
|
+
**Background has no listener by design.** A cleared session is deliberately abandoned: the admin wipes its chat store on `session:cleared`, so nobody on that connection would ever consume a buffered notification. Unlike the detach path — which buffers precisely because a reconnect within the grace window expects stream continuity — there is no reattach here; a later re-open subscribes fresh and gets the full snapshot from `SessionUIStore` / disk. The buffering handler this used to register (its `unsubscribe` discarded, its `pendingEvents` never drained) leaked one listener per "New session" click.
|
|
38
|
+
|
|
37
39
|
## `/session new` (session:clear) flow
|
|
38
40
|
|
|
39
|
-
Source: `packages/server/src/ws/handler.ts` (`
|
|
41
|
+
Source: `packages/server/src/ws/handler.ts` (`handleSessionClear`)
|
|
40
42
|
|
|
41
43
|
```
|
|
42
44
|
User clicks /session new
|
|
@@ -44,45 +46,29 @@ User clicks /session new
|
|
|
44
46
|
▼
|
|
45
47
|
session:clear handler
|
|
46
48
|
│
|
|
47
|
-
├─
|
|
48
|
-
│
|
|
49
|
-
├─ create inline bgHandler:
|
|
50
|
-
│ (event, state, turnId) => bufferDetachedNotification(event, pendingEvents)
|
|
49
|
+
├─ saveSession(client) ← persist current UIState to file
|
|
51
50
|
│
|
|
52
|
-
├─
|
|
53
|
-
│
|
|
51
|
+
├─ client.unsubscribeEvents?.() ← release the live WS listener
|
|
52
|
+
│ client.unsubscribeEvents = null and register NOTHING in its place
|
|
54
53
|
│
|
|
55
|
-
├─ backgroundSaves.set(prevSessionId, () =>
|
|
54
|
+
├─ backgroundSaves.set(prevSessionId, () => saveSession(client))
|
|
56
55
|
│ ← register a save fn for when user switches back or disconnects
|
|
57
56
|
│
|
|
58
|
-
├─
|
|
59
|
-
└─ send { type: 'session:cleared' }
|
|
57
|
+
├─ client.sessionId = null
|
|
58
|
+
└─ send { type: 'session:cleared' } ← admin bumps its session-list bus on this
|
|
60
59
|
```
|
|
61
60
|
|
|
62
|
-
###
|
|
63
|
-
|
|
64
|
-
Location: `packages/server/src/ws/background-handler.ts`
|
|
65
|
-
|
|
66
|
-
A helper used in contexts outside session:clear (e.g. detached sessions). Takes 3 args:
|
|
67
|
-
|
|
68
|
-
```typescript
|
|
69
|
-
createBackgroundHandler(sessionId: string, projectPath: string | null, sm: SessionManager)
|
|
70
|
-
→ { handler, save, pendingEvents }
|
|
71
|
-
```
|
|
61
|
+
### Where the old session's state keeps coming from
|
|
72
62
|
|
|
73
|
-
|
|
74
|
-
- **save**: reads live UIState from `sm.getUIState(sessionId)`, creates a snapshot, writes to file
|
|
75
|
-
- **pendingEvents**: array of buffered structural events for replay on reconnect
|
|
76
|
-
|
|
77
|
-
The handler does NOT hold its own messageLog/streamBuffer/tokenState. All state lives in SessionManager's UIState, maintained by `reduceIntoUIState → applyEvent`.
|
|
63
|
+
Nothing is lost without a listener: `SessionUIStore.emitEvent` folds every event into the root's `UIState` (`reduceIntoUIState → applyEvent`) and persists it — debounced 500 ms for tool traffic, flushed synchronously on `complete` — **before** it looks up listeners. Listeners are a pure fan-out for live UI; the persistence path is independent of them.
|
|
78
64
|
|
|
79
65
|
### When save fires
|
|
80
66
|
|
|
81
67
|
Background state persists in three scenarios:
|
|
82
68
|
|
|
83
69
|
1. **`backgroundSaves.get(sessionId)?.()` in subscribe** — user switches back to the old session; called before re-attaching the listener
|
|
84
|
-
2. **`backgroundSaves` flush on disconnect** — WS closes, flush every pending bg
|
|
85
|
-
3. **
|
|
70
|
+
2. **`backgroundSaves` flush on disconnect** — WS closes (or errors), flush every pending bg
|
|
71
|
+
3. **SessionUIStore's own persist** — `complete` flushes immediately, other structural events go through the 500 ms debounce
|
|
86
72
|
|
|
87
73
|
## Subscribe (switching back) flow
|
|
88
74
|
|
|
@@ -101,7 +87,6 @@ subscribe handler
|
|
|
101
87
|
│ backgroundSaves.delete(sessionId)
|
|
102
88
|
│
|
|
103
89
|
├─ re-attach the event listener to this session tree
|
|
104
|
-
│ (replaces the bg buffer handler with the live WS handler)
|
|
105
90
|
│
|
|
106
91
|
├─ Load UIState from SessionManager (or from file if not in memory)
|
|
107
92
|
│
|
|
@@ -110,25 +95,27 @@ subscribe handler
|
|
|
110
95
|
|
|
111
96
|
## When disconnect happens while background is still running
|
|
112
97
|
|
|
98
|
+
Both `close` and `error` run the same `cleanupConnection()` (`clients.delete` is the idempotency gate — see [ws.md](ws.md#reconnect-flow)):
|
|
99
|
+
|
|
113
100
|
```
|
|
114
|
-
|
|
101
|
+
cleanupConnection
|
|
115
102
|
│
|
|
103
|
+
├─ clearInterval(keepalive) + terminalManager.detachAll()
|
|
116
104
|
├─ (if agent running) → detach session with bufferDetachedNotification
|
|
117
105
|
├─ (otherwise) → saveSession
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
106
|
+
├─ flush every backgroundSaves
|
|
107
|
+
│ for (const [sid, saveFn] of backgroundSaves)
|
|
108
|
+
│ saveFn()
|
|
109
|
+
└─ stop fileWatcher + gitDirWatcher
|
|
122
110
|
```
|
|
123
111
|
|
|
124
112
|
## Relevant files
|
|
125
113
|
|
|
126
114
|
| File | Relevant code |
|
|
127
115
|
|---|---|
|
|
128
|
-
| `packages/server/src/ws/
|
|
129
|
-
| `packages/server/src/ws/handler.ts` | `session:clear` case — inline bgHandler + `backgroundSaves` registration |
|
|
116
|
+
| `packages/server/src/ws/handler.ts` | `handleSessionClear()` — listener release + `backgroundSaves` registration; `cleanupConnection()` — the shared close/error teardown |
|
|
130
117
|
| `packages/server/src/ws/event-processor.ts` | `sendWsNotification()`, `bufferDetachedNotification()` |
|
|
131
|
-
| `packages/server/src/agents/session-
|
|
118
|
+
| `packages/server/src/agents/session-ui-store.ts` | `emitEvent()`, `reduceIntoUIState()`, `flushPersist()` / `debouncedPersist()`, `registerEventListener()` |
|
|
132
119
|
| `packages/server/src/sessions/ui-log-builder.ts` | `applyEvent()`, `createSaveSnapshot()`, UIState type |
|
|
133
120
|
| `packages/server/src/sessions/session-store.ts` | `saveSessionToFile()`, `loadSessionMessages()` |
|
|
134
121
|
|
|
@@ -142,10 +129,10 @@ disconnect handler
|
|
|
142
129
|
**Root cause**: before #1 was fixed, sub-agent events went through the live WS handler, pushing messages into the new session's state.
|
|
143
130
|
**Fix**: (a) fixing #1 made event routing tree-scoped. (b) session:clear explicitly saves before switching and resets client state.
|
|
144
131
|
|
|
145
|
-
### 3.
|
|
146
|
-
**Root cause**: the function defined `handler` and `save` locally but did not `return { handler, save }`.
|
|
147
|
-
**Fix**: add `return { handler, save, pendingEvents }` at the end.
|
|
148
|
-
|
|
149
|
-
### 4. Stream buffer not flushed before saveSession in session:clear
|
|
132
|
+
### 3. Stream buffer not flushed before saveSession in session:clear
|
|
150
133
|
**Root cause**: in-flight stream text wasn't captured before save.
|
|
151
134
|
**Fix**: UIState reducer now incrementally persists on every structural event — stream text is folded into messageLog by `applyEvent` before save triggers.
|
|
135
|
+
|
|
136
|
+
### 4. One leaked listener per "New session" click (2026-08-07)
|
|
137
|
+
**Root cause**: `session:clear` registered a buffering background handler whose `unsubscribe` was discarded and whose `pendingEvents` nobody ever drained, so every click added a permanent listener (and an ever-growing array) to the abandoned session tree.
|
|
138
|
+
**Fix**: release the listener and register nothing — see [Background has no listener by design](#three-event-handler-states). `ws/background-handler.ts` (the `createBackgroundHandler` utility, by then only used by this path) was deleted with it.
|
|
@@ -14,7 +14,7 @@ cron_jobs (global db) registered at boot:
|
|
|
14
14
|
▼ │ │ registerCronDispatcher
|
|
15
15
|
spawn `halo cli -n -s cron-<id>` │ dispatcher.ts (registry)
|
|
16
16
|
│ stdout │ ▲
|
|
17
|
-
▼ └──────────────┘ dispatchToTargets(
|
|
17
|
+
▼ └──────────────┘ dispatchToTargets(rawText, targets, workspacePath)
|
|
18
18
|
cron_runs (audit row + log file) ───────────────┘
|
|
19
19
|
```
|
|
20
20
|
|
|
@@ -29,11 +29,20 @@ Schedules are **durable** — the source of truth is the `cron_jobs` table; in-m
|
|
|
29
29
|
- **Recurring**: standard cron expression (`job.schedule`), optional `timezone`.
|
|
30
30
|
- **One-shot**: `job.runAt` (epoch ms) — croner fires once at that instant. After it completes, `finalize` sets `enabled=0` so it never re-fires. A `runAt` already in the past at schedule time is marked `lastRunStatus='missed'` and disabled (rather than firing immediately or retrying every reconcile).
|
|
31
31
|
|
|
32
|
+
### Trigger-mode exclusivity (the db contract)
|
|
33
|
+
|
|
34
|
+
Exactly **one** of `schedule` / `runAt` is set per job. `runner.ts` `scheduleJob` silently prefers `runAt` when both are present, and at-mode's post-fire auto-disable (`enabled=0`) would then kill the recurring schedule too — so the invariant is enforced at every write point, not just at create. `PUT /api/cron/jobs/:id` validates the **merged** row (partial PUT bodies make this non-obvious):
|
|
35
|
+
|
|
36
|
+
- Both sides in one body → `400 schedule and runAt are mutually exclusive`.
|
|
37
|
+
- One side only → treated as a **mode switch**: the other side is implicitly cleared (`schedule` set ⇒ `runAt=null`; `runAt` set ⇒ `schedule=''`).
|
|
38
|
+
- Clearing the active side with nothing to replace it (`schedule: ''` on a recurring job, `runAt: null` on an at-mode job) → `400 schedule or runAt required`. A triggerless job would sit `enabled` and never fire.
|
|
39
|
+
- A non-number, non-null `runAt` is rejected (`400`) *before* the exclusivity check — otherwise garbage types slip past `typeof body.runAt === 'number'` and land in the db as-is.
|
|
40
|
+
|
|
32
41
|
### Hot-reload + out-of-band edits
|
|
33
42
|
|
|
34
43
|
REST route mutations call `scheduleJob` / `unscheduleJob` directly. But the `cron` skill (and manual ops) edit `cron.db` over a *different* sqlite connection. `reconcileFromDb` (10s timer) catches those:
|
|
35
44
|
|
|
36
|
-
- Fast path: `PRAGMA data_version` flips only when *another* connection commits. If unchanged since last pass, skip the full select entirely — one pragma read (~µs) vs. a select-all + per-row fingerprint compare.
|
|
45
|
+
- Fast path: `PRAGMA data_version` flips only when *another* connection commits. If unchanged since last pass, skip the full select entirely — one pragma read (~µs) vs. a select-all + per-row fingerprint compare. The pragma (and the run-pruning `NOT IN (subquery)` below) needs the raw better-sqlite3 handle, which the runner gets from the shared `rawSqlite(db)` helper (`db/raw-sqlite.ts`) instead of re-spelling drizzle's `$client` / `.session.client` cast at each site — it did, three times, each with its own hand-written method shape.
|
|
37
46
|
- On change: diff db rows against the in-memory `_fingerprint` map (`enabled|schedule|runAt|timezone`); schedule new rows, unschedule deleted ones, re-instantiate croner only for rows whose fingerprint changed. Broadcasts a coalesced `cron:job_changed` per affected job.
|
|
38
47
|
|
|
39
48
|
## Execution (`runJob`)
|
|
@@ -47,7 +56,7 @@ Each fire:
|
|
|
47
56
|
5. Tee stdout + stderr live to `~/.halo/global/logs/cron/<runId>.log`; capture stdout to memory for dispatch.
|
|
48
57
|
6. Enforce a **3600s timeout** via a Node timer (cross-platform, not the Linux-only `timeout(1)`), with a **two-phase kill**: SIGTERM the direct child first — the cli has a SIGTERM handler that stops the harness, repairs + flushes the session, kills its own shell_exec process groups, and exits 143 — then, if it's still alive after a 30s grace, SIGKILL the entire descendant tree (POSIX ppid-walk over one `ps` snapshot; `taskkill /T /F` on Windows) so no grandchild survives to double-write the workspace on the next fire. A ppid walk, not a process-group kill: the cli sandbox spawns shell_exec workers `detached` into their own groups, so `kill(-pid)` would miss them. Timed-out runs still report exit code 124. Generous because overlap-of-the-same-job is already blocked up-front in step 1; this is just the long-stop reaper for a truly stuck child.
|
|
49
58
|
7. Classify: exit 124 → `timeout`; non-zero → `failed`; zero but empty stdout → `failed` ("nothing to dispatch"); else `succeeded`.
|
|
50
|
-
8. On success only, `dispatchToTargets(stdout, targets)
|
|
59
|
+
8. On success only, `dispatchToTargets(stdout, targets, job.workspacePath)` — the raw stdout goes down as-is; `MEDIA:` extraction happens per-target inside the dispatcher (see Dispatch model). If every target fails, downgrade to `failed`.
|
|
51
60
|
9. `finalize`: write the terminal `cron_runs` row, update `cron_jobs.lastRun*`, disable one-shot jobs, broadcast. The `_inflight` entry is released in `finally`, regardless of outcome.
|
|
52
61
|
|
|
53
62
|
The cli executable is resolved via `resolveHaloCli()` (`$HALO_CLI` override; `halo.cmd` on Windows to dodge the GUI `Halo.exe` on PATH).
|
|
@@ -72,12 +81,18 @@ Only the kill grace is asynchronous; the boot-blocking part is one small select
|
|
|
72
81
|
```ts
|
|
73
82
|
interface CronChannelDispatcher {
|
|
74
83
|
channelType: string
|
|
75
|
-
dispatch
|
|
84
|
+
supportsMedia?: boolean // true = dispatch actually delivers `media` attachments
|
|
85
|
+
dispatch: (accountId, text, chatId?, media?: CronMedia) => Promise<DispatchResult[]>
|
|
76
86
|
listTargets?: () => CronTargetOption[]
|
|
77
87
|
}
|
|
78
88
|
```
|
|
79
89
|
|
|
80
90
|
- `dispatchToTargets` is **never throws** — every target's outcome (ok / error) is captured into a `DispatchResult[]` persisted to `cron_runs.dispatch_results`. One channel being down doesn't block the others.
|
|
91
|
+
- **`MEDIA:` attachments**: the runner passes raw stdout down untouched; `dispatchToTargets(rawText, targets, workspacePath)` extracts the agent's `MEDIA:<path>` marker lines once, then decides **per target** (one run can mix both kinds of channel):
|
|
92
|
+
- `supportsMedia: true` (wechat, slack) → marker-stripped text + a `CronMedia { paths, workspacePath }`; the channel sends real file uploads, one `DispatchResult` row per attachment. Both skip the text send when a marker-only run leaves it empty.
|
|
93
|
+
- no `supportsMedia` (telegram, feishu) → the **original text with `MEDIA:` lines intact** — the attachment degrades to a visible path instead of silently vanishing.
|
|
94
|
+
- `CronMedia.workspacePath` is the **job's** workspace (where the run produced its files), not the target account's — it's the sandbox root for `isMediaPathAllowed` (paths outside workspace/OS-tmp are rejected with a failed result row).
|
|
95
|
+
- Wiring telegram/feishu up for real requires refactoring first, not copying: telegram's file send is an inline `sendPhoto/…/sendDocument` switch in `handler.ts`'s responder; feishu's `sendFeishuMedia` is anchored to an inbound `messageId` a cron run doesn't have. Extract the send-to-chat half in the channel module, then flip `supportsMedia: true` (details in each `cron-dispatcher.ts` header).
|
|
81
96
|
- **`chatId` semantics**: when a job is created from inside a chat, the target pins to that conversation (`chatId` set). When unset, the channel's own fan-out / default-recipient logic decides (e.g. Telegram fans out to its whitelist; Slack/Feishu require an explicit `chatId`).
|
|
82
97
|
- `listTargets()` aggregates across all dispatchers (`listAllCronTargets`) to feed the admin create-form dropdown without the routes knowing any channel.
|
|
83
98
|
|
|
@@ -223,6 +223,20 @@ Per-workspace mutex in ticker prevents two applies from racing on the same works
|
|
|
223
223
|
apply-<id>.zip
|
|
224
224
|
```
|
|
225
225
|
|
|
226
|
+
**Every path above is built by a `paths.ts` builder — no inline joins.** The
|
|
227
|
+
layout used to be assembled with `path.join(ws, '.halo', 'evo', …)` at ~40 sites
|
|
228
|
+
across archive / enqueue / evo-wrapper / ticker / routes, which is how the same
|
|
229
|
+
directory got spelled two ways. Workspace-rooted builders (`wsEvoDir`,
|
|
230
|
+
`wsEvoRunDir`, `wsEvoApplyDir`, `wsEvoArchiveDir`, `wsEvoHistoryDir`, …) were
|
|
231
|
+
already there; the refactor added four **artifact-dir-relative** ones —
|
|
232
|
+
`evoSandboxDir(artifactDir)`, `evoSandboxHaloDir(artifactDir)`,
|
|
233
|
+
`evoRegressDir(applyDir)`, `evoRegressRunDir(applyDir, runId)`. They take the
|
|
234
|
+
run-*or*-apply dir rather than `(workspace, id)` because run mode and apply mode
|
|
235
|
+
lay their sandbox out identically inside their own artifact dir (`buildEvoSandbox`
|
|
236
|
+
is deliberately called with either). A dedicated equivalence test
|
|
237
|
+
(`test/paths-evo-equivalence.test.ts`) pins the builders against the old
|
|
238
|
+
literals.
|
|
239
|
+
|
|
226
240
|
`evo-context.json` format:
|
|
227
241
|
|
|
228
242
|
```json
|
|
@@ -254,6 +268,8 @@ Active rows (pending/running/awaiting_review/approved/syncing) never archived. `
|
|
|
254
268
|
|
|
255
269
|
Archive daemon runs at server boot (1 min delay) + daily. Idempotent.
|
|
256
270
|
|
|
271
|
+
Zipping is **pure JS** (`jszip`, already a repo dependency via admin). It used to `spawnSync('zip', …)`, which depends on a system binary Windows installs don't have — so every archive pass failed there forever and artifacts grew unbounded. `zipDir()` walks the tree manually rather than using a recursive readdir, because the layout has to match what `zip -rq <out> .` produced and two cases need explicit handling: empty dirs get explicit folder entries, and **directory symlinks are not recursed** (an agent runs inside the sandbox during evaluation and could leave a cycle). File symlinks store the target's contents (what system `zip` without `-y` did); broken links are skipped instead of failing the pass. Contents are buffered per file (`createReadStream` for every entry would open all fds up front — EMFILE risk) and the zip is streamed to disk. On failure the partial zip is deleted, and `archiveRun`/`archiveApply` skip both the `rmDir` and the `archived_at` stamp, so the next pass retries from a clean state.
|
|
272
|
+
|
|
257
273
|
## Integration with SessionManager
|
|
258
274
|
|
|
259
275
|
Each evo/score/apply agent is flagged `internal: true` in `agent.yaml`. SessionManager skips loading workspace platform prompts (USER.md, INSTRUCTIONS.md, INDEX.md, prompts/all|root|bootstrap) for internal agents — only their own AGENT.md + wrapper brief reaches the LLM. Reduces token cost and removes workspace-rule bleed-through.
|
|
@@ -268,7 +284,7 @@ After apply publishes to main, **no explicit session-release step needed**. Sess
|
|
|
268
284
|
| `/packages/server/src/evolution/ticker.ts` | 496 | Stateless 30s scheduler. Mark timeouts, claim pending, spawn wrappers. Per-workspace apply mutex. Broadcast status changes to admin UI. |
|
|
269
285
|
| `/packages/server/src/evolution/spawn.ts` | 36 | Real spawner: detached Node child running evo-wrapper.js. Override-able for testing. |
|
|
270
286
|
| `/packages/server/src/evolution/evo-wrapper.ts` | 2000+ | Wrapper orchestrator. 3-phase run mode (draft/dry-run/score). Apply mode (merge/regress/publish). Heartbeat every 60s. Handles Windows command-line limits via stdin briefs. |
|
|
271
|
-
| `/packages/server/src/evolution/archive.ts` |
|
|
287
|
+
| `/packages/server/src/evolution/archive.ts` | 313 | Archive job: 14d → zip (pure-JS jszip) + delete, 30d → purge. Runs daily + at boot. Idempotent. |
|
|
272
288
|
|
|
273
289
|
## Sandbox Model
|
|
274
290
|
|
|
@@ -306,7 +322,7 @@ Top-level "Evolution" tab shows:
|
|
|
306
322
|
|
|
307
323
|
- **List**: evolution_runs + latest apply per run. Sortable by created_at / status / score.avg. Filterable by status. Joins runs+applies to show consolidated view.
|
|
308
324
|
- **Detail**: patch.md, score.json, test scenario, assembled brief context, diff against current target.
|
|
309
|
-
- **Approve**: Opens dialog, optional reviewer_hint.
|
|
325
|
+
- **Approve**: Opens dialog, optional reviewer_hint. Flips the run to `approved` **and** inserts the pending `evolution_applies` row **inside one `db.transaction`** — it's a single reviewer decision, and half of it (run flipped, apply missing) strands the run in a status the ticker never advances.
|
|
310
326
|
- **Reject**: Sets status='rejected'.
|
|
311
327
|
- **Retry**: Resets row to pending, requires new hint. Allowed from any terminal status except `applied`.
|
|
312
328
|
- **Manual Delete**: Removes artifacts (live dir + archive zip, whichever exists) + DB row immediately. Blocked on active rows (pending/running/approved/syncing).
|
|
@@ -68,7 +68,7 @@ Files: `packages/server/src/channels/feishu/`
|
|
|
68
68
|
|
|
69
69
|
- `types.ts` — FeishuAccount, FeishuEventEnvelope, FeishuMessageEvent interfaces
|
|
70
70
|
- `accounts.ts` — DAL (listAccounts / getAccount / insertAccount / updateAccount / deleteAccount / findAccountByAppId)
|
|
71
|
-
- `handler.ts` — Lark.WSClient setup, event dispatch, message handling,
|
|
71
|
+
- `handler.ts` — Lark.WSClient setup, event dispatch, message handling, mention detection, media ingestion; session routing + listener/route bookkeeping come from `channels/shared/inbound.ts` (`InboundBridge` / `deliverInbound` / `dispatchChannelCommand`)
|
|
72
72
|
- `event-adapter.ts` — AgentSessionEvent stream → coalesced Feishu message replies (buffer + flush at paragraph boundaries)
|
|
73
73
|
- `api.ts` — HTTP client (tenant token caching / getBotInfo / sendMessage / replyMessage / uploadImage / uploadFile / downloadResource / decryptWebhookBody / searchFeishuTargets / openLongConnection)
|
|
74
74
|
- `cron-dispatcher.ts` — registers the cron dispatcher, requires explicit chatId targets
|
|
@@ -114,11 +114,11 @@ The SDK wraps protobuf marshalling; we just register an `EventDispatcher` callba
|
|
|
114
114
|
6. Download inbound images, save to workspace, generate base64 payloads
|
|
115
115
|
7. Strip mention markup from text (Feishu inserts `<at user_id="ou_xxx">@bot</at>` or `@_user_123` tokens)
|
|
116
116
|
8. Slash command dispatch (p2p only — group threads don't benefit from `/session new`, `/session list`, etc. since each thread is already its own session)
|
|
117
|
-
9.
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
117
|
+
9. Hand off to `deliverInbound` ([shared skeleton](telegram.md#shared-inbound-skeleton)), which does the rest:
|
|
118
|
+
- Create or retrieve the active session for this thread (with inherited access level) + apply the goal-mode route overlay
|
|
119
|
+
- If the session is compacting → reply "⏳ 正在整理上下文,请稍后再发消息(通常 30 秒内完成)"; if busy → "已收到,会在当前轮结束后处理。" — hint only, the message is delivered either way
|
|
120
|
+
- Refresh the reply route (`{inboundMessageId, isP2P, chatId}`) and attach the `FeishuResponder` listener once (it reads the route lazily at send time)
|
|
121
|
+
- `sm.sendUserMessage(sessionId, agentInput, images?)` with `[channel: feishu | user: <id> | thread: <rootId>]` context
|
|
122
122
|
|
|
123
123
|
## Slash commands (native Feishu /commands)
|
|
124
124
|
|
|
@@ -147,6 +147,19 @@ Feishu's text message limit (~5000 chars), smaller than Slack. Strategy:
|
|
|
147
147
|
- Media markers (`MEDIA:<path>`) intercepted and sent via native media upload
|
|
148
148
|
- Sub-agent events dropped (visible in web UI only)
|
|
149
149
|
|
|
150
|
+
**Chunk sends are serialized per responder** (same fix as Slack, audit A-L3):
|
|
151
|
+
splitting produces several chunks in one synchronous loop and each is its own
|
|
152
|
+
send API call, so firing them concurrently gave arrival order no guarantee. Each
|
|
153
|
+
responder appends every chunk to a `sendTail` promise chain — chunk *n+1* waits
|
|
154
|
+
for *n* to settle — and a rejected link is absorbed so one failed send doesn't
|
|
155
|
+
stall the remainder (`dispatchChunk` logs it).
|
|
156
|
+
|
|
157
|
+
`close()` returns that drain promise and `InboundBridge` holds the session's
|
|
158
|
+
reply route until it settles, then deletes it; the `stopAccount` / `closeAll()`
|
|
159
|
+
path relies on the same per-listener teardown rather than clearing all routes at
|
|
160
|
+
once. Before this, the route was gone as soon as the agent turn returned and
|
|
161
|
+
trailing chunks of a long reply were dropped.
|
|
162
|
+
|
|
150
163
|
## Media support
|
|
151
164
|
|
|
152
165
|
**Inbound:** text, images (base64 attached to agent message), files (noted as `[文件: name]` markers), voice, video, rich posts.
|
|
@@ -79,7 +79,7 @@ Gates, in order:
|
|
|
79
79
|
| Max rounds | 10 | `round >= caps.maxRounds` |
|
|
80
80
|
| Wall time | 4h | `now − startedAt > caps.maxWallMs` |
|
|
81
81
|
| No-progress breaker | 3 | 3 consecutive byte-identical reports |
|
|
82
|
-
| Token budget | off unless set at attach | W's `totalOutputTokens − tokenBaseline > caps.maxTokens` (G's own consumption is not metered) |
|
|
82
|
+
| Token budget | off unless set at attach | W's `totalOutputTokens − tokenBaseline > caps.maxTokens` (G's own consumption is not metered) — read from W's `agent_sessions.total_output_tokens` column, selected in the same query as `goal_session_id`, so the delivery point that runs at **every** root turn end never opens a session file |
|
|
83
83
|
|
|
84
84
|
Breach → `halted` + `haltReason`, back-pointer cleared (**revoking the lateral edge** — G's further `query_session` calls are rejected in code), and the report is delivered under a `[Goal HALTED: …]` header instructing G to produce a halt diagnosis instead of more work.
|
|
85
85
|
|
|
@@ -92,7 +92,7 @@ A fifth cap is enforced inside the tools rather than the delivery point: **deleg
|
|
|
92
92
|
Injected only for the `goal` agent; schemas in [dev/tools.md → Goal tools](../dev/tools.md#goal-tools). Every callback **re-reads goal state from the db** — never a cached copy — so a halt / pause / clear that landed while G was mid-turn is enforced on its very next tool call.
|
|
93
93
|
|
|
94
94
|
- `goal_context` — read the binding + counters. During `intake` it also embeds `workerRecent`: the worker's last 20 non-empty user/assistant messages (transcript `role=system` noise skipped), each truncated to 400 chars, 8K chars total budget applied newest-first, plus `workerMessageCount`. This is how G seeds the intake conversation without the user re-explaining — and without parsing a 150 KB session JSON (the dogfood failure that motivated embedding it). Running goals don't embed it; G works off delivered round reports.
|
|
95
|
-
- `goal_attach` — the hinge from intake conversation to running loop, callable from any channel (no button). Preconditions: status `intake` + `GOAL_SPEC.md` written. Stamps the spec hash, records W's token baseline, applies cap overrides pinned during intake, flips to `running`, and dispatches the round-1 kickoff to W.
|
|
95
|
+
- `goal_attach` — the hinge from intake conversation to running loop, callable from any channel (no button). Preconditions: status `intake` + `GOAL_SPEC.md` written. Stamps the spec hash, records W's token baseline (from W's `agent_sessions.total_output_tokens`, the same mirrored column the guardrail reads — so baseline and meter can't drift apart), applies cap overrides pinned during intake, flips to `running`, and dispatches the round-1 kickoff to W.
|
|
96
96
|
- `goal_decide` — records a delegated decision as `decision-<n>.md` in the goal dir *before* relaying the answer; counts against the cap of 5.
|
|
97
97
|
- `goal_finish` — final acceptance: `running → done`, dissolves the binding; G then writes the final report as its reply (which must list every delegated decision).
|
|
98
98
|
- `query_session` (goal-scoped) — the lateral edge: only the bound worker is reachable; only while `running`; a `[Goal work order · round N/cap]` header is prepended in code.
|
|
@@ -117,6 +117,13 @@ js/
|
|
|
117
117
|
- `/api/show/state` — 无活跃 UIState 的会话(idle/stopped/重启后)token
|
|
118
118
|
从会话文件头读取(mtime 缓存,不随轮询刷盘),不显示 0;含 `messageCount` 字段
|
|
119
119
|
|
|
120
|
+
> **`messageCount` 与 admin 的 `exchangeCount` 是两个语义,不要互相对齐**:
|
|
121
|
+
> 这里读的是会话文件里的 `messageCount`(活跃 UI 日志的长度),城市只把它
|
|
122
|
+
> 当"这个市民最近有多热闹"的活跃度信号,compact 归档后会变小是可以接受的;
|
|
123
|
+
> admin 会话列表用的是 sqlite 镜像列 `exchange_count`(整个生命期的主用户
|
|
124
|
+
> 轮次数,归档后不缩)。城市这边继续走文件头是有意的:它必须能在**没有活跃
|
|
125
|
+
> SessionManager**的情况下降级出图,不依赖 db 镜像何时写入。
|
|
126
|
+
|
|
120
127
|
> **`observer` token 的权限面**:除了 `/show/state` 的聚合计数外,`observer`
|
|
121
128
|
> 还能调 `/show/session` 读取**任意 workspace** 任意会话的明细——最近 40 条
|
|
122
129
|
> 消息(每条截断到 600 字,tool input 截到 200 字)。它是"看板/监控"用的
|
|
@@ -29,18 +29,25 @@ These sessions also do **not** get an `agent_sessions` row in the workspace's `h
|
|
|
29
29
|
"contextTokens": 85000,
|
|
30
30
|
"totalOutputTokens": 12000,
|
|
31
31
|
"parentSessionId": null,
|
|
32
|
+
"archiveCount": 2,
|
|
33
|
+
"archivedUserCount": 30,
|
|
32
34
|
"messages": [...],
|
|
33
35
|
"rawMessages": [...],
|
|
34
36
|
"output": "..."
|
|
35
37
|
}
|
|
36
38
|
```
|
|
37
39
|
|
|
38
|
-
- **messages**: event log format (written by the WS handler) — context / usage / tool_call / tool_result / agent_start/done, with full debug info
|
|
40
|
+
- **messages**: event log format (written by the WS handler) — context / usage / tool_call / tool_result / agent_start/done, with full debug info. Assistant rows persist their tool calls **only** in `contentBlocks` (interleaved with text / thinking); the flat `toolCalls` array is no longer written — it duplicated every tool's input *and* output byte-for-byte (measured: 66 MiB of 340 MiB across 195 sessions). Readers are blocks-first with `toolCalls` as a pure legacy fallback (`messageToolCalls()` in `sessions/session-types.ts`; see [storage.md](storage.md#sessionmessage))
|
|
39
41
|
- **rawMessages**: Bedrock API shape (written by SessionManager `saveAgentState`) — raw user/assistant turns with toolUse/toolResult blocks
|
|
40
42
|
- **output**: accumulated assistant text
|
|
43
|
+
- **archiveCount / archivedUserCount**: UI-log archiving bookkeeping — absent until the first archive. See [UI-log archiving](#ui-log-archiving)
|
|
41
44
|
|
|
42
45
|
`saveSessionToFile()` uses read-merge-write so both halves survive. When loading, the event log `messages` takes priority; only when `messages` is empty (e.g. a sub-session tracked only by SessionManager) does `rawMessages` get converted to display format.
|
|
43
46
|
|
|
47
|
+
The file is written as **compact JSON** (no 2-space indent) — nothing reads it by eye and every reader is `JSON.parse`. All five writers to the path switch together (`saveSessionToFile`, the title rewrite, `saveAgentState`, `createSession`'s title pre-seed, the cold `deleteExchange` edit) so the shape doesn't flip depending on which write landed last.
|
|
48
|
+
|
|
49
|
+
**`messageCount` has one writer.** It is the length of the active file's `messages` (the UI log) and is filled only by `saveSessionToFile`. `saveAgentState` used to write `agent.messages.length` (raw LLM history — a different, larger number) into the same key, so the value flapped between two meanings depending on which write landed last; the raw half no longer touches it. Note it *shrinks* when a compact archives history — the list-visible lifetime count is `exchangeCount` (main user turns + `archivedUserCount`), mirrored into sqlite, not this field.
|
|
50
|
+
|
|
44
51
|
Full field list in [storage.md](storage.md).
|
|
45
52
|
|
|
46
53
|
### Exchange deletion (soft UI + hard raw)
|
|
@@ -54,6 +61,8 @@ Full field list in [storage.md](storage.md).
|
|
|
54
61
|
|
|
55
62
|
**Ordinal alignment (the sharp edge).** `userOrdinal` is the 0-based index of the target user turn, counted **excluding `taskId` (sub-agent) messages** — matching the admin's `isMainConversationMessage` filter, which is what the frontend counts on. Both the ordinal-locate loop and the duplicate-rank loop skip `taskId` messages; if they didn't, a sub-agent's injected user turn in the root log would drift the count and delete the wrong exchange.
|
|
56
63
|
|
|
64
|
+
**Archived logs are refused (`'archived'`).** `userOrdinal` is positional over the log both sides can see, so it only means anything while client and server agree where the log *starts*. Archiving moves that start forward, and a chat panel open across the compact still counts from the pre-archive top — the same ordinal then maps onto a different turn on the server. The payload carries no id or anchor to disambiguate, so `deleteExchange` refuses instead of guessing: `readArchiveCount() > 0` → `'archived'`. It is read **from the file header, not memory** — the commit marker on disk is the truth about what was archived. The WS layer surfaces it as an error frame with `code: 'archived'` (see [ws.md](ws.md)); the admin renders the message verbatim, without the `Error:` prefix it adds to unexpected failures. The real fix belongs in the ordinal protocol (an anchor rather than a bare index), not here.
|
|
65
|
+
|
|
57
66
|
**Memory/disk sync.** A live session mutates `agent.messages` in place then `saveAgentState`; a cold session is edited directly on the `.json` file (read-merge-write). Rejected with `running` / `compacting` while a turn is in flight (mutating raw mid-turn would corrupt the in-flight conversation). Refresh is push-based: the active session gets a `state:snapshot`, other open sessions pick it up via the existing `.halo/sessions/` file watcher — no new WS message. Entry point: WS `exchange:delete` → `handler.ts:handleExchangeDelete` (see [ws.md](ws.md)).
|
|
58
67
|
|
|
59
68
|
## SessionManager
|
|
@@ -218,9 +227,18 @@ Table `agent_sessions` holds metadata only (no runtime state):
|
|
|
218
227
|
| created_at / updated_at | INTEGER | Epoch ms |
|
|
219
228
|
| stopped_at | INTEGER | null = active |
|
|
220
229
|
| archived_at | INTEGER | null = not archived |
|
|
230
|
+
| goal / goal_session_id | TEXT | Goal mode: binding JSON on G's row, back-pointer on W's — see [goal-mode.md](goal-mode.md) |
|
|
231
|
+
| title / exchange_count / context_tokens / total_output_tokens | TEXT / INTEGER | List-visible metadata mirrored from the session file header; null = row predates the columns |
|
|
221
232
|
|
|
222
233
|
Status is derived from memory (`promise !== null`) — not stored.
|
|
223
234
|
|
|
235
|
+
**The four mirrored columns** exist so the session list doesn't have to open (and `JSON.parse`) every session file just to render a row — at a few hundred sessions of ~1 MiB each that was the dominant cost of `GET /api/sessions/logs`. Invariants:
|
|
236
|
+
|
|
237
|
+
- **One write seam.** `saveSessionToFile()` returns the header it just wrote (`SessionFileMeta`); `SessionManager.persistSessionFile` — the single funnel every UI-log persist goes through — hands it to `mirrorSessionMeta()`. Nothing else writes these columns except the PATCH rename path, which sets `title` alongside the file rewrite.
|
|
238
|
+
- **Idempotent, and never touches `updated_at`.** `mirrorSessionMeta` selects first and skips the UPDATE when all four values already match; a missing row is a no-op return (sub-sessions of a deleted tree). It deliberately leaves `updated_at` alone — that column drives list ordering and "last activity", and a metadata mirror is not activity.
|
|
239
|
+
- **`exchange_count` is a lifetime count**, `countMainUserMessages(messages) + archivedUserCount`, so it doesn't shrink when a compact archives history (unlike the file's `messageCount`).
|
|
240
|
+
- **Lazy backfill.** The list route treats `exchange_count === null` as "pre-migration row", reads the file once via `readSessionFileMeta`, mirrors it, and never pays that cost again.
|
|
241
|
+
|
|
224
242
|
## Conversation repair
|
|
225
243
|
|
|
226
244
|
File: `packages/server/src/agents/conversation-repair.ts`
|
|
@@ -335,12 +353,44 @@ Self-compact **deep-snapshots the keep-region before running the summarize turn*
|
|
|
335
353
|
|
|
336
354
|
The pre-run snapshot is load-bearing, not an optimization: `agent.run()` coalesces a new user turn *into* the trailing user message when one already exists (a mid-turn `tool_result`, or pending user input) rather than appending a separate message — so the throwaway "Summarize the conversation…" instruction can land *inside* the last kept message. Rebuilding from the post-run array (the old `slice(cut, preRunLen)`) therefore left the instruction stuck in the kept tail, and the model answered it as a real reply on the next turn (an unprompted "conversation summary"). Snapshotting the keep-region *before* the run sidesteps where the instruction lands entirely.
|
|
337
355
|
|
|
338
|
-
As a final byte-trimming step, self-compact runs **micro-compact** over that snapshot (`microCompactMessages(cleanRecent, 1)`). Each kept-recent message may still carry a large tool result (e.g. a 50 KB `file_read`), so after summarizing it would otherwise re-cross the threshold immediately. Micro keeps only the newest `tool_result`'s content and clears the rest in place, preserving `tool_use_id` pairing so the next API call stays valid — no extra LLM round-trip. Micro-compact is **not** a standalone compaction path; `selfCompactSession` is its only call site
|
|
356
|
+
As a final byte-trimming step, self-compact runs **micro-compact** over that snapshot (`microCompactMessages(cleanRecent, 1)`). Each kept-recent message may still carry a large tool result (e.g. a 50 KB `file_read`), so after summarizing it would otherwise re-cross the threshold immediately. Micro keeps only the newest `tool_result`'s content and clears the rest in place, preserving `tool_use_id` pairing so the next API call stays valid — no extra LLM round-trip. Micro-compact is **not** a standalone compaction path; `selfCompactSession` is its only call site — self-compact is the entry point, micro is its tail cleanup.
|
|
339
357
|
|
|
340
|
-
All paths share the same split logic: keep the last `keepMessages` turns, advance the cut forward past any orphan `tool_result`-first user message (otherwise the next API call gets `unexpected tool_use_id`).
|
|
358
|
+
All paths share the same split logic — the private `compactCut(messages)` on SessionManager (single source of truth, 3 call sites: `selfCompactSession` + the two feasibility gates below): keep the last `keepMessages` turns, advance the cut forward past any orphan `tool_result`-first user message (otherwise the next API call gets `unexpected tool_use_id`). The tail loop only ever moves the cut *up*, so `cut === 0` is exactly the `messages.length <= keepMessages` "nothing to compact" case.
|
|
359
|
+
|
|
360
|
+
**Notification contract — a "Compacting context…" preflight is only announced when compaction will actually run, and always gets an outcome line.** Both self-compact entry points (`maybeAutoCompact`, manual `compactSession`) check `compactCut() === 0` *before* emitting the preflight and bail silently when there's nothing to compact. After the preflight, every path closes it out: success pairs it with `Auto-compacted N older messages` / `Context compacted: …`; an empty LLM summary (a thinking-heavy model can legally spend its entire response on non-text blocks) emits `Compaction skipped — no summary produced`; a thrown summarize call emits `Compaction failed — context unchanged` (auto path swallows the error, manual path rethrows after emitting). Previously the preflight fired first and `selfCompactSession`'s silent `return null` paths left an orphan "Compacting context…" with no outcome — see `memory/2026-08-04-compact-preflight-orphan.md` for the production forensics.
|
|
341
361
|
|
|
342
362
|
Config (see `config.compact`): `keepMessages` / `maxSummaryInput` / `maxMessageSlice` / `summarizeTimeoutSec` — editable in Settings → General → compact.
|
|
343
363
|
|
|
364
|
+
**Self-compact also archives the UI log.** `selfCompactSession` calls `uiStore.archiveOldMessages(session.id)` for **root sessions only** (`!session.parentId`), right before the compaction notice is emitted so the notice lands in the kept exchange rather than inside the archived segment. Compaction is the one path that already means "history shrinks here", which is why archiving hangs off it rather than off a timer or its own sweep — the size threshold below decides whether the call does anything. See [UI-log archiving](#ui-log-archiving).
|
|
365
|
+
|
|
366
|
+
## UI-log archiving
|
|
367
|
+
|
|
368
|
+
Files: `packages/server/src/sessions/session-archive.ts` (segment IO) · `agents/session-ui-store.ts` `archiveOldMessages` (the write) · `routes/session-archive.ts` (the read)
|
|
369
|
+
|
|
370
|
+
`rawMessages` has compaction; the UI log had nothing, so it only ever grew — a long-lived session measured 6.9 MB of UI messages out of a 7.4 MB file, all of it re-read and re-written on every persist. Archiving moves the older exchanges out to `<seg>.arch.<N>.json.gz` beside the active `<seg>.json`, keeping the newest exchange in place. The active file's shape doesn't change, so every existing reader keeps working with no edit.
|
|
371
|
+
|
|
372
|
+
**Trigger: file size.** A compact archives only when the active `<seg>.json` is larger than `ARCHIVE_SIZE_THRESHOLD = 3 MB` (one `statSync`; a missing file counts as 0, so the check is also the "nothing written yet" guard). Below it, `archiveOldMessages` returns 0 and touches nothing, however many exchanges have piled up.
|
|
373
|
+
|
|
374
|
+
Bytes, not an exchange count, because bytes are the actual problem — file size is what makes a log slow to read, parse and ship — and per-exchange size varies by ~three orders of magnitude (a bare "yes" versus a turn carrying several 50 KB tool results), so a count is a badly distorted proxy: the same "15 exchanges" is 40 KB in one session and 40 MB in another. It is also self-limiting: after an archive the file restarts near-empty, so it must grow another 3 MB of real content before the next segment, which bounds segment count by total bytes written rather than by how often the user compacts. Still no relation to `config.compact.keepMessages` (that counts raw LLM messages — a different granularity with no honest mapping to UI rows), and no setting: one constant, tuned once.
|
|
375
|
+
|
|
376
|
+
**Split point.** Over the threshold, `archiveSplitIndex(messages, 1)` keeps only the **newest main user exchange** (`role === 'user'` without `taskId` — the same subset the admin renders and `deleteExchange` counts ordinals over) and cuts at its start, so a turn's responses are never split from their user message. A log holding a single exchange yields cut 0 and is left alone: there is nothing to move without splitting the turn the user is looking at, so an oversized *single* exchange is deliberately not archived.
|
|
377
|
+
|
|
378
|
+
Accepted edge, documented so it isn't "fixed": `rawMessages` shares the active file, so if raw alone approaches 3 MB (rare — compaction is what bounds it) the file can still exceed the threshold right after the UI log is archived, and the next compact writes another very small segment. Harmless churn; guarding it would mean second-guessing which half owns the bytes.
|
|
379
|
+
|
|
380
|
+
**The in-flight turn can't be archived** — by construction, not by a guard: the running turn's user message is the newest main user message, so keeping the newest exchange always keeps it.
|
|
381
|
+
|
|
382
|
+
**Crash consistency — three parts:**
|
|
383
|
+
|
|
384
|
+
1. **Segment first, active file second.** `writeArchiveSegment(n)` writes a path nothing references yet; only then is the active file rewritten with the kept exchange. The reverse order could drop messages before their archive exists.
|
|
385
|
+
2. **`archiveCount` in the active file is the commit marker.** A segment with `N > archiveCount` was never committed and no reader may reference it (both the WS anchor and the HTTP route check this). So a crash between the two steps leaves an unreachable orphan plus an active file that still holds every message — nothing lost. The next compact re-derives the same `N` and overwrites the orphan, which is why there's no reconciliation pass.
|
|
386
|
+
3. **Read-back verify, else roll memory back.** `persistUIState` swallows IO errors like every session write, so `archiveOldMessages` re-reads `archiveCount` afterwards; if it is `< n` the truncated in-memory `messageLog` is restored from the two halves. Without that, the *next* ordinary persist would write the short log under the old count — that, not the crash, is how messages would actually go missing.
|
|
387
|
+
|
|
388
|
+
Segments are immutable and append-only (`N` from 1, monotonically). Deletion is filesystem-driven: `deleteArchiveSegments()` globs `<seg>.arch.*.json.gz` in the directory rather than trusting `archiveCount` or any in-memory map, so crash-orphaned segments go too. Both session-file delete paths (`deleteSessionFile`, `findAndDeleteSessionFile`) call it.
|
|
389
|
+
|
|
390
|
+
**Read side (scroll-up loading).** The admin still loads a session the way it always did — the whole active file — and fetches segments only when the user scrolls to the top: `GET /api/sessions/logs/:id/archive/:n?projectId=` returns `{ messages }` for one whole segment, gunzipped server-side. The anchor is `archiveCount`, delivered on the `state:snapshot` of **subscribe / reattach only** (one header read per session open); per-turn snapshots and the post-`exchange:delete` snapshot deliberately omit it. The client walks `archiveCount` → 1 and stops at "no earlier messages"; pulled segments are cached (immutable, so never re-requested) and a failed fetch keeps the cursor so the same segment can be retried.
|
|
391
|
+
|
|
392
|
+
**The anchor moves up, never down.** `noteArchiveAnchor` no-ops for an equal or lower count (keeping per-turn snapshots — which omit the field, read as 0 — and ordinary re-subscribes idempotent), but a **higher** count re-anchors. Needed because a compact that fires while the session is open archives a new segment, and the next subscribe/reattach snapshot both carries the higher count *and* replaces the view with the shrunken active log: with a permanently pinned anchor those just-archived turns were in neither place — gone from the view and below a cursor counting down from the old anchor — showing a hole until the user reopened the session. Re-anchoring restarts the walk instead of extending it (prepends are oldest-first, so a newer segment can't splice onto an already-pulled tail); the re-pull costs no requests because `segmentCache` still holds them. Archived history renders collapsed and **read-only** — no Delete button, which matches `deleteExchange` refusing archived logs.
|
|
393
|
+
|
|
344
394
|
## Model message format dependencies
|
|
345
395
|
|
|
346
396
|
A sub-agent's `rawMessages` is stored directly as the runtime's `agent.messages`, whose format depends on the underlying provider. When swapping models, watch these:
|
|
@@ -60,7 +60,7 @@ Files: `packages/server/src/channels/slack/`
|
|
|
60
60
|
|
|
61
61
|
- `types.ts` — SlackAccount, SlackSocketEnvelope, SlackMessageEvent interfaces
|
|
62
62
|
- `accounts.ts` — DAL (thin wrapper over `channels/shared/accounts.ts` for Slack-specific mappings)
|
|
63
|
-
- `handler.ts` — Socket Mode connection + event routing, mention filtering
|
|
63
|
+
- `handler.ts` — Socket Mode connection + event routing, mention filtering; session resolution + per-thread listener/route bookkeeping come from `channels/shared/inbound.ts` (`InboundBridge` / `deliverInbound` / `dispatchChannelCommand`)
|
|
64
64
|
- `event-adapter.ts` — AgentSessionEvent → Slack messages (buffer + flush; split at 35k chars)
|
|
65
65
|
- `api.ts` — HTTP client (auth.test / openSocketModeConnection / chat.postMessage / files.upload / files.download / searchSlackTargets)
|
|
66
66
|
- `cron-dispatcher.ts` — CronDispatcher registration; requires an explicit Slack chatId per dispatch
|
|
@@ -103,9 +103,7 @@ Unlike Telegram (grammy polling) or WeChat (HTTP long-poll), Socket Mode elimina
|
|
|
103
103
|
7. Strip the `<@botUserId>` mention prefix from the text
|
|
104
104
|
8. (DMs only) Check if text is a slash command (starts with `/` or `!`); if so, dispatch via `channels/shared/commands.ts` and return early
|
|
105
105
|
9. Download and ingest any attached files (save images + files to workspace media store; images base64'd for vision)
|
|
106
|
-
10.
|
|
107
|
-
11. Register a `SlackResponder` event listener (once per session, torn down on stopAccount)
|
|
108
|
-
12. Queue the user message with SessionManager
|
|
106
|
+
10. Hand off to `deliverInbound` ([shared skeleton](telegram.md#shared-inbound-skeleton)): resolve or create the session for this {channel, thread}, apply the goal-mode route overlay, send a busy hint if needed, refresh the reply route `{channelId, replyTs}`, attach the `SlackResponder` listener once (torn down on `stopAccount` via `bridge.closeAll()`), then queue the user message with SessionManager
|
|
109
107
|
|
|
110
108
|
### Outbound (SlackResponder)
|
|
111
109
|
|
|
@@ -120,6 +118,22 @@ Unlike Telegram (grammy polling) or WeChat (HTTP long-poll), Socket Mode elimina
|
|
|
120
118
|
4. Errors and system notices flush immediately (don't wait for complete)
|
|
121
119
|
5. Sub-agent events (taskId set) are dropped (visible in web UI only)
|
|
122
120
|
|
|
121
|
+
**Chunk sends are serialized per responder.** One `flushBuffer` can produce
|
|
122
|
+
several chunks in a single synchronous loop, and each chunk is a separate
|
|
123
|
+
`chat.postMessage` call. Those calls used to be fired without awaiting the
|
|
124
|
+
previous one, so Slack could accept them out of order and a long reply arrived
|
|
125
|
+
scrambled (audit A-L3). Each responder now keeps a `sendTail` promise chain and
|
|
126
|
+
appends every chunk to it, so chunk *n+1* only goes out after *n* settles; a
|
|
127
|
+
rejected link is absorbed (`dispatchChunk` logs its own send failures) so one
|
|
128
|
+
bad send can't stall the rest of the reply.
|
|
129
|
+
|
|
130
|
+
`close()` therefore returns that drain promise, and `InboundBridge` keeps the
|
|
131
|
+
session's reply route registered until it settles before deleting it — including
|
|
132
|
+
on the `stopAccount` / `closeAll()` path, which for the same reason has no
|
|
133
|
+
blanket `routes.clear()`. Previously the route was dropped the moment the agent
|
|
134
|
+
turn returned, so the tail of a split reply had nowhere to send and was silently
|
|
135
|
+
dropped.
|
|
136
|
+
|
|
123
137
|
## Account binding & session keying
|
|
124
138
|
|
|
125
139
|
### DM sessions
|
|
@@ -155,9 +169,9 @@ If no explicit chatId is provided, dispatch fails with a clear message. This pre
|
|
|
155
169
|
Slack channel is a peer with all other channels. The handler:
|
|
156
170
|
|
|
157
171
|
1. Calls `registry.getOrCreate(workspacePath)` to get the SessionManager for the bound workspace
|
|
158
|
-
2. Registers a `SlackResponder` listener on the session via `sm.registerEventListener
|
|
172
|
+
2. Registers a `SlackResponder` listener on the session via the shared `InboundBridge` (`ensureListener` → `sm.registerEventListener`); the responder reads its `{channelId, replyTs}` route lazily at send time, so a thread that later receives a message from elsewhere can't get stale replies
|
|
159
173
|
3. Calls `sm.sendUserMessage(sessionId, agentInput, images?, accessLevel)` to queue inbound messages
|
|
160
|
-
4. Calls `sm.isSessionCompacting(sessionId)` / `sm.isSessionRunning(sessionId)` to report status
|
|
174
|
+
4. Calls `sm.isSessionCompacting(sessionId)` / `sm.isSessionRunning(sessionId)` to report status (via the shared `busyHint`)
|
|
161
175
|
|
|
162
176
|
The registry caches SessionManager instances by workspace path, so multiple bots in the same workspace share the same SessionManager and can coexist in the same sessions (each runs their own responder listener).
|
|
163
177
|
|
|
@@ -69,6 +69,16 @@ Precedence: workspace > global, **at folder granularity**. For the same id, a wo
|
|
|
69
69
|
|
|
70
70
|
All `.halo/...` path math goes through `packages/server/src/paths.ts` (e.g. `wsHaloDir(ws)`, `wsEvoRunDir(ws, runId)`, `globalInternalSessionFile(agentId, seg)`, `cronLogFile(runId)`). Treat `paths.ts` as the canonical reference for the layout above; if the layout changes, update `paths.ts` first and call sites follow.
|
|
71
71
|
|
|
72
|
+
Most builders are workspace-rooted (`ws*`), but the evo tree also needs the
|
|
73
|
+
layout *inside* one run/apply dir, so four builders take the **artifact dir**
|
|
74
|
+
instead of `(workspace, id)`: `evoSandboxDir(artifactDir)`,
|
|
75
|
+
`evoSandboxHaloDir(artifactDir)`, `evoRegressDir(applyDir)`,
|
|
76
|
+
`evoRegressRunDir(applyDir, runId)`. Run mode and apply mode lay their sandbox
|
|
77
|
+
out identically inside their own dir, which is why one signature serves both.
|
|
78
|
+
They replaced ~40 inline `path.join(ws, '.halo', 'evo', …)` sites across
|
|
79
|
+
archive / enqueue / evo-wrapper / ticker / routes; `test/paths-evo-equivalence.test.ts`
|
|
80
|
+
pins them against the old literals.
|
|
81
|
+
|
|
72
82
|
### Workspace runtime lock
|
|
73
83
|
|
|
74
84
|
`<workspace>/.halo/runtime.lock` — a pid marker that records which process owns this workspace's *runtime* (gates `reconcileOrphansOnBoot`; see [session.md](session.md#boot-reconcile-of-crash-orphans-reconcileorphansonboot)). It exists because `~/.halo/global/server.lock` can't arbitrate the case of two servers with different `HALO_HOME` sharing one workspace — the ownership marker has to live in the shared resource itself. Implementation: `packages/server/src/agents/workspace-runtime-lock.ts`.
|
|
@@ -99,12 +109,24 @@ Path: `.halo/sessions/{agentId}/{sessionId}.json`
|
|
|
99
109
|
"contextTokens": 5975,
|
|
100
110
|
"totalOutputTokens": 6058,
|
|
101
111
|
"parentSessionId": "parent_id", // sub-sessions only
|
|
112
|
+
"archiveCount": 2, // committed UI-log archive segments; absent = never archived
|
|
113
|
+
"archivedUserCount": 30, // main user turns that moved into segments
|
|
102
114
|
"messages": [SessionMessage], // UI event-log format (written by WS handler / UIState reducer)
|
|
103
115
|
"rawMessages": [AnthropicMessage], // raw Bedrock API shape (written by SessionManager saveAgentState)
|
|
104
116
|
"output": "..." // accumulated assistant text from the latest turn
|
|
105
117
|
}
|
|
106
118
|
```
|
|
107
119
|
|
|
120
|
+
Written as **compact JSON** (no indent) by all five writers of this path — nothing reads it by eye. `messageCount` is `messages.length` and has a single writer (`saveSessionToFile`); it *shrinks* when a compact archives history, so the lifetime count for listings is `exchangeCount` (mirrored to sqlite, see [agent_sessions](#agent_sessions)). See [session.md](session.md#session-file-format).
|
|
121
|
+
|
|
122
|
+
### Archive segment file (UI-log archiving)
|
|
123
|
+
|
|
124
|
+
Path: `.halo/sessions/{agentId}/{sessionId}.arch.{N}.json.gz`
|
|
125
|
+
|
|
126
|
+
gzipped `JSON.stringify(SessionMessage[])` — the plain array, no wrapper object, oldest-first, in the same order the messages had in the active file. `N` counts up from 1. A segment is **immutable once committed** and the active file's `archiveCount` is the commit marker: `N > archiveCount` means the segment was written but the commit never landed, and no reader may reference it. Deleting a session deletes segments by **globbing `{sessionId}.arch.*.json.gz`** — not by iterating `archiveCount` — so crash-orphaned uncommitted segments are cleaned up too.
|
|
127
|
+
|
|
128
|
+
A segment is written by a compact only when the active `.json` exceeded `ARCHIVE_SIZE_THRESHOLD` (3 MB), and then holds everything except the newest main exchange — so segment sizes are uneven by design and their count is bounded by total bytes written, not by compact frequency. Design + read path in [session.md](session.md#ui-log-archiving).
|
|
129
|
+
|
|
108
130
|
### SessionMessage
|
|
109
131
|
|
|
110
132
|
```typescript
|
|
@@ -128,7 +150,7 @@ interface SessionMessage {
|
|
|
128
150
|
taskId?: string // Sub-agent task ID
|
|
129
151
|
|
|
130
152
|
// assistant-only
|
|
131
|
-
toolCalls?: ToolCallEntry[]
|
|
153
|
+
toolCalls?: ToolCallEntry[] // READ-ONLY legacy — no longer written (see below)
|
|
132
154
|
contentBlocks?: ContentBlockEntry[]
|
|
133
155
|
|
|
134
156
|
// tool_call-only
|
|
@@ -191,11 +213,18 @@ Assistant rendering priority:
|
|
|
191
213
|
2. No contentBlocks, has `toolCalls` → tools first, then content (legacy)
|
|
192
214
|
3. Neither → render only `content`
|
|
193
215
|
|
|
216
|
+
### `toolCalls` is write-deprecated (read-only)
|
|
217
|
+
|
|
218
|
+
Assistant turns are persisted with `contentBlocks` only. `toolCalls` held a second copy of every tool's input **and** output, byte-for-byte identical to the blocks copy: across 195 sessions it was 66 MiB of a 340 MiB corpus (19.4%), and dropping it shrank the corpus to 257 MiB (-24.4%). The three UI-log writers (`flushAssistantMessage`, `flushCompletedAssistantMessage`, `createSaveSnapshot` in `sessions/ui-log-builder.ts`) no longer attach it.
|
|
219
|
+
|
|
220
|
+
Every reader goes through `messageToolCalls(msg)` (`sessions/session-types.ts`): blocks-first, `toolCalls` only when `contentBlocks` is absent — a **fallback, never a supplement**. This matters because the legacy copy was not merely redundant, it was sometimes wrong: of 433 entries where the two copies disagreed, 242 were provably a *different* call's output landing in the legacy array (the write-side merge keyed on tool name, so parallel or repeated calls to the same tool cross-wired). Blocks was right in the spot-checks where they diverged; legacy was 0-for-11. So old files still render, but where they disagree the blocks copy wins by design.
|
|
221
|
+
|
|
194
222
|
### Backwards compatibility
|
|
195
223
|
|
|
196
224
|
1. Missing `type`: infer from `role` + existing fields
|
|
197
|
-
2. Missing `contentBlocks`: fall back to the `toolCalls` + `content` split layout
|
|
225
|
+
2. Missing `contentBlocks`: fall back to the `toolCalls` + `content` split layout — the only reason `toolCalls` is still read
|
|
198
226
|
3. Missing `version`: treat as v0, apply inference rules
|
|
227
|
+
4. Missing `archiveCount`: session never archived (0 segments)
|
|
199
228
|
|
|
200
229
|
### Change rules
|
|
201
230
|
|
|
@@ -535,6 +564,13 @@ The databases hold session metadata indexes and workspace-scoped preferences (e.
|
|
|
535
564
|
| created_at / updated_at | INTEGER | Unix ms |
|
|
536
565
|
| stopped_at | INTEGER | Stopped timestamp (null = active) |
|
|
537
566
|
| archived_at | INTEGER | Archived timestamp (null = not archived) |
|
|
567
|
+
| goal / goal_session_id | TEXT | Goal mode — binding JSON on the goal session's row, back-pointer on the worker's |
|
|
568
|
+
| title | TEXT | Mirror of the session file's `title` |
|
|
569
|
+
| exchange_count | INTEGER | Mirror: main user turns over the session's lifetime (kept + archived) |
|
|
570
|
+
| context_tokens | INTEGER | Mirror of the file's `contextTokens` |
|
|
571
|
+
| total_output_tokens | INTEGER | Mirror of the file's `totalOutputTokens` |
|
|
572
|
+
|
|
573
|
+
The last four are a **read cache for the session list** — `GET /api/sessions/logs` renders rows from these columns instead of opening and parsing every session file. Written only by `mirrorSessionMeta()` off the single persist seam (`SessionManager.persistSessionFile`) and by the PATCH-rename path; the write is skipped when all four already match, and it never touches `updated_at` (that column means "last activity", and a metadata mirror is not activity). `null` = row predates the columns → the list route backfills from the file once. Details in [session.md](session.md#sqlite-metadata).
|
|
538
574
|
|
|
539
575
|
**`disabled_items`** — per-workspace disable state for agents and skills (allows the same global item to be independently toggled in each workspace)
|
|
540
576
|
|
|
@@ -590,3 +626,4 @@ All channel types (telegram, web, wechat, slack, feishu) share one table. Common
|
|
|
590
626
|
| v1+ | 2026-05-12 | Restructured `~/.halo/`: sensitive files (settings.yaml, config.yaml, channels/) moved to `secrets/`; non-sensitive config remains in `global/`. `migrateSecrets()` auto-migrates on first startup. Access level expanded to three tiers: `full` / `workspace` / `readonly` (agent_sessions + channel_accounts). Tool execution routed through bwrap sandbox for non-full access levels. `agent_sessions.working_dir` now stored as workspace-relative path. |
|
|
591
627
|
| v1+ | 2026-05-13 | Settings i18n + UI polish: compact keys renamed camelCase → snake_case (`keep_messages`, `max_summary_input`, `max_message_slice`, `summarize_timeout_sec`). Added `description_zh` on all leaf nodes and `description`/`description_zh` on branch nodes (replaces `_hint`/`_hint_zh`). Added `secret: true` attribute for masked input. Added Tavily to default params. Admin UI fully i18n'd (agents, skills, settings, nav). |
|
|
592
628
|
| v1+ | 2026-05-13 | Added `packages/cli` — standalone CLI/TUI client with embedded agent loop (no server required). Imports server agent-core via subpath exports. Session prefix: `cli_`. Added `exports` + `typesVersions` to server `package.json`. |
|
|
629
|
+
| v1+ | 2026-08-08 | Session file slimming (no version bump — additive + reader-compatible): assistant `toolCalls` no longer written (`contentBlocks` is the single copy, `toolCalls` stays read-only legacy); files written as compact JSON; UI-log archiving adds header fields `archiveCount` / `archivedUserCount` plus `{sessionId}.arch.{N}.json.gz` segment files; `agent_sessions` gains mirrored `title` / `exchange_count` / `context_tokens` / `total_output_tokens` columns so listing doesn't parse every file. |
|