@turmind/halo 1.5.11 → 1.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/admin-out/404.html +1 -1
- package/admin-out/_next/static/chunks/{941-4719e7343b54b9bd.js → 147-d4200c70445aabb6.js} +1 -1
- package/admin-out/_next/static/chunks/58.d26b1c4f14404226.js +1 -0
- package/admin-out/_next/static/chunks/989-ddba0eb72566da23.js +1 -0
- package/admin-out/_next/static/chunks/app/{layout-5ec6c5f713a8562b.js → layout-ea161287d934ba96.js} +1 -1
- package/admin-out/_next/static/chunks/app/page-be47983ba8ff20b1.js +1 -0
- package/admin-out/_next/static/chunks/webpack-74f577c1b094f942.js +1 -0
- package/admin-out/_next/static/css/3b4dcd0ac6b8eae8.css +1 -0
- package/admin-out/_next/static/css/7b736202f0b1dbfd.css +1 -0
- package/admin-out/index.html +1 -1
- package/admin-out/index.txt +6 -5
- package/bundled-docs/design/a2a.md +286 -0
- package/bundled-docs/design/agent.md +6 -4
- package/bundled-docs/design/agentcore.md +194 -122
- package/bundled-docs/design/architecture.md +2 -2
- package/bundled-docs/design/canvas-extensions.md +59 -12
- package/bundled-docs/design/cron.md +22 -11
- package/bundled-docs/design/observability.md +12 -4
- package/bundled-docs/design/prompt-system.md +10 -5
- package/bundled-docs/design/relay.md +1 -1
- package/bundled-docs/design/session.md +7 -2
- package/bundled-docs/design/storage.md +40 -10
- package/bundled-docs/design/web.md +5 -56
- package/bundled-docs/design/ws.md +1 -1
- package/bundled-docs/dev/add-model-provider.md +15 -9
- package/bundled-docs/dev/api.md +93 -14
- package/bundled-docs/dev/deploy.md +5 -1
- package/bundled-docs/dev/desktop-packaging.md +11 -5
- package/bundled-docs/dev/env.md +9 -5
- package/bundled-docs/dev/previews.md +26 -0
- package/bundled-docs/dev/tools.md +93 -3
- package/bundled-docs/guide/agents.md +6 -6
- package/bundled-docs/guide/channels/web.md +0 -26
- package/bundled-docs/guide/cli.md +45 -8
- package/bundled-docs/guide/delegation-and-access.md +3 -2
- package/bundled-docs/guide/skills.md +13 -7
- package/bundled-docs/guide/workspace.md +1 -1
- package/bundled-docs/requirements/agent.md +6 -1
- package/bundled-docs/requirements/chat.md +9 -3
- package/bundled-docs/requirements/command.md +4 -0
- package/bundled-docs/requirements/editor.md +28 -8
- package/bundled-docs/requirements/skill.md +15 -9
- package/bundled-docs/requirements/workspace.md +6 -0
- package/dist/index.js +6081 -3013
- package/package.json +1 -1
- package/templates/models/anthropic.yaml +109 -13
- package/templates/models/aws-bedrock-claude-invoke.yaml +76 -11
- package/templates/models/aws-bedrock-mantle.yaml +60 -28
- package/templates/models/aws-bedrock-openai.yaml +146 -29
- package/templates/models/deepseek.yaml +7 -3
- package/templates/models/doubao.yaml +72 -20
- package/templates/models/hunyuan.yaml +74 -14
- package/templates/models/kimi.yaml +63 -3
- package/templates/models/mimo-token-plan-china.yaml +9 -1
- package/templates/models/minimax.yaml +9 -3
- package/templates/models/openai.yaml +120 -17
- package/templates/models/qwen.yaml +90 -4
- package/templates/models/zhipu.yaml +9 -1
- package/templates/prompts/all/RUNTIME.md +2 -1
- package/templates/schema.sql +1 -1
- package/templates/skills/cron/SKILL.md +38 -2
- package/templates/skills/cron/manage-cron.py +115 -7
- package/templates/skills/send-file/SKILL.md +1 -1
- package/templates/skills/send-file/a2a.md +13 -0
- package/templates/skills/skill/SKILL.md +22 -11
- package/templates/skills/workspace/share.md +5 -2
- package/templates/skills/workspace/stage.py +51 -14
- package/admin-out/_next/static/chunks/220.7566bc89f3d98993.js +0 -1
- package/admin-out/_next/static/chunks/989-86b0f677f69ccf3b.js +0 -1
- package/admin-out/_next/static/chunks/app/page-4792aa1c79f7e7f1.js +0 -1
- package/admin-out/_next/static/chunks/webpack-efd27999305b080a.js +0 -1
- package/admin-out/_next/static/css/97f3ac570664d044.css +0 -3
- /package/admin-out/_next/static/{GO2W5JoiC46FIxLjtFAXx → j6buZnQDPISEFgtAuTGyl}/_buildManifest.js +0 -0
- /package/admin-out/_next/static/{GO2W5JoiC46FIxLjtFAXx → j6buZnQDPISEFgtAuTGyl}/_ssgManifest.js +0 -0
|
@@ -29,6 +29,7 @@ Schedules are **durable** — the source of truth is the `cron_jobs` table; in-m
|
|
|
29
29
|
|
|
30
30
|
- **Recurring**: standard cron expression (`job.schedule`), optional `timezone`, optional **active window** `[activeFrom, activeUntil)` (epoch ms, `active_from` / `active_until`, NULL = unbounded; `CRON_MIGRATIONS` slot 2). The window is passed to croner as native `startAt` / `stopAt` (`activeWindowOptions` in `runner.ts`; `startAt` is `activeFrom − 1 ms` because croner's startAt is exclusive), so fires outside it simply never happen — no `cron_runs` row. "Pause until X" = push `activeFrom` to X; there is no skip-list for a gap in the middle. Past `activeUntil` the job stays `enabled` (so it can be extended later) but `scheduleJob` creates no croner and logs `past activeUntil; not scheduled` once — the id goes into `_expired` so reconcile's "fingerprint cached but not in `_active`" retry branch treats it as handled instead of re-scheduling (and re-logging) every 10s. The REST `nextRunAt` uses the same options, so the list shows the first post-pause fire; the admin badges a job "active from …" / "ended". Run-now ignores the window. Recurring only: a window together with `runAt` is rejected at every write surface (REST on the merged row, skill helper).
|
|
31
31
|
- **One-shot**: `job.runAt` (epoch ms) — croner fires once at that instant. After it completes, `finalize` sets `enabled=0` so it never re-fires. A `runAt` already in the past at schedule time is marked `lastRunStatus='missed'` and disabled (rather than firing immediately or retrying every reconcile). The `run_at` column is `CRON_MIGRATIONS` slot 0 in `db/cron-db.ts`, run through the shared `runMigrations` / `PRAGMA user_version` mechanism (see [storage.md](storage.md#schema-change-rules)).
|
|
32
|
+
- **Folder + stateless** (`CRON_MIGRATIONS` slot 4, v5): `cron_jobs.working_dir` — the folder the run focuses on, passed as `halo cli --dir` (same semantics as a sub-agent's `working_dir`: the folder's `.halo/INSTRUCTIONS.md` chain joins the prompt, plus a `Working directory:` line); stored workspace-relative, NULL = the workspace root; validated at every write surface (inside the workspace, existing — REST re-checks a stored folder when `workspacePath` moves). `cron_jobs.stateless` (0/1) — **new session every run**: each fire gets a fresh root session `cron-<id>-<base36 ms>` (`perRunSessionId`, same no-double-prefix rule as `effectiveSessionId`), so nothing accumulates across fires; mutually exclusive with `session_id` (`400` on the merged row). `cron_runs.session_id` — the session that run used, the only record of a stateless run's id (NULL on rows from before v5).
|
|
32
33
|
|
|
33
34
|
### Trigger-mode exclusivity (the db contract)
|
|
34
35
|
|
|
@@ -44,28 +45,38 @@ Exactly **one** of `schedule` / `runAt` is set per job. `runner.ts` `scheduleJob
|
|
|
44
45
|
REST route mutations call `scheduleJob` / `unscheduleJob` directly. But the `cron` skill (and manual ops) edit `cron.db` over a *different* sqlite connection. `reconcileFromDb` (10s timer) catches those:
|
|
45
46
|
|
|
46
47
|
- Fast path: `PRAGMA data_version` flips only when *another* connection commits. If unchanged since last pass, skip the full select entirely — one pragma read (~µs) vs. a select-all + per-row fingerprint compare. The pragma (and the run-pruning `NOT IN (subquery)` below) needs the raw better-sqlite3 handle, which the runner gets from the shared `rawSqlite(db)` helper (`db/raw-sqlite.ts`) instead of re-spelling drizzle's `$client` / `.session.client` cast at each site — it did, three times, each with its own hand-written method shape.
|
|
47
|
-
- On change: diff db rows against the in-memory `_fingerprint` map (`enabled
|
|
48
|
+
- On change: diff db rows against the in-memory `_fingerprint` map (`jobFingerprint`: enabled, schedule, runAt, timezone, activeFrom, activeUntil, agentId, userPrompt, workingDir, stateless); schedule new rows, unschedule deleted ones, re-instantiate croner only for rows whose fingerprint changed. Broadcasts a coalesced `cron:job_changed` per affected job.
|
|
48
49
|
|
|
49
50
|
## Execution (`runJob`)
|
|
50
51
|
|
|
51
52
|
Each fire:
|
|
52
53
|
|
|
53
|
-
1. **Concurrency guard**: resolve the job's session first (`effectiveSessionId` — `job.sessionId`, else the job's own session, see step 4), then check three conditions in order; the first hit inserts a `cron_runs` row with `status='skipped'` and that `failureReason`, broadcasts, and bails without spawning. Applies to both scheduled fires and manual run-now clicks.
|
|
54
|
+
1. **Concurrency guard**: resolve the job's session first (`effectiveSessionId` — `job.sessionId`, else the job's own session, see step 4; a stateless job's fresh `perRunSessionId`, which the two session-level checks never hit), then check three conditions in order; the first hit inserts a `cron_runs` row with `status='skipped'` and that `failureReason`, broadcasts, and bails without spawning. Applies to both scheduled fires and manual run-now clicks.
|
|
54
55
|
- Same `jobId` already running in this process (in-memory `_inflight` set) → `previous run still in progress`.
|
|
55
56
|
- Another job's cli already on the same session (`_inflightSessions`, keyed `realpath(workspace) + sessionId` so two spellings of one workspace collide) → `session <id> busy: another cron run is in progress`. Two jobs may pick the same session.
|
|
56
57
|
- The server itself is mid-turn anywhere in that session's tree (`hasActiveWorkInTree` on the workspace's live SessionManager, looked up with the registry's `peek` — cache-only, never builds a manager, whose constructor would run the boot reconcile as a side effect) → `session <id> busy: a turn is running in it`. In practice this is a picked session someone is chatting in.
|
|
57
58
|
|
|
58
59
|
Two overlapping writers would double-write the same on-disk session state; SessionManager's per-session lock is in-process and can't see across cli children, so these sets are the cheapest place to enforce serialization. They are in-memory, but their restart blind spot is covered: the boot-time orphan sweep (below) re-enters stale jobIds and session keys before croner re-arms. The runner reaches the server's managers through `setCronSessionRegistry(registry)` (called in `index.ts` next to `setRelayRegistry`); unset — tests, a bare CLI — means there are no server-side sessions to guard.
|
|
59
|
-
2. Insert a `cron_runs` row with `status='running'` up front (UI sees it mid-flight); broadcast `cron:run_changed`.
|
|
60
|
-
3. Check `job.workspacePath` exists — fail loudly if the workspace was moved/deleted.
|
|
61
|
-
4. Spawn `halo cli -a <agent> -s <session> -w <workspace> [--access <level>]`, prompt on **stdin** (avoids Windows argv length limits), with a fixed `[Unattended cron run — nobody will answer questions …]` line stamped ahead of `user_prompt` on every fire (`UNATTENDED_PREAMBLE` in `runner.ts`) — the model can't see the `cron-*` session id, and prompts typed into the admin Cron form carry no such notice, so the runner adds it rather than trusting the prompt author. **Session strategy**: every job runs in one stable session — its own by default, or an existing **root** session picked on the job (`cron_jobs.session_id`, nullable; `CRON_MIGRATIONS` slot 1 in `db/cron-db.ts`). The job's own session id **is the job id** (generated ids are already `cron-<ts>-<rand>`; a custom id without that prefix gets `cron-<id>`) — `effectiveSessionId` in `runner.ts` is the one copy of the rule, and the REST list returns its result as `effectiveSessionId` so the admin never re-derives it. Older builds always prefixed, giving `cron-cron-…`; `CRON_MIGRATIONS` slot 3 (v4) pins every job that ever spawned a cli (`last_run_id` set, or a non-skipped run row — covers a first run in flight across the upgrade restart) to that session via `session_id`, so its history and the orphan sweep's `-s` fingerprint stay the same; jobs that never ran take the new rule. The cli's `-s` is create-on-missing, so the first fire creates the session with the job's agent; every later fire *resumes the same session
|
|
60
|
+
2. Insert a `cron_runs` row with `status='running'` up front (UI sees it mid-flight), recording the session it runs in (`cron_runs.session_id`); broadcast `cron:run_changed`.
|
|
61
|
+
3. Check `job.workspacePath` exists — fail loudly if the workspace was moved/deleted. Same for the job folder (`job.workingDir`, re-resolved with `resolveWorkingDir`): moved / deleted since the job was saved → `failed` with `folder not found: …`, nothing spawned.
|
|
62
|
+
4. Spawn `halo cli -a <agent> -s <session> -w <workspace> --dir=<folder|.> [--access <level>]`, prompt on **stdin** (avoids Windows argv length limits), with a fixed `[Unattended cron run — nobody will answer questions …]` line stamped ahead of `user_prompt` on every fire (`UNATTENDED_PREAMBLE` in `runner.ts`) — the model can't see the `cron-*` session id, and prompts typed into the admin Cron form carry no such notice, so the runner adds it rather than trusting the prompt author. **Session strategy**: every job runs in one stable session — its own by default, or an existing **root** session picked on the job (`cron_jobs.session_id`, nullable; `CRON_MIGRATIONS` slot 1 in `db/cron-db.ts`). The job's own session id **is the job id** (generated ids are already `cron-<ts>-<rand>`; a custom id without that prefix gets `cron-<id>`) — `effectiveSessionId` in `runner.ts` is the one copy of the rule, and the REST list returns its result as `effectiveSessionId` so the admin never re-derives it. Older builds always prefixed, giving `cron-cron-…`; `CRON_MIGRATIONS` slot 3 (v4) pins every job that ever spawned a cli (`last_run_id` set, or a non-skipped run row — covers a first run in flight across the upgrade restart) to that session via `session_id`, so its history and the orphan sweep's `-s` fingerprint stay the same; jobs that never ran take the new rule. The cli's `-s` is create-on-missing, so the first fire creates the session with the job's agent; every later fire *resumes the same session*, so the conversation accumulates and is reviewable in the admin Sessions tab. A **stateless** job (`cron_jobs.stateless`) skips all of this: every fire creates a fresh session (`perRunSessionId`), recorded on the run row.
|
|
63
|
+
|
|
64
|
+
**Agent lock.** The runner always passes `-a <job agent>`, and the cli refuses `-s <existing>` with a different explicit `-a` (a session's agent never changes). So once a stateful job's session exists, the job's agent is fixed to the session's: the REST create / update routes return `400 session <id> already runs agent <x>; …` when the **merged** row's agent differs from the session's (`sessionAgentConflict`; the skill helper exits with the same message). The check only runs when the body actually changes one of its inputs — agent, session, stateless or workspace, compared as stored values (the admin form resends every field) — so a job saved before the rule, or hand-edited into a mismatch, still takes unrelated edits such as `{enabled:false}`; disabling is always allowed. Such a legacy mismatch is not repaired: each of its runs fails with the cli's refusal on stderr as `failureReason`, until the agent is changed back, the session deleted, or the job switched to stateless (which has no lock). The list GET returns the session's agent as `sessionAgentId` (null: stateless / session not created yet), which the admin form locks the agent field on, or flags with an amber notice when it already differs.
|
|
65
|
+
|
|
66
|
+
**Folder.** `--dir=<folder>` is always passed, as **one** token (a folder starting with `-` would otherwise read as a flag), with `.` for the workspace root, so the cli applies it on create *and* re-points a resumed session in place (`SessionManager.setWorkingDir`) — changing a job's folder moves its existing session too. The log header records `dir=`.
|
|
67
|
+
|
|
68
|
+
**Access level.** For a session that already exists the runner reads its `access_level` from the workspace db (`readSessionAccessLevel`, null column = `full`) and passes it as `--access`: the cli defaults to `full` and persists any level that differs from the row, so without it one cron fire would permanently promote a `readonly` / `workspace` session. A session the cli is about to create gets no `--access` (full, as before). The log header records `session=` (with `(stateless)`) / `access=`.
|
|
69
|
+
|
|
70
|
+
**Session lookup** (`readSessionRow`, shared by the access level, `sessionAgentId` and the agent lock). It runs per job on every list GET and `cron:job_changed` push, so it never goes through `getWorkspaceDb` (realpath + `ensureWorkspaceHalo` scaffolding per call — and a GET must not scaffold): one read-only better-sqlite3 handle per workspace path, with the rows cached and dropped whenever `PRAGMA data_version` moves. The handle never writes, so any commit by anyone — the cli child creating the session (`finalize` reads it right after), a Sessions-tab delete, another process — moves the version. **Absent vs unreadable**: no `.halo/halo.db` or no row → null ("no session yet"); any other failure (corrupt / locked db) throws `could not read session <id> from <ws>: …`, so a read error never passes for "no session" — which would spawn a restricted session at `full`, or let the agent lock through. At fire time the run is finalized `failed` with that message and nothing is spawned; the REST write points return it as the `400` (the skill helper reads an unreadable db as no session — the fire still refuses); display (GET `sessionAgentId`, `cron:job_changed`) reads it as null.
|
|
71
|
+
|
|
72
|
+
The child's pid is written back onto the `running` row (`cron_runs.pid`, nullable) — this is what makes the orphan sweep possible after a server crash. Note the **dual-writer layout** this creates: the cli child and the server share the workspace's session files. The server side is view-only for these sessions — admin subscribes seed a UIState from disk, and the WS detach-save skips it via the dirty gate (see [session.md](session.md#ws-disconnect-resilience)); without that gate a frozen server snapshot would overwrite the cli child's newer messages (the cron-session UI-log truncation incident). Live streaming of a cli-driven session into the admin is out of scope — events don't cross processes; the Sessions tab re-reads the file on open.
|
|
62
73
|
|
|
63
74
|
A **picked** session breaks the view-only assumption: the server may hold a live in-memory copy (it was chatted in / opened before, and `ensureSession` keeps it resident), and its raw history predates the cli's turn — the next server-side save would write the cron turn away. So once the cli exits the runner re-`peek`s the workspace's manager (it may have been built mid-run) and calls `SessionManager.forgetExternalWrite(sessionId)`: if the tree is idle, it drops the root + `root>…` sessions from `sessions` **without saving** and drops the UIState, so the next view / message re-reads disk. If a turn started in the session while the cli ran (step 1 only covers turns already running at spawn), it returns false and nothing is dropped — both processes wrote the same files, the last save wins, and the runner logs a `console.warn` that one of the two turns may be missing. That window is the accepted cost of sharing a session with a job; the admin form hint and the `cron` skill both say so.
|
|
64
75
|
5. Tee stdout + stderr live to `~/.halo/global/logs/cron/<runId>.log`; capture stdout to memory for dispatch. Since 1.1.5 the cli's stdout is the **final reply of the last root turn** as raw markdown (stdout is not a TTY, so no terminal styling; the interim text the model emits before tool calls and the wrap-ups of earlier drained turns are not included — see [session.md](session.md#message-queue-and-drain) "Final-text flag"). Before that it was every streamed text block of the whole run, which for a director session absorbing a dozen sub-agent reports reached ~40 KB and blew the wechat gateway ceiling.
|
|
65
76
|
6. Enforce a **per-job timeout** (`job.timeoutSec ?? 3600`, the nullable `cron_jobs.timeout_sec` validated 60–21600 at every write surface; a change takes effect on the job's next fire, no croner re-instantiation needed) via a Node timer (cross-platform, not the Linux-only `timeout(1)`), with a **two-phase kill**: SIGTERM the direct child first — the cli has a SIGTERM handler that stops the harness, repairs + flushes the session, kills its own shell_exec process groups, and exits 143 — then, if it's still alive after a 30s grace, SIGKILL the entire descendant tree (POSIX ppid-walk over one `ps` snapshot; `taskkill /T /F` on Windows) so no grandchild survives to double-write the workspace on the next fire. A ppid walk, not a process-group kill: the cli sandbox spawns shell_exec workers `detached` into their own groups, so `kill(-pid)` would miss them. Timed-out runs still report exit code 124. Generous because overlap-of-the-same-job is already blocked up-front in step 1; this is just the long-stop reaper for a truly stuck child.
|
|
66
77
|
7. Classify: exit 124 → `timeout`; non-zero → `failed`; zero but empty stdout → `failed` ("nothing to dispatch" — with the final-only stdout contract this means the last turn produced no wrap-up text, not that the agent never spoke); else `succeeded`.
|
|
67
78
|
8. On success only, `dispatchToTargets(stdout, targets, job.workspacePath)` — the raw stdout goes down as-is; `MEDIA:` extraction happens per-target inside the dispatcher (see Dispatch model). If every target fails, downgrade to `failed`.
|
|
68
|
-
9. `finalize`: write the terminal `cron_runs` row, update `cron_jobs.lastRun*`, disable one-shot jobs, broadcast. The `_inflight` and `_inflightSessions` entries are released in `finally`, regardless of outcome.
|
|
79
|
+
9. `finalize`: write the terminal `cron_runs` row, update `cron_jobs.lastRun*`, disable one-shot jobs, broadcast. The `cron:job_changed` it sends carries `workingDir` / `stateless` / `sessionAgentId` (`jobChangedFields`, as do the REST mutations' and reconcile's) — `sessionAgentId` read after the run, so a first fire's push already has the agent of the session it just created. The `_inflight` and `_inflightSessions` entries are released in `finally`, regardless of outcome.
|
|
69
80
|
|
|
70
81
|
The cli executable is resolved via `resolveHaloCli()` (`$HALO_CLI` override; `halo.cmd` on Windows to dodge the GUI `Halo.exe` on PATH).
|
|
71
82
|
|
|
@@ -75,8 +86,8 @@ The step-1 guard is per-process, so it has one blind spot: the server dies (SIGK
|
|
|
75
86
|
|
|
76
87
|
`startCronDaemon` calls `sweepOrphanRuns` **before** `reloadAll`. For each stale row:
|
|
77
88
|
|
|
78
|
-
1.
|
|
79
|
-
2. If the row has a pid (POSIX): **identity-check before killing** — `ps -p <pid> -o args=` must carry the session id as a whole argv token (the `-s <session>` value; whole-token so `cron-a` never matches `cron-ab`), so a recycled pid never kills an innocent process. If the job
|
|
89
|
+
1. Take the session the orphan was spawned on from the run row (`cron_runs.session_id` — the only source for a stateless job's per-run id; a row from before that column falls back to the job row's effective session, a job deleted since to the default rule on its id), then re-enter its jobId into `_inflight` and its session key into `_inflightSessions` first — the core invariant: while the orphan may be alive, no new cli for that job *or for any other job on that session* is spawned (a fire in that window is recorded as `skipped`, same as normal overlap).
|
|
90
|
+
2. If the row has a pid (POSIX): **identity-check before killing** — `ps -p <pid> -o args=` must carry the session id as a whole argv token (the `-s <session>` value; whole-token so `cron-a` never matches `cron-ab`), so a recycled pid never kills an innocent process. If the session came from the job row (pre-v5 run row) and was re-pointed after the orphan was spawned, the fingerprint won't match and the orphan is left alone — the safe direction. Verified → SIGTERM (graceful path, same as timeout phase 1); still alive after a 30s async grace (`unref`ed timer, identity re-checked — pid reuse again) → SIGKILL the descendant tree.
|
|
80
91
|
3. No pid / already dead / fingerprint mismatch / Windows: no kill, log only. Windows deliberately marks-only — there's no cheap command-line identity check (WMI is slow and sometimes policy-blocked), and a full-tree `taskkill /T /F` on a reused pid is irreversible while a double-written session is fixable; the orphan-survives-server-death window is also naturally smaller there.
|
|
81
92
|
4. Either way the row is finalized as `failed` (`failureReason` notes the orphan disposition, `completedAt` stamped) so the UI stops showing a ghost `running`, and the jobId / session key are released once the orphan is confirmed dead (or immediately when nothing was killable).
|
|
82
93
|
|
|
@@ -112,7 +123,7 @@ interface CronChannelDispatcher {
|
|
|
112
123
|
|
|
113
124
|
## Key files
|
|
114
125
|
|
|
115
|
-
- `packages/server/src/cron/runner.ts` — scheduler: croner lifecycle, `runJob` spawn/capture/finalize, reconcile, retention
|
|
126
|
+
- `packages/server/src/cron/runner.ts` — scheduler: croner lifecycle, `runJob` spawn/capture/finalize, reconcile, retention; `effectiveSessionId` / `perRunSessionId`, the read-only session lookup (`readSessionRow`) behind `sessionAgentIdFor` / `sessionAgentConflict`.
|
|
116
127
|
- `packages/server/src/cron/dispatcher.ts` — channel-agnostic dispatch registry (`registerCronDispatcher`, `dispatchToTargets`, `listAllCronTargets`).
|
|
117
128
|
- `packages/server/src/db/cron-db.ts` — `cron_jobs` + `cron_runs` schema, `CRON_MIGRATIONS`, `getCronDb`.
|
|
118
129
|
- `packages/server/src/agents/session-manager.ts` — `forgetExternalWrite` (drop the stale in-memory copy of a session a cli child just wrote).
|
|
@@ -121,6 +132,6 @@ interface CronChannelDispatcher {
|
|
|
121
132
|
|
|
122
133
|
## Scope
|
|
123
134
|
|
|
124
|
-
Supported: recurring (cron expr) + one-shot (`runAt`) schedules; per-timezone; optional active window for recurring jobs (`activeFrom` / `activeUntil` — "pause until", "send until"); per-job configurable timeout (`timeoutSec`, 60–21600s, default 3600); multi-target fan-out; stable accumulating session per job — its own (id = job id) or a picked existing root session (`sessionId`); run-now (manual trigger); admin UI history with per-run logs + dispatch results.
|
|
135
|
+
Supported: recurring (cron expr) + one-shot (`runAt`) schedules; per-timezone; optional active window for recurring jobs (`activeFrom` / `activeUntil` — "pause until", "send until"); per-job configurable timeout (`timeoutSec`, 60–21600s, default 3600); multi-target fan-out; stable accumulating session per job — its own (id = job id) or a picked existing root session (`sessionId`) — or a fresh session every run (`stateless`); a per-job folder (`workingDir`, the run's `--dir`); run-now (manual trigger); admin UI history with per-run logs + dispatch results.
|
|
125
136
|
|
|
126
|
-
Not supported: sub-minute schedules below croner's resolution; in-process job execution (always a fresh cli child); running in a sub-session (`>` paths — root sessions only); a conflict-free shared session — a message sent into a picked session while its cron run is in flight can lose one of the two turns (logged, not prevented; see step 4).
|
|
137
|
+
Not supported: sub-minute schedules below croner's resolution; in-process job execution (always a fresh cli child); running in a sub-session (`>` paths — root sessions only); changing a stateful job's agent once its session exists (agent lock); a conflict-free shared session — a message sent into a picked session while its cron run is in flight can lose one of the two turns (logged, not prevented; see step 4).
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## Overview
|
|
4
4
|
|
|
5
|
-
A vendor-neutral observability layer built on the official OpenTelemetry JS SDK. One setting, `general.observability.endpoint`, turns on OTLP http/protobuf export of traces, metrics and logs to whatever collector the operator points at — empty means fully off. Principle: zero AWS (or any vendor) code in the server — SigV4 signing, backend endpoints, and resource enrichment all live in the collector, not in halo.
|
|
5
|
+
A vendor-neutral observability layer built on the official OpenTelemetry JS SDK. One setting, `general.observability.endpoint`, turns on OTLP http/protobuf export of traces, metrics and logs to whatever collector the operator points at — empty means fully off. It applies to every process that runs the agent loop: the server, `halo cli` and `halo tui` — and so the cron jobs and evolution runs, which are spawned `halo cli` children. Principle: zero AWS (or any vendor) code in the server — SigV4 signing, backend endpoints, and resource enrichment all live in the collector, not in halo.
|
|
6
6
|
|
|
7
7
|
## Configuration
|
|
8
8
|
|
|
@@ -13,7 +13,9 @@ A vendor-neutral observability layer built on the official OpenTelemetry JS SDK.
|
|
|
13
13
|
| `general.observability.headers` | string, **secret** | `''` | Extra headers for every OTLP request, comma-separated `k=v` (e.g. `authorization=Bearer …`). Restart required. |
|
|
14
14
|
| `general.observability.capture_content` | boolean | `false` | Put prompt / completion / tool argument+result text on spans (`gen_ai.*.messages` etc.). Off = metadata only (model, tokens, latency, tool names). Restart required. |
|
|
15
15
|
|
|
16
|
-
All four keys are `globalOnly` and take effect on server restart — `otel.ts` reads them once at boot, before anything else touches the OTel API.
|
|
16
|
+
All four keys are `globalOnly` and take effect on server restart — `otel.ts` reads them once at boot, before anything else touches the OTel API. A `halo cli` / `halo tui` process reads them the same way at its own start, so a change reaches the next run.
|
|
17
|
+
|
|
18
|
+
**Which process exported.** Every process shares `service.name`, so the resource also carries `halo.process` = `server` \| `cli` \| `tui` (`HaloProcess` in `otel.ts`; `initObservability()` defaults to `server`). Cron and evolution children are `halo cli` runs, so they report `cli`; their `session.id` (`cron-<jobId>`, or the `__evo_agent__`-style agent name) tells them apart from a hand-run cli. The startup log line names it: `[Observability] OTLP export → … (service.name=…, halo.process=cli, …)`.
|
|
17
19
|
|
|
18
20
|
**Env-var mapping.** `initObservability()` maps settings onto the standard `OTEL_*` env vars with `??=`, so an operator who already exports them (or `--require`s an external distro) wins:
|
|
19
21
|
|
|
@@ -35,17 +37,23 @@ No `OTEL_EXPORTER_OTLP_ENDPOINT` after the mapping → `enabled=false`, `initObs
|
|
|
35
37
|
| module | role |
|
|
36
38
|
|---|---|
|
|
37
39
|
| `packages/server/src/observability/otel.ts` | Bootstrap. `initObservability()` / `shutdownObservability()`, the `enabled` gate, `tracer` / `getMeter()` / `otelLogger` / `captureContent()` exports. |
|
|
38
|
-
| `packages/server/src/observability/otel-sdk.ts` | The ONE module that
|
|
40
|
+
| `packages/server/src/observability/otel-sdk.ts` | The ONE module that loads the SDK packages and OTLP exporters (`import()`ed inside `registerSdk`, see below). |
|
|
39
41
|
| `packages/server/src/observability/genai-spans.ts` | The span model: `beginTurn` / `onAgentEvent` / `recordRetry` / `endTurn`. |
|
|
40
42
|
| `packages/server/src/agents/session-manager.ts` | Calls the four hooks above from the turn loop (see below). |
|
|
41
43
|
| `packages/server/src/logger.ts` | Bridges every `logger.*` call into an OTel LogRecord when `enabled`. |
|
|
42
44
|
| `packages/server/src/index.ts` | `await initObservability()` (before `initLogger()`) and `await shutdownObservability()` (in `gracefulShutdown`). |
|
|
45
|
+
| `packages/cli/src/harness.ts` · `index.ts` | `initRuntime('cli' \| 'tui')` calls `initObservability(<process>)` before `initLogger()` (the list-only `agents` / `sessions` commands pass nothing, so stay off); the exit flushes below. The server package exports `./observability/otel` for this. |
|
|
43
46
|
|
|
44
47
|
**Hook call sites in `session-manager.ts`** (all in `runAgentTurn`): `beginTurn(session, message)` at the start, `onAgentEvent(session, event)` inside the agent event loop (one call per `AgentEvent`), `recordRetry(kind)` at the 7 retry catch sites, `endTurn(session, {error?})` once after the retry loop (the turn's single exit point, success or failure).
|
|
45
48
|
|
|
46
49
|
**The `enabled` gate.** `otel.ts` exports a single `enabled: boolean`, set once by `initObservability()`. Every hook in `genai-spans.ts` and the `logger.ts` bridge checks it first — an unconfigured server pays nothing beyond one boolean read per hook call.
|
|
47
50
|
|
|
48
|
-
**Why
|
|
51
|
+
**Why the SDK is loaded by dynamic import.** The SDK module graph costs ~120ms, which every unconfigured `halo` start (`--version` included) would otherwise pay. `otel.ts` only `import()`s `otel-sdk.ts` after confirming an endpoint is configured, and `otel-sdk.ts` itself `import()`s the SDK / exporter packages inside `registerSdk` rather than at the top: the npm bundle (esbuild) inlines `otel-sdk.ts`, which would hoist top-level imports of those external packages into the bundle's static imports and load them on every start. So the unconfigured path (server, `halo cli`, `halo tui`) never loads the SDK.
|
|
52
|
+
|
|
53
|
+
**Flush on exit (cli / tui).** A short-lived process must push its batched spans / metrics / logs before it goes. `shutdownObservability()` force-flushes and shuts down every provider, is idempotent (a signal can land while the end-of-run flush is in flight), and is **capped at 3 s** (`SHUTDOWN_CAP_MS`) so a dead collector delays an exit but never hangs it; it resolves `false` when the cap fired, and is instant when observability is off. The 3 s sits inside the cron runner's 30 s and the evolution wrapper's 10 s SIGTERM→SIGKILL grace. Three exit paths:
|
|
54
|
+
- **natural end** (one-shot run done, TUI quit, a thrown error): a `beforeExit` hook registered by `initRuntime` flushes; when the cap fired the exporters' retry timers still hold the event loop, so the hook then calls `process.exit()` itself (stdout has drained by then). `beforeExit` never fires on `process.exit()`.
|
|
55
|
+
- **SIGINT / SIGTERM**: the handlers stop the harness first (ending the turn, so its spans are closed and in the batch), then `exitAfterFlush(130 | 143)` (`harness.ts`) flushes and exits. A second SIGINT still hard-exits at once.
|
|
56
|
+
- Any other `process.exit()` skips the flush. On Windows `child.kill('SIGTERM')` delivers no catchable signal, so a timed-out cron run there loses its last batch.
|
|
49
57
|
|
|
50
58
|
**Why the meter is lazy.** The `@opentelemetry/api` tracer and logger are proxies — they stay no-op until a real provider is registered, then transparently start forwarding, so `tracer` / `otelLogger` can be module-level consts. The metrics API has **no proxy**: a `Meter` fetched via `metrics.getMeter()` before `setGlobalMeterProvider()` runs is a permanent no-op forever after, even once a provider registers later. `getMeter()` in `otel.ts` is therefore a function, not a const, and `genai-spans.ts` calls it lazily on first metric emission (`instruments()`, cached after).
|
|
51
59
|
|
|
@@ -29,7 +29,7 @@ Entry point: `session-agent-builder.ts` `SessionAgentBuilder.buildAgentInstance(
|
|
|
29
29
|
├── USER.md ← optional, overrides global USER.md
|
|
30
30
|
├── INSTRUCTIONS.md ← project-level (overrides global INSTRUCTIONS.md)
|
|
31
31
|
├── INDEX.md ← project overview + doc index
|
|
32
|
-
├── <subdir>/.halo/INSTRUCTIONS.md ← directory-scoped: baked into the system prompt when a sub-agent's
|
|
32
|
+
├── <subdir>/.halo/INSTRUCTIONS.md ← directory-scoped: baked into the system prompt when a session's working_dir (sub-agent, or a root's --dir / cron folder) points here; injected per-turn via user @scope otherwise
|
|
33
33
|
├── prompts/ ← optional, directory-level override of global prompts/
|
|
34
34
|
│ ├── bootstrap/ ← overrides global prompts/bootstrap/ if dir exists
|
|
35
35
|
│ ├── all/ ← overrides global prompts/all/ if dir exists
|
|
@@ -78,7 +78,7 @@ agent's AGENT.md), because the workspace folder replaces the global one whole.
|
|
|
78
78
|
**Override rule.** The workspace-root INSTRUCTIONS.md suppresses the global one (same override as USER.md / AGENT.md), so a cloned workspace is self-contained — its conventions travel with the repo and don't depend on the machine's global file. When the workspace root has no INSTRUCTIONS.md, the global one is used.
|
|
79
79
|
|
|
80
80
|
**Sub-directory INSTRUCTIONS.md — two paths, by who supplies the directory** (see [Directory-scoped instructions](#directory-scoped-instructions-scope) below):
|
|
81
|
-
- **A sub-agent's `
|
|
81
|
+
- **A session's `working_dir`** — a sub-agent's from `start_session`, or a root session's from `halo cli` / `halo tui --dir` or a cron job's folder — is persistent session identity (stored in the DB, restored on resume), so its directory-chain INSTRUCTIONS.md are baked into the **system prompt every turn** — the agent never forgets the rules of the directory it lives in.
|
|
82
82
|
- **A user's `@scope <dir>`** is ad-hoc, so it's injected **per-turn** into that one message only (loop-scoped — relevant to the turn that asked for it, not a permanent part of identity).
|
|
83
83
|
|
|
84
84
|
### INDEX.md (**project root only**)
|
|
@@ -146,7 +146,7 @@ Missing directory or read failure: warn + use built-in fallback.
|
|
|
146
146
|
3. The live agent roster (`## Your Team`), slotted directly behind AGENT.md (see [Agent roster](#agent-roster) below). Empty string for non-delegating agents (no `team` / empty `team`) / internal agents, so the section is dropped.
|
|
147
147
|
4. `## User Instructions` — `~/.halo/global/INSTRUCTIONS.md` (suppressed when the workspace root has its own — see Step 2)
|
|
148
148
|
5. `## User Instructions` — `<ws>/.halo/INSTRUCTIONS.md` (workspace root).
|
|
149
|
-
6. `## User Instructions` —
|
|
149
|
+
6. `## User Instructions` — the session's `working_dir` directory-chain INSTRUCTIONS.md (`loadScopeBody`, plain markdown, headed `### <dir>`; a sub-agent's, or a root's `--dir` / cron folder), folded into the same region right after #4/#5 so the order reads general → specific. Empty when there's no working_dir / it's the root / it has no sub-dir file. (User `@scope` does NOT come through here — it injects per-turn into the message, wrapped in `<workspace-instructions>`.)
|
|
150
150
|
7. `## Project Knowledge` — `<ws>/.halo/INDEX.md` (skipped entirely when no INDEX.md exists)
|
|
151
151
|
|
|
152
152
|
Global and workspace-root INSTRUCTIONS share the same `## User Instructions` heading: they're mutually exclusive (workspace overrides global, see Step 2), so only one ever lands in a prompt and there's no sibling to disambiguate from. A sub-agent's `working_dir` directory-chain (#6) reuses the same heading and is a third, *additive* layer in that region — sub-dir rules add to, never replace, the global/ws-root base, so the order reads general → specific. The roster rides as section 3 — passing it through `composeMdPrompt` rather than concatenating it afterward gives it the same `---` separators as every other section (no glue-to-next-block).
|
|
@@ -163,7 +163,7 @@ mdPrompt ← incl. roster + working_dir
|
|
|
163
163
|
+ allPrompt
|
|
164
164
|
```
|
|
165
165
|
|
|
166
|
-
(The `working_dir` directory-chain INSTRUCTIONS.md live *inside* `mdPrompt` — folded into the `## User Instructions` region by `composeMdPrompt` — not after the `Working directory:` tagline. That tagline is just a one-line focus marker.)
|
|
166
|
+
(The `working_dir` directory-chain INSTRUCTIONS.md live *inside* `mdPrompt` — folded into the `## User Instructions` region by `composeMdPrompt` — not after the `Working directory:` tagline. That tagline is just a one-line focus marker, present when a root session has a folder — `halo cli` / `halo tui --dir`, a cron job's folder — other than the workspace root.)
|
|
167
167
|
|
|
168
168
|
The roster is computed once (`canDelegate(yaml)` → `buildAgentRoster(agentId, team)`, else `''`, where `canDelegate` is "not internal AND a non-empty `team`") and handed to `composeMdPrompt`, so it lands inside `mdPrompt` behind AGENT.md. Only when there's no AGENT.md at all (the fallback branch) is a non-empty roster appended at the tail instead — there's no MD layer to slot it behind.
|
|
169
169
|
|
|
@@ -194,6 +194,8 @@ If `mdPrompt` is empty:
|
|
|
194
194
|
- Use `yamlConfig.system_prompt` if present
|
|
195
195
|
- Otherwise use the hard-coded default string
|
|
196
196
|
|
|
197
|
+
This branch carries no `working_dir`: no `Working directory:` line, root or sub-agent (`--dir` / cron folder included). The folder's INSTRUCTIONS.md chain is part of `mdPrompt`, so the branch is only reached when that chain is empty too — an agent with a custom `system_prompt` and no AGENT.md whose folder *does* have a chain gets a non-empty `mdPrompt` and the normal branch above, where `system_prompt` is not used.
|
|
198
|
+
|
|
197
199
|
## Agent roster
|
|
198
200
|
|
|
199
201
|
`buildAgentRoster(selfAgentId, team)` ([session-agent-builder.ts](../../../packages/server/src/agents/session-agent-builder.ts)) builds a live team block listing the agents this session can delegate to — one `- \`<id>\` — <name>: <description>` line per teammate.
|
|
@@ -287,6 +289,7 @@ Shared core: [`loadScopeBody(workspaceRoot, relDir)`](../../../packages/server/s
|
|
|
287
289
|
| Trigger | Who | Lifetime | How it reaches the model |
|
|
288
290
|
|---|---|---|---|
|
|
289
291
|
| `working_dir` on `start_session` | parent agent | **persistent** — every turn, for the session's life | `composeMdPrompt` folds the **bare `loadScopeBody` markdown** into the `## User Instructions` region, right after the global/ws-root layer ([`session-agent-builder.composeSystemPrompt`](../../../packages/server/src/agents/session-agent-builder.ts)). No `<workspace-instructions>` wrapper — the section heading already frames it. `working_dir` is stored on `agent_sessions.working_dir` and restored on resume, so the rules are rebuilt into the system prompt on every `buildAgentInstance`. |
|
|
292
|
+
| `--dir` on `halo cli` / `halo tui`, a cron job's folder | user / cron runner | **persistent**, same as above | The root session's `agent_sessions.working_dir`, set on create and re-pointed in place on resume (`SessionManager.setWorkingDir`); same `composeMdPrompt` path. |
|
|
290
293
|
| `@scope <dir>` in a message | user (TUI / admin / channels) | **one-shot** — only the turn it rides on | `loadScopeInstructions` wraps the body in a self-describing `<workspace-instructions dir=... note=...>` block (it lands in the message stream, so it needs the tag + note to read as turn context); `SessionManager.expandScopeMarkers` strips the marker from the visible text and prepends the block to that message. Not re-injected on later turns. |
|
|
291
294
|
|
|
292
295
|
Why the split: `working_dir` is the sub-agent's persistent identity (it lives in that directory for its whole life), so its rules belong in the system prompt where they're present every turn — a first-turn-only injection would let the agent forget them after a few turns. `@scope` is ad-hoc context a user attaches to one message, so it stays loop-scoped. (The earlier `scope` args on `query_session` / `interrupt_session` were removed — they let a parent temporarily swap a child's directory rules, which contradicts working_dir being persistent identity.)
|
|
@@ -383,6 +386,8 @@ Schema declared by each package + values stored centrally is the same model VSCo
|
|
|
383
386
|
## working_dir behaviour
|
|
384
387
|
|
|
385
388
|
- `start_session` accepts a `working_dir` argument (absolute or workspace-relative)
|
|
389
|
+
- A root session takes one too: `halo cli` / `halo tui --dir <path>` (a cron job's folder rides on it), same checks; on a resumed session it re-points the stored value via `SessionManager.setWorkingDir` (`--dir .` = back to the root), refused while the session is running
|
|
390
|
+
- `/session new` (server-side `execNew`) gives the new session the folder of the issuing session's root, re-validated (gone → root)
|
|
386
391
|
- Default: `workspaceRoot`
|
|
387
392
|
- Enforced inside workspace (path traversal is rejected)
|
|
388
393
|
- Non-existent directory → spawn refused
|
|
@@ -394,7 +399,7 @@ Schema declared by each package + values stored centrally is the same model VSCo
|
|
|
394
399
|
|
|
395
400
|
- **Override precedence**: workspace > global (USER.md, AGENT.md, INSTRUCTIONS.md, prompts/)
|
|
396
401
|
- **agents/ & skills/ override**: whole-folder. `<ws>/.halo/agents/<id>/` (or `skills/<id>/`) exists → serves that agent/skill entirely (agent.yaml + AGENT.md, or SKILL.md + resources), global folder ignored, no per-file fallback
|
|
397
|
-
- **INSTRUCTIONS.md**: workspace root INSTRUCTIONS.md suppresses global; subdirectory chain stacks on top — in the system prompt every turn when it's a
|
|
402
|
+
- **INSTRUCTIONS.md**: workspace root INSTRUCTIONS.md suppresses global; subdirectory chain stacks on top — in the system prompt every turn when it's a session's `working_dir` (persistent; sub-agent, or a root's `--dir`), or per-turn in the message via user `@scope` (one-shot)
|
|
398
403
|
- **prompts/ override**: `<ws>/.halo/prompts/<scope>/` directory exists → entirely replaces global's same scope (directory-level, no per-file merge)
|
|
399
404
|
- **INDEX.md is project-root only**: subdirectory INDEX files are not auto-loaded
|
|
400
405
|
- **Root vs sub-agent**: determined by `!parentId`
|
|
@@ -61,7 +61,7 @@ No quiet gate (the session is by definition not done) and **`reply_to` is kept**
|
|
|
61
61
|
|
|
62
62
|
## What it deliberately doesn't do
|
|
63
63
|
|
|
64
|
-
- **No cross-server reach.** Same `SessionManagerRegistry`, same process. Different server →
|
|
64
|
+
- **No cross-server reach.** Same `SessionManagerRegistry`, same process. Different server → [A2A](a2a.md), which reuses the same `reply_to` column (`{ a2a: taskId }`), quiet gate and interim doors.
|
|
65
65
|
- **No session discovery beyond a flat root list.** `relay_list` returns a workspace's root sessions (id / agent / title / status, newest 100) so the secretary can reuse an existing conversation; it doesn't walk trees or search transcripts — that's the admin's job.
|
|
66
66
|
- **No broadcast / fan-out helper.** The secretary calls `relay_send` once per department; the reports come back individually with the workspace + session id in the header, so it can tell them apart without extra bookkeeping.
|
|
67
67
|
- **No ACL beyond full access.** A full-access secretary can reach every workspace on the box — that's the deployment's trust boundary already (it can `shell_exec` into them anyway).
|
|
@@ -83,6 +83,7 @@ Manages every agent session's lifecycle (root + sub-agent). Each session is 1:1
|
|
|
83
83
|
| Method | Purpose |
|
|
84
84
|
|---|---|
|
|
85
85
|
| `createSession(agentId, parentId, description, agentName?, explicitId?, workingDir?, accessLevel?, title?)` | Create a session (SQLite + memory). `workingDir` = absolute path at runtime, stored as workspace-relative in DB; null = project root. `accessLevel` = `'readonly'`, `'workspace'`, or `null` (full). |
|
|
86
|
+
| `setWorkingDir(sessionId, workingDir)` | Re-point a session's persisted `working_dir` (`--dir` on a resumed root session). `workingDir` absolute or null = root; returns `{from, to}` (workspace-relative) or null when it's already the target (no write). Refuses while the session is running; an idle in-memory copy is released so the next `ensureSession` rebuilds the prompt with the new folder |
|
|
86
87
|
| `sendUserMessage(sessionId, text, images?, accessLevel?)` | Send a message — run immediately if idle, queue if busy; returns `'running'` or `'queued'` |
|
|
87
88
|
| `compactSession(sessionId, opts?)` | LLM-summary compact (`opts.onProgress` for UI progress) |
|
|
88
89
|
| `interruptSession(sessionId)` | Abort the in-flight turn now (fire-and-forget); `interruptRequested` is set so the unwind repairs rather than errors, then the queued message drains. Shared by esc and the `interrupt_session` tool |
|
|
@@ -182,6 +183,8 @@ The orphan reconcile above fixes the *children*'s `stoppedAt`; nobody told the *
|
|
|
182
183
|
|
|
183
184
|
**Eager boot** (`index.ts`): rather than waiting for someone to open a workspace, right after constructing `SessionManagerRegistry` (before `bootChannels`) the server calls `listRunningWorkspaces()` and for each workspace with leftover rows whose `.halo` dir still exists: `claimWorkspaceRuntime(ws)` first — if another live server owns it, `continue` (rows stay for a later boot, and no SessionManager is built, since caching a non-owner SM would freeze "not owner" for the whole process lifetime) — otherwise `registry.getOrCreate(ws)`, which fires the same constructor chain (claim → reconcile → retained `sweepActiveGoals` → `sweepInterruptedRuns`). An interrupted root with nobody around is exactly the one that must nudge itself without a human opening the tab first.
|
|
184
185
|
|
|
186
|
+
**AgentCore A2A mode** (`a2a/agentcore.ts`): the ledger is `<HALO_WORKSPACE>/.halo/runs.db` on the EFS mount instead of `~/.halo/global/runs.db` (a microVM's home dies with it), opened only once the workspace lease is held; the sweep runs on lease acquisition rather than at boot, and the first `POST /` waits for its nudges to start (`nudgesSettled`) before it is served — see [agentcore.md](agentcore.md). That file joins the workspace-relative hidden list (`WORKSPACE_HIDDEN_FILES`).
|
|
187
|
+
|
|
185
188
|
`runs.db`/`-wal`/`-shm` join the built-in sandbox hidden-files list (`DEFAULT_HIDDEN_FILES`, always included; `general.sandbox.hidden_files` only adds to it) alongside `evo.db` / `cron.db` (see [storage.md](storage.md)). Known gap carried from the design doc: `<ISO>` in the nudge is sweep time, not the actual interruption time — for a workspace that sat locked by another server before this boot could claim it, that's a real but accepted drift.
|
|
186
189
|
|
|
187
190
|
### Message queue and drain
|
|
@@ -190,7 +193,7 @@ The orphan reconcile above fixes the *children*'s `stoppedAt`; nobody told the *
|
|
|
190
193
|
|
|
191
194
|
A `QueuedMessage` is `{ text, sourceSessionId?, images? }`: `sourceSessionId` is set for agent→agent entries (drives the `(from: session X)` prefix and the sibling-status suffix) and absent for user messages; `images` is the user multimodal payload (agent entries have none).
|
|
192
195
|
|
|
193
|
-
**runSession loop** (`runSession(sessionId, message)`, `message: string | ContentBlock[]`): runs the opening turn when there is one, then drains. An empty **string** message means "the work is already in `messageQueue`" (the `querySession` idle path) — it skips the opening turn and goes straight to drain. The per-turn reset (fresh `toolCallLog` / `warnedToolHashes`, `interruptRequested = false`) runs before the opening turn; `drainQueue` repeats the same reset before each merged batch so a stale interrupt flag never leaks across turns. Server-only: right before `session.promise = runFn()` it inserts the session id into the run ledger, and in the `finally` (right after `session.promise = null`, before emitting `complete`) it deletes it — see [Run ledger](#run-ledger--restart-nudge-for-interrupted-roots-
|
|
196
|
+
**runSession loop** (`runSession(sessionId, message)`, `message: string | ContentBlock[]`): runs the opening turn when there is one, then drains. An empty **string** message means "the work is already in `messageQueue`" (the `querySession` idle path) — it skips the opening turn and goes straight to drain. The per-turn reset (fresh `toolCallLog` / `warnedToolHashes`, `interruptRequested = false`) runs before the opening turn; `drainQueue` repeats the same reset before each merged batch so a stale interrupt flag never leaks across turns. Server-only: right before `session.promise = runFn()` it inserts the session id into the run ledger, and in the `finally` (right after `session.promise = null`, before emitting `complete`) it deletes it — see [Run ledger](#run-ledger--restart-nudge-for-interrupted-roots-haloglobalrunsdb) above.
|
|
194
197
|
|
|
195
198
|
**Observability.** `runAgentTurn` is wrapped by `beginTurn` / `endTurn` from `observability/genai-spans.ts` — one `invoke_agent` span per turn, with `chat` / `execute_tool` child spans created from the `onAgentEvent` hook inside the event loop; see [design/observability.md](observability.md).
|
|
196
199
|
|
|
@@ -280,7 +283,7 @@ Table `agent_sessions` holds metadata only (no runtime state):
|
|
|
280
283
|
| agent_id | TEXT | Agent YAML ID |
|
|
281
284
|
| agent_name | TEXT | Display name |
|
|
282
285
|
| description | TEXT | Task description |
|
|
283
|
-
| working_dir | TEXT | Workspace-relative path; null = project root |
|
|
286
|
+
| working_dir | TEXT | Workspace-relative path; null = project root. A sub-agent's `working_dir` from `start_session`, or a root session's folder (`halo cli` / `halo tui --dir`, a cron job's folder) — set on create, re-pointed in place on resume by `setWorkingDir` |
|
|
284
287
|
| access_level | TEXT | `'readonly'`, `'workspace'`, or null; null = full access |
|
|
285
288
|
| created_at / updated_at | INTEGER | Epoch ms |
|
|
286
289
|
| stopped_at | INTEGER | null = active |
|
|
@@ -349,6 +352,8 @@ In the admin, `/session new` (also `/clear`, "+ New Session") does not touch the
|
|
|
349
352
|
|
|
350
353
|
Before 1.5.3-alpha this was the WS `session:clear` frame (save, release the listener, unbind the connection); it was removed when one connection started carrying every open tab. Other channels' `/session new` still creates the session server-side and returns `switchTo`.
|
|
351
354
|
|
|
355
|
+
That server-side `execNew` (`channels/shared/commands.ts`) keeps the folder: the new session inherits the `working_dir` of the **root** of the session the command was issued in — `ctx.sessionId` when the caller passes it (the TUI harness does, so a sub-agent tab resolves to its root), else the channel's active session (`findActiveSessionId`). The folder is re-validated with `resolveWorkingDir`; one that no longer exists starts the new session at the workspace root, with a `[Commands] /new: folder of <root> not kept, …` log line.
|
|
356
|
+
|
|
352
357
|
See [background-dispatch.md](background-dispatch.md).
|
|
353
358
|
|
|
354
359
|
## WS disconnect resilience
|
|
@@ -26,8 +26,9 @@ Defines the persisted-data format for every Halo surface. Format changes must re
|
|
|
26
26
|
│ │ # — global so they don't pollute any workspace's halo.db
|
|
27
27
|
│ │ └── <sessionId>.json
|
|
28
28
|
│ ├── evo.db # Cross-workspace evolution queue (evolution_runs + evolution_applies)
|
|
29
|
-
│ ├── cron.db # Cross-workspace cron jobs + run history (cron_jobs + cron_runs)
|
|
29
|
+
│ ├── cron.db # Cross-workspace cron jobs + run history (cron_jobs + cron_runs; schema in design/cron.md, migrations below)
|
|
30
30
|
│ ├── runs.db # Run ledger: (workspace, session_id) rows for server-driven runs in flight; steady state empty (running_sessions)
|
|
31
|
+
│ ├── a2a.db # A2A: inbound tasks, push configs + outbox, outbound dispatches (see design/a2a.md)
|
|
31
32
|
│ ├── extensions/<id>/ # Installed Canvas preview extensions (halo-extension.json + bundle)
|
|
32
33
|
│ ├── logs/ # Runtime logs (server.log)
|
|
33
34
|
│ │ ├── evo/ # Per-evo-run wrapper logs
|
|
@@ -37,6 +38,7 @@ Defines the persisted-data format for every Halo surface. Format changes must re
|
|
|
37
38
|
├── secrets/ # Sensitive files (not mounted into sandbox)
|
|
38
39
|
│ ├── settings.yaml # Global settings (API keys, credentials)
|
|
39
40
|
│ ├── config.yaml # System config (admin password, server settings)
|
|
41
|
+
│ ├── a2a-remotes.yaml # A2A remote list (server-wide, full-only; see design/a2a.md)
|
|
40
42
|
│ └── channels/
|
|
41
43
|
│ └── channels.db # Channel accounts DB
|
|
42
44
|
|
|
@@ -66,6 +68,9 @@ Defines the persisted-data format for every Halo surface. Format changes must re
|
|
|
66
68
|
├── assets/<channel>/inbound/<accountId>/<date>/ # Inbound media per channel (image/voice/video/file)
|
|
67
69
|
├── runtime.lock # Workspace runtime ownership marker (pid) — see below
|
|
68
70
|
├── halo.db # Per-workspace sqlite (sessions metadata, disabled-items)
|
|
71
|
+
├── a2a.db # AgentCore mode only: A2A tasks + push outbox (instead of ~/.halo/global/a2a.db) — design/agentcore.md
|
|
72
|
+
├── runs.db # AgentCore mode only: the run ledger (instead of ~/.halo/global/runs.db)
|
|
73
|
+
├── agentcore.lease # AgentCore mode only: single-writer heartbeat lease file
|
|
69
74
|
└── docs/ # Project docs (requirements/design/dev/test/plans)
|
|
70
75
|
```
|
|
71
76
|
|
|
@@ -402,8 +407,10 @@ A value of the form `<<ENV_NAME>>` is replaced at read time with `process.env.EN
|
|
|
402
407
|
| `general.limits.tool_result_ui_chars` | 65536 | Per-tool-result cap on the content **stored for UI display** (admin/web chat panel). Far larger than the LLM cap so a normal command's full output stays visible, but bounded so a multi-MB `cat` can't bloat the session file / WS payload / browser render; excess truncated with a marker pointing at `file_read` |
|
|
403
408
|
| `general.limits.terminal_scrollback_bytes` | 50000 | Off-screen scrollback bytes retained per detached persistent terminal |
|
|
404
409
|
| `general.sandbox.hidden_dirs` | `''` | Extra dirs hidden from workspace/readonly sessions — bwrap tmpfs overlay (Linux) / Seatbelt deny (macOS). **Appended** to the built-in `DEFAULT_HIDDEN_DIRS` (`tools/sandbox.ts`: `~/.halo/secrets,~/.aws,~/.ssh,~/.gnupg,~/.docker,~/.config/gh,~/.halo/global/internal-sessions,~/.halo/global/logs`), which are always included. Full sessions, the admin file explorer and the terminal are not affected |
|
|
405
|
-
| `general.sandbox.hidden_files` | `''` | Extra files hidden from workspace/readonly sessions — bwrap bind of the empty `~/.halo/.sandbox-empty`, reads as empty (Linux) / Seatbelt deny (macOS). **Appended** to the built-in `DEFAULT_HIDDEN_FILES` (`~/.npmrc,~/.bash_history,~/.gitconfig,~/.git-credentials,~/.netrc,~/.halo/global/{evo,cron,runs}.db` + `-wal` / `-shm`), which are always included. Full sessions, the admin file explorer and the terminal are not affected |
|
|
410
|
+
| `general.sandbox.hidden_files` | `''` | Extra files hidden from workspace/readonly sessions — bwrap bind of the empty `~/.halo/.sandbox-empty`, reads as empty (Linux) / Seatbelt deny (macOS). **Appended** to the built-in `DEFAULT_HIDDEN_FILES` (`~/.npmrc,~/.bash_history,~/.gitconfig,~/.git-credentials,~/.netrc,~/.halo/global/{evo,cron,runs,a2a}.db` + `-wal` / `-shm`), which are always included. Full sessions, the admin file explorer and the terminal are not affected |
|
|
406
411
|
| `general.logging.level` | warn | `debug` / `info` / `warn` / `error`. Restart required |
|
|
412
|
+
| `general.a2a.public_url` | `''` | Global-only. Origin peers use to reach this server for A2A (no trailing slash): the agent card's interface URL and the default push-webhook base for `a2a_send`. Empty = derived from the request host (no outbound dispatch without it or a remote's `push_base`). Env `HALO_A2A_PUBLIC_URL` overrides. See [a2a.md](a2a.md) |
|
|
413
|
+
| `general.a2a.url_allowlist` | `100.64.0.0/10,*.ts.net` | Global-only. CIDRs / host patterns A2A egress may reach besides public https (push webhooks, remote cards and RPC, inbound image URLs). Private / loopback only when listed; plain http only to listed hosts / ranges; link-local always refused. See [a2a.md](a2a.md#push-delivery-and-url-policy) |
|
|
407
414
|
| `general.observability.endpoint` | `''` | OTLP base URL of an OpenTelemetry collector (e.g. `http://localhost:4318`); empty = off. Restart required. See [observability.md](observability.md) |
|
|
408
415
|
| `general.observability.service_name` | `halo` | OTel resource `service.name`. Restart required |
|
|
409
416
|
| `general.observability.headers` | `''` | **Secret.** Extra OTLP request headers, comma-separated `k=v`. Restart required |
|
|
@@ -427,6 +434,8 @@ Paths: `~/.halo/global/models/<providerId>.yaml` (bundled, seeded from `template
|
|
|
427
434
|
|
|
428
435
|
**Merge rule** (one implementation, `packages/server/src/models/registry.ts` `loadProviders`, behind the server registry, the Settings secrets forms, `halo setup` and the hub installer): per provider `id` the copy with the higher `revision` wins; on a tie the `models.d` copy wins; a yaml without `revision` counts as 0 (users' own custom yamls in `models/`). A `models.d` copy whose `runtime:` this build doesn't know is skipped with a `[Config]` log, so the bundled copy stays in effect. `halo setup` merges the package's `templates/models/` with `models.d/` the same way.
|
|
429
436
|
|
|
437
|
+
**Hub install** (`halo models install <dir> [--yes]`, `models/install.ts`): plan, then write `models.d/<id>.yaml` via tmp + rename. A yaml is refused (others still go ahead) for an unknown `runtime:` ("needs a newer halo"), a missing / non-integer `revision`, a `secrets[].default` other than empty / `<<ENV_NAME>>`, or bad model fields — `contextWindow` / `maxContextWindow` not positive integers, `contextWindow` > `maxContextWindow`, `compressAt` outside (0, 1], `endpoints` not a list of `https://` URLs (reason shaped `models.<id>.<field> …`). An older revision than the copy in effect is skipped, an identical one is up to date. **Consent**: the whole run writes nothing and needs `--yes` (exit 3) when an installed provider is new, its `defaultEndpoint` / `endpointPresets` change, or a model's `endpoints:` gains a URL the provider didn't use anywhere before (not in its old `defaultEndpoint`, presets or any model's list). Removed URLs, and added ones the provider already used, are listed but need no consent.
|
|
438
|
+
|
|
430
439
|
**Live reload**: `models/watcher.ts` watches `models.d/` (non-recursive, 300 ms debounce; created at boot), drops the registry cache and broadcasts `models:changed` when the merged view changed; admin pages re-fetch `/agent-configs/models` and the settings schema. `models/` isn't watched — only template refresh writes it, before the registry is first read.
|
|
431
440
|
|
|
432
441
|
```yaml
|
|
@@ -438,11 +447,11 @@ displayName: AWS Bedrock Claude (Invoke API) # shown in UI
|
|
|
438
447
|
displayName_zh: AWS Bedrock Claude(Invoke API) # optional zh override
|
|
439
448
|
description: Invokes Bedrock via InvokeModel (non-streaming)
|
|
440
449
|
description_zh: 通过 Bedrock InvokeModel(非流式)调用
|
|
441
|
-
defaultEndpoint: https://bedrock-runtime.us-
|
|
450
|
+
defaultEndpoint: https://bedrock-runtime.us-east-1.amazonaws.com
|
|
442
451
|
defaultModelId: global.anthropic.claude-sonnet-4-6 # picked when user switches to this provider
|
|
443
|
-
endpointPresets:
|
|
444
|
-
- https://bedrock-runtime.us-west-2.amazonaws.com
|
|
452
|
+
endpointPresets: # endpoint dropdown for models without their own `endpoints:`
|
|
445
453
|
- https://bedrock-runtime.us-east-1.amazonaws.com
|
|
454
|
+
- https://bedrock-runtime.us-west-2.amazonaws.com
|
|
446
455
|
|
|
447
456
|
# Server-side credentials. Stored at <provider-id>.secrets.<key> in
|
|
448
457
|
# settings.yaml; never injected into agent prompts or shell_exec.
|
|
@@ -459,7 +468,11 @@ secrets:
|
|
|
459
468
|
models:
|
|
460
469
|
- id: global.anthropic.claude-sonnet-4-6
|
|
461
470
|
displayName: Claude Sonnet 4.6
|
|
471
|
+
# endpoints: [https://…us-east-1…] # optional — only when the model is served on a subset; narrows endpointPresets
|
|
462
472
|
maxOutputTokens: 64000
|
|
473
|
+
contextWindow: 272000 # recommended window for a new agent (≤ 272000, ≤ maxContextWindow)
|
|
474
|
+
maxContextWindow: 1000000 # the model's true limit; absent → contextWindow
|
|
475
|
+
compressAt: 0.9 # compaction ratio for a new agent, (0, 1]
|
|
463
476
|
capabilities:
|
|
464
477
|
image: true # supports image input
|
|
465
478
|
video: false
|
|
@@ -479,18 +492,33 @@ models:
|
|
|
479
492
|
- { value: medium, label: Medium }
|
|
480
493
|
- { value: high, label: High }
|
|
481
494
|
- { value: max, label: Max }
|
|
495
|
+
verbosity: # Responses-API providers only (bedrock-mantle runtime)
|
|
496
|
+
default: medium
|
|
497
|
+
presets:
|
|
498
|
+
- { value: low, label: Low }
|
|
499
|
+
- { value: medium, label: Medium }
|
|
500
|
+
- { value: high, label: High }
|
|
482
501
|
```
|
|
483
502
|
|
|
503
|
+
**Per-model `endpoints:`** — `resolveModelEndpoints(provider, model)` (`models/model-endpoints.ts`; the admin form keeps a byte-identical copy in `features/agents/model-endpoints.ts`, pinned by `test/model-endpoints.test.ts`): a non-empty list is authoritative for that model — it replaces `endpointPresets` in the dropdown, and the model's default endpoint is `defaultEndpoint` when the list contains it, else the list's first entry. No list → provider presets and `defaultEndpoint`. Used by scaffolding / `halo setup` (`modelBlockFromProvider`) and the agent form. Runtime never falls back: a session needs `model.endpoint` in agent.yaml.
|
|
504
|
+
|
|
505
|
+
**Context fields** (`resolveModelContext` / `resolveEffectiveContext`, `config.ts`): `contextWindow` is the *recommended* window, kept ≤ 272000 even for 1M-class models (input past 272K bills at long-context rates on most of them); `maxContextWindow` is the true limit (absent → `contextWindow`); `compressAt` is the model's compaction ratio. At runtime a session's budget is agent.yaml `context.maxTokens` / `compressAt`, else the global defaults — `config.model.maxContextTokens` (272000) capped at the model's max, and `general.compact.compress_at` (0.9) — then `maxTokens` is clamped to the model's max (warned, `[SessionAgentBuilder] … clamped`). Same resolution for live sessions (`buildModelRuntime`) and cold ones (`getContextConfig`). `PUT /agent-configs/:id/yaml` refuses a new `context.maxTokens` above the model's max (400); an over-max value already on disk for the same model stays editable.
|
|
506
|
+
|
|
507
|
+
**Scaffold-only defaults** — the registry's recommended `contextWindow` / `compressAt` and `capabilities.verbosity.default` are written into agent.yaml when an agent is created (new-agent scaffold, `halo setup` rebind, the form's model / provider switch); runtime never reads them for an existing agent, so a registry update can't move a running agent. A new agent's `context:` = `contextWindow` (else the runtime default) and compressAt = explicitly set global `compact.compress_at` > model `compressAt` > 0.9 (`explicitCompressAt()`, `agent-configs.ts` `scaffoldContext`; settings.yaml only holds keys a user wrote, so absent = never set). Verbosity at runtime comes only from agent.yaml `model.verbosity`; absent → MantleAgent sends `low`.
|
|
508
|
+
|
|
484
509
|
### Capability defaults — what each `default*` field controls
|
|
485
510
|
|
|
486
511
|
| Path | Used when | Fallback |
|
|
487
512
|
|---|---|---|
|
|
488
513
|
| `<provider>.defaultModelId` | User switches provider; form picks this model | `models[0].id` |
|
|
489
|
-
| `<provider>.defaultEndpoint` | Same; endpoint input | `''` |
|
|
514
|
+
| `<provider>.defaultEndpoint` | Same; endpoint input (narrowed by the model's `endpoints:`, see above) | `''` |
|
|
515
|
+
| `models[].contextWindow` | New agent / model switch with an untouched or over-max window → `context.maxTokens` | runtime default (272000 capped at max) |
|
|
516
|
+
| `models[].compressAt` | New agent (unless the global `compress_at` is set) / model switch with an untouched ratio → `context.compressAt` | global `compress_at`, 0.9 |
|
|
490
517
|
| `capabilities.promptCaching.default` | Toggle on / provider switch into a caching-capable model | `ttlPresets[0].value` |
|
|
491
518
|
| `capabilities.thinking.defaultEnabled` | New agent or provider switch with no prior thinking config | `false` |
|
|
492
519
|
| `capabilities.thinking.default` | Thinking toggled on, adaptive mode | `effortPresets[0].value` |
|
|
493
520
|
| `capabilities.thinking.defaultBudgetTokens` | Thinking toggled on, manual mode | `8192` |
|
|
521
|
+
| `capabilities.verbosity.default` | New agent / provider or model switch with no valid prior value → `model.verbosity` | none written; runtime `low` |
|
|
494
522
|
|
|
495
523
|
Existing agent yaml values always win over these defaults — switching providers carries the user's effort/ttl over when valid for the new model. The defaults only fire when there's nothing to carry, or the carry-over isn't valid in the new vocabulary (e.g. Bedrock `medium` → Kimi `enabled/disabled` mismatch resets to the new provider's `default`).
|
|
496
524
|
|
|
@@ -558,7 +586,7 @@ interface SkillMeta {
|
|
|
558
586
|
}
|
|
559
587
|
```
|
|
560
588
|
|
|
561
|
-
Load precedence
|
|
589
|
+
Load precedence: workspace > global > extension, **whole-folder override** (the agent rule, plus a third, read-only source: `~/.halo/global/extensions/<extId>/skills/<id>/`, see [canvas-extensions.md → Extension skills](canvas-extensions.md#extension-skills)). A higher `skills/<id>/` folder replaces the lower skill wholesale — SKILL.md plus every sibling resource file — with no per-file fallback.
|
|
562
590
|
|
|
563
591
|
## SQLite databases
|
|
564
592
|
|
|
@@ -600,7 +628,7 @@ The last four are a **read cache for the session list** — `GET /api/sessions/l
|
|
|
600
628
|
|---|---|---|
|
|
601
629
|
| item_type | TEXT NOT NULL | `'agent'` or `'skill'` |
|
|
602
630
|
| item_id | TEXT NOT NULL | Agent/skill ID |
|
|
603
|
-
| scope | TEXT NOT NULL | `'global'` or `'
|
|
631
|
+
| scope | TEXT NOT NULL | `'global'`, `'workspace'` or `'extension'` (a skill shipped by an installed extension, `agents/skill-sources.ts`) — which source the item comes from. `'extension'` added by `HALO_MIGRATIONS` v2 (table rebuild, rows kept) |
|
|
604
632
|
| disabled_at | INTEGER NOT NULL | Unix ms |
|
|
605
633
|
| | | **PK**: `(item_type, item_id, scope)` |
|
|
606
634
|
|
|
@@ -634,8 +662,9 @@ All channel types (telegram, web, wechat, slack, feishu, wecom) share one table.
|
|
|
634
662
|
### Schema change rules
|
|
635
663
|
|
|
636
664
|
- `schema.sql` / each db's `CREATE_SQL` always describes the FULL current shape, so a fresh db is complete after the CREATEs alone
|
|
637
|
-
- Every change to an already-existing db gets a numbered slot in that file's ordered migration list — `HALO_MIGRATIONS` (`db/index.ts`), `CRON_MIGRATIONS` (`db/cron-db.ts`); `channel-db.ts` / `evo-db.ts` / `runs-db.ts` currently pass an empty list. `runMigrations(sqlite, list)` (`db/migrate.ts`) runs `list[user_version..]` in order, each in its own transaction, stamping `PRAGMA user_version = i + 1` after each
|
|
638
|
-
- A fresh db is also at `user_version 0` and runs the whole list, so every slot must be a no-op against the current shape (`addColumnIfMissing`, `CREATE … IF NOT EXISTS`)
|
|
665
|
+
- Every change to an already-existing db gets a numbered slot in that file's ordered migration list — `HALO_MIGRATIONS` (`db/index.ts`), `CRON_MIGRATIONS` (`db/cron-db.ts`), `A2A_MIGRATIONS` (`db/a2a-db.ts`); `channel-db.ts` / `evo-db.ts` / `runs-db.ts` currently pass an empty list. `runMigrations(sqlite, list)` (`db/migrate.ts`) runs `list[user_version..]` in order, each in its own transaction, stamping `PRAGMA user_version = i + 1` after each
|
|
666
|
+
- A fresh db is also at `user_version 0` and runs the whole list, so every slot must be a no-op against the current shape (`addColumnIfMissing`, `CREATE … IF NOT EXISTS`). A CHECK constraint can't be altered in SQLite: the slot rebuilds the table (create `_v2`, copy rows, drop, rename) and skips when `sqlite_master` already shows the new constraint — `HALO_MIGRATIONS` v2 (`disabled_items.scope` admits `'extension'`) is the pattern
|
|
667
|
+
- Current slots worth knowing: `HALO_MIGRATIONS` v2 — the `disabled_items` rebuild above (rows kept). `CRON_MIGRATIONS` v5 — `cron_jobs.working_dir TEXT NULL` (job folder → `halo cli --dir`), `cron_jobs.stateless INTEGER NOT NULL DEFAULT 0` (1 = fresh session every run), `cron_runs.session_id TEXT NULL` (the session that run used); column semantics in [cron.md](cron.md)
|
|
639
668
|
- Append only — never reorder or edit a shipped slot. A db opened by a newer halo (user_version > list length) logs a warning and continues
|
|
640
669
|
- Do not drop or rename existing columns
|
|
641
670
|
- New channel types: add rows with a new `channel_type` value + define config shape in the channel's `accounts.ts` adapter
|
|
@@ -653,3 +682,4 @@ All channel types (telegram, web, wechat, slack, feishu, wecom) share one table.
|
|
|
653
682
|
| v1+ | 2026-08-08 | Session file slimming (no version bump — additive + reader-compatible): assistant `toolCalls` no longer written (`contentBlocks` is the single copy, `toolCalls` stays read-only legacy); files written as compact JSON; UI-log archiving adds header fields `archiveCount` / `archivedUserCount` plus `{sessionId}.arch.{N}.json.gz` segment files; `agent_sessions` gains mirrored `title` / `exchange_count` / `context_tokens` / `total_output_tokens` columns so listing doesn't parse every file. |
|
|
654
683
|
| v1+ | 2026-09-18 | Relay + liveness (additive, no version bump): `agent_sessions.reply_to` column (idempotent ALTER); session file gains `lastActivityAt` beside `output`, read back by `get_session_output` for released sessions. |
|
|
655
684
|
| v1+ | 2026-09-20 | Schema migrations moved from ad-hoc boot-time ALTERs to `PRAGMA user_version` + ordered per-db migration lists (`db/migrate.ts`); existing dbs pick up at slot 0 and every slot is idempotent. Session-message + WS frame types moved to `@turmind/halo-core/protocol` (re-exported from their old server paths; no wire change). |
|
|
685
|
+
| v1+ | 2026-10-10 | Model registry (additive): per-model `endpoints:`, `maxContextWindow`, `compressAt`; recommended `contextWindow` ≤ 272000; registry context / compressAt / verbosity defaults are scaffold-only; installer validates model fields and asks consent for URLs new to a provider. `HALO_MIGRATIONS` v2: `disabled_items.scope` admits `'extension'`. `CRON_MIGRATIONS` v5: `cron_jobs.working_dir`, `cron_jobs.stateless`, `cron_runs.session_id`. |
|
|
@@ -10,13 +10,12 @@ Browser-based access to Halo via token-authenticated HTTP API. Supports SSE stre
|
|
|
10
10
|
Halo server (9527) ──┤── channels/telegram/ ├── SessionManager
|
|
11
11
|
└── channels/web/ ─┘ (per workspace, via Registry)
|
|
12
12
|
↕ HTTP + SSE
|
|
13
|
-
|
|
14
|
-
or any HTTP client
|
|
13
|
+
any HTTP client (custom frontend, halo-city, ACP adapter)
|
|
15
14
|
```
|
|
16
15
|
|
|
17
16
|
Common slash commands (`/help`, `/evo`, and the object commands `/session`, `/agent`, `/skill`, `/workspace`) live in `channels/shared/commands.ts`; each channel handler is a thin adapter.
|
|
18
17
|
|
|
19
|
-
Web channel is a public HTTP API on Halo.
|
|
18
|
+
Web channel is a public HTTP API on Halo.
|
|
20
19
|
|
|
21
20
|
## Data model
|
|
22
21
|
|
|
@@ -63,7 +62,7 @@ File: `packages/server/src/routes/web.ts`
|
|
|
63
62
|
|
|
64
63
|
All public endpoints require `x-token` header (or `?token=` query param) with a valid account token. No cookie auth needed.
|
|
65
64
|
|
|
66
|
-
> These paths are listed in `PUBLIC_PATHS` (`middleware/auth.ts`) so they bypass the admin cookie gate: `/api/web/chat`, `/api/web/sessions` (POST mint + GET list), `/api/web/stop`, `/api/web/history`, `/api/web/subscribe`, `/api/web/file`, and `/api/show/state` (the [halo-city](../../../halo-city/) snapshot — see [dev/api.md](../dev/api.md#show-world-snapshot)). The server CORS allowlist includes the `x-token` header, so browser-based custom frontends (
|
|
65
|
+
> These paths are listed in `PUBLIC_PATHS` (`middleware/auth.ts`) so they bypass the admin cookie gate: `/api/web/chat`, `/api/web/sessions` (POST mint + GET list), `/api/web/stop`, `/api/web/history`, `/api/web/subscribe`, `/api/web/file`, and `/api/show/state` (the [halo-city](../../../halo-city/) snapshot — see [dev/api.md](../dev/api.md#show-world-snapshot)). The server CORS allowlist includes the `x-token` header, so browser-based custom frontends (e.g. halo-city) can call these cross-origin.
|
|
67
66
|
|
|
68
67
|
### POST `/api/web/chat`
|
|
69
68
|
|
|
@@ -96,7 +95,7 @@ data: {"type":"error","error":"..."}
|
|
|
96
95
|
|
|
97
96
|
#### SSE keepalive
|
|
98
97
|
|
|
99
|
-
`/web/chat` and `/web/subscribe` write an SSE comment line `: keepalive` every 15 s (`SSE_KEEPALIVE_MS` / `streamWithKeepalive` in `routes/web.ts`; the timer is cleared when the stream ends). A long tool call can otherwise leave the stream silent long enough for an idle reverse proxy / CDN to cut it. Comment lines aren't events — `EventSource` and any spec-following parser drop them; a hand-rolled parser must skip lines that don't start with `data
|
|
98
|
+
`/web/chat` and `/web/subscribe` write an SSE comment line `: keepalive` every 15 s (`SSE_KEEPALIVE_MS` / `streamWithKeepalive` in `routes/web.ts`; the timer is cleared when the stream ends). A long tool call can otherwise leave the stream silent long enough for an idle reverse proxy / CDN to cut it. Comment lines aren't events — `EventSource` and any spec-following parser drop them; a hand-rolled parser must skip lines that don't start with `data:`.
|
|
100
99
|
|
|
101
100
|
> **Batch-boundary `complete` (must be absorbed, never closes the stream)**: when a root session drains a queue of multiple messages, the server runs N merged turns and emits an internal `complete` with `batchBoundary: true` between rounds (see [session.md](session.md#message-queue-and-drain)). The SSE generators in `channels/web/handler.ts` (`listenSession().events()` plus the `createMediaBuffer()` event processor) flush the just-finished round's text on a `batchBoundary` complete but **do not** send a `complete` SSE frame and **do not** set `done` — the response stays open for the next round; only the **terminal** (unmarked) `complete` closes the HTTP stream. Without this guard a producer→sub-agent fan-out would truncate the web client after the first round. The `batchBoundary` flag is therefore an internal server-side event marker only — it is never serialized into the SSE payload a client sees, so a custom frontend just consumes one ordinary `complete` at the end. (ACP rides on this channel, so it inherits the same safe behavior.)
|
|
102
101
|
|
|
@@ -110,7 +109,7 @@ By default each token is bound 1:1 to the workspace its admin row configured, an
|
|
|
110
109
|
- `sessionId` (string, optional): explicit halo session id. Bypasses the account's active-session pointer entirely. **Gated by ownership**: `full` tokens may address any id; readonly / workspace tokens only ids their own account minted (`web_<accountId>_*`), plus the goal session (`goal_<ts>`) bound to one of those — goal mode routes that session's chat there and the `session` frame names it, so re-attach / history / stop must reach it. Anything else is refused with HTTP 403 `{error: "session not owned by this token"}` at the route layer (before the SSE stream opens), on `/web/chat`, `/web/stop`, `/web/history` and `/web/subscribe` alike. If the session doesn't yet exist, the server creates it with the supplied id (so callers can pre-mint stable ids and address them across reconnects).
|
|
111
110
|
- `agentId` (string, optional): only consulted when the request is creating a new session (no row yet for `sessionId`). Picks the agent profile to bootstrap with. Defaults to `default`.
|
|
112
111
|
|
|
113
|
-
|
|
112
|
+
Simple clients needn't use any of these; per-token defaults apply. Three accepted transports per request, lowest-priority first:
|
|
114
113
|
|
|
115
114
|
1. Query string: `?workspace=…&sessionId=…&agentId=…`
|
|
116
115
|
2. Headers: `x-workspace`, `x-session-id`, `x-agent-id`
|
|
@@ -188,56 +187,6 @@ Protected by cookie auth (admin panel). File: `packages/server/src/routes/web.ts
|
|
|
188
187
|
|
|
189
188
|
---
|
|
190
189
|
|
|
191
|
-
## packages/web-demo — Standalone Demo Frontend
|
|
192
|
-
|
|
193
|
-
An independent Express app that provides a browser UI for the web channel. It holds the Halo token server-side and exposes its own password-based auth.
|
|
194
|
-
|
|
195
|
-
### Deployment
|
|
196
|
-
|
|
197
|
-
```bash
|
|
198
|
-
cd packages/web-demo
|
|
199
|
-
node server.js
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
### Environment variables
|
|
203
|
-
|
|
204
|
-
| Variable | Required | Description |
|
|
205
|
-
|---|---|---|
|
|
206
|
-
| `HALO_API` | Yes | Halo server URL (e.g. `http://localhost:9527`) |
|
|
207
|
-
| `HALO_TOKEN` | Yes | Token from Halo admin panel (Web settings) |
|
|
208
|
-
| `HALO_WEB_DEMO_PASSWORD` | No | Password for web-demo login. Empty = open access |
|
|
209
|
-
| `PORT` | No | Listen port (default: `9528`) |
|
|
210
|
-
|
|
211
|
-
### Security model
|
|
212
|
-
|
|
213
|
-
- **Proxy mode (default)**: `HALO_TOKEN` and `HALO_API` are server-side only, never exposed to the browser; frontend uses same-origin relative paths (`/chat`, `/history`, etc.); web-demo server proxies all requests to Halo, injecting the token in `x-token` header. Session auth: HMAC token in `x-session` header + localStorage. All proxy routes including `GET /file` are gated by the auth middleware.
|
|
214
|
-
- **Session TTL: 7 days**, enforced without server-side storage — the signed payload *is* the issue timestamp, so `isValidSession` re-parses it and rejects anything older than `SESSION_TTL_MS`. A leaked token therefore stops working on its own; the frontend gets a 401 and falls back to the login view (`handleAuthFail → logout`). Previously the signature alone was checked, making every issued token valid forever.
|
|
215
|
-
- **Direct-connect mode (opt-in)**: the gear panel takes a halo server URL + web-channel token; the browser then calls `<server>/api/web/*` directly with `x-token`, bypassing the proxy (no password step). The token is stored in that browser's localStorage by explicit user choice — the UI says so next to the field. One slot; clearing it returns to proxy mode. Works because the server CORS reflects any origin and allowlists `x-token`.
|
|
216
|
-
|
|
217
|
-
### Proxy routes
|
|
218
|
-
|
|
219
|
-
| Frontend path | Upstream Halo path |
|
|
220
|
-
|---|---|
|
|
221
|
-
| `POST /chat` | `POST /api/web/chat` |
|
|
222
|
-
| `POST /stop` | `POST /api/web/stop` |
|
|
223
|
-
| `GET /history` | `GET /api/web/history` |
|
|
224
|
-
| `GET /subscribe` | `GET /api/web/subscribe` |
|
|
225
|
-
| `GET /file` | `GET /api/web/file` |
|
|
226
|
-
|
|
227
|
-
### Features
|
|
228
|
-
|
|
229
|
-
- Markdown rendering for assistant text (marked@12, gfm + breaks) with typewriter streaming; user text stays escaped plain text
|
|
230
|
-
- SSE streaming with collapsible thinking / tool-call step blocks
|
|
231
|
-
- Auto-reconnect: if session is running on page load, subscribes to live stream; history renders rebuild-from-scratch (never append)
|
|
232
|
-
- Message queue: can send while agent is running, messages queue and send sequentially
|
|
233
|
-
- Slash commands (`/help`, `/session`, `/agent`, `/skill`, `/workspace`), session/agent switcher menus
|
|
234
|
-
- Image upload + camera capture (camera button is mobile-only — the `capture` attribute is a no-op on desktop, where it would duplicate the file picker)
|
|
235
|
-
- Voice recording (webm audio saved to workspace, path given to agent)
|
|
236
|
-
- Stop/interrupt button
|
|
237
|
-
- i18n: auto-detects browser language (`navigator.language`), toggle button in header, persists to `localStorage`
|
|
238
|
-
- Mobile-first layout: 100dvh-safe, safe-area insets, ≥44px touch targets, bottom-sheet menus/settings on small widths, visual language shared with `packages/agentcore-demo`
|
|
239
|
-
- Page load needs internet for two CDN deps (tailwindcss, marked) — no longer fully self-contained
|
|
240
|
-
|
|
241
190
|
## Integration guide (custom frontend)
|
|
242
191
|
|
|
243
192
|
To build your own frontend against Halo's web channel:
|