@turmind/halo 0.1.9

package/bundled-docs/plans/md-and-agent-refactor.md

@@ -0,0 +1,197 @@
+# Refactor plan: drop is_default + working_dir + MD-load simplification + externalise system prompts
+
+**Status**: shipped (2026-04 / 2026-05). All phases A–F landed: `priority` field replaced `is_default`, `agent_sessions.working_dir` column added, MD-loader simplified (INDEX.md root-only, INSTRUCTIONS.md stacks per directory level), system prompts externalised under `prompts/{bootstrap,all,root}/`. This doc is kept as a design log — see the reference state in [design/prompt-system.md](../design/prompt-system.md).
+**Filed**: 2026-04-27
+
+This doc preserves every design decision, rationale, and trade-off so we can rapidly recover context if it's lost.
+
+---
+
+## Background
+
+Halo uses `is_default` as a synonym for "root agent", which has bad knock-on effects:
+- Prompt injection logic, tool behaviour, and UI default selection all bind to this field
+- User-defined "direct-chat agents" can't receive USER.md, PLATFORM_KNOWLEDGE, or other root-only prompts
+- Hard-coded system prompts (BOOTSTRAP / TOOL_GUIDELINES / PLATFORM_KNOWLEDGE / ORCHESTRATOR_GUIDELINES) live in source — users wanting to customise must edit source
+- INDEX.md reserves a subdirectory-loading interface but never calls it (dead code)
+- INSTRUCTIONS.md only supports two layers (global + project); no subdirectory granularity
+
+---
+
+## Core decisions
+
+### 1. Drop `is_default`
+
+- The `is_default` field is deprecated. YAML uses `priority: number` instead — default 0, higher first.
+- The `default` agent seed stays, `priority: 99` (remains the initial default selection).
+- All `isDefault`-semantic branches become `!parentId` ("root agent") checks.
+- `list_agents` no longer filters `default`; agents can spawn themselves (same agent_id, independent session instances).
+- Delete protection: at least 1 agent in the global scope; workspace scope can be emptied.
+- UI default selection: sort by `priority desc, name asc`, pick the first.
+- Remove the "default" badge.
+
+### 2. Add `working_dir`
+
+- New `working_dir` argument on `start_session`
+- Accepts absolute or workspace-relative paths
+- Default: `workspaceRoot` (project root)
+- Enforce resolved path within workspace (reject traversal)
+- Non-existent directory → reject spawn
+- `agent_sessions` gets a `workingDir TEXT` column (null = project root)
+- On resume, DB → `buildAgentInstance`
+
+### 3. INDEX.md — project-root-only
+
+- `<ws>/.halo/INDEX.md` is the sole source, injected for every agent
+- Sub-directory `INDEX.md` files are **not auto-loaded**; user / agent uses `file_read` on demand
+- Drop `subDirIndex` logic in `resolveMdPaths` (dead code)
+- Rationale: multi-level INDEX is an untested requirement; users reference sub-INDEX from the root INDEX and the agent reads on demand (progressive disclosure)
+
+### 4. INSTRUCTIONS.md — stacked by level
+
+- From global → workspaceRoot → every level of the workingDir path, load `.halo/INSTRUCTIONS.md`
+- Injection order: outer → inner (global first, workingDir last)
+- Each block labelled with its path relative to workspaceRoot (root itself uses `.`)
+- Rationale: INSTRUCTIONS encode "how to work here"; directory depth is a natural scope, more specific wins
+
+### 5. Externalise system prompts
+
+The four hard-coded constants (BOOTSTRAP / TOOL_GUIDELINES / PLATFORM_KNOWLEDGE / ORCHESTRATOR_GUIDELINES) move to:
+
+```
+~/.halo/global/prompts/
+├── bootstrap/   ← only injected for root agent + needsBootstrap
+├── all/         ← injected for every agent
+└── root/        ← only injected for root agent
+```
+
+- Subdirectory categorisation decides injection scope
+- Inside each subdirectory, every `.md` is concatenated by filename ascending
+- User extends by dropping MDs in the relevant directory (e.g. `prompts/all/my-style-guide.md`)
+- Startup seed: first run writes four defaults; existing files preserved
+- Live-read (every `buildAgentInstance`)
+- Missing: warn + skip (don't break the server)
+- `tool-guidelines.ts` remains the seed source + hard-coded fallback
+- **Global only**; workspace doesn't override
+- **Language**: seed content stays English (already is); UI copy stays English; section headers stay English; Chinese only in developer docs
+
+---
+
+## Target directory layout
+
+### Global
+
+```
+~/.halo/global/
+├── USER.md                    ← bootstrap-generated
+├── INSTRUCTIONS.md            ← user-global preferences
+├── prompts/
+│   ├── bootstrap/BOOTSTRAP.md
+│   ├── all/TOOL_GUIDELINES.md
+│   └── root/PLATFORM_KNOWLEDGE.md, ORCHESTRATOR_GUIDELINES.md
+├── agents/<id>/{agent.yaml, AGENT.md}
+└── skills/<id>/SKILL.md
+```
+
+### Workspace
+
+```
+<ws>/.halo/
+├── USER.md                       ← optional, overrides global
+├── INSTRUCTIONS.md               ← project-root preferences
+├── INDEX.md                      ← sole project index
+├── <subdir>/.halo/INSTRUCTIONS.md  ← per-subdir preferences, stacked
+├── agents/<id>/{agent.yaml, AGENT.md}
+└── skills/<id>/SKILL.md
+```
+
+---
+
+## Root vs sub-agent behaviour matrix
+
+| Injected content | Root (`!parentId`) | Sub-agent |
+|---|---|---|
+| USER.md | ✓ | ✗ |
+| AGENT.md | ✓ | ✓ |
+| INSTRUCTIONS.md (global + stacked) | ✓ | ✓ |
+| INDEX.md (project root) | ✓ | ✓ |
+| `prompts/all/` | ✓ | ✓ |
+| `prompts/root/` | ✓ | ✗ |
+| `prompts/bootstrap/` | ✓ (only when no USER.md) | ✗ |
+| `"workspace at..."` | ✓ | ✓ |
+| `"Working directory: ..."` | only when workingDir != root | always |
+
+---
+
+## Execution phases
+
+Grouped into 6 todo buckets:
+
+### A. is_default → priority (backend + frontend)
+- session-manager.ts: drop isDefault; replace prompt branches with `!parentId`; merge tool order; drop `list_agents` filter
+- agent-loader.ts: YAML gains priority; `scanAvailableAgents` returns priority
+- agent-configs.ts: `parseAgentYaml` reads priority; sort by priority desc + name asc; seed template gains `priority: 99`; DELETE adds last-global guard
+- frontend types.ts + api-client.ts: rename fields
+- chat-panel.tsx: default-selection logic; remove "default" badge
+- agent-management-main.tsx + agent-form.tsx: delete isDefault UI
+
+### B. working_dir feature
+- `start_session` schema adds `working_dir`
+- `createSession` / `buildAgentInstance` pass it through
+- DB migration: `agent_sessions` adds `workingDir` column
+- Resume restores from DB
+
+### C. md-loader simplification
+- INDEX.md reads only `<ws>/.halo/INDEX.md`; drop `subDirIndex`
+- INSTRUCTIONS.md becomes `workspaceInstructionsChain: Array<{rel, content}>`
+- `composeMdPrompt` expands the chain as multiple sections, each header labelled with relative path (root = `.`)
+- Implement `collectInstructionsChain(workspaceRoot, workingDir)`
+
+### D. Externalise system prompts
+- Create `~/.halo/global/prompts/{bootstrap,all,root}/`
+- `init.ts` startup seeds four MD files (content from `tool-guidelines.ts`)
+- New `loadSystemPrompts()` reads live per subdir, filename-ascending concat
+- Replace `import { ... } from './tool-guidelines'` with `loadSystemPrompts()` calls
+- Missing: warn + fall back to the hard-coded value (keep `tool-guidelines.ts`)
+
+### E. Doc sync
+- CLAUDE.md: describe the MD load flow, agent model
+- `.halo/docs/backend/md-loading-flow.md` (already in target state)
+- `.halo/docs/knowledge-system.md`: directory layout, injection rules
+- `.halo/docs/data-storage-protocol.md`: agent YAML fields + new agent_sessions column
+- `.halo/docs/session-tools-reference.md`: `start_session` schema
+- `.halo/docs/test-case.md`: updated test cases
+
+### F. Deployment smoke test
+- `pnpm --filter core/server/web build`
+- Kill 9527, restart with Node 22 (PATH=$HOME/.nvm/versions/node/v22.21.1/bin:$PATH)
+- Smoke: /api/health, WS connect, send one message end-to-end
+- Verify: root agent receives prompts/root, sub-agent doesn't; INSTRUCTIONS stacking works; working_dir wires through; deletion guard fires
+
+---
+
+## Key constraints and risks
+
+- **Node version**: must be v22+ (better-sqlite3 native binding bound to v22). Scripts must load nvm.
+- **Compatibility**: legacy YAML's `is_default: true` stays readable but ignored; DB's `is_default` column stays untouched, just unread.
+- **Backwards-compat seed**: `default` agent still auto-seeds with id `default`, but YAML uses `priority: 99`.
+- **Path safety**: `working_dir` must be verified inside workspace (block `../` escapes).
+- **Cache key**: with merged tool list, `cache_control` still goes on the last tool — stability unaffected.
+- **Bootstrap idempotence**: once USER.md exists, `needsBootstrap` is false forever, `prompts/bootstrap/` no longer injects.
+- **Self-spawn allowed**: `session_id` depth limit (`maxNestingDepth`) already guards against infinite recursion.
+
+---
+
+## Rollback plan
+
+If something breaks in production:
+- Git revert to pre-refactor commit
+- `~/.halo/global/prompts/` directory is harmless to older versions (they don't read it)
+- `agent_sessions.workingDir` column harmless (old code reads null → falls back to workspaceRoot)
+- YAML `priority` harmless to older code (unknown field = default 0); `is_default: true` can be seeded back into the default agent's YAML
+
+---
+
+## Todo index
+
+See the todo list. All 12 pending items run A → F; the last one is the smoke test.

package/bundled-docs/plans/remote-mac-worker.md

@@ -0,0 +1,196 @@
+# Remote Mac Worker — Design Blueprint
+
+**Status**: not started, design initially frozen
+**Filed**: 2026-04-28
+**Priority**: future work, not at the front of the roadmap
+
+## Scope
+
+Wire the user's Mac into Halo — **not** as a remote code workspace, but as a **capability provider**, letting server-side agents tap into native Mac capabilities (screenshot, camera, browser, WPS, iMessage, calendar, …).
+
+The key contrast with openclaw / opencode / Claude Code:
+- Those tools: agent loop on the local machine, LLM remote
+- This plan: **agent loop + LLM both on the server, Mac is just a "capability execution environment"**
+
+It's "Mac as a service", not "LLM as a service".
+
+## Architecture choice: Swiss-army knife + skill accumulation
+
+Rejected alternatives (discussed and dropped):
+- **Static capability plugin** (a fixed preset tool set) → capability boundary is rigid; agents have no recourse for novel capabilities
+- **Computer Use pixel-level** (screenshot + click coordinates) → unfriendly to the 2018 Intel Mac (slow + expensive); universality bought with token cost
+
+Settled on **Swiss-army knife + skills**:
+
+```
+Low-level primitives (registered by the Worker)
+  - mac_shell(cmd)            → arbitrary shell commands
+  - mac_applescript(script)   → arbitrary AppleScript
+  - mac_jxa(script)           → JavaScript for Automation (AppleScript modernised)
+  - mac_filesystem            → file_read/write/list etc.
+
+Middle layer — skills (preset + user-written + agent-accumulated)
+  - mac_screenshot
+  - mac_take_photo
+  - mac_open_wps
+  - mac_safari_goto
+  - mac_read_imessage
+  - …(agent first explores new scenarios, then asks user to save them as skills after verification)
+
+Surface / UI
+  - Settings shows the connected worker + its registered primitives
+  - Each skill can be enabled/disabled
+```
+
+Why:
+- Covers ~99% of Mac apps (mature AppleScript ecosystem)
+- Friendly to old Intel Mac hardware (no continuous-screenshot + vision-inference load)
+- Plays naturally with Halo's existing Skill system
+- High capability ceiling (Turing-complete)
+
+## Worker protocol outline
+
+WebSocket-based reverse connection (Mac only needs outbound; no public IP needed):
+
+```
+Mac                         Halo Server (9527)
+halo-worker  ──── WSS ───▶ /worker endpoint
+                              ↕
+                           worker-pool
+                              ↕
+                           agent tool dispatch
+```
+
+Worker registration payload:
+
+```json
+{
+  "type": "register",
+  "workerId": "peter-mac",
+  "primitives": ["shell", "applescript", "jxa", "filesystem"],
+  "os": "macos",
+  "osVersion": "14.x",
+  "arch": "x64",
+  "home": "/Users/peter"
+}
+```
+
+Server reads `primitives` and dynamically assembles the tool set it injects into the agent.
+
+Protocol messages to support:
+
+```
+Worker → Server
+  - register
+  - tool_result (id, ok, result)
+  - tool_stream (id, chunk)     // shell stdout streaming
+  - pong
+
+Server → Worker
+  - tool_call (id, tool, args, dry_run?)
+  - tool_cancel (id)
+  - ping
+```
+
+**Tool results must support multiple shapes**: text / image (base64 PNG) / file (binary) / structured (JSON). No longer the "all-tool-results-are-strings" simplification.
+
+## Security model (mandatory)
+
+**Dynamic model + Swiss-army knife = high risk**. Required guards:
+
+### 1. Frontend approval
+
+Every `mac_shell` / `mac_applescript` execution pops a frontend confirmation:
+- Show the command text
+- Allow once / allow for this session / allow permanently for this command pattern
+
+Like Claude Desktop / Cursor's tool approval UI.
+
+### 2. Server-side dangerous-command blocking
+
+Regex scan for:
+- `rm -rf`, `sudo`, `curl|bash` pipes, `nc`, `mkfs`, `dd`
+- AppleScript `delete file`, `empty trash`, `quit application`
+→ Force popup, no "session allowlist"
+
+### 3. Worker-side path validation
+
+`mac_filesystem`-family tools still use the `validateWorkspacePath` approach from `workspace-tools.ts`; the worker validates itself, doesn't trust server-supplied absolute paths.
+
+### 4. Capability switches
+
+Settings lets the user:
+- Disable `mac_shell` (keep only AppleScript)
+- Block specific commands entirely (`rm` / `sudo` / `curl`)
+- Disable an entire sensitive skill
+
+## Hardware constraints (user's 2018 Intel Mac)
+
+- Latest macOS (exact version unknown — will probe)
+- **Computer Use pixel-vision unsuitable** — old GPU, slow image processing
+- AppleScript / shell / osascript are native CLI — no perf issues
+- `node-pty` needs to be built x64 (not arm64)
+
+## Time estimate (revised)
+
+The earlier "2-3 days" was wishful; realistic estimate:
+
+| Phase | Time | Deliverable |
+|---|---|---|
+| MVP: protocol + shell + filesystem | 1 week | Agent can run commands / read files on Mac |
+| + AppleScript + screenshot | 3 days | Agent controls any Mac app + screenshots |
+| + Safari control, WPS control | 3 days | "Open this page and screenshot it" works end-to-end |
+| + Camera, iMessage | 3-5 days | Take photos, read/send messages |
+| UI: worker management + approvals | 1 week | Frontend can see worker status, approve tool calls |
+| Polish + package CLI + deploy docs | 1-2 weeks | Shippable |
+
+**Total: 4-6 full-time weeks.**
+
+Suggested split:
+- **First window (2 weekend days)**: MVP only. Get "agent executes `ls` on Mac and sees the result". Stop to evaluate whether to continue.
+- **Second window (2-3 weeks)**: up to "screenshot + Safari + WPS control" — already a strong demo.
+
+## MVP checklist
+
+Start and stop here; do nothing else until it lands:
+
+- [ ] New `packages/worker` workspace, add to pnpm
+- [ ] New `packages/server/src/workers/` with worker-pool + `/worker` WS endpoint
+- [ ] Protocol: register / tool_call / tool_result / ping-pong
+- [ ] Worker CLI: WS connect, register, ping, one `mac_shell` primitive
+- [ ] Server: Project table gains `execution_target` (local / worker:<id>)
+- [ ] Agent tool dispatch abstraction: `LocalExecutor` vs `RemoteExecutor`
+- [ ] Frontend: Settings hard-codes one worker entry + shows worker status
+- [ ] End-to-end: agent runs `ls ~` on Mac successfully
+
+Stop once these 8 are green. **Do not expand scope.**
+
+## Decision log vs Anthropic Computer Use
+
+We discussed Computer Use (Claude Sonnet 3.5+'s vision + coordinate-click capability) and decided not to go that route, because:
+
+1. User's machine is 2018 Intel Mac; vision inference is slow
+2. Each step's screenshot burns tokens → costly
+3. Non-deterministic (potential misclicks)
+4. Generic, but overkill for macOS's mature AppleScript ecosystem
+5. Halo runs on Bedrock Claude; it's supported but latency/cost is worse than direct API
+
+**Not opposed to adding it later** as another primitive ("vision fallback"), but not the main path.
+
+## Reference projects
+
+- **Anthropic computer-use-demo** (Docker demo, for protocol reference)
+- **OpenInterpreter 01** (open-source local agent, primitive design)
+- **Claude Desktop + MCP** (tool approval UX)
+- **Self-Operating Computer** (coordinate-click fallback, if we ever add it)
+- **Dagger** / **dstack.ai** (worker pool management)
+
+## Questions to answer before starting
+
+1. Mac's specific macOS version (gates which APIs are available)
+2. Which three skills to prioritise packaging (pick 3 from screenshot / Safari / WPS / iMessage)
+3. Approval UI granularity (session allowlist vs per-command pattern)
+4. Worker auth (where to store tokens, rotation)
+5. Multi-worker support (1 server to N workers)
+
+Answer these first before any code.

package/bundled-docs/plans/roadmap.md

@@ -0,0 +1,40 @@
+# Roadmap
+
+## Model Providers
+- [x] OpenAI (GPT series)
+- [ ] Google Gemini
+- [ ] OpenRouter (multi-model aggregator)
+- [ ] Ollama (local models)
+
+Extension interface is ready (`ModelRuntime`). Adding providers is integration work, not architecture work.
+
+## Channels
+- [ ] Discord
+- [x] Slack
+- [ ] Email
+- [ ] SMS
+
+Channel adapter pattern established (Web, WeChat, Telegram, Slack, Feishu already working). New channels follow the same `CommandContext` interface.
+
+## CLI / TUI
+- [x] CLI mode (headless, SSH-friendly)
+- [x] TUI (terminal UI with interactive chat)
+
+Enables usage without a browser. Important for SSH-into-server workflows.
+
+## Deployment
+- [ ] Dockerfile + docker-compose
+- [ ] One-click deploy templates (Fly.io / Render / Railway)
+
+Currently manual (`node dist/index.js`). Containerization needed for distribution.
+
+## Shared Workspace
+- [ ] Workspace templates (clone/fork a pre-configured workspace)
+- [ ] Workspace marketplace or registry
+- [ ] Readonly workspace sharing with token auth
+
+Core differentiator. Share an entire configured workspace (agents + skills + context + docs), not just individual tools or skills.
+
+## Quality
+- [ ] Test suite (unit + integration)
+- [ ] CI pipeline