@turmind/halo 0.1.9

package/bundled-docs/dev/env.md

@@ -0,0 +1,158 @@
+# Development Environment
+
+## Ports
+
+- **8080** is code-server — don't touch, don't occupy, don't kill
+- **9527** is Hono (Halo server); one process serves API + WebSocket + static frontend
+
+## Browser testing
+
+- URL: `http://localhost:9527/?folder=<url-encoded absolute workspace path>`
+- Password: set via `halo setup` (stored as scrypt hash in `~/.halo/secrets/config.yaml server.password`) **or** `HALO_PASSWORD` env (plaintext — preferred for Docker / CI). Env wins over hash if both are set.
+- Workspace path: any absolute directory you want to bind
+
+## Deploy commands
+
+### Frontend build (web only)
+
+```bash
+cd packages/admin && npx next build --no-lint && node scripts/copy-monaco.mjs
+```
+
+### Backend redeploy (server only)
+
+```bash
+cd packages/server && npx tsc
+kill -9 $(ss -tlnp | grep ':9527' | grep -oP 'pid=\K\d+') 2>/dev/null
+sleep 1
+nohup node dist/index.js >> /dev/null 2>&1 &   # HALO_PASSWORD read from env/config
+```
+
+### Full deploy in one go
+
+```bash
+cd packages/admin && npx next build --no-lint && node scripts/copy-monaco.mjs
+cd ../server && npx tsc
+kill -9 $(ss -tlnp | grep ':9527' | grep -oP 'pid=\K\d+') 2>/dev/null
+sleep 1
+nohup node dist/index.js >> /dev/null 2>&1 &   # HALO_PASSWORD read from env/config
+```
+
+## Prerequisites
+
+- Node.js ≥ 22 (better-sqlite3 native binding is bound to v22)
+- pnpm ≥ 9
+- AWS credentials configured (`~/.aws/credentials` or env vars) with Bedrock access
+- bubblewrap (`sudo apt install bubblewrap`) — OS-level sandbox for non-full access levels. Without it, only app-level path validation is active and `shell_exec` is blocked for non-full sessions
+
+### nvm PATH
+
+If the shell doesn't auto-load nvm:
+
+```bash
+export PATH=$HOME/.nvm/versions/node/v22.21.1/bin:$PATH
+```
+
+Your deployment scripts should also prepend this line.
+
+## Runtime directories
+
+- SQLite: `data/halo.db` (auto-created on first start)
+- Session files: `.halo/sessions/{agentId}/{sessionId}.json`
+- Global config: `~/.halo/global/`
+- Per-project config: `<workspace>/.halo/`
+
+## Configuration sources
+
+Three config file types, precedence **env vars > config.yaml / settings.yaml > code defaults**:
+
+| File | Scope | Contents |
+|---|---|---|
+| `~/.halo/secrets/config.yaml` | System | Port, password, CORS, timeouts, limits, logging — "infrastructure" settings |
+| `~/.halo/secrets/settings.yaml` | User | Model, region, session behaviour — "preferences" |
+| `<project>/.halo/settings.yaml` | Project | Overrides of global settings |
+
+`init.ts` seeds these on first run with a per-category policy. Refresh trigger: `halo setup` always re-runs the seed, and the server's startup check re-runs it automatically when `~/.halo/global/.template-version` is behind the bundled `TEMPLATE_VERSION` (see `init.ts:TEMPLATE_VERSION` + `index.ts` startup block).
+
+- **Always overwritten** (platform-owned, refreshed when the template version moves): `~/.halo/global/{prompts,models,docs}/`, `INSTRUCTIONS.md`, the built-in agent ids (`default`, `executor`, `deep-executor`, `__evo_agent__`, `__score__`, `__apply_agent__`), built-in skill ids (`agent`, `skill`, `workspace`, `cron`, `acp`, `send-file`, `self`, `aws-knowledge`, `nova-web-search`, `halo`).
+- **Built-in agents** keep the user's `model:` block on overwrite — the admin UI lets users change which model an agent uses, and that choice survives upgrades.
+- **Optional skills** (`tavily-web-search`) install only when picked via `halo setup`; the opt-in list is `~/.halo/global/.installed-optional-skills`. Picked skills are force-overwritten alongside the always-overwritten set.
+- **`secrets/config.yaml`** is leaf-merged: existing leaf `value`s preserved, new leaves added when a server upgrade introduces them.
+- **`secrets/settings.yaml`** is created empty if missing and never touched again. Defaults live in `settings-schema.ts`.
+
+## Environment variable overrides
+
+Source: `packages/server/src/config.ts`
+
+| Variable | Default | Config path | Description |
+|---|---|---|---|
+| `HALO_PORT` | `9527` | `config.yaml server.port` | Hono listen port |
+| `HALO_PASSWORD` | (none) | `config.yaml server.password` (scrypt hash) | Plaintext login password. When set, takes precedence over the stored hash and bypasses scrypt entirely — intended for Docker / systemd / CI. The hash is set by `halo setup` for interactive installs. |
+| `HALO_CORS_ORIGINS` | empty (reflect any origin) | `config.yaml server.cors_origins` | CORS allowlist (comma-separated). Empty = reflect any incoming Origin so credentials work cross-origin. Set explicit list to enforce strict CORS. |
+| `HALO_FRONTEND_DIR` | `packages/admin/out` | — | Static frontend dir (resolved as absolute path from project root) |
+| `HALO_MAX_CONTEXT_TOKENS` | `200000` | — | Model max context |
+| `HALO_SHELL_TIMEOUT` | `120000` | `config.yaml timeout.shell_exec` | Shell command timeout (ms) |
+| `HALO_WEB_FETCH_TIMEOUT` | `10000` | `config.yaml timeout.web_fetch` | web_fetch timeout |
+| `HALO_MODEL_TIMEOUT` | `1800000` | `config.yaml timeout.model_request` | Per model-call wall-clock cap (ms, 30 min). A bare fetch has no default timeout, so a half-open connection would hang the request forever; on expiry the call aborts and the agent loop retries with backoff. |
+| `HALO_SESSION_GRACE` | `300000` | `config.yaml timeout.session_grace` | WS-disconnect session grace |
+| `HALO_TERMINAL_GRACE` | `300000` | `config.yaml timeout.terminal_grace` | Terminal disconnect grace |
+| `HALO_MAX_CACHED_SESSIONS` | `50` | — | In-memory session cache |
+| `HALO_LOG_MAX_SIZE` | `10485760` | `config.yaml logging.max_file_size` | Log file size cap (10 MB) |
+| `HALO_LOG_MAX_FILES` | `5` | `config.yaml logging.max_files` | Retained rotated logs |
+
+settings.yaml only (no env override):
+- `general.session.max_queue_size` (default 256)
+- `general.session.max_nesting_depth` (default 16)
+- `general.agent.max_retries` (default 5)
+- `general.compact.keep_messages` (default 5) — recent messages to keep uncompacted
+- `general.compact.max_summary_input` (default 15000) — local compaction fallback input cap
+- `general.compact.max_message_slice` (default 800) — local compaction per-message cap
+- `general.compact.summarize_timeout_sec` (default 300) — self-compact timeout
+- `general.sandbox.hidden_dirs` (default `~/.halo/secrets,~/.aws,~/.ssh,~/.gnupg,~/.docker`) — bwrap tmpfs overlays, scope: global
+- `general.sandbox.hidden_files` (default `~/.npmrc,~/.bash_history,~/.gitconfig`) — bwrap /dev/null binds, scope: global
+- `general.logging.level` (default `warn`) — log level: debug | info | warn | error
+
+`<provider-id>.secrets.*` (server-only, hard-rejected by `{{}}` substitution; declared in `models/<provider-id>.yaml` `secrets:`):
+- `aws-bedrock-claude-invoke.secrets.access_key_id` / `.secret_access_key` — AWS / Bedrock credentials
+- `kimi.secrets.api_key` — Kimi (Moonshot AI)
+- `deepseek.secrets.api_key` — DeepSeek
+
+`<skill-id>.params.*` (referenceable from skills via `{{<skill-id>.params.<key>}}`, or short form `{{params.<key>}}` inside SKILL.md; declared in `skills/<id>/config.yaml`):
+- `tavily-search.params.api_key` — example only; declared by whichever skill needs it
+
+Hardcoded (config.ts, no override):
+- `auth.tokenMaxAge` — 14 days
+- `auth.refreshAfter` — 24 hours
+- `model.compressAt` — default 0.8 (auto-compact threshold; configurable via `general.compact.compress_at` in settings.yaml)
+
+settings.yaml-driven `general.limits.*` (no env override):
+- `general.limits.shell_output_bytes` (default 5 MiB)
+- `general.limits.web_fetch_bytes` (default 50 KiB)
+- `general.limits.grep_default_matches` (default 50)
+- `general.limits.tool_result_render_chars` (default 8000)
+- `general.limits.ws_event_buffer` (default 5000)
+- `general.limits.terminal_scrollback_bytes` (default 50000)
+
+Two ways to authenticate:
+
+1. **Interactive** — run `halo setup`, pick "Set / change password". The value is scrypt-hashed and written to `~/.halo/secrets/config.yaml server.password`. Best for personal installs.
+2. **Env** — export `HALO_PASSWORD=...` before launching the server. The plaintext value is compared directly (no hashing) and takes precedence over any stored hash. Best for Docker / systemd / CI where a secrets manager already protects the env.
+
+## Verify
+
+```bash
+curl http://localhost:9527/api/health
+# expect: {"status":"ok", ...}
+
+curl -s -o /dev/null -w "%{http_code}" http://localhost:9527
+# expect: 200
+```
+
+## Coding conventions
+
+- TypeScript strict, ESM only
+- camelCase variables/functions, PascalCase types, kebab-case filenames
+- React functional components; styling via Tailwind only; UI prefers shadcn/ui
+- Log format `[Module] message`
+- Bedrock model ID: `global.anthropic.claude-sonnet-4-6`, default region `us-east-1` (configured per agent via `agent.yaml model.endpoint`)
+- File operations are sandboxed by bwrap (OS-level) + `assertPathAllowed` (app-level fallback) for non-full sessions

package/bundled-docs/dev/logging.md

@@ -0,0 +1,85 @@
+# Logging
+
+File-backed logs: intercepts console, auto-rotates.
+
+File: `packages/server/src/logger.ts`
+
+## Architecture
+
+On startup the logger intercepts `console.log` / `console.error` / `console.warn` / `console.debug`. Every call writes twice:
+1. The original console (stdout/stderr) — live monitoring
+2. A disk log file — persistent history
+
+## Log locations
+
+| Scenario | Directory | File |
+|---|---|---|
+| Default (no workspace) | `~/.halo/logs/` | `server.log` |
+| Workspace open | `<project>/.halo/logs/` | `server.log` |
+
+When the WS handler receives a `subscribe` or `chat` with `projectId`, it calls `setLogDir()` to switch to the project's log directory.
+
+## Log format
+
+```
+2026-04-19T10:30:00.000Z INFO  [Server] Hono server listening on http://localhost:9527
+2026-04-19T10:30:01.234Z ERROR [WS] Chat error: ThrottlingException
+2026-04-19T10:30:02.567Z WARN  [Orchestrator] Context overflow, auto-compacting...
+```
+
+Every line: `ISO timestamp` + `level (DEBUG/INFO/WARN/ERROR)` + `message`.
+
+## Rotation
+
+When the log file exceeds the cap:
+1. Delete `server.log.{maxFiles}` (oldest)
+2. Shift `server.log.N` → `server.log.N+1`
+3. Rename `server.log` → `server.log.1`
+4. New `server.log` starts fresh
+
+### Config
+
+| Config key | Default | Env var | Purpose |
+|---|---|---|---|
+| `logging.maxFileSize` | 10 MB | `HALO_LOG_MAX_SIZE` | Rotation threshold |
+| `logging.maxFiles` | 5 | `HALO_LOG_MAX_FILES` | Retained rotated files |
+
+Defined in `packages/server/src/config.ts`.
+
+## Initialization
+
+```typescript
+import { initLogger } from './logger.js'
+initLogger()
+```
+
+Called once from `packages/server/src/index.ts` at startup. Steps:
+1. Create the log directory
+2. Replace console methods with the interceptor
+3. Each interceptor calls the original method and appends to the log file
+
+## Workspace switching
+
+```typescript
+import { setLogDir } from './logger.js'
+setLogDir('/path/to/project')
+```
+
+Called from `packages/server/src/ws/handler.ts` when the client subscribes to a workspace.
+
+## Error handling
+
+Log writes use synchronous `fs.appendFileSync` and fail silently — logging must not crash the server.
+
+## Viewing logs
+
+```bash
+# live stdout (if started with nohup > /tmp/server.log)
+tail -f /tmp/server.log
+
+# file logs (always written, regardless of nohup redirection)
+tail -f ~/.halo/logs/server.log
+
+# project-specific logs
+tail -f /path/to/your-project/.halo/logs/server.log
+```

package/bundled-docs/dev/previews.md

@@ -0,0 +1,225 @@
+# Preview Plugin System
+
+Canvas renders non-text files via a **plugin registry**. Each plugin declares the extensions it handles and a React component. Adding a new file type = write one plugin, register it. No changes to the core framework or Canvas panel.
+
+Location: [`packages/admin/src/features/editor/previews/`](../../../packages/admin/src/features/editor/previews/)
+
+## Layout
+
+```
+previews/
+├── FilePreview.tsx           Public entry. Looks up plugin by extension,
+│                             renders <Suspense><Component/></Suspense>.
+├── types.ts                  PreviewPlugin, PreviewProps
+├── registry.ts               register() / getPlugin() / registeredExtensions() / isHeavyPreview()
+├── ui/
+│   ├── preview-shell.tsx     Standard header (filename + Open-as-Text + Download + extraToolbar)
+│   ├── use-preview-fetch.ts  Hook: fetch + AbortController + parse, returns {data, error, loading}
+│   └── print.ts              Pop-up print helper
+├── workers/                  Parse workers (one per heavy format)
+│   ├── worker-client.ts      Generic WorkerClient<T> class — id-routed postMessage
+│   ├── xlsx.worker.ts
+│   └── docx.worker.ts
+└── plugins/
+    ├── index.ts              Registers all built-in plugins
+    ├── pdf.tsx               Metadata (id, extensions, lazy Component)
+    ├── pdf-view.tsx          Actual React component
+    ├── docx.tsx / docx-view.tsx
+    ├── xlsx.tsx / xlsx-view.tsx
+    ├── pptx.tsx / pptx-view.tsx
+    └── media.tsx / media-view.tsx
+```
+
+**Two-file-per-plugin pattern**: `foo.tsx` is tiny metadata (no runtime deps). `foo-view.tsx` holds the component and its heavy dependencies. The metadata file uses `React.lazy()` so the view file (and its deps) only loads when a user actually opens that file type.
+
+## PreviewPlugin interface
+
+```typescript
+interface PreviewPlugin {
+  id: string                    // stable id, e.g. 'pdf'
+  extensions: readonly string[] // lowercase, no dot — e.g. ['xlsx', 'xls', 'csv']
+  Component: React.ComponentType<PreviewProps>
+  heavy?: boolean               // true = main-thread-heavy; active-only mount, skip MRU cache
+}
+
+interface PreviewProps {
+  name: string         // full filename including extension
+  path: string         // relative workspace path (or absolute for /tmp files)
+  viewUrl: string      // for inline viewing, supports HTTP Range
+  downloadUrl: string  // for forced download (used by the shell's Download button)
+  onOpenAsText?: () => void  // set when the file can also be force-opened as text
+}
+```
+
+## Adding a new file type
+
+### 1. Create the view component
+
+```tsx
+// plugins/foo-view.tsx
+'use client'
+
+import type { PreviewProps } from '../types'
+import { PreviewShell } from '../ui/preview-shell'
+import { usePreviewFetch } from '../ui/use-preview-fetch'
+
+export function FooPreview(props: PreviewProps) {
+  const { name, viewUrl, downloadUrl, onOpenAsText } = props
+  const { data, error, loading } = usePreviewFetch(viewUrl, async (buf) => {
+    // Parse `buf` here. For heavy parsing, call into a Worker (see Workers below).
+    return parseFoo(buf)
+  })
+  return (
+    <PreviewShell
+      name={name}
+      downloadUrl={downloadUrl}
+      onOpenAsText={onOpenAsText}
+      loading={loading}
+      error={error}
+    >
+      {data && <div className="h-full overflow-auto">{/* render your data */}</div>}
+    </PreviewShell>
+  )
+}
+```
+
+### 2. Declare the plugin
+
+```tsx
+// plugins/foo.tsx
+'use client'
+
+import { lazy } from 'react'
+import type { PreviewPlugin } from '../types'
+
+export const fooPlugin: PreviewPlugin = {
+  id: 'foo',
+  extensions: ['foo', 'foobar'],
+  Component: lazy(() => import('./foo-view').then((m) => ({ default: m.FooPreview }))),
+}
+```
+
+### 3. Register it
+
+```ts
+// plugins/index.ts
+import { fooPlugin } from './foo'
+register(fooPlugin)
+```
+
+Done. The Canvas panel will:
+- Treat `.foo` / `.foobar` as non-text (routes to preview instead of Monaco)
+- Mount `FooPreview` inside an MRU cache (up to 5 concurrent plugins cached)
+
+## PreviewShell — the standard header
+
+Every plugin should wrap its content in `<PreviewShell>` for consistency. The shell gives you:
+- Filename on the left
+- Your plugin-specific buttons via `extraToolbar` (right-aligned, before the standard buttons)
+- Standard Open-as-Text + Download buttons
+- Loading overlay (pass `loading={true}`)
+- Error state (pass `error={'…'}` — the children are hidden)
+
+```tsx
+<PreviewShell
+  name={name}
+  downloadUrl={downloadUrl}
+  onOpenAsText={onOpenAsText}
+  extraToolbar={myButtons}     // optional
+  loading={loading}
+  error={error}
+>
+  {content}
+</PreviewShell>
+```
+
+Plugin-specific buttons use the shared `<ToolbarButton>`:
+
+```tsx
+<ToolbarButton onClick={...} title="Print">
+  <Printer className="h-3 w-3" />
+  <span>Print</span>
+</ToolbarButton>
+```
+
+## Workers — parsing off the main thread
+
+If your format needs CPU-heavy parsing (non-trivial ArrayBuffer → structured data), run it in a Web Worker so the UI stays responsive even with several tabs parsing concurrently.
+
+### 1. Write the worker
+
+```ts
+// workers/foo.worker.ts
+/// <reference lib="webworker" />
+
+type Req = { id: number; buf: ArrayBuffer; meta?: unknown }
+type Res = { id: number; ok: true; data: FooResult } | { id: number; ok: false; error: string }
+
+self.addEventListener('message', (e: MessageEvent<Req>) => {
+  const { id, buf } = e.data
+  try {
+    const data = parseFoo(buf)
+    ;(self as any).postMessage({ id, ok: true, data } satisfies Res)
+  } catch (err) {
+    ;(self as any).postMessage({ id, ok: false, error: String(err) } satisfies Res)
+  }
+})
+```
+
+### 2. Wire it in the view
+
+```tsx
+import { WorkerClient } from '../workers/worker-client'
+
+let client: WorkerClient | null = null
+function getClient() {
+  if (!client) {
+    client = new WorkerClient(
+      () => new Worker(new URL('../workers/foo.worker.ts', import.meta.url), { type: 'module' }),
+    )
+  }
+  return client
+}
+
+// Inside your component:
+const { data } = usePreviewFetch(viewUrl, (buf, signal) =>
+  getClient().call<FooResult>(signal, buf),
+)
+```
+
+`WorkerClient.call(signal, buf, meta?)` handles id routing, transfer of the ArrayBuffer, and abort (when the caller's signal fires, the reply is dropped).
+
+## `heavy: true` — when to use it
+
+Set `heavy: true` when the preview:
+- Needs DOM access (can't run in a Worker — e.g. `pptx-preview` draws to canvas)
+- Is the dominant memory cost of the page (large canvas, many cached elements)
+
+Effect: the MRU cache skips this plugin. Only the **active** instance mounts; switching away unmounts the component (releases memory and stops any in-flight work). Switching back re-fetches/re-renders.
+
+Only pptx currently uses this. Don't set it by default — the MRU cache gives much faster switches.
+
+## `onOpenAsText`
+
+When the user right-clicks a previewable file in Explorer and picks "Open as Text", Canvas closes the preview tab and opens the raw file in Monaco. The preview plugin receives this as `props.onOpenAsText` — pass it straight through to `PreviewShell` and the button appears automatically.
+
+## Server side
+
+All previews fetch via `GET /api/files/download?inline=1` which:
+- Streams the file (doesn't read entire buffer into memory)
+- Supports HTTP `Range` (206 Partial Content) for video/audio seek + progressive load
+- Aborts the read when the client disconnects (tab closed / URL changed)
+
+See [`dev/api.md`](api.md) for the route details.
+
+## Testing a new plugin
+
+1. Drop a test file into your workspace
+2. Click it in the Explorer — Canvas should open it in preview mode automatically
+3. Verify:
+   - Loading overlay appears briefly
+   - Download button works (forces attachment download)
+   - Open-as-Text button works (closes preview, opens in Monaco)
+   - Closing the preview tab cleanly aborts any in-flight fetch/parse
+   - Opening a second file of the same type reuses the worker (no second worker instance — check DevTools → Application → Service Workers / the process graph)
+   - Switching between up to 5 preview tabs is instant (MRU cache); the 6th oldest unmounts