@intent-systems/nexus 2026.1.5-3

package/docs/testing-philosophy.md

@@ -0,0 +1,211 @@
+# Nexus Testing Philosophy
+
+> **No mocks. Real integration tests. Agent-driven verification.**
+
+## Core Principles
+
+### 1. Mocks Are Technical Debt
+
+Mocks diverge from reality. They test your assumptions, not actual behavior. They require maintenance when APIs change. **We don't use mocks in Nexus.**
+
+Instead:
+- **Unit tests** test pure functions with real inputs/outputs
+- **Integration tests** hit real APIs (gated behind env vars)
+- **Agent tests** use actual agents to verify behavior
+
+### 2. Real API Calls > Mock Responses
+
+```typescript
+// ❌ BAD - Mock that will drift from reality
+vi.mock("@anthropic-ai/sdk", () => ({
+  Anthropic: vi.fn().mockImplementation(() => ({
+    messages: { create: vi.fn().mockResolvedValue({ /* fake response */ }) }
+  }))
+}));
+
+// ✅ GOOD - Real API call, gated for CI
+describe.skipIf(!process.env.ANTHROPIC_API_KEY)("ODU Integration", () => {
+  it("should delegate from IA to EA", async () => {
+    const controlPlane = new ODUControlPlane({ /* real config */ });
+    const result = await controlPlane.sendMessage({
+      sessionKey: "test:integration",
+      message: "Analyze this codebase",
+    });
+    expect(result.success).toBe(true);
+  });
+});
+```
+
+### 3. Agent-Driven Verification
+
+For complex behavior, define tests in markdown that an agent executes:
+
+```markdown
+# Test: ODU Delegation Flow
+
+## Setup
+1. Initialize ODUControlPlane with test config
+2. Ensure ~/nexus/home workspace exists
+
+## Steps
+1. Send message: "Create a file called test.txt with 'hello world'"
+2. Wait for EA to complete
+3. Verify file exists at ~/nexus/home/test.txt
+4. Verify file contents equal "hello world"
+
+## Cleanup
+1. Delete test.txt
+```
+
+These markdown test specs can be executed by Ralph or any agent with file system access.
+
+### 4. Test Categories
+
+| Category | What | When to Run | API Calls |
+|----------|------|-------------|-----------|
+| **Unit** | Pure functions, utilities | Always (CI) | None |
+| **Integration** | Real API calls | When `ANTHROPIC_API_KEY` set | Yes |
+| **Agent** | Full agent flows | Manual / nightly | Yes |
+| **E2E** | Complete user journeys | Pre-release | Yes |
+
+### 5. Environment Gating
+
+```typescript
+// Skip if no API key (CI-safe)
+describe.skipIf(!process.env.ANTHROPIC_API_KEY)("Real API Tests", () => {
+  // These only run when you have credentials
+});
+
+// Run always (pure logic)
+describe("Session Path Resolution", () => {
+  // No external dependencies
+});
+```
+
+### 6. Test Commands
+
+```bash
+# Unit tests only (CI)
+pnpm test
+
+# Integration tests (requires API keys)
+ANTHROPIC_API_KEY=sk-... pnpm test:integration
+
+# Agent verification (runs Ralph)
+pnpm test:agent
+
+# Full E2E suite
+pnpm test:e2e
+```
+
+## What We Test
+
+### ✅ Test These
+
+- **Path resolution** - `resolveStateDir()`, `resolveSessionDir()`
+- **Config parsing** - Loading and validating `nexus.json`
+- **Session storage** - Writing/reading JSONL history
+- **Compaction** - Archiving and summarization
+- **Skill loading** - Parsing SKILL.md, gating logic
+- **Queue semantics** - steer/followup/collect modes
+- **ODU delegation** - IA → EA message routing
+
+### ❌ Don't Test These (With Mocks)
+
+- Claude API responses (use real API or skip)
+- OAuth flows (use real auth or skip)
+- File system operations (use real temp dirs)
+- Network requests (use real endpoints or skip)
+
+## Writing New Tests
+
+### 1. Start with the real thing
+
+```typescript
+it("should compact session and preserve recent messages", async () => {
+  // Create real session with real history
+  const agentId = "test-agent";
+  const sessionId = `test-${Date.now()}`;
+  
+  // Write real history
+  for (let i = 0; i < 20; i++) {
+    await appendToHistory(agentId, sessionId, {
+      turn_id: `turn-${i}`,
+      role: i % 2 === 0 ? "user" : "assistant",
+      content: `Message ${i}`,
+      timestamp: new Date().toISOString(),
+    });
+  }
+  
+  // Real compaction (may call Claude API if ANTHROPIC_API_KEY set)
+  const result = await compactSession(agentId, sessionId, {
+    keepRecentMessages: 5,
+  });
+  
+  // Verify real results
+  expect(result.turnsArchived).toBe(15);
+  expect(result.turnsKept).toBe(5);
+  
+  // Cleanup
+  await fs.rm(resolveSessionDir(agentId, sessionId), { recursive: true });
+});
+```
+
+### 2. Gate expensive operations
+
+```typescript
+it.skipIf(!process.env.ANTHROPIC_API_KEY)(
+  "should generate summary using Claude",
+  async () => {
+    // This test actually calls Claude
+  }
+);
+```
+
+### 3. Use temp directories
+
+```typescript
+let tempDir: string;
+
+beforeEach(async () => {
+  tempDir = await fs.mkdtemp(path.join(os.tmpdir(), "nexus-test-"));
+  process.env.NEXUS_STATE_DIR = tempDir;
+});
+
+afterEach(async () => {
+  await fs.rm(tempDir, { recursive: true, force: true });
+  delete process.env.NEXUS_STATE_DIR;
+});
+```
+
+## Agent Test Specs
+
+For complex flows, write markdown specs in `test/specs/`:
+
+```
+test/
+├── specs/
+│   ├── odu-delegation.md
+│   ├── session-compaction.md
+│   └── skill-loading.md
+└── agent-runner.ts  # Executes specs
+```
+
+The agent runner parses these specs and executes them, reporting pass/fail.
+
+## Migration from Mocks
+
+If you find existing mock-based tests:
+
+1. **Identify what's being mocked** - Is it an API? File system? Network?
+2. **Replace with real calls** - Gate behind env vars if needed
+3. **Use temp directories** - For file system tests
+4. **Delete the mock** - No mock should survive
+
+## Summary
+
+- **No mocks** - They lie
+- **Real APIs** - Gated behind env vars
+- **Agent verification** - For complex flows
+- **Temp directories** - For file system tests
+- **Markdown specs** - Human-readable test definitions

package/docs/thinking.md

@@ -0,0 +1,46 @@
+---
+summary: "Directive syntax for /think + /verbose and how they affect model reasoning"
+read_when:
+  - Adjusting thinking or verbose directive parsing or defaults
+---
+# Thinking Levels (/think directives)
+
+## What it does
+- Inline directive in any inbound body: `/t <level>`, `/think:<level>`, or `/thinking <level>`.
+- Levels (aliases): `off | minimal | low | medium | high`
+  - minimal → “think”
+  - low → “think hard”
+  - medium → “think harder”
+  - high → “ultrathink” (max budget)
+  - `highest`, `max` map to `high`.
+
+## Resolution order
+1. Inline directive on the message (applies only to that message).
+2. Session override (set by sending a directive-only message).
+3. Global default (`agent.thinkingDefault` in config).
+4. Fallback: low for reasoning-capable models; off otherwise.
+
+## Setting a session default
+- Send a message that is **only** the directive (whitespace allowed), e.g. `/think:medium` or `/t high`.
+- That sticks for the current session (per-sender by default); cleared by `/think:off` or session idle reset.
+- Confirmation reply is sent (`Thinking level set to high.` / `Thinking disabled.`). If the level is invalid (e.g. `/thinking big`), the command is rejected with a hint and the session state is left unchanged.
+
+## Application by agent
+- **Embedded Pi**: the resolved level is passed to the in-process Pi agent runtime.
+
+## Verbose directives (/verbose or /v)
+- Levels: `on|full` or `off` (default).
+- Directive-only message toggles session verbose and replies `Verbose logging enabled.` / `Verbose logging disabled.`; invalid levels return a hint without changing state.
+- Inline directive affects only that message; session/global defaults apply otherwise.
+- When verbose is on, agents that emit structured tool results (Pi, other JSON agents) send each tool result back as its own metadata-only message, prefixed with `<emoji> <tool-name>: <arg>` when available (path/command); the tool output itself is not forwarded. These tool summaries are sent as soon as each tool finishes (separate bubbles), not as streaming deltas. If you toggle `/verbose on|off` while a run is in-flight, subsequent tool bubbles honor the new setting.
+
+## Related
+- Elevated mode docs live in [`docs/elevated.md`](https://docs.nexus.bot/elevated).
+
+## Heartbeats
+- Heartbeat probe body is `HEARTBEAT`. Inline directives in a heartbeat message apply as usual (but avoid changing session defaults from heartbeats).
+
+## Web chat UI
+- The web chat thinking selector mirrors the session's stored level from the inbound session store/config when the page loads.
+- Picking another level applies only to the next message (`thinkingOnce`); after sending, the selector snaps back to the stored session level.
+- To change the session default, send a `/think:<level>` directive (as before); the selector will reflect it after the next reload.

package/docs/timezone.md

@@ -0,0 +1,40 @@
+---
+summary: "Timezone handling for agents, envelopes, and prompts"
+read_when:
+  - You need to understand how timestamps are normalized for the model
+  - Configuring the user timezone for system prompts
+---
+
+# Timezones
+
+Nexus standardizes timestamps so the model sees a **single reference time**.
+
+## Message envelopes (UTC)
+
+Inbound messages are wrapped in an envelope like:
+
+```
+[Provider ... 2026-01-05T21:26Z] message text
+```
+
+The timestamp in the envelope is **always UTC**, with minutes precision.
+
+## Tool payloads (raw provider data)
+
+Tool calls (`discord.readMessages`, `slack.readMessages`, etc.) return **raw provider timestamps**.
+These are typically UTC ISO strings (Discord) or UTC epoch strings (Slack). We do not rewrite them.
+
+## User timezone for the system prompt
+
+Set `agent.userTimezone` to tell the model the user's local time zone. If it is
+unset, Nexus resolves the **host timezone at runtime** (no config write).
+
+```json5
+{
+  agent: { userTimezone: "America/Chicago" }
+}
+```
+
+The system prompt includes:
+- `User timezone: America/Chicago`
+- `Current user time: 2026-01-05 15:26`

package/docs/tools.md

@@ -0,0 +1,346 @@
+---
+summary: "Agent tool surface for Nexus (browser, canvas, nodes, cron) replacing nexus-* skills"
+read_when:
+  - Adding or modifying agent tools
+  - Retiring or changing nexus-* skills
+---
+
+# Tools (Nexus)
+
+Nexus exposes **first-class agent tools** for browser, canvas, nodes, and cron.
+These replace the old `nexus-*` skills: the tools are typed, no shelling,
+and the agent should rely on them directly.
+
+## Disabling tools
+
+You can globally allow/deny tools via `agent.tools` in `nexus.json`
+(deny wins). This prevents disallowed tools from being sent to providers.
+
+```json5
+{
+  agent: {
+    tools: {
+      deny: ["browser"]
+    }
+  }
+}
+```
+
+## Tool profiles (base allowlist)
+
+`agent.tools.profile` sets a **base tool allowlist** before `agent.tools.allow`/`agent.tools.deny`.
+Per-agent override: `routing.agents.<agentId>.tools.profile`.
+
+Profiles:
+- `minimal`: `sessions_list` only
+- `coding`: `group:fs`, `group:runtime`, `group:sessions`, `group:memory`, `image`
+- `messaging`: `group:messaging`, `sessions_list`, `sessions_history`, `sessions_send`
+- `full`: no restriction (same as unset)
+
+Example (messaging-only by default, allow Slack + Discord tools too):
+```json5
+{
+  agent: {
+    tools: {
+      profile: "messaging",
+      allow: ["slack", "discord"]
+    }
+  }
+}
+```
+
+Example (coding profile, but deny exec/process everywhere):
+```json5
+{
+  agent: {
+    tools: {
+      profile: "coding",
+      deny: ["group:runtime"]
+    }
+  }
+}
+```
+
+Example (global coding profile, messaging-only support agent):
+```json5
+{
+  agent: { tools: { profile: "coding" } },
+  routing: {
+    agents: {
+      support: {
+        tools: { profile: "messaging", allow: ["slack"] }
+      }
+    }
+  }
+}
+```
+
+## Tool groups (shorthands)
+
+Tool policies (global, per-agent, sandbox) support `group:*` entries that expand
+to multiple tools. Use these in `agent.tools.allow` / `agent.tools.deny`.
+
+Available groups:
+- `group:runtime`: `exec`, `process`
+- `group:fs`: `read`, `write`, `edit`, `apply_patch`
+- `group:sessions`: `sessions_list`, `sessions_history`, `sessions_send`, `sessions_spawn`
+- `group:memory`: `memory_search`, `memory_get`
+- `group:ui`: `browser`, `canvas`
+- `group:automation`: `cron`, `gateway`
+- `group:messaging`: `discord`, `slack`
+- `group:nodes`: `nodes`
+- `group:nexus`: all built-in Nexus tools (excludes provider plugins)
+
+Example (allow only file tools + browser):
+```json5
+{
+  agent: {
+    tools: {
+      allow: ["group:fs", "browser"]
+    }
+  }
+}
+```
+
+## Tool inventory
+
+### `bash`
+Run shell commands in the workspace.
+
+Core parameters:
+- `command` (required)
+- `yieldMs` (auto-background after timeout, default 10000)
+- `background` (immediate background)
+- `timeout` (seconds; kills the process if exceeded, default 1800)
+- `elevated` (bool; run on host if elevated mode is enabled/allowed)
+- Need a real TTY? Use the tmux skill.
+
+Notes:
+- Returns `status: "running"` with a `sessionId` when backgrounded.
+- Use `process` to poll/log/write/kill/clear background sessions.
+
+### `process`
+Manage background bash sessions.
+
+Core actions:
+- `list`, `poll`, `log`, `write`, `kill`, `clear`, `remove`
+
+Notes:
+- `poll` returns new output and exit status when complete.
+- `log` supports line-based `offset`/`limit` (omit `offset` to grab the last N lines).
+
+### `browser`
+Control the dedicated nexus browser.
+
+Core actions:
+- `status`, `start`, `stop`, `tabs`, `open`, `focus`, `close`
+- `snapshot` (aria/ai)
+- `screenshot` (returns image block + `MEDIA:<path>`)
+- `act` (UI actions: click/type/press/hover/drag/select/fill/resize/wait/evaluate)
+- `navigate`, `console`, `pdf`, `upload`, `dialog`
+
+Profile management:
+- `profiles` — list all browser profiles with status
+- `create-profile` — create new profile with auto-allocated port (or `cdpUrl`)
+- `delete-profile` — stop browser, delete user data, remove from config (local only)
+- `reset-profile` — kill orphan process on profile's port (local only)
+
+Common parameters:
+- `controlUrl` (defaults from config)
+- `profile` (optional; defaults to `browser.defaultProfile`)
+Notes:
+- Requires `browser.enabled=true` (default is `true`; set `false` to disable).
+- Uses `browser.controlUrl` unless `controlUrl` is passed explicitly.
+- All actions accept optional `profile` parameter for multi-instance support.
+- When `profile` is omitted, uses `browser.defaultProfile` (defaults to "nexus").
+- Profile names: lowercase alphanumeric + hyphens only (max 64 chars).
+- Port range: 18800-18899 (~100 profiles max).
+- Remote profiles are attach-only (no start/stop/reset).
+- `snapshot` defaults to `ai`; use `aria` for the accessibility tree.
+- `act` requires `ref` from `snapshot --format ai`; use `evaluate` for rare CSS selector needs.
+- Avoid `act` → `wait` by default; use it only in exceptional cases (no reliable UI state to wait on).
+- `upload` can optionally pass a `ref` to auto-click after arming.
+- `upload` also supports `inputRef` (aria ref) or `element` (CSS selector) to set `<input type="file">` directly.
+
+### `canvas`
+Drive the node Canvas (present, eval, snapshot, A2UI).
+
+Core actions:
+- `present`, `hide`, `navigate`, `eval`
+- `snapshot` (returns image block + `MEDIA:<path>`)
+- `a2ui_push`, `a2ui_reset`
+
+Notes:
+- Uses gateway `node.invoke` under the hood.
+- If no `node` is provided, the tool picks a default (single connected node or local mac node).
+- A2UI is v0.8 only (no `createSurface`); the CLI rejects v0.9 JSONL with line errors.
+- Quick smoke: `nexus canvas a2ui push --text "Hello from A2UI"`.
+
+### `nodes`
+Discover and target paired nodes; send notifications; capture camera/screen.
+
+Core actions:
+- `status`, `describe`
+- `pending`, `approve`, `reject` (pairing)
+- `notify` (macOS `system.notify`)
+- `camera_snap`, `camera_clip`, `screen_record`
+- `location_get`
+
+Notes:
+- Camera/screen commands require the node app to be foregrounded.
+- Images return image blocks + `MEDIA:<path>`.
+- Videos return `FILE:<path>` (mp4).
+- Location returns a JSON payload (lat/lon/accuracy/timestamp).
+
+### `image`
+Analyze an image with the configured image model.
+
+Core parameters:
+- `image` (required path or URL)
+- `prompt` (optional; defaults to "Describe the image.")
+- `model` (optional override)
+- `maxBytesMb` (optional size cap)
+
+Notes:
+- Only available when `agent.imageModel` is configured (primary or fallbacks).
+- Uses the image model directly (independent of the main chat model).
+
+### `cron`
+Manage Gateway cron jobs and wakeups.
+
+Core actions:
+- `status`, `list`
+- `add`, `update`, `remove`, `run`, `runs`
+- `wake` (enqueue system event + optional immediate heartbeat)
+
+Notes:
+- `add` expects a full cron job object (same schema as `cron.add` RPC).
+- `update` uses `{ id, patch }`.
+
+### `gateway`
+Restart the running Gateway process (in-place).
+
+Core actions:
+- `restart` (sends `SIGUSR1` to the current process; `nexus gateway`/`gateway-daemon` restart in-place)
+
+Notes:
+- Use `delayMs` (defaults to 2000) to avoid interrupting an in-flight reply.
+
+### `sessions_list` / `sessions_history` / `sessions_send`
+List sessions, inspect transcript history, or send to another session.
+
+Core parameters:
+- `sessions_list`: `kinds?`, `limit?`, `activeMinutes?`, `messageLimit?` (0 = none)
+- `sessions_history`: `sessionKey`, `limit?`, `includeTools?`
+- `sessions_send`: `sessionKey`, `message`, `timeoutSeconds?` (0 = fire-and-forget)
+
+Notes:
+- `main` is the canonical direct-chat key; global/unknown are hidden.
+- `messageLimit > 0` fetches last N messages per session (tool messages filtered).
+- `sessions_send` waits for final completion when `timeoutSeconds > 0`.
+- `sessions_send` runs a reply‑back ping‑pong (reply `REPLY_SKIP` to stop; max turns via `session.agentToAgent.maxPingPongTurns`, 0–5).
+- After the ping‑pong, the target agent runs an **announce step**; reply `ANNOUNCE_SKIP` to suppress the announcement.
+
+### `discord`
+Send Discord reactions, stickers, or polls.
+
+Core actions:
+- `react` (`channelId`, `messageId`, `emoji`)
+- `reactions` (`channelId`, `messageId`, optional `limit`)
+- `sticker` (`to`, `stickerIds`, optional `content`)
+- `poll` (`to`, `question`, `answers`, optional `allowMultiselect`, `durationHours`, `content`)
+- `permissions` (`channelId`)
+- `readMessages` (`channelId`, optional `limit`/`before`/`after`/`around`)
+- `sendMessage` (`to`, `content`, optional `mediaUrl`, `replyTo`)
+- `editMessage` (`channelId`, `messageId`, `content`)
+- `deleteMessage` (`channelId`, `messageId`)
+- `threadCreate` (`channelId`, `name`, optional `messageId`, `autoArchiveMinutes`)
+- `threadList` (`guildId`, optional `channelId`, `includeArchived`, `before`, `limit`)
+- `threadReply` (`channelId`, `content`, optional `mediaUrl`, `replyTo`)
+- `pinMessage`/`unpinMessage` (`channelId`, `messageId`)
+- `listPins` (`channelId`)
+- `searchMessages` (`guildId`, `content`, optional `channelId`/`channelIds`, `authorId`/`authorIds`, `limit`)
+- `memberInfo` (`guildId`, `userId`)
+- `roleInfo` (`guildId`)
+- `emojiList` (`guildId`)
+- `roleAdd`/`roleRemove` (`guildId`, `userId`, `roleId`)
+- `channelInfo` (`channelId`)
+- `channelList` (`guildId`)
+- `channelCreate` (`guildId`, `name`, optional `type`, `parentId`, `topic`, `position`, `nsfw`)
+- `channelEdit` (`channelId`, optional `name`, `topic`, `position`, `parentId`, `nsfw`, `rateLimitPerUser`)
+- `channelDelete` (`channelId`)
+- `channelMove` (`guildId`, `channelId`, optional `parentId`, `position`)
+- `categoryCreate` (`guildId`, `name`, optional `position`)
+- `categoryEdit` (`categoryId`, optional `name`, `position`)
+- `categoryDelete` (`categoryId`)
+- `voiceStatus` (`guildId`, `userId`)
+- `eventList` (`guildId`)
+- `eventCreate` (`guildId`, `name`, `startTime`, optional `endTime`, `description`, `channelId`, `entityType`, `location`)
+- `timeout` (`guildId`, `userId`, optional `durationMinutes`, `until`, `reason`)
+- `kick` (`guildId`, `userId`, optional `reason`)
+- `ban` (`guildId`, `userId`, optional `reason`, `deleteMessageDays`)
+
+Notes:
+- `to` accepts `channel:<id>` or `user:<id>`.
+- Polls require 2–10 answers and default to 24 hours.
+- `reactions` returns per-emoji user lists (limited to 100 per reaction).
+- `discord.actions.*` gates Discord tool actions; `roles`, `channels`, and `moderation` default to `false`.
+- `searchMessages` follows the Discord preview spec (limit max 25, channel/author filters accept arrays).
+- The tool is only exposed when the current provider is Discord.
+
+## Parameters (common)
+
+Gateway-backed tools (`canvas`, `nodes`, `cron`):
+- `gatewayUrl` (default `ws://127.0.0.1:18789`)
+- `gatewayToken` (if auth enabled)
+- `timeoutMs`
+
+Browser tool:
+- `controlUrl` (defaults from config)
+
+## Recommended agent flows
+
+Browser automation:
+1) `browser` → `status` / `start`
+2) `snapshot` (ai or aria)
+3) `act` (click/type/press)
+4) `screenshot` if you need visual confirmation
+
+Canvas render:
+1) `canvas` → `present`
+2) `a2ui_push` (optional)
+3) `snapshot`
+
+Node targeting:
+1) `nodes` → `status`
+2) `describe` on the chosen node
+3) `notify` / `camera_snap` / `screen_record`
+
+## Safety
+
+- Avoid `system.run` (not exposed as a tool).
+- Respect user consent for camera/screen capture.
+- Use `status/describe` to ensure permissions before invoking media commands.
+
+## How the model sees tools (pi-mono internals)
+
+Tools are exposed to the model in **two parallel channels**:
+
+1) **System prompt text**: a human-readable list + guidelines.
+2) **Provider tool schema**: the actual function/tool declarations sent to the model API.
+
+In pi-mono:
+- System prompt builder: `packages/coding-agent/src/core/system-prompt.ts`
+  - Builds the `Available tools:` list from `toolDescriptions`.
+  - Appends skills and project context.
+- Tool schemas passed to providers:
+  - OpenAI: `packages/ai/src/providers/openai-responses.ts` (`convertTools`)
+  - Anthropic: `packages/ai/src/providers/anthropic.ts` (`convertTools`)
+  - Gemini: `packages/ai/src/providers/google-shared.ts` (`convertTools`)
+- Tool execution loop:
+  - Agent loop: `packages/ai/src/agent/agent-loop.ts`
+  - Validates tool arguments and executes tools, then appends `toolResult` messages.
+
+In Nexus:
+- System prompt append: `src/agents/system-prompt.ts`
+- Tool list injected via `createNexusCodingTools()` in `src/agents/pi-tools.ts`