@vellumai/assistant 0.8.4 → 0.8.6

package/AGENTS.md

@@ -14,7 +14,9 @@ When you introduce a new env var that the assistant process needs to read at run
 
 ## Daemon startup philosophy
 
-The daemon must **never** block startup under _any circumstance_. All possible errors should be logged so that the assistant can recover from it's corrupted state after the fact.
+The daemon must **never** block startup due to **subsystem** failures (DB, Qdrant, plugins, feature flags, etc.). If an individual subsystem fails, log the error and continue in degraded mode so the process remains reachable for health checks and diagnostics.
+
+**Exception — duplicate daemon detection:** If the daemon cannot establish **any** client-facing transport because another daemon already holds both the IPC socket and HTTP port, it must exit immediately. A daemon with no transport is unmanageable (invisible to health checks, unreachable by stop commands) yet still runs background jobs (scheduler, memory worker, background wake) against the shared database, causing duplicate side effects.
 
 ## Post-execution hooks
 
@@ -39,6 +41,7 @@ Routes in `src/runtime/routes/` are being migrated to a **shared `ROUTES` array*
 The CLI and daemon communicate over a Unix domain socket using **length-prefixed binary framing**: each frame is a 4-byte big-endian length followed by a payload. Messages use a JSON envelope `{ id, method, params?, headers? }` for requests and `{ id, result?, error?, headers? }` for responses.
 
 Three response shapes are supported:
+
 - **JSON-only**: a single JSON frame (no `content-length` or `transfer-encoding` header).
 - **Binary**: a JSON envelope with `headers: { "content-length": "<n>" }` followed by one binary frame of exactly `n` bytes.
 - **Chunked streaming**: a JSON envelope with `headers: { "transfer-encoding": "chunked" }` followed by one or more binary frames, terminated by a zero-length frame.
@@ -55,6 +58,35 @@ Some routes are IPC-only (defined in `src/ipc/routes/`, not in the shared array)
 
 The module-level dependency-injection pattern (`registerFooDeps()`) used by some IPC routes is a known antipattern. New IPC-only routes should avoid it.
 
+## SQLite WAL checkpointing
+
+**Never run `PRAGMA wal_checkpoint(TRUNCATE)` from a subprocess** (i.e. from `runAsyncSqlite` or any other path that opens a fresh SQLite connection while the daemon's in-process connection is live).
+
+TRUNCATE has a side effect documented in the SQLite source but not in most surface-level docs: after writing all committed WAL pages to the main database file, if the resulting WAL file is empty, SQLite **unlinks the `.wal` (and `.shm`) file from the directory**. This is fine when the connection running the checkpoint is the only connection — the next opener creates a fresh WAL. It is **not** fine when a peer connection (the daemon) still holds open file descriptors to the original WAL inode: the unlink orphans those fds into a "ghost WAL" only the daemon can reach, every subsequent connection creates a new `.wal`/`.shm` pair on disk, and you get split-brain — daemon and outside readers see different data, no errors logged on either side. The symptom in `/proc/<daemon-pid>/fd/` is the unmistakable `(deleted)` suffix on the daemon's WAL/SHM entries while the main DB fd looks normal.
+
+Use these instead:
+
+- **`PRAGMA wal_checkpoint(FULL)`** when you need flush-completion (export readers, migration snapshots — anywhere downstream reads the main `.db` file). FULL blocks until all committed WAL pages are written to the main DB but does _not_ truncate the WAL file size, so the unlink side effect does not fire.
+- **`PRAGMA wal_checkpoint(PASSIVE)`** for background/amortized housekeeping where partial progress is acceptable. PASSIVE makes no attempt to acquire locks that would conflict with active readers, so it never reaches the empty-WAL state in a contested environment.
+- **Autocheckpoint** (`PRAGMA wal_autocheckpoint = <pages>`, default 1000) handles the routine "keep the WAL bounded" case without any explicit PRAGMA call.
+
+TRUNCATE _is_ safe on the daemon's own long-lived in-process connection (e.g. at startup or during shutdown when no peer connections exist). Today `src/memory/db-init.ts` and `src/daemon/shutdown-handlers.ts` legitimately use it. **Do not copy that pattern into any code path that runs via `runAsyncSqlite` or otherwise spawns a separate process.**
+
 ## Code comments
 
 When writing or updating comments, **do not reference code that has been removed.** Comments should describe the current state of the codebase, not narrate its history. Avoid phrases like "no longer does X", "previously used Y", or "was removed in PR Z" — future readers should not need to understand past implementations to understand the current code.
+
+## Test machinery isolation
+
+**Test files and test helpers (anything under `__tests__/`) must not import from `src/` except for the production module they are testing.** No transitive imports from `src/` into helpers, preloads, or shared test utilities.
+
+The rule exists because test machinery and production code have **inverted invariants**: production assumes the workspace exists and is real; tests assume the workspace is a per-process temp dir that's safe to destroy. When a helper reaches into `src/` to set up state, it ties those invariants together — and a future change to either side can silently break the isolation the helper was supposed to provide. The May 2026 DB-ghost incidents (3 in 4 days) all traced back to test code touching production state through this kind of coupling.
+
+Concretely:
+
+- **Test helpers** (e.g. `src/__tests__/*-test-helpers.ts`) use only node stdlib, `bun:test`, and sibling helpers. If they need to manipulate shared state that production code also reads, both sides declare a typed slot under `globalThis.vellumAssistant.*` and read/write that slot independently. The slot shape is duplicated by design — the helper and the production module both reference the namespace, neither imports the other.
+- **The test preload** (`src/__tests__/test-preload.ts`) is even stricter: it must not import from `src/` at all. Its only static imports are node stdlib, `bun:test`, and helpers in `src/__tests__/`. Importing from a source module risks running its import-time side effects before the workspace override is set.
+- **The preload verifier** (`src/__tests__/test-preload-verifier.ts`) runs after the main preload and asserts the override took effect (`VELLUM_WORKSPACE_DIR` must resolve under `os.tmpdir()`).
+- **Destructive ops** (e.g. `rmSync(dbPath, ...)`) in tests must call `assertNotLiveDb(path)` from `src/__tests__/assert-not-live-db.js` immediately before the destructive call. The check is a per-callsite belt to the preload-verifier suspenders.
+
+When in doubt: **if a piece of test infrastructure can live in `__tests__/` without reaching into `src/`, it must.** Reach for source-code coupling only when there is no `__tests__/`-only alternative that achieves the same invariant.

package/ARCHITECTURE.md

@@ -23,7 +23,7 @@ This document owns assistant-runtime architecture details. The repo-level archit
 
 ### Safe Storage Limits
 
-Safe storage limits are gated by the assistant feature flag `safe-storage-limits`, default off. When the flag is off, the disk pressure guard reports a disabled status and no runtime path blocks work, injects cleanup guidance, or changes tool access.
+Safe storage limits protect the workspace volume from running out of disk. The disk pressure guard samples storage continuously and blocks work, injects cleanup guidance, and narrows tool access while the workspace is critically full.
 
 **Disk pressure state:** `src/daemon/disk-pressure-guard.ts` samples workspace storage usage every 60 seconds through `src/util/disk-usage.ts`. At or above the 95% critical threshold it creates an in-memory lock with `lockId`, usage snapshot, `acknowledged`, `overrideActive`, `effectivelyLocked`, and the blocked capabilities `agent-turns`, `background-work`, and `remote-ingress`. The lock clears when usage drops below the threshold or the process restarts. `acknowledgeDiskPressureLock()` only lets the guardian enter cleanup mode; `overrideDiskPressureLock()` requires the exact phrase `I understand the risks` and disables the effective lock while usage remains critical.
 
@@ -807,7 +807,7 @@ The assistant feature-flag resolver (`src/config/assistant-feature-flags.ts`) is
 
 1. `~/.vellum/protected/feature-flags.json` overrides (local) or gateway HTTP API (Docker/containerized) — written by the gateway's PATCH endpoint
 2. Defaults registry `defaultEnabled` — from the unified registry (`meta/feature-flags/feature-flag-registry.json`, filtered to `scope: "assistant"`)
-3. `true` — unknown/undeclared flags with no persisted override default to enabled
+3. `false` — unknown/undeclared flags with no override fail closed
 
 **Storage:** Flags are persisted in `~/.vellum/protected/feature-flags.json` (local) or `GATEWAY_SECURITY_DIR/feature-flags.json` (Docker), managed by the gateway's `/v1/feature-flags` API (see [`gateway/ARCHITECTURE.md`](../gateway/ARCHITECTURE.md)). The daemon's config watcher monitors the protected directory and hot-reloads flag changes, so mutations take effect on the next tool resolution or session.
 
@@ -1898,7 +1898,7 @@ graph TB
     end
 
     subgraph "SSE Transport"
-        SSE_ROUTE["SSE Route<br/>GET /v1/events[?conversationKey=...]<br/>(events-routes.ts)<br/>──────────────────────<br/>ReadableStream + CountQueuingStrategy(16)<br/>Heartbeat every 30 s<br/>Slow-consumer shed"]
+        SSE_ROUTE["SSE Route<br/>GET /v1/events[?conversationId=... | ?conversationKey=...]<br/>(events-routes.ts)<br/>──────────────────────<br/>ReadableStream + CountQueuingStrategy(16)<br/>Heartbeat every 30 s<br/>Slow-consumer shed"]
     end
 
     subgraph "Clients"

package/bunfig.toml

@@ -3,4 +3,9 @@ exact = true
 
 [test]
 root = "./src"
-preload = ["./src/__tests__/test-preload.ts"]
+# Order matters: the verifier reads state set by the main preload, so it
+# MUST be listed second. See test-preload-verifier.ts for context.
+preload = [
+  "./src/__tests__/test-preload.ts",
+  "./src/__tests__/test-preload-verifier.ts",
+]

package/docs/browser-use-architecture-phase2.md

@@ -213,5 +213,5 @@ CDP factory: candidate succeeded, backend is now sticky
 Filter runtime logs with:
 
 ```bash
-grep "cdp-factory" ~/.vellum/workspace/data/logs/vellum.log
+grep -h "cdp-factory" "$VELLUM_WORKSPACE_DIR"/data/logs/assistant-*.log
 ```

package/docs/credential-execution-service.md

@@ -172,11 +172,11 @@ Audit logs record every materialization event with: grant ID, credential ID, too
 
 CES and the assistant share contract definitions and credential-storage abstractions through three private packages in `packages/`:
 
-| Package                          | Purpose                                                                                                                                                                                                                    | Consumers                            |
-| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
-| `@vellumai/service-contracts`    | RPC protocol types, method names, protocol version constant, grant shapes, credential handle types, and rendering helpers. Consumed via explicit domain subpaths (e.g. `@vellumai/service-contracts/credential-rpc`). | `assistant/`, `credential-executor/` |
-| `@vellumai/credential-storage`   | Credential store read API (static secrets and OAuth runtime), unified credential handle abstraction                                                                                                                         | `assistant/`, `credential-executor/` |
-| `@vellumai/egress-proxy`         | Session-scoped egress proxy lifecycle (create, start, stop, env-var injection)                                                                                                                                             | `assistant/`, `credential-executor/` |
+| Package                        | Purpose                                                                                                                                                                                                               | Consumers                            |
+| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
+| `@vellumai/service-contracts`  | RPC protocol types, method names, protocol version constant, grant shapes, credential handle types, and rendering helpers. Consumed via explicit domain subpaths (e.g. `@vellumai/service-contracts/credential-rpc`). | `assistant/`, `credential-executor/` |
+| `@vellumai/credential-storage` | Credential store read API (static secrets and OAuth runtime), unified credential handle abstraction                                                                                                                   | `assistant/`, `credential-executor/` |
+| `@vellumai/egress-proxy`       | Session-scoped egress proxy lifecycle (create, start, stop, env-var injection)                                                                                                                                        | `assistant/`, `credential-executor/` |
 
 These packages are the **only** allowed shared-code path between the assistant and CES. Direct source imports between `assistant/` and `credential-executor/` remain banned. The packages are built locally via `workspace:*` references and copied into the CES Docker image at build time (`COPY packages/ ...` in `credential-executor/Dockerfile`).
 
@@ -229,7 +229,7 @@ These invariants are enforced by guard tests and code review:
 4. **Grants and audit logs are CES-internal**: The assistant cannot read CES grant tables or audit logs directly. CES exposes grant status and audit summaries via RPC responses.
 5. **No generic authenticated HTTP clients in secure commands**: `curl`, `wget`, `httpie`, interpreters, and shell trampolines are structurally denied as secure command entrypoints. This is checked at manifest validation and re-checked at execution time.
 6. **Managed CES container runs as non-root**: The CES Docker image runs as `uid 1001` (user `ces`). The CES data volume is owned by this user.
-7. **Single-connection bootstrap socket**: In managed mode, CES accepts exactly one connection on the bootstrap socket, then unlinks it. No second process can connect.
+7. **Single active bootstrap connection**: In managed mode, CES accepts one connection on the bootstrap socket and unlinks the socket path while that connection is live, so no second process can connect concurrently. CES is a long-lived sidecar: when the active session ends (the assistant disconnects or its container restarts), CES re-binds the socket and awaits the assistant's reconnection rather than shutting down. At most one connection is ever active; the sidecar only exits on SIGTERM/SIGINT.
 
 ## Rollout
 

package/docs/plugins.md

@@ -548,9 +548,10 @@ middleware. Each is optional.
 
 An array of `Tool` objects. The bootstrap registers them with the global
 tool registry after `init()` succeeds, stamping `origin: "plugin"` and
-`ownerPluginId: <plugin.name>` so they live in a ref-count namespace
-disjoint from real skills (a plugin whose `manifest.name` happens to
-match a skill id cannot collide with that skill's registrations).
+`owner: { kind: "plugin", id: <plugin.name> }` so they live in a ref-count
+namespace disjoint from real skills (a plugin whose `manifest.name`
+happens to match a skill id cannot collide with that skill's
+registrations).
 
 ```typescript
 const myPlugin: Plugin = {

package/knip.json

@@ -4,7 +4,8 @@
     "src/**/__tests__/**/*.ts",
     "scripts/**/*.ts",
     "src/daemon/main.ts!",
-    "src/plugin-api/index.ts!"
+    "src/plugin-api/index.ts!",
+    "src/api/index.ts!"
   ],
   "project": ["src/**/*.ts!", "src/**/*.tsx!", "scripts/**/*.ts"],
   "ignoreDependencies": [

package/node_modules/@vellumai/skill-host-contracts/src/client.ts

@@ -502,9 +502,7 @@ export class SkillHostClient implements SkillHost {
 
     socket.on("close", () => {
       this.socket = null;
-      const err = new Error(
-        "SkillHostClient: socket closed before response",
-      );
+      const err = new Error("SkillHostClient: socket closed before response");
       for (const { reject, timer } of this.pending.values()) {
         clearTimeout(timer);
         reject(err);
@@ -1057,9 +1055,7 @@ export class SkillHostClient implements SkillHost {
         // Fire-and-forget close RPC — we don't await the ack because the
         // server also tears down on socket close, which is the fallback.
         if (self.socket && !self.socket.destroyed) {
-          self
-            .call(SUBSCRIBE_CLOSE_METHOD, { subscribeId: id })
-            .catch(swallow);
+          self.call(SUBSCRIBE_CLOSE_METHOD, { subscribeId: id }).catch(swallow);
         }
       },
     };
@@ -1081,10 +1077,6 @@ export class SkillHostClient implements SkillHost {
             defaultRiskLevel: t.defaultRiskLevel,
             category: t.category,
             executionTarget: t.executionTarget,
-            executionMode: t.executionMode ?? "proxy",
-            ownerSkillId: t.ownerSkillId ?? this.options.skillId,
-            ownerSkillBundled: t.ownerSkillBundled,
-            ownerSkillVersionHash: t.ownerSkillVersionHash,
           };
         });
         // Cache the provider so `skill.dispatch_tool` can resolve a tool
@@ -1094,10 +1086,17 @@ export class SkillHostClient implements SkillHost {
           "skill.dispatch_tool",
           this.dispatchTool.bind(this),
         );
+        // Skill ownership travels at the params level (not per-tool) — the
+        // daemon-side registry stamps the owner from `skillId` into its
+        // own `ownersByName` map keyed by tool name. The `Tool` objects on
+        // the wire stay free of owner metadata so callers cannot spoof
+        // ownership by forging a field on the manifest.
+        //
         // Fire-and-forget; registration failures surface in the daemon log.
-        this.call("host.registries.register_tools", { tools: manifests }).catch(
-          swallow,
-        );
+        this.call("host.registries.register_tools", {
+          skillId: this.options.skillId,
+          tools: manifests,
+        }).catch(swallow);
       },
       registerSkillRoute: (route: SkillRoute): SkillRouteHandle => {
         // Cache the route by its regex source — `skill.dispatch_route` uses

package/node_modules/@vellumai/skill-host-contracts/src/skill-host.ts

@@ -194,6 +194,9 @@ export interface ProvidersFacet {
 // Memory
 // ---------------------------------------------------------------------------
 
+/** Valid message roles for `memory.addMessage`. */
+export type MessageRole = "user" | "assistant" | "system";
+
 /**
  * Callable signature for `memory.addMessage`. Mirrors the daemon's
  * `addMessage()` (in `assistant/src/memory/conversation-crud.ts`) shape.
@@ -202,7 +205,7 @@ export interface ProvidersFacet {
  */
 export type InsertMessageFn = (
   conversationId: string,
-  role: string,
+  role: MessageRole,
   content: string,
   metadata?: Record<string, unknown>,
   opts?: { skipIndexing?: boolean },

package/node_modules/@vellumai/skill-host-contracts/src/tool-types.ts

@@ -28,6 +28,22 @@
 
 export type ExecutionTarget = "sandbox" | "host";
 
+/** The kind of extension that owns a tool. Core tools have no owner. */
+export type OwnerKind = "skill" | "mcp" | "plugin";
+
+/**
+ * Identifies which extension owns a tool (skill / plugin / MCP server).
+ * Tracked by the daemon's tool registry keyed by tool name, not stored on
+ * the `Tool` object itself. The registry function `getToolOwner(name)`
+ * returns this shape; ownership is established at `register_tools` time
+ * via the `host.registries.register_tools` IPC method.
+ */
+export interface OwnerInfo {
+  kind: OwnerKind;
+  /** ID of the owning extension (skill id / plugin name / MCP server id). */
+  id: string;
+}
+
 export enum RiskLevel {
   Low = "low",
   Medium = "medium",
@@ -419,20 +435,6 @@ export interface Tool {
   description: string;
   category: string;
   defaultRiskLevel: RiskLevel;
-  /** When set to 'proxy', the tool is forwarded to a connected client rather than executed locally. */
-  executionMode?: "local" | "proxy";
-  /** Whether this tool is a core built-in, provided by a skill, contributed by a plugin, or from an MCP server. */
-  origin?: "core" | "skill" | "mcp" | "plugin";
-  /** If origin is 'skill', the ID of the owning skill. */
-  ownerSkillId?: string;
-  /** If origin is 'mcp', the ID of the owning MCP server. */
-  ownerMcpServerId?: string;
-  /** If origin is 'plugin', the name of the owning plugin. */
-  ownerPluginId?: string;
-  /** Content-hash of the owning skill's source at registration time. */
-  ownerSkillVersionHash?: string;
-  /** Whether the owning skill is bundled with the daemon (trusted first-party). */
-  ownerSkillBundled?: boolean;
   /** Declared execution target from the skill manifest. Used by resolveExecutionTarget
    * to accurately label lifecycle events for skill-provided tools. */
   executionTarget?: ExecutionTarget;