@vellumai/assistant 0.8.12 → 0.9.0-staging.2

package/docs/architecture/integrations.md

@@ -486,14 +486,14 @@ The proxy subsystem is fully wired, including credential injection. The session
 
 ## Conversation Disk View — Filesystem-Based Conversation Access
 
-The conversation disk view projects conversation metadata, messages, and attachments to a browsable filesystem layout under `~/.vellum/workspace/conversations/`. This enables the assistant to search, read, and manipulate conversation data (including media attachments) using standard file tools (`read_file`, `glob`, `grep`) rather than dedicated asset search tools.
+The conversation disk view projects conversation metadata, messages, and attachments to a browsable filesystem layout under `$VELLUM_WORKSPACE_DIR/conversations/`. This enables the assistant to search, read, and manipulate conversation data (including media attachments) using standard file tools (`read_file`, `glob`, `grep`) rather than dedicated asset search tools.
 
 ### Directory Layout
 
 Each conversation is projected to a directory named `{isoDate}_{id}`:
 
 ```
-~/.vellum/workspace/conversations/
+$VELLUM_WORKSPACE_DIR/conversations/
   2025-01-15T10-30-00.000Z_abc123/
     meta.json             # Conversation metadata (id, title, type, channel, timestamps)
     messages.jsonl        # Flattened message log (one JSON object per line)
@@ -506,7 +506,7 @@ Each conversation is projected to a directory named `{isoDate}_{id}`:
 
 The disk view is updated at the daemon level, not automatically by the DB CRUD layer. Conversation creation, metadata updates, and deletion are synced from `conversation-crud.ts`, but message sync (`syncMessageToDisk`) is only called from daemon-level code paths (e.g. `conversation-messaging.ts`) — not from the CRUD `addMessage()` function. This means `messages.jsonl` reflects messages processed through the daemon's messaging pipeline, not every message write. All disk writes are best-effort; failures are logged but never thrown, so the disk view cannot break DB operations.
 
-> **Privacy note:** Conversation disk-view files live under `~/.vellum/workspace/conversations/`. Workspace files are not included in diagnostic log exports ("Send logs to Vellum"). For conversation-scoped exports, conversation data is included as structured JSON tables (e.g. `messages.json`, `llm-request-logs.json`), not as a raw database dump.
+> **Privacy note:** Conversation disk-view files live under `$VELLUM_WORKSPACE_DIR/conversations/`. Workspace files are not included in diagnostic log exports ("Send logs to Vellum"). For conversation-scoped exports, conversation data is included as structured JSON tables (e.g. `messages.json`, `llm-request-logs.json`), not as a raw database dump.
 
 ```mermaid
 sequenceDiagram

package/docs/architecture/memory.md

@@ -445,7 +445,7 @@ graph TB
 
 ### Key design decisions
 
-- **Scope**: Sandbox workspace only (`~/.vellum/workspace`). Non-recursive — only top-level directories.
+- **Scope**: Sandbox workspace only (`$VELLUM_WORKSPACE_DIR`). Non-recursive — only top-level directories.
 - **Bounded**: Maximum 120 directory entries (`MAX_TOP_LEVEL_ENTRIES`). Excess is truncated with a note.
 - **Prepend, not append**: The workspace block is prepended to the user message content so that Anthropic cache breakpoints continue to land on the trailing user text block, preserving prompt cache efficiency.
 - **Persists in history**: The injected `<workspace>` block persists in conversation history, giving the model workspace grounding across turns. Legacy `<workspace_top_level>` blocks from pre-change history are stripped for backward compatibility.
@@ -512,7 +512,7 @@ graph TB
 
 ## Workspace Git Tracking — Change Management
 
-The workspace sandbox (`~/.vellum/workspace`) is automatically tracked by a per-workspace git repository. Every file change made by the assistant is captured in structured commits, providing a full audit trail and natural undo/history exploration via standard git commands.
+The workspace sandbox (`$VELLUM_WORKSPACE_DIR`) is automatically tracked by a per-workspace git repository. Every file change made by the assistant is captured in structured commits, providing a full audit trail and natural undo/history exploration via standard git commands.
 
 ### Architecture overview
 

package/docs/architecture/security.md

@@ -94,13 +94,13 @@ When `autoApproveUpTo` is `"none"`, `skill_load` without a matching rule is alwa
 
 Skills that use existing system tools (`bash`, `file_read`, `web_fetch`, etc.) **do not expand the assistant's capability surface**. The assistant already has access to these tools based on its trust rules; a skill that teaches `curl https://api.example.com/v1/endpoint -d "..."` presents identical risk to a user asking the assistant to run that same command directly. The risk is governed entirely by the bash risk classifier and the user's `autoApproveUpTo` threshold — the same path as any other bash invocation.
 
-The threat vectors that skills *do* introduce are:
+The threat vectors that skills _do_ introduce are:
 
-| Threat | Mitigation |
-|---|---|
+| Threat                                                                                                  | Mitigation                                                                                                              |
+| ------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
 | **Skill source file mutations** — editing a skill's own source files could inject behavior on next load | `isSkillSourcePath()` escalates `file_write`/`file_edit` targeting skill roots to **High**, requiring explicit approval |
-| **New skill-origin tools** — tools registered and invoked through the skill IPC contract | Skill-origin tools with no matching trust rule are always **prompted**, regardless of their risk level |
-| **Credential storage** — a skill collecting and storing an API key | Mediated by the `credential_store` approval flow |
+| **New skill-origin tools** — tools registered and invoked through the skill IPC contract                | Skill-origin tools with no matching trust rule are always **prompted**, regardless of their risk level                  |
+| **Credential storage** — a skill collecting and storing an API key                                      | Mediated by the secure credential prompt flow (`assistant credentials prompt`)                                          |
 
 What skills do **not** introduce:
 
@@ -278,11 +278,11 @@ The `allowOneTimeSend` config gate (default: `false`) enables a secondary "Send
 
 ### Storage Layout
 
-| Component           | Location                                             | What it stores                                                                                                                                                   |
-| ------------------- | ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Secret values       | CES credential store or encrypted file store         | Encrypted credential values keyed as `credential/{service}/{field}`. Stored via CES RPC (primary), CES HTTP (containerized), or encrypted file store (fallback). |
-| Credential metadata | `~/.vellum/workspace/data/credentials/metadata.json` | Service, field, label, policy (allowedTools, allowedDomains), timestamps                                                                                         |
-| Config              | `~/.vellum/workspace/config.*`                       | `secretDetection` settings: enabled, blockIngress, allowOneTimeSend                                                                                              |
+| Component           | Location                                               | What it stores                                                                                                                                                   |
+| ------------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Secret values       | CES credential store or encrypted file store           | Encrypted credential values keyed as `credential/{service}/{field}`. Stored via CES RPC (primary), CES HTTP (containerized), or encrypted file store (fallback). |
+| Credential metadata | `$VELLUM_WORKSPACE_DIR/data/credentials/metadata.json` | Service, field, label, policy (allowedTools, allowedDomains), timestamps                                                                                         |
+| Config              | `$VELLUM_WORKSPACE_DIR/config.*`                       | `secretDetection` settings: enabled, blockIngress, allowOneTimeSend                                                                                              |
 
 ### Key Files
 

package/docs/runbook-trusted-contacts.md

@@ -90,7 +90,7 @@ Access requests are stored in the `channel_guardian_approval_requests` table. Us
 ### Via SQLite CLI
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT id, channel, requester_external_user_id, requester_chat_id, \
    guardian_external_user_id, status, tool_name, created_at, expires_at \
    FROM channel_guardian_approval_requests \
@@ -101,7 +101,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Check all access requests (including resolved)
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT id, channel, requester_external_user_id, status, \
    decided_by_external_user_id, created_at \
    FROM channel_guardian_approval_requests \
@@ -114,7 +114,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 Verification challenges are stored in `channel_verification_sessions`. Active sessions have `status = 'awaiting_response'` and `expires_at > now`.
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT id, channel, status, identity_binding_status, \
    expected_external_user_id, expected_chat_id, expected_phone_e164, \
    expires_at, created_at \
@@ -158,7 +158,7 @@ curl -s -X PATCH "$BASE/v1/contact-channels/$CHANNEL_ID" \
 If the HTTP API is unavailable:
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "UPDATE contact_channels \
    SET status = 'revoked', revoked_reason = 'Emergency operator revocation', \
    updated_at = $(date +%s)000 \
@@ -172,7 +172,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 If a user is getting "invalid or expired code" errors, they may be rate-limited:
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT * FROM channel_guardian_rate_limits \
    WHERE external_user_id = 'TARGET_USER_ID' \
    OR chat_id = 'TARGET_CHAT_ID' \
@@ -182,7 +182,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Reset rate limits for a user
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "DELETE FROM channel_guardian_rate_limits \
    WHERE external_user_id = 'TARGET_USER_ID' AND channel = 'telegram';"
 ```
@@ -190,7 +190,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Check verification challenge state
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT id, channel, status, identity_binding_status, \
    expected_external_user_id, expected_chat_id, expected_phone_e164, \
    expires_at, consumed_by_external_user_id \
@@ -215,7 +215,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Check if the access request notification was delivered
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT ne.id, ne.source_event_name, ne.dedupe_key, ne.created_at, \
    nd.channel, nd.status, nd.confidence \
    FROM notification_events ne \
@@ -227,7 +227,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Check delivery records
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT ndel.id, ndel.channel, ndel.status, ndel.error_message, \
    ndel.created_at, ne.source_event_name \
    FROM notification_deliveries ndel \
@@ -239,7 +239,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 ### Check lifecycle signals
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "SELECT source_event_name, source_channel, dedupe_key, created_at \
    FROM notification_events \
    WHERE source_event_name LIKE 'ingress.trusted_contact.%' \
@@ -295,7 +295,7 @@ curl -s -X POST "$BASE/v1/contacts" \
 Expired sessions are already invisible to the verification flow (filtered by `expires_at`), but you can clean them up:
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "DELETE FROM channel_verification_sessions \
    WHERE expires_at < $(date +%s)000 \
    AND status IN ('awaiting_response', 'pending_bootstrap');"
@@ -306,7 +306,7 @@ sqlite3 ~/.vellum/workspace/data/db/assistant.db \
 The `sweepExpiredGuardianApprovals()` timer handles this automatically every 60 seconds, but manual cleanup:
 
 ```bash
-sqlite3 ~/.vellum/workspace/data/db/assistant.db \
+sqlite3 "$VELLUM_WORKSPACE_DIR"/data/db/assistant.db \
   "UPDATE channel_guardian_approval_requests \
    SET status = 'expired' \
    WHERE status = 'pending' AND expires_at < $(date +%s)000;"

package/docs/skills.md

@@ -6,7 +6,7 @@ This document describes the security model for the Vellum Assistant skill system
 
 Skills extend the assistant's capabilities by providing instructions (via `SKILL.md`) and optional custom tools (via `TOOLS.json`). Skills can be **bundled** (shipped with the application), **managed** (user-installed via `scaffold_managed_skill`), **workspace** (project-local), or **extra** (additional directories configured by the user).
 
-For managed skills, the installed source of truth is a valid directory at `~/.vellum/workspace/skills/<id>/` containing a top-level `SKILL.md` with standardized frontmatter. The assistant parses that frontmatter at startup and when skill directories change, then seeds Memory V2 skill entries under `skills/<id>` so the assistant can discover available skills from memory. The legacy `SKILLS.md` index is removed by workspace migration and is no longer created by install or scaffold paths.
+For managed skills, the installed source of truth is a valid directory at `$VELLUM_WORKSPACE_DIR/skills/<id>/` containing a top-level `SKILL.md` with standardized frontmatter. The assistant parses that frontmatter at startup and when skill directories change, then seeds Memory V2 skill entries under `skills/<id>` so the assistant can discover available skills from memory. The legacy `SKILLS.md` index is removed by workspace migration and is no longer created by install or scaffold paths.
 
 Because skills can introduce arbitrary tool behavior, they are subject to stricter permission defaults than core tools.
 
@@ -70,7 +70,7 @@ Version-specific rules are more secure but require re-approval after every skill
 
 Writing to skill source files is treated as a **high-risk** operation by the risk classifier. The `isSkillSourcePath()` function detects whether a file path falls under any known skill directory:
 
-- **Managed skills**: `~/.vellum/workspace/skills/`
+- **Managed skills**: `$VELLUM_WORKSPACE_DIR/skills/`
 - **Bundled skills**: The application's built-in `bundled-skills/` directory
 - **Workspace skills**: Project-local skill directories
 - **Extra skills**: Additional roots configured by the user
@@ -116,10 +116,10 @@ Acceptance is idempotent and recorded in `trust.json`. The bundle does not inclu
 
 Tools can execute in two contexts:
 
-| Target    | Description                                                                            |
-| --------- | -------------------------------------------------------------------------------------- |
-| `sandbox` | Isolated execution within `~/.vellum/workspace` (Docker container or OS-level sandbox) |
-| `host`    | Direct execution on the host machine                                                   |
+| Target    | Description                                                                              |
+| --------- | ---------------------------------------------------------------------------------------- |
+| `sandbox` | Isolated execution within `$VELLUM_WORKSPACE_DIR` (Docker container or OS-level sandbox) |
+| `host`    | Direct execution on the host machine                                                     |
 
 Trust rules can include an `executionTarget` field to bind the rule to a specific context. A rule without `executionTarget` matches both sandbox and host invocations.
 

package/docs/workflows-testing.md

@@ -0,0 +1,221 @@
+# Workflows — Manual e2e / Live-Verification Runbook
+
+This runbook covers the parts of the workflow engine that automated tests do not:
+real provider calls, real journaled resume across a restart, capability
+containment under a live model, flag-off inertness, and persona-leaf voice. Unit
+and integration coverage lives under `assistant/src/workflows/*.test.ts` and
+`assistant/src/__tests__/`; this is the human-in-the-loop pass that runs **last**,
+on throwaway instances first, and only touches a real inbox or a production
+instance at the very end.
+
+Work through the steps in order. The `workflows` flag is **off by default**, so
+every step that exercises the engine first enables it on a throwaway instance.
+
+> **Safety rule:** Nothing in this runbook touches a real inbox, a real workspace,
+> or a production instance until every throwaway-instance step has passed. The
+> production/persona instance is the **last** step.
+
+---
+
+## 1. Hatch a throwaway instance and enable the flag
+
+Hatch a disposable Docker instance built from local source:
+
+```
+vellum hatch --remote docker --source .
+```
+
+Note the instance name it prints (e.g. `vellum-<adjective>-<animal>`). Use it as
+`--assistant <name>` for everything below, or set it active.
+
+Enable the `workflows` flag via that instance's feature-flag override file. The
+override lives in the instance's `protected/feature-flags.json` (overrides here
+win over the registry default — see the feature-flag-overrides gotcha):
+
+```jsonc
+// .../<instance>/protected/feature-flags.json
+{ "workflows": true }
+```
+
+Restart the instance so it re-reads the override, then confirm it is up:
+
+```
+vellum ps
+```
+
+Sanity-check that the surface is now live: `vellum workflows runs --assistant <name>`
+should return an (empty) table rather than a 404.
+
+---
+
+## 2. Synthetic-corpus run (no Gmail/Slack dependency)
+
+Drive a `run_workflow` over a **fixture item list passed as `args`** — no external
+integration. Ask the assistant (via `vellum events`-visible conversation, the app,
+or a CLI message) to run a small workflow such as the triage example from
+[`workflows.md`](./workflows.md), passing a dozen synthetic items in `args.items`.
+
+Watch the run:
+
+```
+vellum workflows runs  --assistant <name>          # find the runId
+vellum workflows show  <run-id> --assistant <name>  # status + agent/token counts
+vellum events          --assistant <name>           # workflow_progress / workflow_completed
+```
+
+Confirm:
+
+- The run reaches `completed`.
+- `agentsSpawned` matches the number of leaves the script should have launched.
+- A completion summary is injected back into the originating conversation.
+
+---
+
+## 3. Inspect the evidence
+
+Query the instance's database directly (the live DB is under the instance's
+`workspace/data/db/assistant.db`).
+
+**Leaf cost attribution** — confirm leaves ran on the cost-optimized model and
+tally spend:
+
+```sql
+SELECT model, COUNT(*) AS calls,
+       SUM(input_tokens)  AS in_tok,
+       SUM(output_tokens) AS out_tok,
+       SUM(estimated_cost_usd) AS usd
+FROM llm_usage_events
+WHERE call_site = 'workflowLeaf'
+GROUP BY model;
+```
+
+**Request shape** — confirm the schema path forced a tool and used a cheap model;
+for persona leaves, confirm identity + memory were injected:
+
+```sql
+SELECT id, created_at
+FROM llm_request_logs
+ORDER BY created_at DESC
+LIMIT 20;
+```
+
+Inspect a few `request_payload` bodies:
+
+- A **schema** leaf request carries `tool_choice` forcing the synthetic
+  `emit_result` tool and a cheap model id.
+- A **persona** leaf request's system prompt carries the assistant identity and a
+  `<memory>` block; an **anonymous** leaf's does not.
+
+---
+
+## 4. Restart-resume (journal replay)
+
+Start a **long** synthetic run (enough leaves that it is still in flight for a few
+seconds). While it is running, restart the assistant with **SIGTERM**:
+
+> **Never SIGKILL the assistant.** SIGKILL leaves WAL pages unmerged and forces a
+> costly recovery on the next start (and can corrupt an in-flight run). Use a
+> graceful stop / `vellum sleep` / SIGTERM with grace.
+
+After restart, the run that was in flight when the process stopped is no longer
+`running` — at startup the assistant reconciles every orphaned `running` row to
+`interrupted` (status only; the accounting counters are preserved). Resume is
+**not automatic**. Confirm the run shows up as interrupted, then trigger an
+explicit resume:
+
+```bash
+# The crashed run is now interrupted, not running.
+vellum workflows runs --status interrupted
+
+# Trigger the resume (or: manage_workflows with action="resume" from chat).
+vellum workflows resume <run-id>
+```
+
+Resuming re-invokes the engine with the **same `runId`**, so the completed prefix
+replays from the journal. Assert:
+
+- Resuming does **not** double the `agentsSpawned` count or the
+  `llm_usage_events` rows for `call_site = 'workflowLeaf'` — the completed prefix
+  replays from the journal rather than re-spawning leaves. The agent count
+  **carries** across the restart (it seeds from the persisted run row) instead of
+  resetting to zero.
+- Only leaves that had not completed before the restart produce new usage rows.
+- The run transitions `interrupted` → `running` → a terminal status.
+
+```sql
+-- Before and after the resume, this count should not grow for already-done leaves.
+SELECT COUNT(*) FROM llm_usage_events WHERE call_site = 'workflowLeaf';
+
+-- The journal holds one row per completed (run_id, seq); resuming does not duplicate them.
+SELECT run_id, COUNT(*) FROM workflow_journal GROUP BY run_id;
+```
+
+---
+
+## 5. Capability containment
+
+Author a workflow whose leaf tries to use a **side-effecting tool that is absent
+from its manifest** (e.g. a leaf prompt that pushes the model toward `file_write`
+or a send tool, with `capabilities.tools` left empty). Run it and confirm:
+
+- The forbidden tool invocation is **hard-denied** inside the leaf — it never
+  executes. The leaf gets an error result, not a permission prompt (there are no
+  per-call prompts inside a run).
+- Declaring a **forbidden** tool (`subagent_spawn`, `run_workflow`,
+  `manage_workflows`, `manage_secure_command_tool`) in `capabilities.tools` fails
+  the run synchronously at start — `run_workflow` returns an error and **no**
+  `running` row is created.
+
+---
+
+## 6. Flag-off inertness
+
+On a **default** instance (no `workflows` override, or set back to `false` and
+restart), confirm the whole surface is gone:
+
+- `run_workflow` and `manage_workflows` are **absent** from the tool set (the
+  assistant cannot call them).
+- The routes 404:
+
+  ```
+  vellum workflows runs --assistant <default-instance>   # request fails (404)
+  ```
+
+- A scheduler `workflow`-mode job is **rejected** (the engine gate throws before
+  any run is launched).
+
+---
+
+## 7. Real-data smoke (throwaway TEST account)
+
+Only after steps 1–6 pass: run the full path against a **throwaway TEST Google or
+Slack account** with about a dozen messages — never a real account. Drive the
+end-to-end flow:
+
+1. The assistant enumerates the relevant skill (e.g. `gmail`).
+2. It authors and launches a `run_workflow` over the real messages.
+3. Leaves synthesize results.
+4. Any side-effecting action (label, draft, reply) goes through the declared
+   capability manifest and the normal audited-action path.
+
+Confirm the run completes, the evidence queries from step 3 look right against
+real content, and every side effect was one the manifest declared.
+
+---
+
+## 8. Production / persona instance — LAST
+
+Flip the `workflows` flag on a real (e.g. persona) instance **only after** the
+throwaway instances above pass, and only after the instance is on current code:
+
+1. `git pull` and **restart** the instance so it runs the merged code (a restart
+   alone re-runs stale code — see the deploy gotcha).
+2. Enable the flag and restart.
+3. Start with the **smallest real slice** — a few items, dry-run actions where the
+   tool supports it — before any larger or side-effecting run.
+4. **Human-eval the persona-leaf voice**: read a persona leaf's output and confirm
+   it reads as the assistant, not as a generic worker.
+
+Stop and reassess if any run's `agentsSpawned` approaches `maxAgentsPerRun`, if
+spend (step 3 query) is higher than expected, or if any side effect was not one
+the manifest declared.