@hybridaione/hybridclaw 0.13.1 → 0.15.0

package/CHANGELOG.md

@@ -2,6 +2,176 @@
 
 ## Unreleased
 
+## [0.15.0](https://github.com/HybridAIOne/hybridclaw/tree/v0.15.0) - 2026-04-29
+
+### Added
+
+- **Backup and restore CLI**: `hybridclaw backup` creates WAL-safe runtime-home
+  archives and `hybridclaw backup restore` validates manifests before
+  rehydrating `~/.hybridclaw` on a fresh or recovered host.
+- **Brand-voice output guard**: A repo-shipped `brand-voice` plugin can flag,
+  rewrite, or block off-brand final responses using configured voice rules,
+  banned phrases or patterns, required phrases, and optional classifier/rewriter
+  models. The bundled `brand-voice` skill helps agents draft within those rules
+  before the output guard fires.
+- **Production Salesforce skill**: The bundled Salesforce skill now ships a
+  fuller read-only helper, metadata/query references, eval scenarios, and
+  server-side secret placeholder handling for inspecting org schema and SOQL
+  records without writing credentials to disk.
+- **Tailscale Funnel tunnel provider and admin status**: Local deployments can
+  use `deployment.tunnel.provider=tailscale`, with `TS_AUTHKEY` resolved from
+  encrypted runtime secrets when needed and kept out of process arguments. The
+  admin console surfaces public URL and tunnel status alongside the existing
+  gateway controls.
+- **Web chat model switcher**: The chat composer can switch models from the
+  browser using discovered provider catalogs, provider icons, model capability
+  metadata, and the same active-session routing used by local slash commands.
+- **Agent org chart, team revisions, and chronological CVs**: Agent metadata
+  can model role, reporting, delegation, and peer relationships; admin agent
+  pages keep restorable team-structure revisions; observed skill history can
+  refresh per-agent CV output.
+- **Warm process pool**: Host and container runners can keep a bounded adaptive
+  pool of idle runtime processes for recently active agents, reducing cold-start
+  latency while respecting max-idle, startup-claim, config-change, and
+  memory-pressure limits.
+- **Trace judge and trace preparation**: Local eval workflows can prepare
+  redacted traces and dispatch them through an auxiliary judge model for skill,
+  leak, and output-quality evaluation foundations.
+- **Config value inspection**: `hybridclaw config get <key>` and `/config get
+  <key>` return one resolved runtime config value without dumping the full
+  config file.
+- **GPT-5.5 model support**: Static and Codex-discovered model catalogs include
+  `gpt-5.5`, `gpt-5.5-pro`, and `openai-codex/gpt-5.5`.
+
+### Changed
+
+- **Gateway health is less fragile**: Gateway status and health endpoints rely
+  on cached provider checks instead of blocking on live model-provider probes,
+  so transient provider failures no longer make the local gateway look down.
+- **Token usage recording is buffered**: Usage events are normalized,
+  size-capped, and batch-flushed asynchronously to reduce hot-path overhead
+  while preserving audit records.
+- **Runtime secrets are scrubbed more consistently**: Host/container agent
+  runtimes share sensitive environment filtering and web-search credential
+  injection so Brave, Perplexity, and Tavily keys are passed only through the
+  intended secret channels.
+- **Trajectory capture is stricter**: Stored skill trajectories run through
+  PII, secret, and confidential-info redaction, and retention can be capped
+  globally or per tenant.
+- **Docs are consolidated under `docs/content`**: The duplicate legacy
+  `docs/development` tree was removed after the content moved into the current
+  docs hierarchy.
+
+### Fixed
+
+- **Loopback API auth is no longer bypassed**: Local OpenAI-compatible API
+  requests require `WEB_API_TOKEN` or `GATEWAY_API_TOKEN`; loopback address
+  alone is not treated as authentication.
+- **Local TUI and gateway token handling is safer**: Generated gateway tokens
+  are persisted once with serialized creation, shared with local TUI/eval
+  clients, and no longer rewritten during later config reloads.
+- **Ngrok tunnel reconnects are quieter**: Reconnect errors are deduplicated by
+  normalized cause, audit run ids are correlated, and tunnel health checks stay
+  enabled by default.
+- **HybridAI-prefixed model names resolve cleanly**: Provider prefix handling
+  recognizes `hybridai/...` model ids without noisy warnings.
+- **Context ring popovers render correctly**: The web chat context usage ring
+  once again shows its detail popover.
+
+## [0.14.0](https://github.com/HybridAIOne/hybridclaw/tree/v0.14.0) - 2026-04-28
+
+### Added
+
+- **Signal channel**: HybridClaw can connect to Signal through a
+  `signal-cli` compatible daemon, with private-by-default DM and group
+  policies, outbound chunk pacing, reconnect handling, admin QR linking, and a
+  full setup guide.
+- **Confidential-info filter and audit leak scanner**: Operators can define
+  NDA-class client, project, person, keyword, and regex rules in
+  `.confidential.yml`; prompts are redacted before model calls, responses are
+  rehydrated for the user, and `hybridclaw audit scan-leaks` can inspect
+  historic audit logs with severity and type filters.
+- **Admin statistics and agent scoreboard**: The admin console adds
+  `/admin/statistics` for session, message, token, cost, and channel trends,
+  plus `/admin/agent-scoreboard` for per-agent skill scores, best skills,
+  reliability, timing, and CV links.
+- **Live context usage controls**: Web chat shows a live context-usage ring,
+  local sessions support `/context`, and compaction headroom is visible before
+  long-running chats hit the model window.
+- **Packaged skill lifecycle**: Production skills can declare manifests with
+  package id, version, capabilities, required credentials, and supported
+  channels. Operators can install, upgrade, uninstall, list revisions, and roll
+  back managed skills with audited snapshots.
+- **Skill autonomy and stakes policy foundations**: `skills.autonomy` records
+  per-agent skill autonomy levels, the container approval policy can classify
+  high-stakes actions, and conditional skill availability can be routed through
+  the generalized policy engine.
+- **Deployment config and ngrok tunnel provider**: Runtime config now declares
+  local or cloud deployment mode, public URLs, tunnel provider intent, and a
+  built-in ngrok tunnel provider backed by the encrypted `NGROK_AUTHTOKEN`
+  secret.
+- **Nix and Homebrew packaging groundwork**: The repo ships a multi-arch Nix
+  flake, NixOS service module, contributor dev shell, packaging notes, and a
+  preview Homebrew formula for future tap publication.
+- **Model metadata, pricing, and monthly usage rollups**: `/model info`,
+  `/usage`, and the admin Models page surface discovered context windows,
+  output limits, capabilities, pricing, and monthly per-model spend when
+  providers expose that metadata.
+- **Headful browser control**: Browser tools can run a visible Chrome session
+  when a user explicitly asks for headed/headful control, while shared browser
+  login profiles stay reusable for automation.
+- **Agent-to-agent and trajectory persistence foundations**: The runtime can
+  persist A2A envelopes and opt-in redacted skill-run trajectories, creating
+  the data trail needed for multi-agent handoffs, skill evaluation, and future
+  workflow tuning.
+
+### Changed
+
+- **Browser chat is more operational**: Chat navigation is session-id driven,
+  recent sessions keep richer snippets, the composer can switch agents, slash
+  result streams render correctly, and context-ring data is shared with the
+  `/context` command.
+- **Agent terminology and profile data are consistent**: The UI and internal
+  persistence moved from coworker compatibility naming to agent naming, while
+  agent configs gained owner, role, and CV fields.
+- **Model and provider status is discovery-led**: Provider catalogs cache
+  runtime discovery, merge pinned entries with discovered models, remove stale
+  static pricing assumptions, and keep status/model-info output focused on the
+  active model.
+- **Approval and policy evaluation is more explicit**: Approval tiers can be
+  influenced by autonomy level and stakes classification, invalid policy
+  regexes and thresholds warn early, and unsafe realpath inspection during
+  approval classification is avoided.
+- **Local diagnostics are more precise**: Gateway debug startup flags can
+  capture raw model responses and last prompts for local troubleshooting, and
+  `doctor` resource hygiene can reclaim stale gateway artifacts more safely.
+- **TUI and status reporting are quieter and more useful**: Proactive polling
+  runs less often, streamed TUI responses preserve visible text, transient tool
+  lines truncate cleanly, and status output includes tokens-per-second and
+  time-to-first-token aware metrics.
+
+### Fixed
+
+- **Web fetch is guarded against SSRF**: Plain HTTP retrieval now enforces
+  private-network protections more consistently before escalating to browser
+  tools.
+- **Headful browser launches require system Chrome**: Visible browser control
+  refuses unstable headed macOS fallback launches and reports the required
+  Chrome executable setup instead.
+- **Voice turns survive relay reconnects**: Twilio voice relay reconnects no
+  longer lose the active turn state while the gateway is handling a call.
+- **Chat history and streaming edge cases are closed**: Result-only slash
+  streams render, tool-call sentinels are stripped before storage, regenerated
+  replies include tools used, context rings stay visible, and `/chat.html`
+  redirects preserve query strings.
+- **Skill lifecycle and manifest handling are stricter**: Managed skill
+  installs require installed status records, validate snapshot entries, cap
+  restored file modes, preserve unknown deployment tunnel providers, and reject
+  upgrades for uninstalled packages.
+- **Channel runtimes shut down more predictably**: WhatsApp and voice shutdown
+  paths cancel stale work, Signal delivery validates daemon/account state, and
+  channel send tools remain scoped to active transports.
+
 ## [0.13.1](https://github.com/HybridAIOne/hybridclaw/tree/v0.13.1) - 2026-04-24
 
 ### Added

package/README.md

@@ -19,9 +19,9 @@ security, and operational visibility. It combines sandboxed execution, secure
 credentials, approvals, persistent memory, and admin surfaces behind a single
 gateway.
 
-Connect it to Discord, Slack, WhatsApp, Telegram, Microsoft Teams, email,
-Twilio voice, or the web. Run it locally, deploy it for business workflows,
-and keep your agents, secrets, and data under your control.
+Connect it to Discord, Slack, Signal, WhatsApp, Telegram, Microsoft Teams,
+email, Twilio voice, or the web. Run it locally, deploy it for business
+workflows, and keep your agents, secrets, and data under your control.
 
 [Quick Start](https://www.hybridclaw.io/docs/getting-started/quickstart) ·
 [Installation](https://www.hybridclaw.io/docs/getting-started/installation) ·
@@ -78,7 +78,7 @@ Open locally:
 
 - Chat UI: `http://127.0.0.1:9090/chat`
 - Admin UI: `http://127.0.0.1:9090/admin` for channels, versioned agent files,
-  scheduler, audit, config, and channel-specific instructions
+  scheduler, audit, statistics, config, and channel-specific instructions
 - Agents UI: `http://127.0.0.1:9090/agents`
 - OpenAI-compatible API: `http://127.0.0.1:9090/v1/models` and `http://127.0.0.1:9090/v1/chat/completions`
 
@@ -95,41 +95,66 @@ Once the gateway is running, open HybridClaw locally:
 - Web Chat: `http://127.0.0.1:9090/chat`
 - Web Chat keeps a recent-session sidebar and can search conversation titles
   with contextual snippets before you reopen an older browser session
+- Web Chat shows live context-window usage, accepts `/context`, and lets you
+  switch the active agent and model from the composer
 - Web Chat accepts `/btw <question>` side questions while a primary run is
   active, so you can ask an ephemeral follow-up without interrupting the
   current run
 - Admin Console: `http://127.0.0.1:9090/admin` for channels, versioned agent files,
-  scheduler, audit, config, and channel-specific instructions
+  scheduler, audit, statistics, config, and channel-specific instructions
 - Agent Dashboard: `http://127.0.0.1:9090/agents`
-- or connect Slack, WhatsApp, Telegram, Discord, Microsoft Teams, Email
+- or connect Slack, Signal, WhatsApp, Telegram, Discord, Microsoft Teams, Email
 
 ## Operator workflows
 
+- Install from npm, source, or the multi-arch Nix flake; a preview Homebrew
+  formula is available for `--HEAD` builds while stable tap publication is
+  prepared.
 - `hybridclaw gateway status` reports sandbox/runtime details, and in
   container mode it includes the configured image name plus the resolved
   version and short image id.
+- `hybridclaw backup` creates a WAL-safe archive of the runtime home, and
+  `hybridclaw backup restore <archive.zip>` validates the archive before
+  replacing local runtime state.
 - `hybridclaw update --yes` upgrades a global npm install and auto-restarts a
   running local gateway with its original launch parameters when possible,
   falling back to `hybridclaw gateway restart` if not.
 - `/admin/agents` edits allowlisted bootstrap markdown files such as
   `AGENTS.md`, keeps saved revisions, and restores earlier versions from the
   browser.
+- `/admin/statistics` reports message, session, token, cost, and channel trends
+  across a selected date range.
+- `/admin/agent-scoreboard` ranks agents by observed skill scores, reliability,
+  timing, best skills, and CV links.
 - `hybridclaw agent config` accepts generated JSON payloads to upsert agent
   metadata, write bootstrap markdown, import profile images into the agent
   workspace, and optionally activate the agent.
 - `/admin/channels` edits transport config, encrypted channel credentials,
-  Twilio voice settings, and per-channel instructions that are injected into
-  prompts at runtime.
+  Signal QR linking, Twilio voice settings, and per-channel instructions that
+  are injected into prompts at runtime.
 - `/admin/approvals` manages approval policies from the browser.
 - `/admin/gateway` reloads runtime config and refreshes secrets from the
-  browser without tearing down the enclosing workspace container; keep
-  `hybridclaw gateway restart` for local/manual full restarts.
+  browser, and shows public URL plus tunnel status, without tearing down the
+  enclosing workspace container; keep `hybridclaw gateway restart` for
+  local/manual full restarts.
+- `/context` and the web chat context ring show current context-window usage,
+  remaining headroom, and compaction counts for the active session.
 - `proactive.delegation.model` can pin delegated work to a different model
   from the parent turn; `/status` shows delegate token totals and local-token
   share when that split is configured.
+- `deployment.mode`, `deployment.public_url`, `deployment.tunnel.provider`, and
+  `deployment.tunnel.health_check_interval_ms` describe local/cloud exposure
+  and tunnel health cadence. The built-in ngrok and Tailscale Funnel providers
+  read `NGROK_AUTHTOKEN` and `TS_AUTHKEY` from the encrypted runtime secret
+  store.
+- `container.warmPool` keeps a bounded adaptive pool of idle host/container
+  runtimes for recently active agents when low cold-start latency matters.
 - `container.persistBashState` controls whether bash tool calls share shell
   state (`cd`, exported env vars, aliases) across turns in the same active
   runtime session; `/admin/config` exposes the same setting as `Persistent bash state`.
+- `hybridclaw audit scan-leaks` scans historical audit logs against optional
+  `.confidential.yml` rules for NDA-class client, project, person, keyword,
+  and regex matches.
 - Generated artifacts remain downloadable and attachable even when the sandbox
   exposes a custom workspace display root such as `/app`.
 - `hybridclaw tui` includes live delegate progress, pulsing tool rows,
@@ -142,6 +167,9 @@ Once the gateway is running, open HybridClaw locally:
   `config.json` becomes invalid.
 - `hybridclaw skill import` supports community sources, local directories,
   and `.zip` archives.
+- `hybridclaw skill install <source>`, `skill upgrade`, `skill revisions`, and
+  `skill rollback` manage packaged business skills with manifests, audit
+  events, and snapshots.
 - The bundled tutorials cover owner, GTM, marketing, sales, DevRel, content,
   invoicing, webinar, and release-launch workflows that can run from the TUI,
   web chat, or connected channels.
@@ -161,6 +189,10 @@ Once the gateway is running, open HybridClaw locally:
   Kimi, MiniMax, DashScope, Xiaomi, Kilo Code, and local backends such as
   Ollama, LM Studio, llama.cpp, and vLLM. Remote OpenAI-compatible providers
   can merge runtime-discovered model catalogs with operator-pinned lists.
+- `/model info`, `/usage monthly`, `/usage model monthly`, and the admin
+  Models page surface discovered context windows, output limits, model
+  capabilities, pricing, and per-model monthly spend where provider metadata is
+  available.
 - Anthropic can run through the direct Messages API with `ANTHROPIC_API_KEY`
   or through the official Claude CLI transport in host sandbox mode.
 - Brave, Perplexity, and Tavily web-search credentials can live in the
@@ -172,9 +204,13 @@ Once the gateway is running, open HybridClaw locally:
 - Skills can be enabled or disabled globally or per channel from
   `hybridclaw skill enable|disable`, TUI `/skill config`, or the admin
   `Skills` page.
+- Packaged skills can declare versioned manifests, capabilities, required
+  credentials, supported channels, and per-agent autonomy policy.
 - Bundled skills include API-backed Google Workspace workflows (`gog`, `gws`),
-  GitHub issue queue processing (`gh-issues`), and editable Excalidraw diagram
-  creation.
+  Salesforce inspection, GitHub issue queue processing (`gh-issues`),
+  brand-voice drafting, and editable Excalidraw diagram creation.
+- The repo-shipped `brand-voice` plugin can flag, rewrite, or block final
+  responses that violate configured voice rules before they reach users.
 - Built-in office skills handle longer PDF creation flows cleanly: the bundled
   PDF creator wraps long lines, honors explicit `\n`, and adds pages
   automatically when reports or invoices spill past the first page.
@@ -210,6 +246,8 @@ Once the gateway is running, open HybridClaw locally:
 ## Security and governance built in
 
 - secure credential storage
+- optional confidential-info redaction before model calls
+- retroactive audit leak scanning
 - sandboxed execution
 - approvals
 - audit trails with hash chain
@@ -233,7 +271,7 @@ Once the gateway is running, open HybridClaw locally:
 
 ## Architecture
 
-- **Gateway service** (Node.js) — shared message/command handlers, SQLite persistence (KV + semantic + knowledge graph + canonical sessions + usage events), scheduler, heartbeat, web/API, loopback OpenAI-compatible API, and channel integrations for Discord, Slack, Microsoft Teams, Telegram, iMessage, WhatsApp, Twilio voice, and email
+- **Gateway service** (Node.js) — shared message/command handlers, SQLite persistence (KV + semantic + knowledge graph + canonical sessions + usage events), scheduler, heartbeat, web/API, loopback OpenAI-compatible API, and channel integrations for Discord, Slack, Signal, Microsoft Teams, Telegram, iMessage, WhatsApp, Twilio voice, and email
 - **TUI client** — thin client over HTTP (`/api/chat`, `/api/command`) with
   a structured startup banner that surfaces model, sandbox, gateway, and
   chatbot context before the first prompt, live delegate status/progress,
@@ -268,6 +306,7 @@ Browse the full manual at
   [Discord](https://www.hybridclaw.io/docs/channels/discord),
   [Slack](https://www.hybridclaw.io/docs/channels/slack),
   [Telegram](https://www.hybridclaw.io/docs/channels/telegram),
+  [Signal](https://www.hybridclaw.io/docs/channels/signal),
   [Email](https://www.hybridclaw.io/docs/channels/email),
   [WhatsApp](https://www.hybridclaw.io/docs/channels/whatsapp),
   [iMessage](https://www.hybridclaw.io/docs/channels/imessage), and

package/SECURITY.md

@@ -101,6 +101,39 @@ Implementation: [src/session/session-key.ts](./src/session/session-key.ts),
 [src/session/session-routing.ts](./src/session/session-routing.ts),
 [src/memory/db.ts](./src/memory/db.ts)
 
+### 4.1) Confidential-Info Filter (NDA / secret-leak detector)
+
+Optional, opt-in filter that prevents NDA-class business data from leaving the
+host:
+
+- Define rules in `.confidential.yml`. The loader checks the current working
+  directory first (`./.confidential.yml`) and then
+  `~/.hybridclaw/.confidential.yml`; first hit wins. The file holds clients,
+  projects, people, keywords, and regex patterns, each tagged with a
+  sensitivity level.
+- Before every prompt is sent to a model, matches are replaced with stable
+  placeholders (`«CONF:CLIENT_001»`); the mapping is held in process memory and
+  forgotten when the request ends.
+- Streaming text deltas and the final response are rehydrated for the user, so
+  the model never sees the original strings but the user sees real names.
+- Disabled via `HYBRIDCLAW_CONFIDENTIAL_DISABLE=1` for debugging or dry-runs.
+
+A retroactive scanner walks existing audit logs to surface possible past leaks
+and assigns a 0-100 risk score:
+
+```bash
+hybridclaw audit scan-leaks                # scan every session
+hybridclaw audit scan-leaks <sessionId>    # scan one session
+hybridclaw audit scan-leaks --level high   # critical/high only
+hybridclaw audit scan-leaks --type in,out  # prompts and model responses only
+hybridclaw audit scan-leaks --json         # machine-readable report
+```
+
+Implementation: [src/security/confidential-rules.ts](./src/security/confidential-rules.ts),
+[src/security/confidential-redact.ts](./src/security/confidential-redact.ts),
+[src/security/confidential-runtime.ts](./src/security/confidential-runtime.ts),
+[src/audit/leak-scanner.ts](./src/audit/leak-scanner.ts).
+
 ### 5) Audit & Tamper Evidence
 
 Security-relevant behavior is written to structured audit logs:

package/config.example.json

@@ -1,23 +1,33 @@
 {
-  "version": 21,
+  "version": 26,
   "security": {
     "trustModelAccepted": false,
     "trustModelAcceptedAt": "",
     "trustModelVersion": "",
     "trustModelAcceptedBy": ""
   },
+  "deployment": {
+    "mode": "local",
+    "public_url": "",
+    "tunnel": {
+      "provider": "manual",
+      "health_check_interval_ms": 30000
+    }
+  },
   "skills": {
     "extraDirs": [],
     "disabled": [],
     "channelDisabled": {
       "discord": [],
       "msteams": [],
+      "signal": [],
       "slack": [],
       "telegram": [],
       "voice": [],
       "whatsapp": [],
       "email": []
-    }
+    },
+    "installed": []
   },
   "tools": {
     "disabled": []
@@ -25,6 +35,7 @@
   "channelInstructions": {
     "discord": "",
     "msteams": "",
+    "signal": "",
     "slack": "",
     "telegram": "",
     "voice": "This is a live phone call. Produce plain spoken text only.\nKeep each reply short and conversational, usually one or two short sentences.\nAbsolutely no markdown, bullets, numbered lists, headings, code fences, tables, JSON, or decorative formatting.\nDo not narrate internal reasoning, planning, tool usage, or stage directions. Say only what the caller should hear.\nDo not spell punctuation, formatting marks, or raw URLs unless the caller explicitly asks for exact characters.",
@@ -35,6 +46,10 @@
   "adaptiveSkills": {
     "enabled": false,
     "observationEnabled": true,
+    "trajectoryCapture": {
+      "enabledAgentIds": [],
+      "storeDir": ""
+    },
     "inspectionIntervalMs": 3600000,
     "observationRetentionDays": 30,
     "trailingWindowHours": 168,
@@ -157,6 +172,18 @@
     "textChunkLimit": 4000,
     "mediaMaxMb": 20
   },
+  "signal": {
+    "enabled": false,
+    "daemonUrl": "http://127.0.0.1:8080",
+    "account": "",
+    "dmPolicy": "allowlist",
+    "groupPolicy": "disabled",
+    "allowFrom": [],
+    "groupAllowFrom": [],
+    "textChunkLimit": 4000,
+    "reconnectIntervalMs": 5000,
+    "outboundDelayMs": 350
+  },
   "whatsapp": {
     "dmPolicy": "pairing",
     "groupPolicy": "disabled",
@@ -357,6 +384,11 @@
       "model": "",
       "maxTokens": 0
     },
+    "eval_judge": {
+      "provider": "auto",
+      "model": "",
+      "maxTokens": 0
+    },
     "mcp": {
       "provider": "auto",
       "model": "",
@@ -380,7 +412,15 @@
     "additionalMounts": "",
     "maxOutputBytes": 10485760,
     "maxConcurrent": 5,
-    "persistBashState": true
+    "persistBashState": true,
+    "warmPool": {
+      "enabled": true,
+      "coldStartBudgetMs": 200,
+      "trafficWindowMs": 3600000,
+      "minIdlePerActiveAgent": 1,
+      "maxIdlePerAgent": 2,
+      "memoryPressureRssMb": 2048
+    }
   },
   "mcpServers": {
     "filesystem": {