vilvona 1.0.0

package/docs/refactor/acp.md

@@ -0,0 +1,298 @@
+---
+summary: "Migration plan for making ACP session and ACPX process ownership explicit"
+read_when:
+  - Refactoring ACP session lifecycle or ACPX process cleanup
+  - Debugging ACPX orphan processes, PID reuse, or multi-gateway cleanup safety
+  - Changing sessions_list visibility for spawned ACP or subagent sessions
+  - Designing ownership metadata for background tasks, ACP sessions, or process leases
+title: "ACP lifecycle refactor"
+sidebarTitle: "ACP lifecycle refactor"
+---
+
+ACP lifecycle currently works, but too much of it is inferred after the fact.
+Process cleanup reconstructs ownership from PIDs, command strings, wrapper
+paths, and the live process table. Session visibility reconstructs ownership
+from session-key strings plus secondary `sessions.list({ spawnedBy })` lookups.
+That makes narrow fixes possible, but it also makes edge cases easy to miss:
+PID reuse, quoted commands, adapter grandchildren, multi-gateway state roots,
+`cancel` versus `close`, and `tree` versus `all` visibility all become separate
+places to rediscover the same ownership rules.
+
+This refactor makes ownership first-class. The goal is not a new ACP product
+surface; it is a safer internal contract for the existing ACP and ACPX behavior.
+
+## Goals
+
+- Cleanup never signals a process unless current live evidence matches an
+  OpenClaw-owned lease.
+- `cancel`, `close`, and startup reaping have distinct lifecycle intents.
+- `sessions_list`, `sessions_history`, `sessions_send`, and status checks use
+  the same requester-owned session model.
+- Multi-gateway installs cannot reap each other's ACPX wrappers.
+- Old ACPX session records keep working during migration.
+- The runtime remains plugin-owned; core does not learn ACPX package details.
+
+## Non-goals
+
+- Replacing ACPX or changing the public `/acp` command surface.
+- Moving vendor-specific ACP adapter behavior into core.
+- Requiring users to manually clean state before upgrading.
+- Making `cancel` close reusable ACP sessions.
+
+## Target Model
+
+### Gateway Instance Identity
+
+Each Gateway process should have a stable runtime instance id:
+
+```ts
+type GatewayInstanceId = string;
+```
+
+It can be generated on Gateway startup and persisted in state for the life of
+that install. It is not a security secret; it is an ownership discriminator used
+to avoid confusing one Gateway's ACP processes with another Gateway's processes.
+
+### ACP Session Ownership
+
+Every spawned ACP session should have normalized ownership metadata:
+
+```ts
+type AcpSessionOwner = {
+  sessionKey: string;
+  spawnedBy?: string;
+  parentSessionKey?: string;
+  ownerSessionKey: string;
+  agentId: string;
+  backend: "acpx";
+  gatewayInstanceId: GatewayInstanceId;
+  createdAt: number;
+};
+```
+
+The Gateway should return these fields on session rows where they are known.
+Visibility filtering should be a pure check over row metadata:
+
+```ts
+canSeeSessionRow({
+  row,
+  requesterSessionKey,
+  visibility,
+  a2aPolicy,
+});
+```
+
+That removes hidden secondary `sessions.list({ spawnedBy })` calls from
+visibility checks. A spawned cross-agent ACP child is requester-owned because
+the row says so, not because a second query happens to find it.
+
+### ACPX Process Leases
+
+Every generated wrapper launch should create a lease record:
+
+```ts
+type AcpxProcessLease = {
+  leaseId: string;
+  gatewayInstanceId: GatewayInstanceId;
+  sessionKey: string;
+  wrapperRoot: string;
+  wrapperPath: string;
+  rootPid: number;
+  processGroupId?: number;
+  commandHash: string;
+  startedAt: number;
+  state: "open" | "closing" | "closed" | "lost";
+};
+```
+
+The wrapper process should receive the lease id and gateway instance id in its
+environment:
+
+```sh
+OPENCLAW_ACPX_LEASE_ID=...
+OPENCLAW_GATEWAY_INSTANCE_ID=...
+```
+
+When the platform allows it, verification should prefer live process metadata
+that cannot be confused by command quoting:
+
+- root PID still exists
+- live wrapper path is under `wrapperRoot`
+- process group matches the lease when available
+- environment contains the expected lease id when readable
+- command hash or executable path matches the lease
+
+If the live process cannot be verified, cleanup fails closed.
+
+## Lifecycle Controller
+
+Introduce one ACPX lifecycle controller that owns process leases and cleanup
+policy:
+
+```ts
+interface AcpxLifecycleController {
+  ensureSession(input: AcpRuntimeEnsureInput): Promise<AcpRuntimeHandle>;
+  cancelTurn(handle: AcpRuntimeHandle): Promise<void>;
+  closeSession(input: {
+    handle: AcpRuntimeHandle;
+    discardPersistentState?: boolean;
+    reason?: string;
+  }): Promise<void>;
+  reapStartupOrphans(): Promise<void>;
+  verifyOwnedTree(lease: AcpxProcessLease): Promise<OwnedProcessTree | null>;
+}
+```
+
+`cancelTurn` requests turn cancellation only. It must not reap reusable wrapper
+or adapter processes.
+
+`closeSession` is allowed to reap, but only after loading the session record,
+loading the lease, and verifying the live process tree still belongs to that
+lease.
+
+`reapStartupOrphans` starts from open leases in state. It may use the process
+table to find descendants, but it should not scan arbitrary ACP-looking
+commands first and then decide they are probably ours.
+
+## Wrapper Contract
+
+Generated wrappers should stay small. They should:
+
+- start the adapter in a process group where supported
+- forward normal termination signals to the process group
+- detect parent death
+- on parent death, send SIGTERM, then keep the wrapper alive until the SIGKILL
+  fallback runs
+- report root PID and process group id back to the lifecycle controller when
+  that is available
+
+Wrappers should not decide session policy. They only enforce local process-tree
+cleanup for their own adapter group.
+
+## Session Visibility Contract
+
+Visibility should use normalized row ownership:
+
+```ts
+type SessionVisibilityInput = {
+  requesterSessionKey: string;
+  row: {
+    key: string;
+    agentId: string;
+    ownerSessionKey?: string;
+    spawnedBy?: string;
+    parentSessionKey?: string;
+  };
+  visibility: "self" | "tree" | "agent" | "all";
+  a2aPolicy: AgentToAgentPolicy;
+};
+```
+
+Rules:
+
+- `self`: only the requester session.
+- `tree`: requester session plus rows owned by or spawned from the requester.
+- `all`: all same-agent rows, a2a-allowed cross-agent rows, and requester-owned
+  spawned cross-agent rows even when general a2a is disabled.
+- `agent`: same agent only, unless an explicit owner relationship says the row
+  belongs to the requester.
+
+This makes `tree` and `all` monotonic: `all` must not hide an owned child that
+`tree` would show.
+
+## Migration Plan
+
+### Phase 1: Add Identity And Leases
+
+- Add `gatewayInstanceId` to Gateway state.
+- Add an ACPX lease store under the ACPX state directory.
+- Write a lease before spawning a generated wrapper.
+- Store `leaseId` on new ACPX session records.
+- Keep existing PID and command fields for old records.
+
+### Phase 2: Lease-First Cleanup
+
+- Change close cleanup to load `leaseId` first.
+- Verify live process ownership against the lease before signaling.
+- Keep the current root PID and wrapper-root fallback only for legacy records.
+- Mark leases `closed` after verified cleanup.
+- Mark leases `lost` when the process is gone before cleanup.
+
+### Phase 3: Lease-First Startup Reaping
+
+- Startup reaping scans open leases.
+- For each lease, verify the root process and collect descendants.
+- Reap verified trees children-first.
+- Expire old `closed` and `lost` leases with a bounded retention window.
+- Keep command-marker scanning only as a temporary legacy fallback, guarded by
+  wrapper root and Gateway instance where possible.
+
+### Phase 4: Session Ownership Rows
+
+- Add ownership metadata to Gateway session rows.
+- Teach ACPX, subagent, background-task, and session-store writers to populate
+  `ownerSessionKey` or `spawnedBy`.
+- Convert session visibility checks to use row metadata.
+- Remove visibility-time secondary `sessions.list({ spawnedBy })` lookups.
+
+### Phase 5: Remove Legacy Heuristics
+
+After one release window:
+
+- stop relying on stored root command strings for non-legacy ACPX cleanup
+- remove command-marker startup scans
+- remove visibility fallback list lookups
+- keep defensive fail-closed behavior for missing or unverifiable leases
+
+## Tests
+
+Add two table-driven suites.
+
+Process lifecycle simulator:
+
+- PID reused by unrelated process
+- PID reused by another Gateway's wrapper root
+- stored wrapper command is shell-quoted, live `ps` command is not
+- adapter child exits, grandchild remains in the process group
+- parent death SIGTERM fallback reaches SIGKILL
+- process listing unavailable
+- stale lease with missing process
+- startup orphan with wrapper, adapter child, and grandchild
+
+Session visibility matrix:
+
+- `self`, `tree`, `agent`, `all`
+- a2a enabled and disabled
+- same-agent row
+- cross-agent row
+- requester-owned spawned cross-agent ACP row
+- sandboxed requester clamped to `tree`
+- list, history, send, and status actions
+
+The important invariant: a requester-owned spawned child is visible wherever
+the configured visibility includes the requester session tree, and `all` is not
+less capable than `tree`.
+
+## Compatibility Notes
+
+Old session records may not have `leaseId`. They should use the legacy
+fail-closed cleanup path:
+
+- require a live root process
+- require wrapper-root ownership when a generated wrapper is expected
+- require command agreement for non-wrapper roots
+- never signal based only on stale stored PID metadata
+
+If a legacy record cannot be verified, leave it alone. Startup lease cleanup and
+the next release window should eventually retire the fallback.
+
+## Success Criteria
+
+- Closing an old or stale ACPX session cannot kill another Gateway's process.
+- Parent death does not leave stubborn adapter grandchildren running.
+- `cancel` aborts the active turn without closing reusable sessions.
+- `sessions_list` can show requester-owned cross-agent ACP children under both
+  `tree` and `all`.
+- Startup cleanup is driven by leases, not broad command-string scans.
+- The focused process and visibility matrix tests cover every edge case that
+  previously required one-off review fixes.

package/docs/refactor/canvas.md

@@ -0,0 +1,131 @@
+---
+summary: "Plan and audit checklist for moving Canvas out of core and into a bundled experimental plugin."
+read_when:
+  - Moving Canvas host, tools, commands, docs, or protocol ownership
+  - Auditing whether Canvas is still core-owned
+  - Preparing or reviewing the experimental Canvas plugin PR
+title: "Canvas plugin refactor"
+---
+
+# Canvas plugin refactor
+
+Canvas is low-use and experimental. Treat it as a bundled plugin, not a core feature. Core may keep generic gateway, node, HTTP, auth, config, and native-client plumbing, but Canvas-specific behavior should live under `extensions/canvas`.
+
+## Goal
+
+Move Canvas ownership to `extensions/canvas` while preserving the current paired-node behavior:
+
+- the agent-facing `canvas` tool is registered by the Canvas plugin
+- Canvas node commands are allowed only when the Canvas plugin registers them
+- A2UI host/source files live under the Canvas plugin
+- Canvas document materialization lives under the Canvas plugin
+- CLI command implementation lives under the Canvas plugin, or delegates through a plugin-owned runtime barrel
+- docs and plugin inventory describe Canvas as experimental and plugin-backed
+
+## Non-goals
+
+- Do not redesign the native app Canvas UI in this refactor.
+- Do not remove Canvas protocol/client support from iOS, Android, or macOS unless a separate product decision says Canvas should be deleted.
+- Do not build a broad plugin service framework only for Canvas unless at least one other bundled plugin needs the same seam.
+
+## Current branch state
+
+Done:
+
+- Added bundled plugin package in `extensions/canvas`.
+- Added `extensions/canvas/openclaw.plugin.json`.
+- Moved the agent `canvas` tool from `src/agents/tools/canvas-tool.ts` to `extensions/canvas/src/tool.ts`.
+- Removed core registration of `createCanvasTool` from `src/agents/openclaw-tools.ts`.
+- Moved Canvas host implementation from `src/canvas-host` to `extensions/canvas/src/host`.
+- Kept `extensions/canvas/runtime-api.ts` as the plugin-owned compatibility barrel for tests, packaging, and external public Canvas helpers.
+- Moved Canvas document materialization from `src/gateway/canvas-documents.ts` to `extensions/canvas/src/documents.ts`.
+- Moved Canvas CLI implementation and A2UI JSONL helpers into `extensions/canvas/src/cli.ts`.
+- Moved Canvas host URL and scoped capability helpers into `extensions/canvas/src`.
+- Moved Canvas node command defaults out of hardcoded core lists and into plugin `nodeInvokePolicies`.
+- Added plugin-owned Canvas host config at `plugins.entries.canvas.config.host`.
+- Moved Canvas and A2UI HTTP serving behind Canvas plugin HTTP route registration.
+- Added generic plugin WebSocket upgrade dispatch for plugin-owned HTTP routes.
+- Replaced Canvas-specific gateway host URL and node capability auth with generic hosted plugin surface and node capability helpers.
+- Added plugin-owned hosted media resolvers so Canvas document URLs resolve through the Canvas plugin instead of core importing Canvas document internals.
+- Added `api.registerNodeCliFeature(...)` so Canvas can declare `openclaw nodes canvas` as a plugin-owned node feature without manually spelling the parent command path.
+- Removed production `src/**` imports of `extensions/canvas/runtime-api.js`.
+- Moved the A2UI bundle source from `apps/shared/OpenClawKit/Tools/CanvasA2UI` to `extensions/canvas/src/host/a2ui-app`.
+- Moved A2UI build/copy implementation under `extensions/canvas/scripts` and replaced root build wiring with generic bundled-plugin asset hooks.
+- Removed the runtime legacy top-level `canvasHost` config alias.
+- Kept the Canvas doctor migration so `openclaw doctor --fix` rewrites old `canvasHost` configs into `plugins.entries.canvas.config.host`.
+- Removed old-agent Canvas protocol compatibility behind gateway protocol v4. Native clients and gateways now use only `pluginSurfaceUrls.canvas` plus `node.pluginSurface.refresh`; the deprecated `canvasHostUrl`, `canvasCapability`, and `node.canvas.capability.refresh` path is intentionally unsupported in this experimental refactor.
+- Updated generated plugin inventory to include Canvas.
+- Added plugin reference docs at `docs/plugins/reference/canvas.md`.
+
+Known remaining core-owned Canvas surfaces:
+
+- Native app Canvas handlers under `apps/` still intentionally consume the Canvas plugin surface
+- native app Canvas protocol/client handlers under `apps/`
+- published artifact output still uses `dist/canvas-host/a2ui` for backwards-compatible runtime lookup, but the copy step is now plugin-owned
+
+## Target shape
+
+`extensions/canvas` should own:
+
+- plugin manifest and package metadata
+- agent tool registration
+- node invoke command policy
+- Canvas host and A2UI runtime
+- Canvas A2UI bundle source and asset build/copy scripts
+- Canvas document creation and asset resolution
+- Canvas CLI implementation
+- Canvas docs page and plugin inventory entry
+
+Core should own only generic seams:
+
+- plugin discovery and registration
+- generic agent tool registry
+- generic node invoke policy registry
+- generic gateway HTTP/auth and WebSocket upgrade dispatch
+- generic hosted plugin surface URL resolution
+- generic hosted media resolver registration
+- generic node capability transport
+- generic config plumbing
+- generic bundled-plugin asset hook discovery
+
+Native apps may keep Canvas command handlers as clients of the protocol. They are not the plugin runtime owner.
+
+## Migration steps
+
+1. Treat `plugins.entries.canvas.config.host` as the plugin-owned config surface.
+2. Update docs so Canvas is described as an experimental bundled plugin.
+3. Run focused Canvas tests, plugin inventory checks, plugin SDK API checks, and build/type gates affected by runtime boundaries.
+
+## Audit checklist
+
+Before calling the refactor complete:
+
+- `rg "src/canvas-host|../canvas-host"` returns no live source imports.
+- `rg "canvas-tool|createCanvasTool" src` finds no core-owned Canvas tool implementation.
+- `rg "canvas.present|canvas.snapshot|canvas.a2ui" src/gateway` finds no hardcoded allowlist defaults outside generic plugin policy tests.
+- `rg "extensions/canvas/runtime-api" src --glob '!**/*.test.ts'` is empty.
+- `rg "canvas-documents" src` is empty.
+- `rg "registerNodesCanvasCommands|nodes-canvas" src` is empty; the Canvas plugin registers `openclaw nodes canvas` through nested plugin CLI metadata.
+- `rg "createCanvasHostHandler|handleA2uiHttpRequest" src/gateway` returns no gateway runtime ownership.
+- `rg "apps/shared/OpenClawKit/Tools/CanvasA2UI|canvas-a2ui-copy|extensions/canvas/src/host/a2ui" scripts .github package.json` finds only compatibility wrappers or plugin-owned paths.
+- `pnpm plugins:inventory:check` passes.
+- `pnpm plugin-sdk:api:check` passes, or generated API baselines are intentionally updated and reviewed.
+- Targeted Canvas tests pass.
+- Changed-lanes tests pass for Canvas host/A2UI paths.
+- PR body explicitly says Canvas is experimental and plugin-backed.
+
+## Verification commands
+
+Use targeted local checks while iterating:
+
+```sh
+pnpm test extensions/canvas/src/host/server.test.ts extensions/canvas/src/host/server.state-dir.test.ts extensions/canvas/src/host/file-resolver.test.ts
+pnpm test src/gateway/server.plugin-node-capability-auth.test.ts src/gateway/server-import-boundary.test.ts
+pnpm test extensions/canvas/src/config-migration.test.ts src/commands/doctor-legacy-config.migrations.test.ts
+pnpm test test/scripts/changed-lanes.test.ts test/scripts/build-all.test.ts extensions/canvas/scripts/bundle-a2ui.test.ts test/scripts/bundled-plugin-assets.test.ts extensions/canvas/scripts/copy-a2ui.test.ts src/infra/run-node.test.ts
+pnpm tsgo:extensions
+pnpm plugins:inventory:check
+pnpm plugin-sdk:api:check
+```
+
+Run `pnpm build` before push if runtime barrel, lazy import, packaging, or published plugin surfaces change.