@mseep/open-computer-use 1.0.0

package/docs/future-architecture/architecture/08-networking.md

@@ -0,0 +1,149 @@
+<!-- SPDX-License-Identifier: FSL-1.1-Apache-2.0 -->
+<!-- Copyright (c) 2025 Open Computer Use Contributors -->
+
+# 08 — Networking
+
+> Network policy, egress proxy, CDP/ttyd routing, ingress.
+> Cross-cuts L3 + L4.
+
+## Principles
+
+1. **Default-deny everywhere.** Every namespace, every sandbox, every direction.
+2. **Single mediated egress path** — the JWT-allowlist proxy (see [07-security.md](./07-security.md)).
+3. **Sandbox is not publicly addressable.** Ever.
+4. **L4 ↔ L3 is mTLS.** L3 ↔ L1 is network-policy-isolated, no app-level auth.
+
+## Topology (k8s target, Phase 5+)
+
+```text
+                  Internet
+                     │
+            ┌────────▼────────┐
+            │   Ingress + WAF │   (public; serves L4 only)
+            └────────┬────────┘
+                     │
+            ┌────────▼────────┐
+            │  L4 Control Plane│
+            └────┬───────┬────┘
+                 │       │
+        mTLS gRPC│       │ HTTPS to S3 / Secrets / KV
+                 │       │
+            ┌────▼────┐  │
+            │   L3    │  │
+            │ Provider│  │
+            └────┬────┘  │
+                 │ k8s API
+            ┌────▼─────────────────────────────────────┐
+            │  Tenant namespace (per tenant_id)        │
+            │  NetworkPolicy: deny all, allow:          │
+            │    - L3 → sandbox pod port (exec API)     │
+            │    - sandbox → egress-proxy svc           │
+            │  No pod-to-pod within namespace either.   │
+            │                                            │
+            │  ┌─────────────┐   ┌─────────────────┐    │
+            │  │  Sandbox A  │   │  Egress proxy   │────┼──► Internet
+            │  │  (L1 agent) │──►│  (JWT validate) │    │   (allowlisted)
+            │  └─────────────┘   └─────────────────┘    │
+            └────────────────────────────────────────────┘
+```
+
+## NetworkPolicy (default per tenant namespace)
+
+- `default-deny-ingress` and `default-deny-egress` on every pod.
+- Allow ingress: from `namespace=control-plane` pods (label-selected) on the sandbox port only.
+- Allow egress: to `namespace=egress` egress-proxy svc on its port only; plus DNS to kube-dns.
+- **No pod-to-pod within the tenant namespace** — workspaces never see each other.
+
+## Egress proxy
+
+- One deployment per cluster (HA-replicated). Service in a dedicated `egress` namespace.
+- Validates per-session JWT issued by L4's secret broker.
+- JWT carries: `session_id`, `allowed_hosts` (or regex), `expiry`.
+- Logs: destination host, decision, JWT id, latency. Sent to audit sink.
+- Reference: [`Michaelliv/agentbox`](https://github.com/Michaelliv/agentbox). Port to Go in Phase 8.
+
+DNS:
+- Allowlist DNS too (egress to kube-dns; kube-dns has its own egress allowlist for resolution).
+- Or: proxy resolves DNS itself, sandbox uses HTTP proxy directly.
+- Decision deferred to Phase 8 research.
+
+## CDP / ttyd routing
+
+Today:
+- Open WebUI / user UI calls L4 (`computer-use-server`) on a public route.
+- L4 proxies CDP WebSocket frames to/from the sandbox's exposed Chromium.
+- Same path for ttyd.
+
+Target (Phase 6+):
+- Same shape: L4 is the only public surface for CDP/ttyd too.
+- L4 ↔ sandbox: mTLS internal. Sandbox's CDP endpoint reachable only from L4 pods (NetworkPolicy).
+- Long-lived WebSocket — L4 must be HA-friendly (sticky sessions via consistent hashing, or session-router lookup on each new connection).
+
+## Ingress (public)
+
+- TLS terminated at ingress (cert-manager + Let's Encrypt for self-hosted; ACM for AWS).
+- WAF in front for public deployments (mod_security, AWS WAF, Cloudflare).
+- Only L4 routes exposed publicly. L3 / L1 / sandbox pods have no public ingress.
+
+## Docker Compose (PoC)
+
+- Phases 0–4: existing compose network; no NetworkPolicy equivalent.
+- Phase 8: optional egress proxy container can be enabled in Compose for local testing of the allowlist pattern.
+
+## What ships, when
+
+| Phase | Network change |
+|---|---|
+| 1–4 | No network topology change (Compose stays as today) |
+| 5 | Helm chart adds NetworkPolicy default-deny + tenant namespace template |
+| 6 | mTLS L4 ↔ L3; ingress/WAF guidance documented |
+| 8 | Egress proxy + JWT signing in L4 + audit sink (prereq for untrusted tier in Phase 9) |
+| 10 | Multi-AZ session routing — snapshot-based recovery on pod failure (not in-memory affinity); multi-region foundations only |
+
+## Multi-region workspace proxies (Phase 10 substrate)
+
+Long-lived CDP / ttyd WebSockets penalize latency hard — a 200 ms RTT makes a Chromium screencast feel underwater. Once the deployment spans more than one region, L4 cannot terminate every user's WebSocket centrally without paying the cross-region tax on every keystroke.
+
+The pattern, lifted from Coder ([`research/03`](../research/03-coder.md)):
+
+```text
+                User UI
+                   │
+                   ▼
+        ┌──────────────────────┐
+        │ Region-local         │
+        │ Workspace Proxy      │   one per region; terminates user-side TLS
+        │ (CDP/ttyd terminator)│   consistent-hashes by session_id
+        └──────────┬───────────┘
+                   │ mTLS, region-local
+                   ▼
+        ┌──────────────────────┐
+        │ L4 (global, multi-AZ)│
+        │ + KV session router  │
+        └──────────┬───────────┘
+                   │
+                   ▼
+        ┌──────────────────────┐
+        │ L3 + sandboxes        │
+        │ in the user's region  │
+        └──────────────────────┘
+```
+
+Properties:
+- **User-perceived latency is region-local.** RTT to the sandbox stays under the regional ceiling.
+- **L4 stays single-pane-of-glass.** Auth, session router, secret broker remain global; the proxies are dumb shovels.
+- **Failure isolation.** A region's proxy can lose its L4 link without dropping in-flight CDP frames (proxy buffers; reconnects when L4 returns).
+- **Consistent-hash by `session_id`.** Within a region, the same session always lands on the same proxy replica. Avoids the `sessionAffinity: ClientIP` anti-pattern called out in [`02-layer4-control-plane.md`](./02-layer4-control-plane.md).
+
+What this implies for earlier phases:
+- The CDP/ttyd transport must already be transparent passthrough (L4 does not parse frames — [ADR-0008](../adr/0008-internal-grpc-external-rest-mcp.md)). Anything that requires L4 to understand the wire breaks here.
+- Session router state must be **externally addressable** (KV, not in-process) so a region-local proxy can resolve `session_id → region`.
+- mTLS between proxy and L4 must be operational — Phase 6 deliverable.
+
+Phase 10 ships one proxy per region; before that, the proxy is just L4 itself (one region). The architecture is forward-compatible: a Phase 6 deployment with no proxies looks like the Phase 10 deployment minus the geographic shard.
+
+## Source
+
+- Internal security notes
+- [07-security.md](./07-security.md)
+- [`docs/future-architecture/references.md`](../references.md) (`agentbox`)

package/docs/future-architecture/architecture/09-templates.md

@@ -0,0 +1,122 @@
+<!-- SPDX-License-Identifier: FSL-1.1-Apache-2.0 -->
+<!-- Copyright (c) 2025 Open Computer Use Contributors -->
+
+# 09 — Sandbox Templates
+
+> The configurable, version-pinned recipe that produces a sandbox.
+> Tenant tier → template → image + runtime + resources + mounts + network + egress allowlist.
+
+## Why templates are first-class
+
+User requirement: *the container itself must be configurable via templates.* The template object decouples *what tenants get* from *how providers implement it*.
+
+## Schema (target)
+
+Conceptually CRD-shaped (k8s) but the same object lives in Compose (YAML file) and DirectCH (TOML). Provider materializes it.
+
+```yaml
+apiVersion: sandbox.openchatcomputeruse.io/v1
+kind: SandboxTemplate
+metadata:
+  name: computer-use-untrusted-v3
+  labels:
+    tier: untrusted
+    purpose: computer-use
+spec:
+  image:
+    ref: ghcr.io/wide-moat/open-computer-use@sha256:abcdef…
+    signature: required           # cosign verified by admission
+
+  runtime_class: kata-ch          # see 04-layer2-runtimes.md
+  envtype: customer-cu            # L3 dispatch key — see 03-layer3-providers.md
+  snapstart_compatible: false     # Phase 10 only; set true when template ships paired squashfs blobs (see 06-storage.md)
+
+  resources:
+    cpu:   "2"
+    mem:   "2Gi"
+    disk:  "10Gi"
+
+  pool:
+    minSize: 2
+    targetSize: 5
+    maxSize: 20
+    idleTimeoutSeconds: 600
+
+  mounts:                         # see 06-storage.md
+    - type: skill
+      refs:
+        - { name: pptx,   sha256: … }
+        - { name: docx,   sha256: … }
+    - type: workspace
+      persistence: ephemeral
+    - type: user-data
+      backend: s3
+      bucket: tenant-{tenant_id}-data
+
+  env_template:                   # rendered at /v1/configure with session ctx
+    PUBLIC_BASE_URL: "{public_base_url}"
+    SUBAGENT_CLI:   "claude"
+    # NO SECRETS HERE — secret broker injects them
+
+  egress_baseline:                # added to per-session JWT allowlist
+    - "*.anthropic.com"
+    - "pypi.org"
+    - "files.pythonhosted.org"
+    - "registry.npmjs.org"
+    - "github.com"
+    - "objects.githubusercontent.com"
+
+  security:
+    runAsNonRoot: false           # sysbox/kata allow safe root
+    seccompProfile: RuntimeDefault
+    dropCapabilities: ["ALL"]
+    addCapabilities: []           # template can request specific caps with justification
+```
+
+## Tenant → template mapping
+
+L4 resolves at session creation:
+
+```python
+template = TemplateResolver.resolve(
+    tenant_tier,          # e.g. "internal-employee", "paid-customer", "trial"
+    workload_kind,        # e.g. "computer-use", "code-exec"
+    region,
+)
+```
+
+Examples:
+- Internal employee + code-exec → `internal-code-sysbox-v2`
+- Paid customer + Computer Use → `customer-cu-kata-ch-v3`
+- Free trial + Computer Use → `trial-cu-kata-fc-v1`
+
+The mapping is policy held in L4 config (DB-backed in Phase 6+). Admin UI edits it.
+
+## Template lifecycle
+
+- **Created** by ops via admin API (Phase 6+) or YAML in Helm values pre-Phase-6.
+- **Validated** at admission: image signature, mount sanity, resource within cluster quotas.
+- **Versioned**: name carries `vN`. Old version stays until its referenced sessions drain. No mutation in place.
+- **Deprecated** via label; new sessions get the latest non-deprecated.
+
+## Two new fields, briefly
+
+- **`envtype`** — the L3 dispatch key. Picks the backend mechanism (Docker Compose vs k8s vs DirectCH, plus the egress-proxy enforcement mode). Values: `dev`, `internal`, `customer-shared`, `customer-cu`, `anthropic-hosted`, `byoc`. Full matrix in [`03-layer3-providers.md`](./03-layer3-providers.md) "Environment-type dispatch (Baku pattern)". Distinct from `runtime_class` — `envtype` says *where it runs*, `runtime_class` says *what isolates it*.
+- **`snapstart_compatible`** — Phase-10-only flag. When `true`, the template's release pipeline produced paired Tier-2 squashfs blobs (`vdb`, `vdc`) alongside the OCI image, and the template is wired for block-device hot-swap on resume ([`06-storage.md`](./06-storage.md) block-device tooling swap). Phase 9 templates are always `false`. Templates without paired blobs are rejected by admission when this flag is `true`.
+
+## Per-phase progression
+
+| Phase | Templates state |
+|---|---|
+| 1 | None — single hardcoded config inherited from today's compose |
+| 2 | One template per provider, declared in code |
+| 3 | Mounts spec real (skills + S3) |
+| 4 | Egress baseline + env_template separated from secrets |
+| 5 | Templates become CRD-shaped in k8s (via `agent-sandbox` `SandboxTemplate`) |
+| 6 | Admin UI CRUD + tenant→template resolver in L4 |
+| 8 | Multi-tier templates (sysbox / gVisor / kata-* fully wired) |
+
+## Source
+
+- Internal design notes (template patterns)
+- [`kubernetes-sigs/agent-sandbox`](https://github.com/kubernetes-sigs/agent-sandbox) — `SandboxTemplate` CRD basis

package/docs/future-architecture/architecture/10-observability.md

@@ -0,0 +1,121 @@
+<!-- SPDX-License-Identifier: FSL-1.1-Apache-2.0 -->
+<!-- Copyright (c) 2025 Open Computer Use Contributors -->
+
+# 10 — Observability
+
+> Metrics, traces, audit log, SLOs.
+> Boring on purpose — use the standard stack.
+
+## Signals
+
+| Signal | Tool | Notes |
+|---|---|---|
+| Metrics | Prometheus | Scrape L3 + L4 + egress proxy + (optionally) L1 |
+| Traces | OpenTelemetry → any OTLP backend | Sample at L4 ingress, propagate through L3 → L1 |
+| Structured logs | stdout/stderr → fluent-bit → object store / Loki | JSON lines |
+| Audit log | dedicated append-only sink | Separate from regular logs; retention ≥ 90d |
+
+## Required metrics
+
+L4:
+- `mcp_requests_total{tool,tenant,status}`
+- `mcp_request_duration_seconds{tool}`
+- `session_create_duration_seconds`
+- `session_active{tenant,template}` (gauge)
+- `secret_rotation_total{kind,status}`
+
+L3:
+- `sandbox_pool_size{template,state}` (state ∈ idle / leased / draining)
+- `sandbox_spawn_duration_seconds{template}`
+- `sandbox_exec_duration_seconds{template}`
+- `sandbox_terminate_total{template,reason}`
+
+L1 (in-sandbox):
+- `agent_exec_total{kind}` where kind ∈ bash/python/file/sub_agent
+- `agent_exec_duration_seconds{kind}`
+
+Egress proxy:
+- `egress_requests_total{decision,destination_class}`
+- `egress_request_duration_seconds`
+
+## SLOs (target)
+
+| SLO | Target |
+|---|---|
+| MCP request success rate | ≥ 99.9% (excluding user-side errors) |
+| Session create latency p99 | < 500 ms (warm pool hit) |
+| Session create latency p99 cold | < 2 s (cold start, kata-ch) |
+| Exec latency p99 | < 50 ms (orchestration overhead, not workload) |
+| CDP frame rate | ≥ 10 fps |
+| Egress proxy latency p99 | < 100 ms |
+
+These match our targets and are validated in Phase 5 (k8s prod) + Phase 9 (kata).
+
+## RAM-based capacity-sizing formula
+
+The first question operators ask is "how many sandboxes does a node hold?" The answer is bounded by **RAM**, not CPU — sandboxes idle most of the time but always reserve their memory request. For `kata-ch` / `kata-fc` specifically, the VMM itself owns a slab of memory that the workload never sees.
+
+```text
+concurrent_sandboxes_per_node = floor(
+    (node_ram_bytes - system_reserve_bytes - kubelet_reserve_bytes)
+    / (template.mem_request_bytes × overcommit_factor + vmm_overhead_bytes)
+)
+```
+
+| Term | Typical value | Notes |
+|---|---|---|
+| `node_ram_bytes` | Per node, e.g. `64 GiB` | The bare-metal node spec |
+| `system_reserve_bytes` | `1–2 GiB` | Kernel, daemons, monitoring agents |
+| `kubelet_reserve_bytes` | `~512 MiB` | k8s overhead per `kube-reserved` + `system-reserved` |
+| `template.mem_request_bytes` | `2 GiB` (default `customer-cu-kata-ch-v3`) | What the template guarantees the workload |
+| `overcommit_factor` | `1.0` for `customer-cu`, up to `1.5` for `internal` | Operators choose; lower = stricter |
+| `vmm_overhead_bytes` | `0` for runc/sysbox, `~20 MiB` for `kata-fc`, `~40 MiB` for `kata-ch` | Per-VM Firecracker / CH process |
+
+Operators size node pools by **solving for `node_ram_bytes`** given a target `concurrent_sandboxes_per_node` and the dominant template. Phase 9 validates the formula on real bare-metal hardware; the dashboard ships node-level "sandbox-density" gauge so the formula can be tuned with field data.
+
+The formula intentionally does **not** include CPU. CPU oversubscription is a separate axis governed by `cpu_request` and HPA; conflating it with RAM here would mislead capacity-planners.
+
+For the Phase 10 frozen-snapshot pool, the analog formula replaces `mem_request × concurrent` with `snapshot_blob_size × pool_size` — RAM cost goes to zero for cold pool entries (they live on disk), only resumed-but-idle sandboxes consume RAM. The formula is updated in the Phase 10 deliverable.
+
+## Distributed tracing
+
+The W3C `traceparent` header crosses every layer boundary; without that, "why was this exec slow?" is unanswerable. Concrete wire requirements:
+
+- **L4 ingress.** Generate a root span per MCP request; attach `traceparent` to every downstream call.
+- **L4 → L3.** Carried on the connect-go metadata (`traceparent` is a first-class metadata key); L3 starts a child span on receive.
+- **L3 → L1.** Carried as a JSON field in the `Configure` / `ToolCall` / `Exec` data-plane frame (the WS protocol doesn't have HTTP headers; the trace context rides inside the message envelope). L1 starts a child span on receive.
+- **Audit-log linkage.** Every audit event carries the `trace_id` of the request that triggered it. This is what lets the "why was this destination egress-blocked?" question reduce to a single trace lookup.
+
+Sampling: ingress samples at a configurable rate (default `1.0` in dev, `0.01` in prod); the rest of the stack honors the upstream sample decision (no resampling). Forced-sample on errors regardless of rate.
+
+OpenTelemetry SDK lands in L4 at Phase 6; L3 at Phase 6+; L1 at Phase 7 (the Rust agent uses `tracing` + `opentelemetry-otlp`). Phase 8 wires the audit-log linkage.
+
+## Audit log
+
+See [07-security.md](./07-security.md) for the mandatory event list and forbidden-content rules.
+
+- Sink: separate from regular logs (e.g., S3 bucket, immutability lock).
+- Retention: ≥ 90 days. SOC-2-aligned.
+- Schema: stable, versioned. Each event carries `event_id`, `ts`, `tenant_id`, `session_id`, `type`, `payload`.
+
+## Health probes
+
+- `/healthz` (liveness) and `/readyz` (readiness) on L4 (already exists) and L3 (Phase 5).
+- L1 agent: `GET /v1/health` is the readiness signal for the warm pool.
+
+## Phase progression
+
+| Phase | Observability change |
+|---|---|
+| 1–3 | Keep current stdout logging |
+| 4 | Audit-event emission added for secret rotation |
+| 5 | Prometheus scrape annotations in Helm + dashboards starter pack |
+| 6 | OpenTelemetry SDK in Go L4 + audit sink wired |
+| 8 | Egress proxy metrics + audit pipeline finalized; 90 d retention enforced |
+| 9 | Kata-tier metrics added; capacity-formula validation on bare-metal pool |
+| 10 | Multi-AZ traces, error-budget burn alerting; multi-region foundations |
+
+## Source
+
+- Internal operations notes
+- [07-security.md](./07-security.md)

package/docs/future-architecture/design-notes.md

@@ -0,0 +1,72 @@
+<!-- SPDX-License-Identifier: BUSL-1.1 -->
+<!-- Copyright (c) 2025 Open Computer Use Contributors -->
+
+# Design Notes
+
+Solution-shaped proposals that are **not yet locked**. This file is the sibling of [`gaps.md`](./gaps.md): `gaps.md` records *problems and risks*, this file records *candidate solutions* to them.
+
+Rules:
+
+- A design note is a **candidate**, not spec. Each one names its owning roadmap phase and must clear that phase's research pass and owner sign-off before any of it lands in `architecture/` or an ADR.
+- If a note conflicts with a file under `architecture/`, **`architecture/` wins** until the owning phase ships (same rule as the [README](./README.md#what-this-document-tree-does-not-do)).
+- Any third-party component named in a note must pass [ADR-0006](./adr/0006-no-agpl-no-bsl-dependencies.md) license vetting (no AGPL, no BSL in direct deps) at phase-research time, not here.
+
+---
+
+## DN-1 — Substrate-independent egress, identity & secret-broker design
+
+> Owning phases: Phase 4 (secret broker), Phase 6 (control plane), Phase 8 (egress proxy).
+> Derived from internal microVM design notes.
+> Status: **candidate.** Pending Phase 4/6/8 research + owner sign-off.
+
+**Goal.** One design for egress control, connectivity/identity, and secret handling that holds across all three deployment substrates — Docker Compose PoC, Kubernetes, microVM — as *one invariant with thin per-substrate enforcement*, not three separate designs.
+
+### 1. One egress invariant, three thin wrappers
+
+Invariant: **default-deny + allowlist-on-connect** — enforce against the resolved IP + TLS SNI, never the DNS name.
+
+| Substrate | Enforcement binding |
+|---|---|
+| Docker Compose | `DOCKER-USER` iptables chain |
+| Kubernetes | `NetworkPolicy` (egress) |
+| microVM | `nftables` |
+
+SNI-based allowlisting implementations to evaluate at Phase 8: HAProxy `req.ssl_sni`, smokescreen, Cilium `toFQDNs`, Envoy `tls_inspector`.
+
+### 2. "Internet, not intranet"
+
+The egress filter must deny RFC1918 + link-local + cloud metadata (`169.254.169.254`), and **must not forget IPv6** (`fc00::/7`, `fe80::/10`). A sandbox that can reach the internet must still not reach the deployment's internal network or the host's metadata endpoint.
+
+### 3. Connectivity + identity
+
+For the cases where a sandbox legitimately needs an *internal* service, do not widen the egress allowlist to the intranet. Instead use a mesh: **Tailscale / Headscale (self-hosted)** — connectivity over a single outbound connection, identity by mesh membership, least-privilege via mesh ACLs. The sandbox reaches exactly the internal services its ACL grants and nothing else.
+
+### 4. Don't expose keys — a broker-gateway
+
+A **broker-gateway lives outside the sandbox.** The workload receives a per-session token; the real `ANTHROPIC_API_KEY` exists only on the gateway. Claude Code (and any model client) is pointed at the gateway via `ANTHROPIC_BASE_URL`. **LiteLLM** sits *behind* the gateway as a token-accounting / usage-metering layer — **not** as the auth or RBAC layer.
+
+It aligns with the [`07-security.md`](./architecture/07-security.md) secret broker (Phase 4) and with an FD-passing hardening philosophy: a compromised sandbox can leak at most a scoped, short-lived session token, never the long-lived provider key.
+
+### 5. Real RBAC
+
+Authorization is a pipeline, and **LiteLLM is not it**:
+
+```text
+IdP (Keycloak)
+  → policy engine (OPA / Casbin)
+    → enforcement points (broker-gateway, egress filter, mesh ACLs)
+      → LiteLLM  (narrow role: token accounting only)
+```
+
+Explicitly: **LiteLLM ≠ RBAC.** It meters; it does not decide who may do what.
+
+### Mapping to the roadmap
+
+| Section | Owning phase | Refines |
+|---|---|---|
+| §1–§2 egress invariant + SSRF deny-set | [Phase 8](./roadmap.md#phase-8) | [`08-networking.md`](./architecture/08-networking.md) |
+| §3 mesh connectivity / identity | none yet — flag for Phase 6/8 scoping | new surface |
+| §4 broker-gateway | [Phase 4](./roadmap.md#phase-4) | [`07-security.md`](./architecture/07-security.md) |
+| §5 RBAC pipeline | [Phase 6](./roadmap.md#phase-6) + Phase 4 | [`02-layer4-control-plane.md`](./architecture/02-layer4-control-plane.md) |
+
+**License note (ADR-0006).** First-pass read: Headscale (BSD-3), Keycloak / OPA / Casbin / Cilium / Envoy (Apache-2), smokescreen (MIT), LiteLLM (MIT) are clear of AGPL/BSL. HAProxy is GPLv2 — acceptable as a standalone deployed service (not linked into our binaries). Confirm every dependency at the owning phase's research pass; this note does not authorize adoption.