groove-dev 0.27.8 → 0.27.11

package/integrations/FEDERATION_PLAN.md

@@ -1,583 +0,0 @@
-# Multi-Server Federation Plan — Cross-Instance Agent Communication
-
-> Status: Planning complete, not yet started
-> Created: 2026-04-12
-> Owner: Rok
-
----
-
-## Current State
-
-Already built:
-- federation.js — Ed25519 keypair per daemon, peer pairing via HTTP key exchange, signed contracts with 5-min replay protection, peer storage on disk
-- connect.js — SSH tunnel to remote daemon (groove connect user@host)
-- CLI commands: groove federation pair/unpair/list/status
-- API endpoints: /api/federation/* (status, peers, pair, unpair, contract send/receive)
-
-Not built:
-- Agent-to-agent communication across instances
-- WebSocket bridge between paired daemons
-- Ambassador agent role
-- IP whitelisting on bridge port
-- Nonce-based replay protection
-- Ephemeral (job-scoped) connections
-- GUI for any federation features
-
----
-
-## Vision
-
-Two or more Groove instances coordinate through ambassador agents. A user spawns an ambassador, it opens an ephemeral bridge to a paired peer, and agents on both sides work together as if they were on the same machine. The connection is job-scoped — it opens when work starts and closes when work finishes. Security is layered: IP whitelist first, then Ed25519 signatures, then nonces, all over Tailscale encryption.
-
----
-
-## The Ambassador Model
-
-### How It Works
-
-1. Pairing (already built): `groove federation pair 100.64.0.2` — both daemons exchange Ed25519 keys, become trusted peers. This is a one-time setup. The peer's IP is stored and becomes the whitelist entry.
-
-2. Spawning an ambassador: User picks "ambassador" role in the Spawn Wizard and selects a paired peer. The ambassador agent gets injected context: remote daemon's agent registry, remote project map, and a message channel to the remote side.
-
-3. Bridge opens: An ephemeral WebSocket connection opens between the two daemons on the bridge port (31416). Both sides verify IP whitelist, then Ed25519 handshake.
-
-4. Agents coordinate: Local and remote agents exchange messages, status updates, scope claims, and project map syncs through the bridge. The ambassador relays and translates.
-
-5. Job completes: All linked agents finish. Bridge auto-closes. WebSocket torn down, port released.
-
-### Use Cases
-
-- API server (VPS) + frontend (local): backend and frontend agents coordinate the build together, sharing API contracts, types, and integration points
-- Monorepo split: one Groove instance per service, ambassadors coordinate cross-service changes
-- Staging + production: agent on staging validates changes before ambassador signals production instance to deploy
-- Team collaboration: two developers on different machines, each running Groove, coordinate through federation
-
----
-
-## Security Architecture
-
-### Layered Defense
-
-```
-Incoming connection to :31416
-       |
-  [Layer 1: IP whitelist] — not on list? TCP RST, logged. Done.
-       |
-  [Layer 2: Tailscale encryption] — WireGuard tunnel, end-to-end
-       |
-  [Layer 3: WebSocket upgrade]
-       |
-  [Layer 4: Ed25519 hello verification] — bad signature? close.
-       |
-  [Layer 5: Nonce + timestamp check] — replay? reject.
-       |
-  [Layer 6: JSON schema validation] — malformed/oversized? reject.
-       |
-  [Layer 7: Route to agent inbox]
-       |
-  [Layer 8: Agent applies scope + safety rules]
-```
-
-### Layer 1 — IP Whitelist (First Gate)
-
-The bridge port checks source IP against the peer allowlist before any protocol logic, any crypto, any parsing. If you're not on the list, connection is dropped at TCP level.
-
-- Allowlist source: federation/peers/*.json (each peer file stores host IP)
-- Each `federation pair` adds one IP, each `federation unpair` removes one
-- Non-whitelisted IPs never reach the WebSocket layer
-- Bridge port appears closed to non-whitelisted IPs (port scan sees nothing)
-- All rejections logged to audit
-
-```
-allowedIPs = federation.getPeers().map(p => p.host)
-if sourceIP not in allowedIPs:
-    drop connection (TCP RST)
-    audit.log('bridge.rejected', { sourceIP, reason: 'not_whitelisted' })
-    return
-```
-
-Eliminates: rogue pairing, port scanning, brute force, flooding from unknown IPs.
-
-### Layer 2 — Tailscale (Transport Encryption)
-
-Primary transport: Tailscale (WireGuard). Provides:
-- End-to-end encryption between machines
-- Stable IPs (100.64.x.x) that don't change
-- NAT traversal (no port forwarding needed)
-- Machine identity verification
-
-Fallback transports:
-- SSH tunnel (groove connect) — already encrypted
-- For non-Tailscale static IPs: TLS required on bridge WebSocket (wss://)
-- bridge.js refuses unencrypted connections to non-Tailscale, non-localhost targets
-
-### Layer 3 — Ed25519 Signatures (Message Authentication)
-
-Every bridge message is signed with the sender daemon's Ed25519 private key (already built in federation.js). Receiving daemon verifies with the stored public key.
-
-Handles: compromised peer forgery (attacker would need the private key).
-
-### Layer 4 — Nonce + Timestamp (Replay Protection)
-
-Every message includes:
-- nonce: UUID v4, unique per message
-- timestamp: ISO 8601, must be within 5-minute window
-
-Receiving daemon:
-- Tracks seen nonces in memory (Set, pruned every 10 minutes)
-- Rejects duplicate nonces
-- Rejects messages outside the 5-minute timestamp window
-- Combined with signature, eliminates replay attacks entirely
-
-### Layer 5 — Input Validation
-
-- JSON schema validation on every received message
-- Max message size: 1MB
-- Max JSON depth: 10 levels
-- No prototype pollution (__proto__, constructor, prototype keys rejected)
-- Payload is text only — no executable code, no serialized objects
-- Bridge never passes payloads to eval, new Function, or child_process
-
-### Layer 6 — Scope and Approval
-
-- Remote peers can only send messages, not execute commands
-- Messages go to agent inbox — the agent decides what to do
-- Agent's own scope restrictions and CLAUDE.md safety rules apply
-- Dangerous cross-instance actions (task handoffs, file syncs) require local user approval
-- Message type allowlist: only hello, agent.status, message, coordinate, sync, heartbeat
-
-### Additional Measures
-
-- Rate limiting: 100 messages/min per peer on bridge
-- Anomaly detection: flag sudden spikes in message volume, auto-pause connection
-- Key rotation: `groove federation rotate-keys` generates new keypair, invalidates old, all peers must re-pair
-- Audit trail: every cross-daemon event logged (send, receive, reject, connect, disconnect)
-
----
-
-## IP Whitelist — Multi-Peer Mesh
-
-Scales naturally with each pairing:
-
-```
-Server A (laptop):       allowlist = [B: 100.64.0.2, C: 100.64.0.3]
-Server B (api-vps):      allowlist = [A: 100.64.0.1, C: 100.64.0.3]
-Server C (frontend-vps): allowlist = [A: 100.64.0.1, B: 100.64.0.2]
-```
-
-Each server only talks to servers it has explicitly paired with. Peer-to-peer, no central hub. Adding a new server: `groove federation pair <new-ip>` on both sides.
-
-### Dynamic IP Handling
-
-| IP Type | Stability | Recommendation |
-|---------|-----------|----------------|
-| Tailscale (100.64.x.x) | Stable, tied to machine identity | Primary path, recommended |
-| Static VPS | Stable | Works perfectly |
-| Dynamic home IP | Changes | Use Tailscale, or re-pair when IP changes |
-
----
-
-## Ephemeral Connections — Job Mode
-
-Connections are tied to a job, not a peer relationship. The pairing (key exchange + IP whitelist) is permanent, but the live WebSocket is temporary.
-
-### Connection Lifecycle
-
-```
-User spawns ambassador with peer + task
-  -> bridge.open(peerId, { jobId, ttl, agents })
-  -> IP whitelist check
-  -> WebSocket connects, Ed25519 handshake
-  -> Agents work, messages flow
-  -> Job completes (all linked agents done)
-  -> bridge.close(peerId, jobId)
-  -> WebSocket torn down
-```
-
-### Auto-Close Triggers
-
-Any of these kills the connection:
-
-| Trigger | Default | Description |
-|---------|---------|-------------|
-| Job complete | Always | All linked agents complete/crash/killed |
-| Idle timeout | 15 min | No messages in either direction |
-| TTL expiry | 2 hours | Hard cap, even if agents still running |
-| Manual disconnect | User action | GUI button or CLI command |
-| Daemon shutdown | Always | Graceful close with signed goodbye |
-
-### Keep-Alive Mode (Opt-In)
-
-For persistent coordination (always-on servers), user sets keepAlive: true when spawning. This disables idle timeout and extends TTL to 24h with auto-renew. Default is ephemeral.
-
----
-
-## Protocol Design — WebSocket Bridge
-
-### Message Envelope
-
-Every message through the bridge:
-
-```json
-{
-  "type": "message",
-  "nonce": "550e8400-e29b-41d4-a716-446655440000",
-  "timestamp": "2026-04-12T20:30:00.000Z",
-  "from": { "daemon": "abc123", "agent": "frontend-1" },
-  "to": { "daemon": "def456", "agent": "backend-3" },
-  "payload": { ... },
-  "signature": "base64-ed25519-signature"
-}
-```
-
-### Message Types
-
-| Type | Direction | Description |
-|------|-----------|-------------|
-| hello | Bidirectional | Initial handshake with agent registry + job metadata |
-| goodbye | Bidirectional | Graceful close with reason |
-| heartbeat | Bidirectional | Keepalive, every 30s, dead peer at 90s |
-| agent.status | Broadcast | Agent spawned/completed/crashed/killed on remote |
-| message | Routed | Direct agent-to-agent message |
-| coordinate | Routed | Scope negotiation, task handoff, shared decision |
-| sync.registry | Request/Response | Full agent registry exchange |
-| sync.projectmap | Request/Response | Journalist project map exchange |
-
-### Connection Flow
-
-```
-Daemon A                              Daemon B
-   |                                     |
-   |--- TCP connect to :31416 ---------->|
-   |                            [IP whitelist check]
-   |<-- TCP accept ---------------------|
-   |                                     |
-   |--- WS upgrade -------------------->|
-   |<-- WS accept ----------------------|
-   |                                     |
-   |--- hello (signed, with registry) ->|
-   |<-- hello (signed, with registry) --|
-   |                                     |
-   |--- message (A:frontend -> B:backend) -->|
-   |<-- message (B:backend -> A:frontend) ---|
-   |                                     |
-   |--- heartbeat --------------------->|
-   |<-- heartbeat ----------------------|
-   |                                     |
-   |--- goodbye (job complete) -------->|
-   |<-- goodbye (ack) ------------------|
-   |--- TCP close --------------------->|
-```
-
----
-
-## Daemon Architecture
-
-### New Files
-
-#### bridge.js — Ephemeral WebSocket Bridge Manager
-
-```
-class Bridge {
-  constructor(daemon)
-    - connections: Map<jobId, Connection>
-    - seenNonces: Set (pruned every 10 min)
-    - bridgeServer: WebSocket.Server on :31416
-
-  open(peerId, { jobId, linkedAgents, ttl, keepAlive })
-    - Validate peer is paired
-    - Enforce transport security (Tailscale/localhost/wss)
-    - Connect WebSocket to peer's bridge port
-    - Send signed hello with job metadata + agent registry
-    - Start idle timer + TTL timer
-    - Return connection handle
-
-  close(jobId)
-    - Send signed goodbye
-    - Close WebSocket
-    - Clean up timers
-    - Audit log
-
-  send(jobId, message)
-    - Add nonce (UUID v4) + timestamp
-    - Sign with Ed25519
-    - Validate message size (<1MB)
-    - Route through connection
-    - Update lastActivity
-
-  receive(rawMessage)
-    - Verify source IP is whitelisted
-    - Verify Ed25519 signature
-    - Check nonce (reject duplicate)
-    - Check timestamp (reject >5min)
-    - Validate JSON schema
-    - Route to target agent's inbox
-    - Emit event for GUI WebSocket
-
-  _onAgentStatusChange(agentId, status)
-    - Broadcast to all connections that include this agent
-    - If all linked agents done, trigger close
-
-  _checkIdle(jobId)
-    - If lastActivity > idleTimeout, close
-
-  _enforceTransportSecurity(peerHost)
-    - Tailscale range (100.64.0.0/10): allow ws://
-    - localhost/127.0.0.1: allow ws://
-    - All others: require wss:// or reject
-
-  _pruneNonces()
-    - Remove nonces older than 10 minutes
-    - Runs on 5-minute timer
-}
-```
-
-### Modified Files
-
-#### federation.js
-
-- Add peer IP to allowlist management
-- Add bridge lifecycle hooks (onPeerPaired, onPeerUnpaired)
-- Add key rotation support
-- Add nonce tracking helpers
-
-#### api.js — New Endpoints
-
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | /api/federation/bridge/status | Bridge connection state per peer |
-| POST | /api/federation/bridge/open | Open bridge to peer (for ambassador spawn) |
-| POST | /api/federation/bridge/close | Close bridge connection |
-| POST | /api/federation/bridge/message | Send message to remote agent |
-| GET | /api/federation/bridge/inbox/:agentId | Pending messages for an agent |
-| GET | /api/federation/remote-agents | List agents on all peers |
-| GET | /api/federation/remote-agents/:peerId | List agents on a specific peer |
-
-#### index.js
-
-- Initialize Bridge after Federation
-- Wire bridge port listener (31416)
-- Pass bridge reference to ProcessManager for ambassador spawning
-
-#### introducer.js
-
-- Inject remote agent context into ambassador agents
-- Include remote project map in agent briefing
-- Add cross-instance team awareness
-
-#### registry.js
-
-- Support "remote" agent entries (ghost agents from peers)
-- Remote agents are read-only, updated via bridge sync
-
-#### process.js
-
-- Ambassador role spawning with federation context
-- Wire agent status changes to bridge broadcasts
-
-### Port Allocation
-
-| Port | Binding | Purpose |
-|------|---------|---------|
-| 31415 | 127.0.0.1 | Main daemon (GUI, API, WebSocket, local agents) |
-| 31416 | Tailscale interface / 0.0.0.0 with IP whitelist | Bridge (peer-to-peer federation) |
-
----
-
-## GUI Design
-
-### A. Spawn Wizard — Federation Section
-
-Appears when at least one peer is paired:
-
-```
-Federation
-[x] Connect to peer: [dropdown: my-vps (100.64.0.2)]
-    Remote agents: backend-3, frontend-2, planner-1
-    Mode: (x) Ambassador  ( ) Direct link
-
-    Ambassador: Full bridge — relays between local and remote teams
-    Direct link: 1:1 channel to a specific remote agent
-```
-
-### B. Agent Config Panel — Federation Tab
-
-New tab alongside Chat, Config, Monitor, Files:
-
-```
-Federation
-Peer: my-vps (100.64.0.2)         [Connected]
-Role: Ambassador
-Job: job-a1b2c3
-Bridge uptime: 14m
-Messages: 47 sent / 52 received
-Last activity: 12s ago
-
-Remote Team:
-  backend-3  (running)  — packages/api/src/**
-  frontend-2 (running)  — packages/web/src/**
-  planner-1  (completed)
-
-[Send Message]  [Sync Project Map]  [Disconnect]
-```
-
-### C. Agent Tree View
-
-Remote agents appear as ghost nodes — dimmed, with a network icon badge. Connected via dashed edges to the local ambassador.
-
-```
-[planner-1] ----> [frontend-1]
-     |                 |
-     |           [ambassador-1] - - - - [remote:backend-3]
-     |                           - - - - [remote:frontend-2]
-     v
-[backend-1]
-```
-
-Clicking a remote ghost node shows read-only status in the detail panel.
-
-### D. Status Bar
-
-When federation bridge is active:
-
-```
-Connected | 3/4 agents | Federation: 1 peer (3 remote agents)
-```
-
-### E. Dashboard — Federation Card
-
-New card in the KPI strip:
-
-| Metric | Value |
-|--------|-------|
-| Peers connected | 1 |
-| Remote agents | 3 |
-| Messages exchanged | 99 |
-| Bridge uptime | 14m |
-| Active jobs | 1 |
-
-### F. Settings — Federation Section
-
-```
-Federation
-  Daemon ID: abc123
-  Keypair: Ready (Ed25519)
-
-  Paired Peers:
-  [my-vps]  100.64.0.2  paired 2026-04-10  [Unpair]
-  [staging] 100.64.0.3  paired 2026-04-11  [Unpair]
-
-  [Pair New Server]  [Rotate Keys]
-
-  Bridge Settings:
-    Default idle timeout: [15] minutes
-    Default TTL: [2] hours
-    Bridge port: [31416]
-```
-
----
-
-## Integration Registry Entry
-
-New entry in integrations-registry.json:
-
-```json
-{
-  "id": "federation",
-  "name": "Cross-Server Federation",
-  "description": "Connect agents across Groove instances for coordinated multi-server builds",
-  "category": "developer",
-  "icon": "network",
-  "tags": ["federation", "multi-server", "cross-instance", "ambassador"],
-  "roles": ["ambassador", "fullstack", "backend", "frontend"],
-  "authType": "none",
-  "envKeys": [],
-  "setupSteps": [
-    "Pair with a remote Groove instance: groove federation pair <ip>",
-    "Both daemons must be on Tailscale or have static IPs",
-    "Spawn an ambassador agent and select the paired peer"
-  ],
-  "requiresApproval": ["send_message", "sync_project", "coordinate_task"],
-  "agentInstructions": "## Federation Integration\n\nYou are an ambassador agent connected to a remote Groove instance.\n\n### API\nSend message: POST http://localhost:31415/api/federation/bridge/message\nBody: {\"jobId\": \"...\", \"to\": {\"agent\": \"remote-agent-id\"}, \"payload\": {...}}\n\nCheck inbox: GET http://localhost:31415/api/federation/bridge/inbox/<your-agent-id>\n\nRemote agents: GET http://localhost:31415/api/federation/remote-agents\n\n### Rules\n- Coordinate, don't command — remote agents have their own scope and safety rules\n- Confirm with user before initiating cross-instance task handoffs\n- Keep messages concise — bridge has rate limits (100/min)"
-}
-```
-
----
-
-## CLI Additions
-
-```
-groove federation pair <ip>        — pair with remote daemon (existing)
-groove federation unpair <id>      — remove peer (existing)
-groove federation list             — list peers (existing)
-groove federation status           — federation status (existing)
-groove federation rotate-keys      — rotate Ed25519 keypair, invalidate old
-groove federation bridge status    — show active bridge connections
-groove federation bridge close <id> — manually close a bridge connection
-```
-
----
-
-## Phased Rollout
-
-### Phase 1 — Bridge + Ambassador (core)
-
-- [ ] bridge.js — WebSocket bridge with IP whitelist, Ed25519 handshake, nonce replay protection
-- [ ] Bridge port listener on 31416 with IP whitelist enforcement
-- [ ] Agent registry sync (see remote agents via bridge)
-- [ ] Direct agent-to-agent messaging through bridge
-- [ ] Ambassador role in spawn wizard
-- [ ] Ephemeral connection lifecycle (idle timeout, TTL, auto-close on job complete)
-- [ ] API endpoints for bridge management
-- [ ] Federation tab in agent config panel
-
-### Phase 2 — Coordination Protocol
-
-- [ ] Cross-instance scope negotiation (remote backend claims API files, local frontend claims UI files)
-- [ ] Task handoff: remote agent completes, hands off to local agent with context
-- [ ] Shared journalist: project map exchange so both sides have full picture
-- [ ] Ghost nodes in agent tree view (remote agents as dimmed nodes)
-- [ ] Federation card on dashboard
-
-### Phase 3 — Polish + Scale
-
-- [ ] Multi-peer mesh (3+ daemons)
-- [ ] Keep-alive mode for persistent coordination
-- [ ] Key rotation command and flow
-- [ ] Anomaly detection (spike in message volume, auto-pause)
-- [ ] Settings panel for federation config
-- [ ] Message history / audit viewer in GUI
-- [ ] Connection quality indicators
-- [ ] TLS fallback for non-Tailscale public IP connections
-
----
-
-## Team Structure
-
-| Role | Scope | Work |
-|------|-------|------|
-| Backend | daemon/src/bridge.js (new), federation.js, api.js, index.js, process.js, introducer.js, integrations-registry.json | Bridge manager, IP whitelist, nonce tracking, API endpoints, ambassador spawning |
-| Frontend | gui/src/components/agents/spawn-wizard.jsx, agent-panel.jsx, gui/src/views/agents.jsx, dashboard.jsx, settings.jsx, stores/groove.js | Federation tab, ghost nodes, spawn wizard federation section, dashboard card, settings panel |
-| QC (fullstack) | All | Verify end-to-end bridge works, build, no regressions, security review |
-
----
-
-## Key Dependencies
-
-```
-# Already in use
-ws                — WebSocket (already used by daemon)
-crypto            — Ed25519 (already used by federation.js)
-minimatch         — scope matching (already used by lockmanager.js)
-
-# New (if TLS fallback needed)
-selfsigned        — generate self-signed certs for wss:// fallback
-```
-
-No new heavy dependencies. The bridge is built on ws (already a dependency) and crypto (built-in Node).
-
----
-
-## Summary
-
-Federation turns Groove from a single-machine orchestrator into a distributed one. The ambassador model keeps it simple — agents don't need to know about networking, they just talk to their ambassador. Security is layered: IP whitelist stops 99% of threats at the TCP level, Ed25519 + nonces handle the rest, and Tailscale encrypts the transport. Connections are ephemeral by default — open for a job, closed when done. The whole system scales by adding peers with `groove federation pair`.