deepdebug-local-agent 1.4.0 → 1.6.0

package/src/la-protocol-v2/contract/README.md

@@ -7,7 +7,7 @@ exist yet. v1 is untouched. The gateway packages `*.schema.json` + `index.json`
 `la-protocol-v2/` (pom resource; both Dockerfiles copy this folder), so this folder stays the only source.
 
 ## Files
-- `envelope.schema.json`: common frame, closed (`additionalProperties:false`). `<type>.schema.json` (21):
+- `envelope.schema.json`: common frame, closed (`additionalProperties:false`). `<type>.schema.json` (22):
   one per allowlisted type, each `$ref`s the envelope and closes `payload`. `defs.schema.json`: shared types.
 - `index.json`: allowlist, direction, signed/control flags, content class per type, and the limits below.
 - `vectors/<type>.vectors.json` (+ `cross-type`): valid and invalid messages with `expect` (`valid`, error `code`,
@@ -79,7 +79,7 @@ schema does not know. That runtime check covers the data frames sent through the
 handshake and control frames (`ready`, `ack`, `error`) are assembled by fixed code and are NOT schema-checked at runtime: the
 test transport validates every frame the session writes, and the tests fail on a violation.
 A fatal error closes the socket with 4400 + the position of its code in the `errorCode` enum (a test pins the order; `timeout` = 4420). Resume works only on the same gateway instance (state is in memory, 90 s); elsewhere `resumed:false`. Not done in
-F3-05a: relaying a v2 stream between gateway instances, the HTTP relay (F3-06), LA handlers (F3-05b/07). Gaps, for
+F3-05a: relaying a v2 stream between gateway instances and the HTTP relay (both done in F3-06, see docs/architecture/la-content-relay.md), LA handlers (F3-05b/07). Gaps, for
 F3-05b to confirm (none changes a schema):
 1. **`byteLength` is the decoded size of `data`**; the gateway refuses a chunk where it differs (`chunk_inconsistent`).
    The F3-22 vectors had 27 for a 26-byte payload and a `max-size-chunk` that decoded to 32769 bytes (fixed in F3-05a); the
@@ -101,3 +101,50 @@ F3-05b to confirm (none changes a schema):
 7. v1 commands are refused for an LA that speaks v2 (`protocol_mismatch`): until F3-06/07 move the callers, a v2 LA
    cannot serve `sendCommand`, so the LA must stay on v1 for tenants whose flows still use v1 commands.
 
+## Opaque refs in `content.request` (F3-06b, F3-07b) - convention, no schema change
+`content.request.payload.ref` stays `string, 1..512, no control characters` (the schema and its vectors are unchanged), but
+the gateway, when `deepdebug.content-relay.enabled` is on, ONLY sends one of three shapes and refuses anything else with a 400
+before the agent is asked (no path ever reaches the Local Agent, the logs of the load balancer or the gateway):
+- **opaque id**: exactly **43 characters** of the base64url alphabet (`A-Z a-z 0-9 - _`, no padding, **no prefix**), i.e. 32
+  bytes. The Local Agent mints it (32 random bytes, or HMAC-SHA256 of the workspace path under a local secret: a stable id per
+  file is fine, a guessable one is not) and keeps the id -> path map LOCALLY; the gateway cannot reverse it. An id is scoped to
+  the `workspaceId` of the envelope: the agent must resolve it inside that workspace only. A shorter or prefixed id (for
+  example a 26-character `cr1_...`) is NOT an opaque id: the gateway answers 400 and never asks.
+- **UUID** (canonical lower-case 8-4-4-4-12), only for the kinds whose subject is minted by VisionHammer:
+  `incident.detail incident.stack chat.session job.stream job.timeline approval.detail`.
+- **`worktree`**, the one symbolic ref, ONLY for `kind: "diff"`: the diff of the whole work tree of the workspace of the
+  envelope (`git diff HEAD`, never a file name). Exact, lower-case; it names nothing of the customer's, so it may appear in a
+  URL. The gateway requires a `workspaceId` with it (400 otherwise: "the work tree" of which workspace?) and refuses it for
+  every other kind. The agent resolves it inside that workspace only, as for an id.
+The Local Agent is the authority: it MUST NOT trust the shape. It validates the workspace (the one in the signed envelope, one
+it serves, else `forbidden`/`not_found`), resolves the id to a path inside that workspace's root (no `..`, no symlink out of
+the root, no sensitive file) and answers `not_found` for an id it does not know. With several agents per tenant the gateway
+asks the one that declared the workspace in `register.workspaces[]` first and tries the next only after `not_found`,
+`forbidden` or `agent_offline` (see docs/architecture/la-content-relay.md). A later contract revision may tighten the schema
+`pattern` once no client sends a path; until then this README is the convention.
+
+## `content.index` (LA -> CP, F3-07b): which ids exist
+An unsolicited, **unsigned** LA -> CP frame (`seq` 1.., never chunked, `agentId` required, like `log.stats`) that announces
+the opaque ids the agent minted for ONE workspace (the `workspaceId` of the envelope, required and non-empty here):
+`payload.entries[]`, 1..256 items `{kind, ref, size, updatedAt?}`, closed. `kind` is the `content.request` kind list; `ref` is
+exactly the 43-character id above; `size` is an integer 0..2^40 bytes of the item (informative: the relay still caps a
+response at 2 MiB); `updatedAt` an optional UTC timestamp. **It cannot carry a name or a path**: no free-form field exists,
+every string is an enum or the 43-character pattern, `additionalProperties:false` everywhere, and `uniqueItems` refuses the
+same entry twice. 256 entries with `updatedAt` are about 36 KiB, one frame; the agent sends more messages for more ids.
+It is metadata about client content, not client content (`contentClass: none`).
+- **What the gateway does** (`deepdebug.content-relay.enabled` on): validates the frame like any other (schema first, then
+  tenant/agent/direction/state/seq), keeps the entries IN MEMORY of that gateway instance for `index-ttl-seconds` (default 15 min,
+  renewed by announcing again), with limits per session, per tenant and per instance, never persists them and never logs an id
+  or a size. The key is a digest of tenant, workspace, kind and id, so a heap dump lists no id.
+- **The answer is the `ack`** (non-fatal, the session goes on): `accepted` = stored; `rejected` + `rate_limited` = a limit was
+  reached, NOTHING of that message was stored (all or nothing), announce less and later; `rejected` + `protocol_unsupported` =
+  this gateway keeps no index (content relay off), stop sending; `rejected` + `internal` = retry later.
+- **What it is for**: the control plane can know which ids exist for a workspace. It is NOT an access list: the agent stays the
+  authority and answers `not_found` / `forbidden` for what it does not serve, known id or not. Optional refusal of unknown ids
+  before asking the agent: `deepdebug.content-relay.require-known-ref`, default **false**, and it must stay false with more than
+  one gateway instance (the index is the memory of the instance that holds the agent's session, a request that lands on another
+  instance would be refused wrongly).
+- **What the agent must do**: announce ids when it mints them (or in a burst after `ready`), stay within 256 entries per frame
+  and 2048 live ids per session, send `content.index` only when its content handlers are on and DLP is `enforce`, and treat the
+  ids as expiring (it answers `not_found` for an id it forgot). It must never put a path or a file name in a frame.
+

package/src/la-protocol-v2/contract/content.index.schema.json

@@ -0,0 +1,92 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://schemas.visionhammer.io/la-protocol-v2/content.index.schema.json",
+  "title": "LA protocol v2: content.index (la->cp)",
+  "description": "Inventory of the opaque content ids the Local Agent minted for ONE workspace (the workspaceId of the envelope): kind, id and size, never a name or a path. The control plane keeps it in memory with a TTL, never persists or logs it.",
+  "type": "object",
+  "allOf": [
+    {
+      "$ref": "envelope.schema.json"
+    }
+  ],
+  "properties": {
+    "type": {
+      "const": "content.index"
+    },
+    "payload": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+        "entries": {
+          "type": "array",
+          "minItems": 1,
+          "maxItems": 256,
+          "uniqueItems": true,
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "properties": {
+              "kind": {
+                "type": "string",
+                "enum": [
+                  "incident.detail",
+                  "incident.stack",
+                  "file",
+                  "diff",
+                  "log.window",
+                  "log.groups",
+                  "chat.session",
+                  "wiki.page",
+                  "job.stream",
+                  "job.timeline",
+                  "approval.detail",
+                  "report"
+                ]
+              },
+              "ref": {
+                "type": "string",
+                "minLength": 43,
+                "maxLength": 43,
+                "pattern": "^[A-Za-z0-9_-]{43}$"
+              },
+              "size": {
+                "type": "integer",
+                "minimum": 0,
+                "maximum": 1099511627776
+              },
+              "updatedAt": {
+                "$ref": "defs.schema.json#/$defs/ts"
+              }
+            },
+            "required": [
+              "kind",
+              "ref",
+              "size"
+            ]
+          }
+        }
+      },
+      "required": [
+        "entries"
+      ]
+    },
+    "seq": {
+      "type": "integer",
+      "minimum": 1,
+      "maximum": 9007199254740991
+    },
+    "workspaceId": {
+      "type": "string",
+      "minLength": 1,
+      "maxLength": 64
+    },
+    "chunk": false,
+    "sig": false,
+    "paramsRaw": false,
+    "agentId": true
+  },
+  "required": [
+    "payload",
+    "agentId"
+  ]
+}

package/src/la-protocol-v2/contract/envelope.schema.json

@@ -34,6 +34,7 @@
         "job.result",
         "content.request",
         "content.response",
+        "content.index",
         "approval.decision",
         "config.update",
         "scan.upload",

package/src/la-protocol-v2/contract/index.json

@@ -99,6 +99,13 @@
       "signed": false,
       "contentClass": "relay"
     },
+    "content.index": {
+      "schema": "content.index.schema.json",
+      "direction": "la->cp",
+      "control": false,
+      "signed": false,
+      "contentClass": "none"
+    },
     "approval.decision": {
       "schema": "approval.decision.schema.json",
       "direction": "cp->la",

package/src/la-protocol-v2/runtime.js

@@ -7,6 +7,7 @@ import fs from "node:fs";
 import path from "node:path";
 import { fileURLToPath } from "node:url";
 import { createV2Link } from "./link.js";
+import { resolveContentHandlersConfig } from "./content-handlers/config.js";
 
 const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
 const WORKSPACE_ID = /^[A-Za-z0-9_.-]{0,64}$/;
@@ -32,9 +33,9 @@ async function diskFreeBytes(dir) {
 }
 
 /**
- * @returns the link, or null (with a warning) when v2 cannot run here: then the tunnel is the plain v1 client.
+ * @returns {Promise} the link, or null (with a warning) when v2 cannot run here: then the tunnel is the plain v1 client.
  */
-export function buildV2Link({ tenantId, agentId, signing, wsManager, getWorkspaceRoot, log = console, env = process.env }) {
+export async function buildV2Link({ tenantId, agentId, signing, wsManager, getWorkspaceRoot, log = console, env = process.env }) {
     const tenant = String(tenantId || "").toLowerCase();
     if (!UUID.test(tenant)) { log.warn("[la-v2] off: the tenant id is not a UUID"); return null; }
     if (!agentId) { log.warn("[la-v2] off: no agent id (authorizationId in ~/.deepdebug/config.json or LOCAL_AGENT_ID)"); return null; }
@@ -42,7 +43,20 @@ export function buildV2Link({ tenantId, agentId, signing, wsManager, getWorkspac
     let lastDisk = 0;
     const refreshDisk = () => diskFreeBytes(getWorkspaceRoot() || process.cwd()).then((v) => { lastDisk = v; });
     refreshDisk();
-    return createV2Link({
+    // F3-07: content.request handlers, behind LOCAL_AGENT_CONTENT_HANDLERS (default off: nothing below changes anything)
+    let contentCfg = resolveContentHandlersConfig(env, (m) => log.warn(m));
+    if (contentCfg.enabled) {
+        // F3-07b: client content leaves this machine only masked. Without LOCAL_AGENT_DLP_MODE=enforce the handlers are NOT
+        // registered and no content.* capability is announced: content.request is acked `rejected protocol_unsupported`, as
+        // with the flag off. There is no opt-out.
+        const { dlpConfigFromEnv } = await import("../dlp/index.js");
+        if (dlpConfigFromEnv(env).mode !== "enforce") {
+            (log.error || log.warn).call(log, "[la-content] ERROR: LOCAL_AGENT_CONTENT_HANDLERS=true but LOCAL_AGENT_DLP_MODE is not enforce: the content handlers are NOT registered and no content capability is announced (content.request answers protocol_unsupported). Set LOCAL_AGENT_DLP_MODE=enforce to serve content.");
+            contentCfg = { enabled: false };
+        }
+    }
+    const contentCaps = contentCfg.enabled ? ["content.diff", "content.file"] : [];
+    const link = createV2Link({
         log,
         sessionOptions: {
             tenantId: tenant,
@@ -51,7 +65,7 @@ export function buildV2Link({ tenantId, agentId, signing, wsManager, getWorkspac
             signing,
             laVersion: agentVersion(),
             // only what this build implements: the transport, the handshake and the v1 commands on the same connection
-            capabilities: ["protocol.v2", "v1.commands", "chunk.base64"],
+            capabilities: ["protocol.v2", "v1.commands", "chunk.base64", ...contentCaps],
             getWorkspaces: () => {
                 const open = (wsManager?.list?.() || []).map((w) => ({
                     workspaceId: String(w.id || ""),
@@ -69,4 +83,9 @@ export function buildV2Link({ tenantId, agentId, signing, wsManager, getWorkspac
             },
         },
     });
+    if (contentCfg.enabled) {
+        const { installContentHandlers } = await import("./content-handlers/install.js");
+        installContentHandlers({ link, tenantId: tenant, wsManager, getWorkspaceRoot, config: contentCfg, env, log });
+    }
+    return link;
 }

package/src/server.js

@@ -647,6 +647,9 @@ const WS_METADATA = resolveWsMetadataConfig();
 const WS_CONNECTION = createConnectionState();
 // LOCAL_AGENT_PROTOCOL_V2 (F3-05b, default off): with it off the tunnel below is the v1 client, untouched.
 const PROTOCOL_V2 = resolveProtocolV2Config();
+if (!PROTOCOL_V2.enabled && /^(true|1|yes|on)$/i.test(String(process.env.LOCAL_AGENT_CONTENT_HANDLERS ?? '').trim())) {
+    console.warn('[la-v2] LOCAL_AGENT_CONTENT_HANDLERS is set but LOCAL_AGENT_PROTOCOL_V2 is off: content handlers stay off');
+}
 if (WS_METADATA.mode === "enforce") {
     EXEC_POLICY.offlineGuard = createOfflinePolicyGuard({
         connection: WS_CONNECTION,
@@ -6985,7 +6988,7 @@ async function startWebSocketTunnel(port, gatewayUrl, authToken, tenantId, agent
     if (PROTOCOL_V2.enabled) {
         try {
             const { buildV2Link } = await import('./la-protocol-v2/runtime.js');
-            v2Link = buildV2Link({ tenantId, agentId, signing: SIGNING, wsManager, getWorkspaceRoot: () => WORKSPACE_ROOT });
+            v2Link = await buildV2Link({ tenantId, agentId, signing: SIGNING, wsManager, getWorkspaceRoot: () => WORKSPACE_ROOT });
         } catch (err) {
             console.warn(`[la-v2] off: ${err.code || err.name} while loading the protocol module`);
         }