deepdebug-local-agent 1.8.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/context.md +2 -1
- package/CLAUDE.md +1 -1
- package/docs/la-protocol-v2.md +19 -2
- package/env.example +5 -0
- package/package.json +1 -1
- package/src/incident-detector/README.md +2 -1
- package/src/incident-detector/aggregator.js +8 -3
- package/src/incident-detector/detector.js +2 -1
- package/src/incident-detector/emitter.js +16 -6
- package/src/incident-detector/index.js +3 -3
- package/src/incident-detector/sources/webhook-source.js +109 -0
- package/src/la-protocol-v2/contract/CONTRACT_SOURCE.json +24 -16
- package/src/la-protocol-v2/contract/README.md +103 -3
- package/src/la-protocol-v2/contract/config.update.schema.json +85 -1
- package/src/la-protocol-v2/contract/hello.schema.json +26 -1
- package/src/la-protocol-v2/contract/incident.candidate.schema.json +11 -0
- package/src/la-protocol-v2/contract/index.json +9 -2
- package/src/la-protocol-v2/contract/ready.schema.json +8 -0
- package/src/la-protocol-v2/contract/register.schema.json +26 -1
- package/src/la-protocol-v2/contract/webhook.forward.schema.json +72 -3
- package/src/la-protocol-v2/contract.js +45 -7
- package/src/la-protocol-v2/llm-key/config.js +29 -0
- package/src/la-protocol-v2/llm-key/handler.js +33 -0
- package/src/la-protocol-v2/llm-key/install.js +21 -0
- package/src/la-protocol-v2/llm-key/store.js +173 -0
- package/src/la-protocol-v2/runtime.js +37 -3
- package/src/la-protocol-v2/session.js +47 -13
- package/src/la-protocol-v2/webhooks/assembler.js +85 -0
- package/src/la-protocol-v2/webhooks/config.js +51 -0
- package/src/la-protocol-v2/webhooks/handler.js +125 -0
- package/src/la-protocol-v2/webhooks/install.js +22 -0
- package/test/contract/la-protocol-v2/vectors/ack.vectors.json +91 -0
- package/test/contract/la-protocol-v2/vectors/config.update.vectors.json +1339 -0
- package/test/contract/la-protocol-v2/vectors/hello.vectors.json +261 -0
- package/test/contract/la-protocol-v2/vectors/incident.candidate.vectors.json +623 -0
- package/test/contract/la-protocol-v2/vectors/ready.vectors.json +599 -0
- package/test/contract/la-protocol-v2/vectors/register.vectors.json +275 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-after-last.json +186 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-body-hash-changes.json +186 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-last-flag-wrong.json +186 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-non-last-too-short.json +152 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-out-of-order.json +152 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-sha-mismatch.json +186 -0
- package/test/contract/la-protocol-v2/vectors/traces/invalid-webhook-chunk-total-changes.json +186 -0
- package/test/contract/la-protocol-v2/vectors/traces/valid-webhook-chunks-in-order.json +218 -0
- package/test/contract/la-protocol-v2/vectors/webhook.forward.vectors.json +1263 -0
- package/test/incident-detector/size-model.test.js +127 -0
- package/test/la-protocol-v2/contract-v21.test.js +107 -0
- package/test/la-protocol-v2/contract.test.js +6 -4
- package/test/la-protocol-v2/helpers/reference-cp.js +8 -4
- package/test/la-protocol-v2/llm-key.test.js +509 -0
- package/test/la-protocol-v2/traces.test.js +3 -3
- package/test/la-protocol-v2/webhook-receiver.test.js +702 -0
package/.claude/context.md
CHANGED
|
@@ -15,7 +15,8 @@ Cloud Run service (`deepdebug-local-agent-qa` / `deepdebug-local-agent`) for the
|
|
|
15
15
|
variant.
|
|
16
16
|
|
|
17
17
|
**Does not:** call LLMs or hold provider keys (the thinking happens in vh-agents via
|
|
18
|
-
vh-litellm)
|
|
18
|
+
vh-litellm). The one exception, behind `LOCAL_AGENT_LLM_KEY` (default off): the VIRTUAL key of the AI
|
|
19
|
+
gateway delivered by `config.update` (F3-09), kept in memory or a 0600 file, never logged; there is no LLM client yet, decide tenant policy, or merge anything.
|
|
19
20
|
|
|
20
21
|
## Neighbours
|
|
21
22
|
|
package/CLAUDE.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
layers, simplicity, testing, review checklist, `node.md`).
|
|
8
8
|
3. `README.md`: features, legacy connectors, configuration.
|
|
9
9
|
|
|
10
|
-
Non-negotiables: no LLM calls and no provider keys here; exec with argument arrays only;
|
|
10
|
+
Non-negotiables: no LLM calls and no provider keys here (the one exception, approved by Edu 2026-11-03, is the VIRTUAL key of the AI gateway delivered by `config.update`, F3-09, `client` tenants only: flag `LOCAL_AGENT_LLM_KEY`, memory by default and a 0600 file in a 0700 dir only with `LOCAL_AGENT_LLM_KEY_FILE`, never in logs or acks; there is still no LLM client); exec with argument arrays only;
|
|
11
11
|
paths confined to the workspace; never log secrets or raw customer log payloads beyond
|
|
12
12
|
what an incident needs; PRs target `main` (merging publishes the npm package).
|
|
13
13
|
Personal, uncommitted notes go in `CLAUDE.local.md` (gitignored).
|
package/docs/la-protocol-v2.md
CHANGED
|
@@ -146,10 +146,27 @@ the same rules as the Java monitor (errorHash, fingerprint, significance, noise;
|
|
|
146
146
|
(9 metadata fields, no text) per errorHash window. Announces the capability `incident.candidate`. The detail stays in a local JSONL store (0600, DLP-masked).
|
|
147
147
|
Needs `LOCAL_AGENT_PROTOCOL_V2=true`. Details, limits and hooks: `src/incident-detector/README.md`. The `incident.detail` content kind is NOT served yet.
|
|
148
148
|
|
|
149
|
+
## Contract 2.1 consumers (F3-09 / F3-12 / F3-11b, flags default false)
|
|
150
|
+
|
|
151
|
+
The contract copy is revision 2.1 (`src/la-protocol-v2/contract/CONTRACT_SOURCE.json` names the gateway commit and branch). With every flag off the agent announces exactly what it
|
|
152
|
+
announced before (`protocol.v2 v1.commands chunk.base64`); `contract.v2.1` is announced only when one of the three features below is on.
|
|
153
|
+
- **LLM key** (`LOCAL_AGENT_LLM_KEY`, capability `llm.key`; `src/la-protocol-v2/llm-key/`): `config.update.llmKey` `set` / `revoke`. The virtual key of the AI gateway is held in a
|
|
154
|
+
private field (memory) or, with `LOCAL_AGENT_LLM_KEY_FILE`, in a 0600 file outside every workspace; it is never in an ack, a log, the audit or a frame. A key whose id is not the
|
|
155
|
+
sha256 of the key, or that has expired, is `rejected schema_violation`. There is NO LLM client in the agent: `link.llm.request(path, init)` is the hook (https endpoint of the control
|
|
156
|
+
plane, Authorization set by the holder, redirects refused) and nothing calls it yet. Memory by default; the file only with `LOCAL_AGENT_LLM_KEY_FILE` (file 0600, directory 0700;
|
|
157
|
+
exception to "no keys here" approved by Edu 2026-11-03, recorded in `CLAUDE.md`). `llm.key.held` (contract 2.1 (e)): at EVERY connect the agent adds this capability to `register` AND `hello`
|
|
158
|
+
only while the holder has a valid, unexpired key (never without `llm.key`); a revoked or expired key stops being announced on the next connection, so the control plane delivers a new one.
|
|
159
|
+
- **Webhooks** (`LOCAL_AGENT_WEBHOOK_FORWARD`, capabilities `webhook.forward` + `webhook.chunked`; `src/la-protocol-v2/webhooks/`): one frame (<= 9216 bytes) or chunks (16 KiB, <= 64, <= 1 MiB)
|
|
160
|
+
rebuilt IN MEMORY with the contract limits (30 s, 4 assemblies, 4 MiB), sha256 verified, handed to the incident detector (`src/incident-detector/sources/webhook-source.js`, then the
|
|
161
|
+
same pipeline and the DLP-masked local store). The ack of the last chunk (or of the single frame) is sent only after the local delivery finished, waiting up to `LOCAL_AGENT_WEBHOOK_DELIVERY_MS`
|
|
162
|
+
(default 10 s, max 14 s; the session does not ack `accepted` after 2 s for this type); past it the ack is `rejected timeout`. Delivered ids (+ sha256 of the body) are remembered in memory for 10 min (max 1024):
|
|
163
|
+
a resend (same id and body) is acked `accepted`, chunk by chunk, without being processed again, so the detector's count is not inflated. Needs `LOCAL_AGENT_INCIDENT_DETECTOR=true`.
|
|
164
|
+
- **Candidate size and model** (no new flag, part of `LOCAL_AGENT_INCIDENT_DETECTOR`): `incident.candidate` carries `stackBytes` and `model` only if `ready.capabilities` listed `incident.candidate.size-model`.
|
|
165
|
+
|
|
149
166
|
## Tests
|
|
150
167
|
|
|
151
|
-
`npm test` (node:test). The incident detector (F3-11b) is in `test/incident-detector/` (`parity.test.js` against `parity.vectors.json`, `units`, `sources`, `wiring`); the rest of v2 in `test/la-protocol-v2/`:
|
|
152
|
-
`contract.test.js` (all
|
|
168
|
+
`npm test` (node:test). Contract 2.1: `contract-v21.test.js`, `webhook-receiver.test.js` (assembler, handler, the 8 traces, end to end), `llm-key.test.js` (including the sweep for the key), `test/incident-detector/size-model.test.js`. The incident detector (F3-11b) is in `test/incident-detector/` (`parity.test.js` against `parity.vectors.json`, `units`, `sources`, `wiring`); the rest of v2 in `test/la-protocol-v2/`:
|
|
169
|
+
`contract.test.js` (all 294 frame vectors of F3-22 + 2.1, valid and invalid, the 54 `file.list` body vectors, plus the drift check), `traces.test.js` (the 34
|
|
153
170
|
traces), `session-inbound.test.js`, `session-outbound.test.js`, `link.test.js`, `tunnel-e2e.test.js` (the real
|
|
154
171
|
`src/server.js` against a reference CP; flag off = v1 frames), and for F3-07 `content-path-guard.test.js`,
|
|
155
172
|
`content-handlers.test.js`, `content-wiring.test.js` (flag, runtime wiring, real `src/server.js`), and for F3-07b
|
package/env.example
CHANGED
|
@@ -92,4 +92,9 @@ LOG_LEVEL=info
|
|
|
92
92
|
# LOCAL_AGENT_INCIDENT_STDIN=true read log lines from the agent's own stdin
|
|
93
93
|
# LOCAL_AGENT_INCIDENT_STORE_DIR=~/.deepdebug/incidents LOCAL_AGENT_INCIDENT_WINDOW_MS=300000 LOCAL_AGENT_INCIDENT_FLUSH_MS=30000
|
|
94
94
|
# LOCAL_AGENT_INCIDENT_MAX_EMITS_PER_MINUTE=60 LOCAL_AGENT_INCIDENT_MAX_PENDING=500 LOCAL_AGENT_INCIDENT_MAX_TRACKED=5000
|
|
95
|
+
# LOCAL_AGENT_WEBHOOK_FORWARD=true receives the webhook bodies the control plane strips and forwards (webhook.forward, one frame <= 9216 bytes or chunks up to 1 MiB),
|
|
96
|
+
# rebuilds them in memory, verifies the sha256 and hands them to the incident detector. Needs LOCAL_AGENT_PROTOCOL_V2=true AND
|
|
97
|
+
# LOCAL_AGENT_INCIDENT_DETECTOR=true (the local consumer). Tighten only: LOCAL_AGENT_WEBHOOK_MAX_BYTES, _MAX_OPEN, _MAX_BUFFERED_BYTES, _ASSEMBLY_MS, _MAX_ENTRIES. LOCAL_AGENT_WEBHOOK_DELIVERY_MS (default 10000, max 14000: how long the final ack waits for the local delivery), _DEDUPE_MS (600000) and _DEDUPE_MAX (1024): resends of a delivered webhookId are acked without reprocessing
|
|
98
|
+
# LOCAL_AGENT_LLM_KEY=true accepts the virtual key of the AI gateway (config.update llmKey): in memory only, never logged or returned in an ack. Needs LOCAL_AGENT_PROTOCOL_V2=true
|
|
99
|
+
# LOCAL_AGENT_LLM_KEY_FILE=/var/lib/deepdebug/llm-key.json optional: keep the key across restarts (mode 0600, directory 0700, MUST be outside every workspace)
|
|
95
100
|
# LOCAL_AGENT_CONTENT_DENY_GLOBS=a/,*.x extra deny globs (they add to the built-in list: .env*, keys, credentials files, secrets/ ...)
|
package/package.json
CHANGED
|
@@ -26,7 +26,8 @@ Flag: `LOCAL_AGENT_INCIDENT_DETECTOR=true` (default **false**; needs `LOCAL_AGEN
|
|
|
26
26
|
|
|
27
27
|
Out: `errorHash` (32 hex = sha256 prefix of the normalised FIRST line, Java `computeGroupHash`), `fingerprint` (64 hex, Java
|
|
28
28
|
`IncidentFingerprintGenerator`), `severity`, `service`, `environment`, `count`, `firstSeen`, `lastSeen` (the window), `logCategory`.
|
|
29
|
-
|
|
29
|
+
Contract 2.1 adds two optional numbers/labels, sent ONLY when the CP listed `incident.candidate.size-model` in `ready.capabilities`: `stackBytes` (UTF-8 size of the biggest stack of the window,
|
|
30
|
+
saturating at 16 MiB) and `model` (`stack-detector-<agent version>`). There is still no field for text. The size is also in the local store (`stackBytes`).
|
|
30
31
|
Stays: message, stack, everything else, in `LOCAL_AGENT_INCIDENT_STORE_DIR` (default `~/.deepdebug/incidents`).
|
|
31
32
|
|
|
32
33
|
## Window semantics (for the control plane, F3-11c)
|
|
@@ -12,17 +12,20 @@
|
|
|
12
12
|
* bounds at most `maxTracked` hashes at once. Beyond it the least recently seen one is closed (final update if it has one
|
|
13
13
|
* pending) and counted as evicted. A hash that comes back after its window closed opens a new window.
|
|
14
14
|
* severity the highest seen in the window. fingerprint/service/logCategory: those of the first occurrence.
|
|
15
|
+
* stackBytes the biggest stack (UTF-8 bytes) seen in the window, saturating at 16 MiB (contract 2.1 (c)); 0 = no stack.
|
|
15
16
|
* Pure: no timers, no I/O. The detector calls tick() on an interval.
|
|
16
17
|
*/
|
|
17
18
|
import { maxSeverity } from "./categorize.js";
|
|
18
19
|
|
|
20
|
+
const MAX_STACK_BYTES = 16 * 1024 * 1024; // contract 2.1: incident.candidate.stackBytes saturates here
|
|
21
|
+
|
|
19
22
|
export function createAggregator({ windowMs, flushMs, maxTracked, now = Date.now, onCandidate }) {
|
|
20
23
|
const states = new Map(); // errorHash -> state (insertion order = least recently seen first)
|
|
21
24
|
const stats = { observed: 0, windows: 0, emitted: 0, evicted: 0 };
|
|
22
25
|
|
|
23
26
|
const snapshot = (s) => ({
|
|
24
27
|
errorHash: s.errorHash, fingerprint: s.fingerprint, severity: s.severity, service: s.service, environment: s.environment,
|
|
25
|
-
count: s.count, firstSeen: s.firstSeen, lastSeen: s.lastSeen, logCategory: s.logCategory, windowStart: s.windowStart,
|
|
28
|
+
count: s.count, firstSeen: s.firstSeen, lastSeen: s.lastSeen, logCategory: s.logCategory, windowStart: s.windowStart, stackBytes: s.stackBytes,
|
|
26
29
|
});
|
|
27
30
|
const send = (s, t) => { s.sentCount = s.count; s.lastEmitAt = t; stats.emitted++; onCandidate(snapshot(s)); };
|
|
28
31
|
const close = (s, t) => { states.delete(s.errorHash); if (s.count > s.sentCount) send(s, t); };
|
|
@@ -31,7 +34,8 @@ export function createAggregator({ windowMs, flushMs, maxTracked, now = Date.now
|
|
|
31
34
|
stats,
|
|
32
35
|
get tracked() { return states.size; },
|
|
33
36
|
/** @returns {{opened: boolean, windowStart: number}} opened = this occurrence opened a new window. */
|
|
34
|
-
observe({ errorHash, fingerprint, severity, service, environment, logCategory, tsMs }) {
|
|
37
|
+
observe({ errorHash, fingerprint, severity, service, environment, logCategory, tsMs, stackBytes = 0 }) {
|
|
38
|
+
const size = Number.isFinite(stackBytes) && stackBytes > 0 ? Math.min(Math.trunc(stackBytes), MAX_STACK_BYTES) : 0;
|
|
35
39
|
const t = now();
|
|
36
40
|
stats.observed++;
|
|
37
41
|
const seenAt = Number.isFinite(tsMs) && tsMs >= 0 ? Math.min(tsMs, t) : t; // never in the future, never before the epoch (the log line is untrusted)
|
|
@@ -43,7 +47,7 @@ export function createAggregator({ windowMs, flushMs, maxTracked, now = Date.now
|
|
|
43
47
|
stats.evicted++;
|
|
44
48
|
close(oldest, t);
|
|
45
49
|
}
|
|
46
|
-
s = { errorHash, fingerprint, severity, service, environment, logCategory, count: 0, sentCount: 0, firstSeen: seenAt, lastSeen: seenAt, windowStart: t, lastEmitAt: 0 };
|
|
50
|
+
s = { errorHash, fingerprint, severity, service, environment, logCategory, count: 0, sentCount: 0, stackBytes: size, firstSeen: seenAt, lastSeen: seenAt, windowStart: t, lastEmitAt: 0 };
|
|
47
51
|
stats.windows++;
|
|
48
52
|
s.count = 1;
|
|
49
53
|
states.set(errorHash, s);
|
|
@@ -54,6 +58,7 @@ export function createAggregator({ windowMs, flushMs, maxTracked, now = Date.now
|
|
|
54
58
|
states.set(errorHash, s);
|
|
55
59
|
s.count++;
|
|
56
60
|
s.severity = maxSeverity(s.severity, severity);
|
|
61
|
+
if (size > s.stackBytes) s.stackBytes = size; // the biggest stack seen in the window (contract 2.1 (c))
|
|
57
62
|
// firstSeen never changes inside a window (it is part of the window identity), even for an out-of-order line
|
|
58
63
|
if (seenAt > s.lastSeen) s.lastSeen = seenAt;
|
|
59
64
|
if (t - s.lastEmitAt >= flushMs) send(s, t);
|
|
@@ -58,7 +58,8 @@ export function createIncidentDetector({ config, emitter, store, audit = null, n
|
|
|
58
58
|
const environment = config.environment || inferEnvironment(service);
|
|
59
59
|
const logCategory = logCategoryOf(message, stack, extractErrorType(message, stack));
|
|
60
60
|
stats.detected++;
|
|
61
|
-
const
|
|
61
|
+
const stackBytes = typeof entry.stacktrace === "string" ? Buffer.byteLength(entry.stacktrace) : 0; // as the source gave it, before the entry cap
|
|
62
|
+
const { windowStart } = aggregator.observe({ errorHash, fingerprint, severity, service, environment, logCategory, tsMs: entry.tsMs, stackBytes });
|
|
62
63
|
// the detail of the occurrence, tied to its window (the store keeps a few samples per window); resolves when queued
|
|
63
64
|
if (await store.add({ errorHash, fingerprint, service, severity, windowStart, message, stack })) stats.stored++;
|
|
64
65
|
}
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* emitter.js - sends `incident.candidate` frames (F3-11b, contract F3-22: la->cp, metadata only).
|
|
3
3
|
*
|
|
4
|
-
* The payload has
|
|
5
|
-
* lastSeen, logCategory
|
|
6
|
-
*
|
|
7
|
-
*
|
|
4
|
+
* The payload has the 9 contract fields: errorHash, fingerprint, severity, service, environment, count, firstSeen,
|
|
5
|
+
* lastSeen, logCategory and, ONLY when the CP announced `incident.candidate.size-model` in `ready.capabilities` (contract 2.1 (c)), two more
|
|
6
|
+
* that carry no text: `stackBytes` (UTF-8 size of the biggest stack of the window) and `model` (detector version, e.g. stack-detector-1.8.0). There is no field for text, a stack, a path or a message, and the session validates every frame against
|
|
7
|
+
* the closed schema before it is written, so even a bug here cannot add one. (window = firstSeen..lastSeen.)
|
|
8
8
|
*
|
|
9
9
|
* DLP service and environment are labels of the customer's own logs; they go through the DLP and a label it would change
|
|
10
10
|
* (or deny) is not sent at all: the candidate is dropped and counted (`dlp`).
|
|
@@ -27,7 +27,11 @@ const RETRY_MS = 5_000;
|
|
|
27
27
|
const MAX_ATTEMPTS = 5;
|
|
28
28
|
const NON_RETRYABLE_REFUSALS = new Set(["schema_violation", "frame_too_large", "payload_too_large", "unknown_type", "wrong_direction"]);
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
const SIZE_MODEL_CAPABILITY = "incident.candidate.size-model";
|
|
31
|
+
const MODEL = /^[A-Za-z0-9][A-Za-z0-9._:+-]{0,47}$/;
|
|
32
|
+
const MAX_STACK_BYTES = 16 * 1024 * 1024;
|
|
33
|
+
|
|
34
|
+
export function createCandidateEmitter({ session, config, workspaceId = "", model = null, env = process.env, audit = null, log = console, now = Date.now, timers = { setTimeout, clearTimeout } }) {
|
|
31
35
|
const warn = (m) => (log.warn || log.log).call(log, m);
|
|
32
36
|
const pending = new Map(); // `${errorHash}|${windowStart}` -> snapshot
|
|
33
37
|
const stats = { submitted: 0, sent: 0, coalesced: 0, droppedFull: 0, droppedDlp: 0, droppedInvalid: 0, droppedGaveUp: 0, droppedStopped: 0, rejected: 0, byCode: {} };
|
|
@@ -59,7 +63,13 @@ export function createCandidateEmitter({ session, config, workspaceId = "", env
|
|
|
59
63
|
const r = protectEgress(label, { env, source: "incident-candidate" });
|
|
60
64
|
if (r.denied || r.text !== label) return null;
|
|
61
65
|
}
|
|
62
|
-
|
|
66
|
+
const payload = { errorHash: s.errorHash, fingerprint: s.fingerprint, severity: s.severity, service, environment, count: s.count, firstSeen: iso(s.firstSeen), lastSeen: iso(s.lastSeen), logCategory: s.logCategory };
|
|
67
|
+
// contract 2.1 (c): only when the CP said (ready.capabilities) it takes them: a 2.0 CP answers the closed schema with schema_violation
|
|
68
|
+
if (typeof session.peerHas === "function" && session.peerHas(SIZE_MODEL_CAPABILITY)) {
|
|
69
|
+
if (Number.isFinite(s.stackBytes)) payload.stackBytes = Math.max(0, Math.min(Math.trunc(s.stackBytes), MAX_STACK_BYTES));
|
|
70
|
+
if (typeof model === "string" && MODEL.test(model)) payload.model = model;
|
|
71
|
+
}
|
|
72
|
+
return payload;
|
|
63
73
|
}
|
|
64
74
|
|
|
65
75
|
function schedule(ms) {
|
|
@@ -23,8 +23,8 @@ export function buildSources(config, { stdin = process.stdin } = {}) {
|
|
|
23
23
|
return sources;
|
|
24
24
|
}
|
|
25
25
|
|
|
26
|
-
export function installIncidentDetector({ link, workspaceId = "", config, env = process.env, log = console, sources = null }) {
|
|
27
|
-
const list = sources || buildSources(config);
|
|
26
|
+
export function installIncidentDetector({ link, workspaceId = "", config, env = process.env, log = console, sources = null, extraSources = [], model = null }) {
|
|
27
|
+
const list = sources || [...buildSources(config), ...extraSources]; // extraSources: push sources owned by someone else (the webhook receiver, F3-12 consumer)
|
|
28
28
|
if (list.length === 0) {
|
|
29
29
|
log.warn("[la-incidents] LOCAL_AGENT_INCIDENT_DETECTOR=true but no source is configured (LOCAL_AGENT_INCIDENT_LOG_FILES, LOCAL_AGENT_INCIDENT_STDIN or LOCAL_AGENT_INCIDENT_LOKI_URL): the detector does not start");
|
|
30
30
|
return null;
|
|
@@ -33,7 +33,7 @@ export function installIncidentDetector({ link, workspaceId = "", config, env =
|
|
|
33
33
|
const audit = createAudit({ file: config.auditFile });
|
|
34
34
|
const store = createDetailStore({ dir: config.storeDir, maxBytes: config.storeMaxBytes, maxQueue: config.storeMaxQueue, messageChars: config.storeMessageChars,
|
|
35
35
|
stackChars: config.storeStackChars, samplesPerWindow: config.samplesPerWindow, env });
|
|
36
|
-
const emitter = createCandidateEmitter({ session: link.session, config, workspaceId: ws, env, audit, log });
|
|
36
|
+
const emitter = createCandidateEmitter({ session: link.session, config, workspaceId: ws, model, env, audit, log });
|
|
37
37
|
const detector = createIncidentDetector({ config, emitter, store, audit, sources: list, log });
|
|
38
38
|
const closeLink = link.close.bind(link);
|
|
39
39
|
link.close = () => { void detector.stop().finally(() => { emitter.close(); }); closeLink(); };
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* webhook-source.js - the incident detector as the LOCAL consumer of the webhooks the control plane strips and forwards (F3-12 consumer).
|
|
3
|
+
*
|
|
4
|
+
* `webhook.forward` hands the (rebuilt, sha256-verified) body here. It is turned into the same entries every other source produces
|
|
5
|
+
* ({severity, message, stacktrace, service, tsMs}) and goes through the SAME pipeline (Java rules, DLP-masked local store, candidates with
|
|
6
|
+
* metadata only): the text of the webhook never leaves this process except into the local store. The ack of the webhook waits for `deliver()`.
|
|
7
|
+
*
|
|
8
|
+
* text/plain lines, joined into records like a log file (EntryAssembler)
|
|
9
|
+
* alertmanager `alerts[]` with status firing: "alertname: summary", severity label critical -> CRITICAL, error/high/major -> ERROR,
|
|
10
|
+
* anything else WARN (the detector ignores it)
|
|
11
|
+
* sentry `data.event` / `event` (+ `data.issue`): "Type: value" and a stack from the exception frames, `level` as severity,
|
|
12
|
+
* project as service
|
|
13
|
+
* other JSON one log-shaped object (message/msg/level/stack... as in parse.js); a body that does not look like one is `unmapped`
|
|
14
|
+
* Only SYNTHETIC payloads in the tests: the real shapes of each tool should be confirmed with captured webhooks before relying on them.
|
|
15
|
+
* Bounded: at most `maxEntries` entries and `maxBytes` of text per webhook. Counters only; never a word of the body in a log.
|
|
16
|
+
*/
|
|
17
|
+
import { EntryAssembler, parseRecord, parseTime } from "../parse.js";
|
|
18
|
+
|
|
19
|
+
const str = (v) => (typeof v === "string" && v !== "" ? v : undefined);
|
|
20
|
+
|
|
21
|
+
function fromAlertmanager(o, maxEntries) {
|
|
22
|
+
const out = [];
|
|
23
|
+
for (const a of Array.isArray(o.alerts) ? o.alerts : []) {
|
|
24
|
+
if (out.length >= maxEntries) break;
|
|
25
|
+
if (!a || typeof a !== "object" || (a.status && a.status !== "firing")) continue;
|
|
26
|
+
const labels = a.labels && typeof a.labels === "object" ? a.labels : {};
|
|
27
|
+
const ann = a.annotations && typeof a.annotations === "object" ? a.annotations : {};
|
|
28
|
+
const name = str(labels.alertname) || "alert";
|
|
29
|
+
const text = str(ann.summary) || str(ann.description) || "";
|
|
30
|
+
const sev = String(labels.severity ?? "").toLowerCase();
|
|
31
|
+
out.push({
|
|
32
|
+
severity: sev === "critical" ? "CRITICAL" : ["error", "high", "major", "page"].includes(sev) ? "ERROR" : "WARN",
|
|
33
|
+
message: text ? `${name}: ${text}` : name,
|
|
34
|
+
stacktrace: null,
|
|
35
|
+
service: str(labels.service) || str(labels.job) || str(labels.app) || str(labels.namespace) || null,
|
|
36
|
+
tsMs: parseTime(a.startsAt),
|
|
37
|
+
});
|
|
38
|
+
}
|
|
39
|
+
return out;
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
function sentryStack(exc) {
|
|
43
|
+
const values = exc && Array.isArray(exc.values) ? exc.values : [];
|
|
44
|
+
const v = values[values.length - 1];
|
|
45
|
+
const frames = v && v.stacktrace && Array.isArray(v.stacktrace.frames) ? v.stacktrace.frames : [];
|
|
46
|
+
if (frames.length === 0) return null;
|
|
47
|
+
return frames.slice().reverse().slice(0, 100).map((f) => ` at ${str(f && f.function) || "?"} (${str(f && (f.filename || f.module)) || "?"}:${Number.isFinite(f && f.lineno) ? f.lineno : 0})`).join("\n");
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function fromSentry(o) {
|
|
51
|
+
const data = o.data && typeof o.data === "object" ? o.data : o;
|
|
52
|
+
const ev = data.event && typeof data.event === "object" ? data.event : data.issue && typeof data.issue === "object" ? data.issue : null;
|
|
53
|
+
if (!ev) return [];
|
|
54
|
+
const exc = ev.exception && typeof ev.exception === "object" ? ev.exception : null;
|
|
55
|
+
const last = exc && Array.isArray(exc.values) ? exc.values[exc.values.length - 1] : null;
|
|
56
|
+
const head = last && str(last.type) ? `${last.type}${str(last.value) ? `: ${last.value}` : ""}` : str(ev.title) || str(ev.message) || str(ev.culprit);
|
|
57
|
+
if (!head) return [];
|
|
58
|
+
const proj = ev.project && typeof ev.project === "object" ? ev.project : null;
|
|
59
|
+
const ts = typeof ev.timestamp === "number" ? ev.timestamp : ev.datetime || ev.firstSeen;
|
|
60
|
+
return [{
|
|
61
|
+
severity: str(ev.level) ? String(ev.level).toUpperCase() : "ERROR",
|
|
62
|
+
message: head.split("\n")[0],
|
|
63
|
+
stacktrace: sentryStack(exc),
|
|
64
|
+
service: str(o.project_name) || str(proj && (proj.slug || proj.name)) || str(ev.project) || null,
|
|
65
|
+
tsMs: parseTime(ts),
|
|
66
|
+
}];
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
/** @returns {{entries:object[], unmapped:boolean}} */
|
|
70
|
+
export function entriesFromWebhook({ source, contentType, body }, { maxEntries = 200, maxBytes = 1048576, maxEntryBytes, maxStackLines } = {}) {
|
|
71
|
+
const text = new TextDecoder("utf-8").decode(body.length > maxBytes ? body.subarray(0, maxBytes) : body);
|
|
72
|
+
if (contentType === "text/plain") {
|
|
73
|
+
const asm = new EntryAssembler({ maxEntryBytes, maxStackLines });
|
|
74
|
+
const out = [];
|
|
75
|
+
for (const line of text.split("\n")) { out.push(...asm.push(line.replace(/\r$/, ""))); if (out.length >= maxEntries) break; }
|
|
76
|
+
out.push(...asm.flush());
|
|
77
|
+
const entries = out.slice(0, maxEntries);
|
|
78
|
+
return { entries, unmapped: entries.length === 0 };
|
|
79
|
+
}
|
|
80
|
+
let o;
|
|
81
|
+
try { o = JSON.parse(text); } catch { return { entries: [], unmapped: true }; }
|
|
82
|
+
if (!o || typeof o !== "object" || Array.isArray(o)) return { entries: [], unmapped: true };
|
|
83
|
+
let entries = [];
|
|
84
|
+
if (source === "alertmanager" || Array.isArray(o.alerts)) entries = fromAlertmanager(o, maxEntries);
|
|
85
|
+
else if (source === "sentry") entries = fromSentry(o);
|
|
86
|
+
else { const r = parseRecord(text); entries = r ? [r] : []; }
|
|
87
|
+
return { entries, unmapped: entries.length === 0 };
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
export function createWebhookSource({ maxEntries = 200, maxBytes = 1048576, maxEntryBytes, maxStackLines, stats = {} } = {}) {
|
|
91
|
+
let consumer = null;
|
|
92
|
+
Object.assign(stats, { received: 0, entries: 0, unmapped: 0, ...stats });
|
|
93
|
+
return {
|
|
94
|
+
name: "webhook",
|
|
95
|
+
stats,
|
|
96
|
+
start(onEntries) { consumer = onEntries; },
|
|
97
|
+
stop() { consumer = null; },
|
|
98
|
+
/** The sink of the webhook receiver. Resolves when every entry went through the detector (its detail queued): the ack waits for it. */
|
|
99
|
+
async deliver(webhook) {
|
|
100
|
+
if (!consumer) { const e = new Error("webhook source is not started"); e.code = "internal"; throw e; }
|
|
101
|
+
stats.received++;
|
|
102
|
+
const { entries, unmapped } = entriesFromWebhook(webhook, { maxEntries, maxBytes, maxEntryBytes, maxStackLines });
|
|
103
|
+
if (unmapped) { stats.unmapped++; return { entries: 0 }; }
|
|
104
|
+
stats.entries += entries.length;
|
|
105
|
+
await consumer(entries);
|
|
106
|
+
return { entries: entries.length };
|
|
107
|
+
},
|
|
108
|
+
};
|
|
109
|
+
}
|
|
@@ -2,15 +2,15 @@
|
|
|
2
2
|
"repo": "VISIONHAMMER/vh-api-gateway",
|
|
3
3
|
"path": "docs/contracts/la-protocol-v2",
|
|
4
4
|
"branch": "release/qa",
|
|
5
|
-
"commit": "
|
|
5
|
+
"commit": "c0d6b44a89d2d18a2627493403c06b9caa7d1a96",
|
|
6
6
|
"note": "Copied by scripts/sync-la-protocol-v2-contract.mjs. Do not edit: change the gateway folder and sync again.",
|
|
7
7
|
"files": {
|
|
8
|
-
"README.md": "
|
|
8
|
+
"README.md": "e247945f859f777a705c5baebecf2a216462a7c2fe1c30598099c96c88cd432b",
|
|
9
9
|
"ack.schema.json": "d6dfdaa8b1ec2dd47db9e858cd047ca860196ab6b0ad350f8ccb5d2596441ccf",
|
|
10
10
|
"alert.schema.json": "01a8a77840be8ab7901ed7d07c3d68a2295bfaeb638694564e19e60be1747054",
|
|
11
11
|
"approval.decision.schema.json": "a9c7bc74d146465afde1d5830839165a02f232b830b5f6cfd8c8e91fb6da1563",
|
|
12
12
|
"audit.event.schema.json": "e9ceb58ea00077151373955957b8319324de488c80da4063f6d0dcb40a8b3ed0",
|
|
13
|
-
"config.update.schema.json": "
|
|
13
|
+
"config.update.schema.json": "432438b0c93ddafa0741e47f99b7357ad5f2b06b606779bf846c282e2f8a5639",
|
|
14
14
|
"content.index.schema.json": "1abc24344435f0f392d5b626c3abb6879d8f9dd2c1d0463840a67634a5caa71e",
|
|
15
15
|
"content.request.schema.json": "440e3675246657e9aae23e51176168bc21c91283b5fcaf6782111a18110bf76e",
|
|
16
16
|
"content.response.schema.json": "c9a66ac9cbbcaec7fc182b3e3fa23b34877bd342592521c7e393779230332231",
|
|
@@ -19,24 +19,24 @@
|
|
|
19
19
|
"error.schema.json": "b1fd663d4a1ea73d88cb575422f15f8c7279001f9385ebb2d5e1889b5d5ef878",
|
|
20
20
|
"file.list.body.schema.json": "dd58ac2af91bbde53e0fd585dc55af0adb01380492d1604f21ed8e972d7ba8d1",
|
|
21
21
|
"heartbeat.schema.json": "8aff84a1ef3ca56bdff8819abb3b0ab01a2547814c2ea2f9ee893cb08f7a08f1",
|
|
22
|
-
"hello.schema.json": "
|
|
23
|
-
"incident.candidate.schema.json": "
|
|
24
|
-
"index.json": "
|
|
22
|
+
"hello.schema.json": "a90a6bf0e9a8a20e4c6edcf2137931cf044846e7924977228275ef648a48934f",
|
|
23
|
+
"incident.candidate.schema.json": "ffec36aca6916079debc2c6060d093819ea3ea660b57c0c92ac3b541a0e99000",
|
|
24
|
+
"index.json": "de79c065217e1fa8430c4266ec19fd57df35fc4182eb7c7246b6f1ea03d2630c",
|
|
25
25
|
"job.assign.schema.json": "fdebb4ae98bf38a2c723281c1cf9477f71d877171df9a55b0baeb0b65244f60c",
|
|
26
26
|
"job.cancel.schema.json": "40ef64373283e17306d857c9a40081689786015294b0b88ab8f68f063bd2cfb8",
|
|
27
27
|
"job.event.schema.json": "493d08f9db4f7ffd4a97403ee33ca4a73a65dc8f774d8f18ef0a137fdb5fc470",
|
|
28
28
|
"job.result.schema.json": "9cbc61211482557283b8f4df44e79011ead1abbacdb2d986138199c747edc043",
|
|
29
29
|
"log.stats.schema.json": "4b33b111946a19793a60756793294e90911b06f7050e48821759f2497ec17641",
|
|
30
|
-
"ready.schema.json": "
|
|
31
|
-
"register.schema.json": "
|
|
30
|
+
"ready.schema.json": "02e51ba5aad3357c9840c14eb45cb86b7aab0c10e349c94c720cda1a0b5b8366",
|
|
31
|
+
"register.schema.json": "bec30062cf99657a1ecb94bad004f502615901e36bc2b0ba62351ed021e059ac",
|
|
32
32
|
"scan.upload.schema.json": "b1bba86ee362f73afc94c8c216977c12f67f770490e81002d14789397692f23b",
|
|
33
33
|
"scan.uploaded.schema.json": "f4c6aed7b9363b3ce1a9e14f9c4f4fb003bcf104b60acf74421dd0c75fc5d229",
|
|
34
|
-
"webhook.forward.schema.json": "
|
|
35
|
-
"vectors/ack.vectors.json": "
|
|
34
|
+
"webhook.forward.schema.json": "9ab66adf31386445c2b76930155ffa6874326395d968582d2345ee7fb5b77f82",
|
|
35
|
+
"vectors/ack.vectors.json": "1756c1169cf5f42c0ab6b1c094071d1f8ccf2fe9efdbc97101e402e93bd65a67",
|
|
36
36
|
"vectors/alert.vectors.json": "0bf179ff390bbecf07a117b80237cb7ac8778864e5780f36d68c8c9a64db8f04",
|
|
37
37
|
"vectors/approval.decision.vectors.json": "338115973bf2c4390d2bb33ce8f4079f3132cd739cd5505e1e873fdc252d2271",
|
|
38
38
|
"vectors/audit.event.vectors.json": "d05c1820103eef041a84eb8b13d9a67a8a5b54b9b54ee3b6e43fd4ef15ed7000",
|
|
39
|
-
"vectors/config.update.vectors.json": "
|
|
39
|
+
"vectors/config.update.vectors.json": "e61865fad9b4efa808e02877c4352d3720c8069a0b31261701b369d8ae11f9cf",
|
|
40
40
|
"vectors/content.index.vectors.json": "f167db30910989c46f317313d651ca59f9cd9d881d2ea81b84912155d18227b7",
|
|
41
41
|
"vectors/content.request.vectors.json": "fdf0f4d9e19f6480e20e40f4652bf8205da2b113620f8461dfaeb0d61add36a0",
|
|
42
42
|
"vectors/content.response.vectors.json": "cbb47cbf8a9512aa5360eea425f0cdd9064334311d37834dc75bddff07dadc5d",
|
|
@@ -44,15 +44,15 @@
|
|
|
44
44
|
"vectors/error.vectors.json": "f5fd7954f2c1736ee0943eb330c2f85b7fdd7a8959e4f043f9cc9e9bcda39aa5",
|
|
45
45
|
"vectors/file.list.body.vectors.json": "da1dcc5d4ec976853161bef4b3947572f36b5526cbe16b838ac4ef72ddded403",
|
|
46
46
|
"vectors/heartbeat.vectors.json": "cfd2e048139c67aa925f1fa94c05326e75248839645c1a1e4d6d3bb60cafa624",
|
|
47
|
-
"vectors/hello.vectors.json": "
|
|
48
|
-
"vectors/incident.candidate.vectors.json": "
|
|
47
|
+
"vectors/hello.vectors.json": "7a50c4e54e6a4ef1227fd7485071bca8ec7b797c3b2907fddda3d8836c3feab6",
|
|
48
|
+
"vectors/incident.candidate.vectors.json": "a78eb4b103f411f1ec64d9eeb4c39623851e710ac0cc20b4d78b2739b7ff78d5",
|
|
49
49
|
"vectors/job.assign.vectors.json": "e510c2cc6d17238d477429c255d828c0339b504e39a3a50d4b6ca980e262d9ef",
|
|
50
50
|
"vectors/job.cancel.vectors.json": "0fc9f99c8662435e9425389068adf418273489dc73928a21eae1b283256b9bc5",
|
|
51
51
|
"vectors/job.event.vectors.json": "03a2a2435918ba561821916bf4e435f2780210900139ecf68091acac6abf200c",
|
|
52
52
|
"vectors/job.result.vectors.json": "05995fbe65e5357705d75e9b660b517f54c6902069d36b62190b1b79f283e7a5",
|
|
53
53
|
"vectors/log.stats.vectors.json": "a8a67e09ef0f87b81f4028b77427ed7afd809c3f8534b8423a712b2ceef9a27f",
|
|
54
|
-
"vectors/ready.vectors.json": "
|
|
55
|
-
"vectors/register.vectors.json": "
|
|
54
|
+
"vectors/ready.vectors.json": "16327f5e5eccc57a0ec635bcde96116feeeeec7ddbc4b621d4d070738cdc48d5",
|
|
55
|
+
"vectors/register.vectors.json": "b612e9462b1e2ac198442afccf450f1535daadc4c32d87f6a99b9752c343c486",
|
|
56
56
|
"vectors/scan.upload.vectors.json": "e8a6f73fe91ac58a3222b68cf28124c1254aa4f00c3ef8f472deb7de31cab76e",
|
|
57
57
|
"vectors/scan.uploaded.vectors.json": "7314ea15b1afa7b9625048fb7e245492d8910bc0dae8a52ea62afe2a1e240382",
|
|
58
58
|
"vectors/traces/invalid-agent-mismatch.json": "5f17b867de71c1bcc6c42723acb53a50338c0e491193499ea25c2220ea4fa558",
|
|
@@ -76,11 +76,19 @@
|
|
|
76
76
|
"vectors/traces/invalid-signature-warn-with-enforce-required.json": "71b5b3dc5dce52417173934db1e0de6542ecae5798e87124d54a8017493312e1",
|
|
77
77
|
"vectors/traces/invalid-tenant-mismatch.json": "dcc04ea423bddef20524ead7bfd146a5d9c42fe474c9216dc38c90544d291151",
|
|
78
78
|
"vectors/traces/invalid-type-outside-allowlist.json": "e492d326cfa044d3fc9f7e48dee63a52b715a62ba4867fc5ce21f1d86e3c435f",
|
|
79
|
+
"vectors/traces/invalid-webhook-chunk-after-last.json": "9a35faaeeafce045a887e643f603ca69950872d62d27d17086513103999aefd6",
|
|
80
|
+
"vectors/traces/invalid-webhook-chunk-body-hash-changes.json": "7ade576a8dc8e9671383e85fbe6aae414863cbb1cde38737a2cb78efce9c9528",
|
|
81
|
+
"vectors/traces/invalid-webhook-chunk-last-flag-wrong.json": "8505b5fbbc0a8dc97021a1c02f9883df95f9401bc18eaa2498331b0b6b7d155c",
|
|
82
|
+
"vectors/traces/invalid-webhook-chunk-non-last-too-short.json": "ff1d64b019c082e7a5c6537fdbf069025a532c764e62c532a24a3b7257f4b6ab",
|
|
83
|
+
"vectors/traces/invalid-webhook-chunk-out-of-order.json": "b6baa056b4e3347deaf0a4b901a126a3f93d20627faaa3951028bd47894715f4",
|
|
84
|
+
"vectors/traces/invalid-webhook-chunk-sha-mismatch.json": "1bb79d0e1794cd791625ed2a7eab154e20150c5d7bee7c86aa97f9bf4aaeab6d",
|
|
85
|
+
"vectors/traces/invalid-webhook-chunk-total-changes.json": "5edf3c1916927f94ef15061a13a6ded8a0168cb0492158f34db8160c0d6bdbe4",
|
|
79
86
|
"vectors/traces/valid-content-chunks-in-order.json": "c5fc301527b062b30c625da2798c7cfab3e3debac29a36f6ec7c699705267ab3",
|
|
80
87
|
"vectors/traces/valid-content-index.json": "9305dc955c4cca1f09158a9c8cf21d9e89d6a6b03646088c547f64a61cd59c30",
|
|
81
88
|
"vectors/traces/valid-handshake-and-job.json": "9aa3c423c496403ca77d63965f1805f0a57ea9f6f8a01351310515b265aeb9cc",
|
|
82
89
|
"vectors/traces/valid-resume-continues-seq.json": "8aa90f24598f8f2d87a5355ec58c3531d16fd11ad41c782f53eca063372b704a",
|
|
83
90
|
"vectors/traces/valid-retransmit-is-idempotent.json": "924e9d296a67f0006c289e7cc978b9529ae1aa03bec2cd331a62f31354a90257",
|
|
84
|
-
"vectors/webhook.
|
|
91
|
+
"vectors/traces/valid-webhook-chunks-in-order.json": "2e5e6313a7fe9b3f7fea23d6a0c928d6bfef69b2a7179f1578230425f451ada9",
|
|
92
|
+
"vectors/webhook.forward.vectors.json": "991a677d72a9c698ed3c8d960af532daf8f3158d77a4bea7d08750ccd8693250"
|
|
85
93
|
}
|
|
86
94
|
}
|
|
@@ -5,13 +5,16 @@ sides. Design: vault `10 Library/Plans/Fase 3 Desenho Tecnico 2026-10-28.md` sec
|
|
|
5
5
|
implements it behind `deepdebug.agent-ws.protocol-v2.enabled` (default off, F3-05a); the LA side (F3-05b) does not
|
|
6
6
|
exist yet. v1 is untouched. The gateway packages `*.schema.json` + `index.json` from this folder as classpath
|
|
7
7
|
`la-protocol-v2/` (pom resource; both Dockerfiles copy this folder), so this folder stays the only source.
|
|
8
|
+
**Revision 2.1** (`index.json` `revision`, Fase 3, last section): optional LLM key delivery, chunked webhooks, size/model in `incident.candidate`, `ready.capabilities`.
|
|
8
9
|
|
|
9
10
|
## Files
|
|
10
11
|
- `envelope.schema.json`: common frame, closed (`additionalProperties:false`). `<type>.schema.json` (22):
|
|
11
12
|
one per allowlisted type, each `$ref`s the envelope and closes `payload`. `defs.schema.json`: shared types.
|
|
12
13
|
- `file.list.body.schema.json` (F3-07c): not a frame type; the schema of the DECODED body of a `content.response` for kind `file.list`
|
|
13
14
|
(below). `vectors/file.list.body.vectors.json` holds its vectors (same format, `schema` names that file).
|
|
14
|
-
- `index.json`: allowlist, direction, signed/control flags, content class per type, and the
|
|
15
|
+
- `index.json`: allowlist, direction, signed/control flags, content class per type (`none`, `relay`, `transit`, `secret`), the limits below and the contract `revision`.
|
|
16
|
+
- Vector files of a BODY that travels inside a frame (not a frame) are named `*.body.vectors.json` and their schema `*.body.schema.json`; the frame
|
|
17
|
+
suites skip them (`ProtocolV2ContractTest` checks them against their own schema, `SchemaLaV2ContractTest` does not read them as frames).
|
|
15
18
|
- `vectors/<type>.vectors.json` (+ `cross-type`): valid and invalid messages with `expect` (`valid`, error `code`,
|
|
16
19
|
optional JSON Schema `keyword`, optional session `rule`). `vectors/traces/*.json`: whole sessions for the stateful rules.
|
|
17
20
|
|
|
@@ -30,7 +33,8 @@ ack timeout 5 s, <= 3 retransmissions; handshake steps 10 s; `ping`/`heartbeat`
|
|
|
30
33
|
(= Redis `la:session` TTL); clock skew <= 60 s. Duplicate detection (gateway, F3-05a): the last 512 `seq`->`msgId` pairs
|
|
31
34
|
per session, in memory (`dedupe-entries`); the 10 min Redis window of the design (`dedupeWindowSeconds` in `index.json`)
|
|
32
35
|
is NOT implemented, a replay older than those 512 frames is `seq_replayed`. CP->LA frames repeat the payload in
|
|
33
|
-
`paramsRaw`, so their effective payload budget is about 20 KiB; larger inputs travel by `ref`.
|
|
36
|
+
`paramsRaw`, so their effective payload budget is about 20 KiB; larger inputs travel by `ref`. Webhook chunking (2.1) has its own limits
|
|
37
|
+
(`webhook*` in `index.json`: 16 KiB raw per chunk, 64 chunks, 1 MiB per webhook, see the last section).
|
|
34
38
|
|
|
35
39
|
## State machine
|
|
36
40
|
`AWAIT_REGISTER -> AWAIT_HELLO -> AWAIT_READY -> READY`, after the WS upgrade authenticates (Bearer, as v1).
|
|
@@ -45,7 +49,9 @@ is NOT implemented, a replay older than those 512 frames is `seq_replayed`. CP->
|
|
|
45
49
|
handshake state, `seq` (same seq + same `msgId` = idempotent duplicate, re-ack; same seq + other `msgId` =
|
|
46
50
|
`seq_replayed`; skip = `seq_gap`), chunks (index 0.., `total` constant, `last` iff last index, none after last; then `payload.byteLength` must equal the size of
|
|
47
51
|
the base64-DECODED `payload.data`, else `chunk_inconsistent`: JSON Schema cannot state it, so the vector files carry
|
|
48
|
-
`expect.rule:"byteLength-decoded"` and both validators apply it).
|
|
52
|
+
`expect.rule:"byteLength-decoded"` and both validators apply it). Since 2.1 two more stateless rules of the same kind:
|
|
53
|
+
`webhook-chunk-size` and `llmKey-id-sha256` (last section); `webhook.forward` chunks are also grouped by `payload.webhookId` and the reassembled
|
|
54
|
+
body must hash to `bodySha256` (`chunk_inconsistent` otherwise).
|
|
49
55
|
- **Errors** (`error{code,retryable}`, then WS close 4400-4499 when fatal): `bad_frame unknown_type schema_violation
|
|
50
56
|
frame_too_large payload_too_large wrong_direction handshake_violation not_ready seq_gap seq_replayed
|
|
51
57
|
chunk_out_of_order chunk_inconsistent tenant_mismatch agent_mismatch params_mismatch signature_mode_insufficient
|
|
@@ -64,6 +70,7 @@ Everything is closed: no free-text fields, no extra properties, free strings <=
|
|
|
64
70
|
hashes, counts). Client content may exist only in three declared fields (`x-content-class`, pinned by the test):
|
|
65
71
|
`content.response.data` (relay, LA->CP, never persisted or logged by the CP), `job.assign.inputs[].inline` and
|
|
66
72
|
`webhook.forward.bodyB64` (transit, CP->LA, `transient:true`, not persisted; `paramsRaw` echoes them).
|
|
73
|
+
Since 2.1 there is one more classified field and it is the only SECRET: `config.update.llmKey.key` (`x-content-class: secret`, see the last section).
|
|
67
74
|
Long non-content strings are marked `x-large-metadata` (`paramsRaw`, `signedPutUrl`, `vhapr1` token). `error.detail` <= 200 chars
|
|
68
75
|
and `audit.event` have no summary text; stack traces, diffs, logs and paths go only through `content.*`.
|
|
69
76
|
|
|
@@ -186,3 +193,96 @@ existing kinds, `ref` convention, ids, envelope and chunking are reused; the cha
|
|
|
186
193
|
answer is HTTP 501 `not_supported`. That announcement is also how the front/CP discovers that listing exists on a given agent.
|
|
187
194
|
- **Compatibility.** Additive: the kind is added to the `content.request` and `content.index` enums, nothing else changes there. The
|
|
188
195
|
`content.request` payload gained no field. With `deepdebug.content-relay.enabled` off nothing is mapped, as before.
|
|
196
|
+
|
|
197
|
+
## Revision 2.1 (Fase 3, contract v2.1): LLM key, chunked webhooks, size and model of a candidate
|
|
198
|
+
Decisions of Edu 2026-11-03 (ADR-0017, adenda (c)). **No new frame type** (still 22), every change is additive and optional, and each is gated by a
|
|
199
|
+
capability so a 2.0 peer keeps working unchanged. The gateway code that uses these fields is a later task (G); here only the contract moved.
|
|
200
|
+
|
|
201
|
+
**Capabilities (negotiation).** The LA announces, in `register` and `hello` `capabilities`: `contract.v2.1` (it has the 2.1 schemas, so it can read the
|
|
202
|
+
new optional `ready.payload.capabilities`), `llm.key` (it handles `config.update.llmKey`) and `webhook.chunked` (it reassembles chunked `webhook.forward`;
|
|
203
|
+
it announces `webhook.forward` too). The CP announces, in `ready.payload.capabilities` (array, <= 32, same name pattern): `incident.candidate.size-model` (it
|
|
204
|
+
accepts `stackBytes` and `model`). Rules: the CP sends `llmKey` only to a session that announced `llm.key`, a chunked webhook only to one that announced
|
|
205
|
+
`webhook.chunked`, and `ready.capabilities` only to one that announced `contract.v2.1` (a 2.0 LA's closed `ready` schema does not know the field, so without
|
|
206
|
+
`contract.v2.1` the `ready` is exactly the 2.0 one); an LA sends `stackBytes`/`model` only if `ready` listed `incident.candidate.size-model` (a 2.0 CP would
|
|
207
|
+
answer the closed schema with `schema_violation`). A name that is absent means 2.0 behaviour.
|
|
208
|
+
One more LA capability, `llm.key.held` (section (e)), tells the CP the LA holds a valid key right now; it is optional and only informs the CP.
|
|
209
|
+
|
|
210
|
+
### (a) LLM key delivery: `config.update.payload.llmKey`
|
|
211
|
+
`config.update` already carried only flags, a DLP hash, a minimum version, a retention and the data-plane mode: it had no way to hand the LA the virtual
|
|
212
|
+
LiteLLM key (F3-09). `llmKey` is a closed `oneOf`:
|
|
213
|
+
- `{op:"set", keyId, endpoint, key, models[], expiresAt}`, all required. `keyId` is the 64-hex lower-case sha256 of the UTF-8 bytes of `key` (it is
|
|
214
|
+
LiteLLM's token id and the CP's `la_llm_keys.key_id`; semantic rule `llmKey-id-sha256`: the LA recomputes it and refuses a mismatch). `endpoint` is the AI
|
|
215
|
+
gateway base URL, `https://` only, host[:port][/path], no credentials, no query, no fragment, <= 256 chars. `key` is 16..256 chars of
|
|
216
|
+
`[A-Za-z0-9._~+/=-]` (no whitespace, no control character). `models` is 1..32 unique aliases (the scope of the key). `expiresAt` is UTC.
|
|
217
|
+
- `{op:"revoke", keyId}`: names the key by id, never repeats it.
|
|
218
|
+
- A `config.update` that carries `llmKey` carries NOTHING else (`maxProperties: 1`), so the frame can be recognised and redacted as a whole.
|
|
219
|
+
- **The key is a secret.** `x-content-class: secret` on `llmKey.key`; `index.json` marks `config.update` `contentClass: "secret"`. It travels only on the v2
|
|
220
|
+
channel (TLS plus the Ed25519 signature) and, between gateway instances, only inside the AEAD-sealed relay (F3-04). It appears twice in the frame, in
|
|
221
|
+
`payload` and in the `paramsRaw` copy: BOTH are secret. The CP never writes it to a log, a metric, a trace, the audit trail, a table or an unsealed Redis
|
|
222
|
+
value (it keeps it only in the in-memory unacked frame until acked or the session ends). The LA keeps it **only in memory or in a file mode `0600`
|
|
223
|
+
(directory `0700`) outside every workspace and outside what the content handlers can serve**, never logs it, never puts it in `hello`, `alert`,
|
|
224
|
+
`audit.event`, `job.event`, `content.response` or an error, and never passes it on a command line.
|
|
225
|
+
- **The ack never returns it.** The answer is the plain `ack` (its payload is only `ackSeq`, `status`, `errorCode`; the contract test pins that no LA->CP
|
|
226
|
+
frame has a field named `key`, `llmKey` or `keyId`): `accepted`, or `rejected` + `schema_violation` when the LA refuses the key (expired
|
|
227
|
+
`expiresAt`, `keyId` not the sha256 of `key`).
|
|
228
|
+
- **Rotation.** A `set` with another `keyId` replaces the key the LA holds, atomically (one key per LA); the same `keyId` again is idempotent. The CP should
|
|
229
|
+
revoke the old key at the AI gateway only after the `accepted` ack of the new one.
|
|
230
|
+
- **Revocation.** The key revoked at the AI gateway is the authoritative revocation; the `revoke` frame only makes the LA drop its copy (delete from memory,
|
|
231
|
+
unlink and overwrite the file) and stop using it. A `revoke` of an id the LA does not hold is a no-op and is acked `accepted` (it never tells which key it
|
|
232
|
+
has). The LA also drops the key at `expiresAt`.
|
|
233
|
+
|
|
234
|
+
### (b) Chunking of `webhook.forward` (CP -> LA)
|
|
235
|
+
2.0 allowed one frame with `bodyB64` <= 12288 chars (9216 raw bytes). That stays: **a body of at most 9216 bytes is sent as ONE frame without `chunk`, exactly
|
|
236
|
+
as in 2.0** (a 2.0 LA reads it). A bigger body (up to **1 MiB**) is chunked, reusing the `content.response` pattern but CP -> LA:
|
|
237
|
+
- Each chunk is a signed `webhook.forward` frame with the envelope `chunk{index,total,last}` (index from 0), its own `seq` and `msgId`, and the same
|
|
238
|
+
`webhookId`, `source`, `contentType`, **`bodyBytes`** (size of the WHOLE decoded body, 0..1048576) and **`bodySha256`** (sha256 of the whole decoded body,
|
|
239
|
+
lower-case hex). `bodyBytes` and `bodySha256` are required in a chunked frame and forbidden in an unchunked one (schema `if/then/else` on `chunk`).
|
|
240
|
+
- **Fixed chunk size**: 16384 raw bytes (`bodyB64` <= 21848 chars; twice in the frame with `paramsRaw` it still fits 48 KiB: a vector pins it).
|
|
241
|
+
`total = max(1, ceil(bodyBytes / 16384))` (<= 64); every non-last chunk has exactly 16384 bytes; the last has `bodyBytes - 16384 * (total - 1)`. That is the
|
|
242
|
+
stateless rule `webhook-chunk-size` (`chunk_inconsistent` when a frame breaks it), so the LA can size the buffer from the first chunk. A body above
|
|
243
|
+
9216 bytes is never sent unchunked; a chunked frame with a small body (`total` 1) is valid.
|
|
244
|
+
- **Order and integrity** (reference model, same codes as `content.response`): chunks in index order, `total` constant, `last` iff the last index, none after
|
|
245
|
+
the last (`chunk_out_of_order` / `chunk_inconsistent`); `bodyBytes`, `bodySha256`, `source`, `contentType` must not change between chunks; on the last
|
|
246
|
+
chunk the reassembled body must have `bodyBytes` bytes and hash to `bodySha256`, else `chunk_inconsistent`. Each chunk is also authentic on its own (signature).
|
|
247
|
+
- **Acks.** Every chunk is a normal v2 frame: the LA acks each by `seq` (piggyback or `ack{ackSeq,status}`), `accepted` = stored. The `accepted` ack of the
|
|
248
|
+
LAST chunk means the body was reassembled and its hash verified: that is the ack the CP waits for before answering the webhook (decision: ack AFTER the
|
|
249
|
+
forward). The LA answers `rejected` + `chunk_inconsistent` / `chunk_out_of_order` / `payload_too_large` / `rate_limited` and drops the assembly; the CP then
|
|
250
|
+
stops sending chunks of that `webhookId` and treats the webhook as not delivered.
|
|
251
|
+
- **LA limits** (`index.json`): an assembly older than 30 s (`webhookAssemblySeconds`) without its last chunk is dropped; at most 4 open assemblies per session
|
|
252
|
+
(`webhookMaxOpenAssemblies`; the CP sends one chunked webhook at a time per session) and 4 MiB buffered; beyond that `rejected` + `rate_limited`. The body
|
|
253
|
+
stays in memory, is never written or logged by the CP, and the LA applies its DLP to it as to any client content.
|
|
254
|
+
- `webhookId` is the group key; a retransmitted chunk (same `seq` and `msgId`) is the existing idempotent duplicate; the same `webhookId` redelivered as a
|
|
255
|
+
whole is the application's dedup (a deterministic `webhookId` is the consumer's job).
|
|
256
|
+
|
|
257
|
+
### (c) `incident.candidate`: `stackBytes` and `model`
|
|
258
|
+
Two OPTIONAL fields, no required field added: `stackBytes` (integer 0..16777216: size in bytes of the stack the LA saw; the LA saturates at the cap) and
|
|
259
|
+
`model` (string 1..48, `^[A-Za-z0-9][A-Za-z0-9._:+-]{0,47}$`: detector/parser version, e.g. `stack-detector-1.8.0`; no space, no slash, so no name, path or
|
|
260
|
+
error text fits). The frame still carries no stack, no message and no name. Sent only after `ready` announced `incident.candidate.size-model`.
|
|
261
|
+
|
|
262
|
+
### (d) `ready.payload.capabilities`
|
|
263
|
+
Optional array (<= 32 capability names). It is what lets (c) be sent safely; it carries no limit and no content. A CP that does not send it is a 2.0 CP, or the
|
|
264
|
+
session did not announce `contract.v2.1`.
|
|
265
|
+
|
|
266
|
+
### (e) `llm.key.held`: the LA says whether it still has its key
|
|
267
|
+
Problem it solves: an LA that keeps the key only in memory (the default) loses it on restart, but the CP's table still says ACTIVE, and the CP only
|
|
268
|
+
delivers again when the key is close to expiry (< 14 days), so the LA would stay without a key. Rather than the CP guessing, the LA announces it.
|
|
269
|
+
- **Name and place.** The capability string `llm.key.held` (it already matches the `capability` pattern, so no schema enum changes), announced in
|
|
270
|
+
`register.payload.capabilities` and in `hello.payload.capabilities`, never in `ready.payload.capabilities` (that list is the CP's). It is optional and
|
|
271
|
+
needs no `contract.v2.1` decision of its own: a CP that does not know it ignores an unknown capability name, as it ignores any other.
|
|
272
|
+
- **Rule for the LA.** It announces `llm.key.held` ONLY when it holds a key that is still valid (`expiresAt` in the future, accepted and
|
|
273
|
+
`keyId`-verified through `config.update.llmKey`) and that it is IN USE (it is the key its LLM calls would use). No key, an expired key, a revoked key
|
|
274
|
+
or a key it refused: it does not announce it. The announcement is a snapshot at connect time: `register` and `hello` of one session carry the same
|
|
275
|
+
answer, and the LA never announces it without also announcing `llm.key` (an LA that cannot receive a key cannot hold one). That dependency is in the
|
|
276
|
+
schema (`if capabilities contains llm.key.held then it must contain llm.key`, `register` and `hello`), so `llm.key.held` alone is `schema_violation`.
|
|
277
|
+
- **Rule for the CP.** At connect, if the capability is NOT announced (in `register` and `hello`; the CP should treat the key as held only when both
|
|
278
|
+
say so), the CP delivers a key (mint, then `config.update.llmKey set`) even when an ACTIVE row exists for that agent; the previous key follows the
|
|
279
|
+
rotation rule of (a) (revoked at the AI gateway only after the `accepted` ack of the new one). If it IS announced, the CP keeps the existing rule:
|
|
280
|
+
nothing to deliver until the key is within 14 days of `expiresAt`. A 2.0 LA, or a 2.1 LA that does not know the capability, never announces it, so
|
|
281
|
+
the CP treats it as "holds no key" and delivers one if the channel is on (`llm.key` announced): the safe direction.
|
|
282
|
+
- **Compatible.** Optional and additive: a register/hello without it is valid (2.0, or 2.1 after a restart), and one with it is valid under 2.1 only
|
|
283
|
+
together with `llm.key`. No frame type, no new field and no new limit. The CP side of this rule is gateway runtime code (a later task, not this contract).
|
|
284
|
+
|
|
285
|
+
### Compatibility and roll-out order
|
|
286
|
+
Everything is additive and optional: a 2.0 frame is valid under the 2.1 schemas (the vectors of 2.0 are unchanged and still pass), and a 2.1 peer talking to a
|
|
287
|
+
2.0 peer sends nothing the 2.0 peer cannot read, because each new field is gated by a capability the other side announced (above). Roll out the gateway that
|
|
288
|
+
has the 2.1 schemas first (it accepts both shapes), then the LAs. The files changed by 2.1 are listed with their sha256 in the PR; the LA re-syncs exactly those.
|