@desplega.ai/agent-swarm 2.5.2 → 2.5.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{acp-adapter-csh0zz3b.js → acp-adapter-gr2fcyyj.js} +3 -3
- package/dist/{actions-6xx079d1.js → actions-wv766ghf.js} +18 -18
- package/dist/{amp-adapter-8n6f0hz8.js → amp-adapter-speb906t.js} +3 -3
- package/dist/{app-ft6r6v23.js → app-ag29n14f.js} +3 -3
- package/dist/{assistant-vt2cyfzd.js → assistant-3x8bz3eg.js} +19 -19
- package/dist/{boot-reembed-r3e808ky.js → boot-reembed-5b3d2eea.js} +1 -1
- package/dist/{boot-reembed-tgrfy27t.js → boot-reembed-jsmfrpzf.js} +1 -1
- package/dist/{boot-scrub-sweep-masdwxkm.js → boot-scrub-sweep-s73gerxy.js} +2 -2
- package/dist/{boot-seeds-0n82qc7q.js → boot-seeds-5zmax2wr.js} +4 -4
- package/dist/{claude-adapter-s7j47k06.js → claude-adapter-hswdnm3t.js} +3 -3
- package/dist/{claude-managed-adapter-gt6pa35d.js → claude-managed-adapter-cvbzj1h6.js} +2 -2
- package/dist/{claude-managed-setup-3hzrgrnk.js → claude-managed-setup-3k8m3hhc.js} +2 -2
- package/dist/{claude-sdk-session-0zrdzyx2.js → claude-sdk-session-fkbyk66x.js} +2 -2
- package/dist/{cli-d172svcw.js → cli-1fc85mvt.js} +4 -4
- package/dist/{cli-3e9vs93g.js → cli-210xd1f4.js} +4 -4
- package/dist/{cli-vaxw3bg0.js → cli-25j01dpj.js} +3 -3
- package/dist/{cli-n6kmqngw.js → cli-35gtk1p6.js} +1 -1
- package/dist/{cli-fgn7yg5r.js → cli-3ngdk5te.js} +1 -1
- package/dist/{cli-7ccf7akh.js → cli-40jsnrjm.js} +2 -2
- package/dist/{cli-b1tv22wr.js → cli-502eqv93.js} +1 -1
- package/dist/{cli-ydg6ep51.js → cli-6n8axjv1.js} +1 -1
- package/dist/{cli-f55b6ky0.js → cli-8s5wx3nz.js} +2 -2
- package/dist/{cli-86zpfje5.js → cli-99xgw26r.js} +1 -1
- package/dist/{cli-xw9q173h.js → cli-9s05422w.js} +1 -1
- package/dist/{cli-h5d22pfs.js → cli-aecdcr77.js} +119 -282
- package/dist/{cli-w9etnnxk.js → cli-bf8ma3f1.js} +120 -50
- package/dist/{cli-15xd0j7w.js → cli-ce0ak9ms.js} +11 -11
- package/dist/{cli-y3dzwyy3.js → cli-e2ar4bp1.js} +71 -20
- package/dist/{cli-b6tnjdvx.js → cli-e8jyq7d8.js} +1 -1
- package/dist/{cli-6y43v79h.js → cli-egjggehm.js} +1 -1
- package/dist/{cli-cf63dnxj.js → cli-ewm3pwbq.js} +2 -2
- package/dist/{cli-bpbcpd8b.js → cli-fsdprwrg.js} +1 -1
- package/dist/{cli-qrjv7t18.js → cli-fzcn3d87.js} +1 -1
- package/dist/{cli-14v1sxxy.js → cli-gpsz7g88.js} +6 -6
- package/dist/{cli-wadzapwb.js → cli-hcpj2xqz.js} +1 -1
- package/dist/{cli-jmcjbjkt.js → cli-hhxjnjzs.js} +1 -1
- package/dist/{cli-j3dwd4kz.js → cli-jjznk3f7.js} +718 -108
- package/dist/{cli-csh8fnbg.js → cli-kxawjrxg.js} +147 -28
- package/dist/{cli-mntxtkhn.js → cli-mz9afxxf.js} +2 -1
- package/dist/{cli-0vgy5wfw.js → cli-n45mr7zh.js} +2 -2
- package/dist/{cli-aaqnqpyw.js → cli-p0rzjkya.js} +2 -2
- package/dist/{cli-3qv8naq8.js → cli-p6zs9h4y.js} +827 -107
- package/dist/{cli-e91vshb1.js → cli-p9rafdrb.js} +4 -4
- package/dist/{cli-edtm741g.js → cli-qgf8emrn.js} +5 -5
- package/dist/{cli-ha22h0eg.js → cli-qzme5949.js} +26 -26
- package/dist/{cli-xzge46sj.js → cli-s49hpn37.js} +2 -2
- package/dist/{cli-b63d08rs.js → cli-s53vp709.js} +1 -1
- package/dist/{cli-cyfgxnvb.js → cli-svrj71pg.js} +18 -16
- package/dist/{cli-4qkg9968.js → cli-vwn81jbs.js} +3 -3
- package/dist/{cli-f3pfn4x3.js → cli-w1zwctj2.js} +1 -1
- package/dist/{cli-kcnwn85g.js → cli-y561h5a9.js} +4 -4
- package/dist/{cli-n2918sed.js → cli-z21whrrg.js} +15 -15
- package/dist/cli.js +16 -16
- package/dist/{codex-adapter-brh2tcxr.js → codex-adapter-n8ka66vf.js} +2 -2
- package/dist/{codex-session-runner-0dnj21qs.js → codex-session-runner-fsebts37.js} +1 -1
- package/dist/{commands-1cnvseqx.js → commands-t04rt2pc.js} +2 -2
- package/dist/{controller-kc7s9xhn.js → controller-rgna04r2.js} +2 -2
- package/dist/{controllers-connect-jzx7t4bn.js → controllers-connect-bqm1rtft.js} +20 -20
- package/dist/{cursor-adapter-qpwp5sfg.js → cursor-adapter-swr2k95y.js} +1 -1
- package/dist/{db-v5ytg5pw.js → db-24ck09xy.js} +1 -1
- package/dist/{db-4vq88jkc.js → db-a4m7haw0.js} +3 -3
- package/dist/{dispatcher-trtcrqae.js → dispatcher-nwf9ss0q.js} +17 -17
- package/dist/{dsh-adapter-f20xwn1z.js → dsh-adapter-kpdnhjst.js} +2 -2
- package/dist/{external-channel-n9bymc2n.js → external-channel-j1e2p34p.js} +2 -2
- package/dist/{grok-adapter-4kpmmnr6.js → grok-adapter-qtjz0whk.js} +4 -4
- package/dist/{handlers-ptsraqa8.js → handlers-19xr6dnz.js} +20 -20
- package/dist/{heartbeat-60qtxtjf.js → heartbeat-vtesda5d.js} +20 -20
- package/dist/{hook-rbktcstg.js → hook-8wqaszda.js} +1 -1
- package/dist/{hook-ngtvh729.js → hook-b2hhj6rd.js} +1 -1
- package/dist/{http-z8xy72gp.js → http-mxcqhhcs.js} +2757 -2448
- package/dist/{index-a9nsavhx.js → index-1r8wfpzs.js} +1 -1
- package/dist/{index-cbxkmfc5.js → index-9pn1pfkf.js} +1 -1
- package/dist/{index-s193j5fh.js → index-kw2mmat7.js} +20 -20
- package/dist/{index-06qkmcx6.js → index-n1vbvta9.js} +17 -9
- package/dist/{index-6srwvd5b.js → index-p4xfz37z.js} +1 -1
- package/dist/{index-1cscynkb.js → index-spwxjgay.js} +17 -17
- package/dist/{index-r70q0xjx.js → index-srg522m6.js} +17 -17
- package/dist/{keepalive-a9srcphj.js → keepalive-dsd8rtgx.js} +3 -3
- package/dist/{lead-3devneqy.js → lead-zyerk1x2.js} +30 -30
- package/dist/{maintenance-0er16h2y.js → maintenance-jqmqwnet.js} +1 -1
- package/dist/{oauth-refresh-sweep-b65jcyw2.js → oauth-refresh-sweep-q2c3bnvm.js} +2 -2
- package/dist/{onboard-wqywz5py.js → onboard-w4yd7s8t.js} +2 -2
- package/dist/{opencode-adapter-e56086m1.js → opencode-adapter-9tppvxar.js} +2 -2
- package/dist/{otel-impl-vk32s884.js → otel-impl-f36byzbd.js} +1 -1
- package/dist/{pricing-refresh-t27anv4z.js → pricing-refresh-xvak5p4v.js} +3 -3
- package/dist/{rbac-roles-bmcgsxjn.js → rbac-roles-682k2mt5.js} +1 -1
- package/dist/{rbac-roles-452665y0.js → rbac-roles-gfn30fb8.js} +2 -2
- package/dist/{render-v2-0ttsk2j4.js → render-v2-hgytz8nn.js} +3 -3
- package/dist/{resume-a26kvr41.js → resume-2dvq348g.js} +7 -5
- package/dist/{session-access-dgh9nkwk.js → session-access-7f21ekq9.js} +4 -4
- package/dist/{setup-9vc9qahm.js → setup-epffxtbk.js} +1 -1
- package/dist/{work-objects-vn8nkxwn.js → work-objects-xz1a5rqk.js} +20 -20
- package/dist/{worker-1x1h8vnm.js → worker-42t8emjt.js} +30 -30
- package/openapi.json +3045 -2456
- package/package.json +4 -4
- package/src/be/approval-config-question.ts +389 -0
- package/src/be/automation-preflight.ts +37 -3
- package/src/be/db/session-tree.ts +73 -15
- package/src/be/db/skill-search.ts +87 -0
- package/src/be/db/tasks/read.ts +29 -2
- package/src/be/db/workflow-runs.ts +54 -7
- package/src/be/db.ts +25 -5
- package/src/be/migrations/239_skills_fts.sql +40 -0
- package/src/be/migrations/240_workflow_run_parent_step_attempt.sql +37 -0
- package/src/be/migrations/241_skills_meta_fts.sql +39 -0
- package/src/be/swarm-config-guard.ts +23 -0
- package/src/extensions/catalog.generated.json +4 -4
- package/src/http/approval-requests.ts +209 -38
- package/src/http/config.ts +22 -37
- package/src/http/sessions.ts +27 -2
- package/src/integrations/chat-sdk/inbound.ts +5 -4
- package/src/providers/amp-adapter.ts +1 -1
- package/src/providers/dsh-adapter.ts +1 -1
- package/src/tests/approval-config-question.test.ts +702 -0
- package/src/tests/chat-sdk-inbound.test.ts +20 -0
- package/src/tests/dsh-adapter.test.ts +1 -1
- package/src/tests/entrypoint-gitlab-credential-helper.test.ts +273 -0
- package/src/tests/extensions-example-comms-adherence-gate.test.ts +321 -12
- package/src/tests/list-endpoint-slimming.test.ts +33 -0
- package/src/tests/migration-240-workflow-run-parent-step-attempt.test.ts +55 -0
- package/src/tests/script-connections.test.ts +6 -0
- package/src/tests/sessions-tree.test.ts +168 -1
- package/src/tests/skill-search-long-content.test.ts +193 -0
- package/src/tests/skill-search.test.ts +173 -22
- package/src/tests/workflow-sub-workflow.test.ts +538 -1
- package/src/tools/request-human-input.ts +27 -5
- package/src/types.ts +7 -0
- package/src/workflows/approval-notifications.ts +10 -3
- package/src/workflows/checkpoint.ts +69 -35
- package/src/workflows/engine.ts +234 -87
- package/src/workflows/executors/base.ts +6 -0
- package/src/workflows/executors/human-in-the-loop.ts +25 -1
- package/src/workflows/executors/sub-workflow.ts +45 -16
- package/src/workflows/recovery.ts +63 -1
- package/src/workflows/resume.ts +60 -21
- package/src/workflows/retry-poller.ts +25 -4
- package/templates/extensions/comms-adherence-gate/README.md +38 -13
- package/templates/extensions/comms-adherence-gate/hooks.ts +272 -69
- package/templates/extensions/comms-adherence-gate/manifest.json +2 -2
- package/templates/skills/turso-interaction/content.md +1 -1
- package/templates/skills/workflow-iterate/content.md +8 -1
- /package/dist/{cli-stxdrqs2.js → cli-avdvgcay.js} +0 -0
- /package/dist/{pi-mono-adapter-tzps8f38.js → pi-mono-adapter-tqsge1ay.js} +0 -0
|
@@ -5,22 +5,30 @@ import { z } from "zod";
|
|
|
5
5
|
* comms-adherence-gate
|
|
6
6
|
*
|
|
7
7
|
* When an agent calls `store-progress` with status "completed" on a task that
|
|
8
|
-
* reaches a human in Slack,
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
* (
|
|
12
|
-
*
|
|
8
|
+
* reaches a human in Slack, or posts to Slack itself (`slack-reply`,
|
|
9
|
+
* `slack-post`, `slack-start-thread`, `slack-update`), judge the text with Jev:
|
|
10
|
+
* against the requesting user's stored comms preferences
|
|
11
|
+
* (users.metadata.$.comms) and what that user actually asked for in their
|
|
12
|
+
* latest message, and against a plain-language rubric that applies to every
|
|
13
|
+
* message whatever preferences exist. An explicit request ("tell me a long
|
|
14
|
+
* joke", "explain in detail") outranks the stored default, so a requested long
|
|
15
|
+
* answer is not a verbosity failure.
|
|
16
|
+
*
|
|
17
|
+
* Threads with several people: a message that @-mentions someone is judged
|
|
18
|
+
* against the first mentioned person with stored preferences, skipping a
|
|
19
|
+
* requester without preferences; otherwise against the requester. `plain` applies to everyone.
|
|
13
20
|
*
|
|
14
21
|
* Transport (config.provider): Jev is served by TypeSafe directly and by
|
|
15
22
|
* OpenRouter's Decisions endpoint. Both take the same {model, state, questions}
|
|
16
23
|
* body and return the same `answers` shape. `auto` uses TYPESAFE_API_KEY when
|
|
17
24
|
* set, else OPENROUTER_API_KEY.
|
|
18
25
|
*
|
|
19
|
-
* modes (config.mode):
|
|
26
|
+
* modes (config.mode for the preference dimensions, config.plainMode for `plain`):
|
|
20
27
|
* off -> never evaluate
|
|
21
28
|
* shadow -> evaluate + record, never block (default)
|
|
22
|
-
* enforce -> record, and block the
|
|
23
|
-
*
|
|
29
|
+
* enforce -> record, and block the call up to maxBlocksPerTask times per task
|
|
30
|
+
* with actionable feedback. The budget is shared by both modes and
|
|
31
|
+
* by every gated tool.
|
|
24
32
|
*
|
|
25
33
|
* Fails OPEN on every error: a judge outage must never wall the swarm.
|
|
26
34
|
* Every verdict lands in KV namespace `ext:comms-gate`.
|
|
@@ -28,8 +36,15 @@ import { z } from "zod";
|
|
|
28
36
|
|
|
29
37
|
export const config = z.object({
|
|
30
38
|
mode: z.enum(["off", "shadow", "enforce"]).default("shadow"),
|
|
31
|
-
/** A dimension scoring below this fails. */
|
|
39
|
+
/** A preference dimension scoring below this fails. */
|
|
32
40
|
threshold: z.number().min(0).max(1).default(0.5),
|
|
41
|
+
/** The `plain` dimension, judged on every message. Independent of `mode`. */
|
|
42
|
+
plainMode: z.enum(["off", "shadow", "enforce"]).default("shadow"),
|
|
43
|
+
/**
|
|
44
|
+
* `plain` scoring below this fails. Jev scores dense status text low and
|
|
45
|
+
* plain text high with little in between, so this sits far below `threshold`.
|
|
46
|
+
*/
|
|
47
|
+
plainThreshold: z.number().min(0).max(1).default(0.25),
|
|
33
48
|
timeoutMs: z.number().int().positive().default(4000),
|
|
34
49
|
maxBlocksPerTask: z.number().int().min(0).default(1),
|
|
35
50
|
provider: z.enum(["auto", "typesafe", "openrouter"]).default("auto"),
|
|
@@ -48,6 +63,13 @@ const NS = "ext:comms-gate";
|
|
|
48
63
|
const TYPESAFE_ENDPOINT = "https://api.typesafe.ai/v1/systemone";
|
|
49
64
|
const OPENROUTER_ENDPOINT = "https://openrouter.ai/api/alpha/decisions";
|
|
50
65
|
const UUID_RE = /^[0-9a-fA-F-]{36}$/;
|
|
66
|
+
/** MCP tools that post agent-authored text to Slack, all with a `message` arg. */
|
|
67
|
+
export const SLACK_TOOLS = new Set([
|
|
68
|
+
"slack-reply",
|
|
69
|
+
"slack-post",
|
|
70
|
+
"slack-start-thread",
|
|
71
|
+
"slack-update",
|
|
72
|
+
]);
|
|
51
73
|
// The dispatcher cancels a handler after 5 s; leave room for the DB read and KV writes.
|
|
52
74
|
const MAX_TIMEOUT_MS = 4000;
|
|
53
75
|
|
|
@@ -96,6 +118,39 @@ const JUDGING_RULES = [
|
|
|
96
118
|
|
|
97
119
|
type Prefs = { verbosity?: unknown; tone?: unknown; language?: unknown } & Record<string, unknown>;
|
|
98
120
|
|
|
121
|
+
/**
|
|
122
|
+
* The `plain` rubric, from the `comms` skill (Simpler and Precise modes). Jev
|
|
123
|
+
* separates dense from plain text reliably only with contrasting examples; a
|
|
124
|
+
* prose-only rubric scored a plain rewrite as low as the dense original.
|
|
125
|
+
*/
|
|
126
|
+
export const PLAIN_RULES = [
|
|
127
|
+
"Examples of NOT plain: 'p50 Δ +120ms vs baseline (n=37)', 'error rate 2.1% → 0.4% (cohort B, ±0.3 at 95%)', 'cost $-42/day', 'first-call hit ratio 0.61', 'drift unmeasured; 2 flaky'.",
|
|
128
|
+
"Examples of plain: 'Pages load 120 ms slower than last week, across 37 runs', 'Errors dropped from 2 in 100 requests to 4 in 1,000', 'We spend $42 a day less', 'PR #123 keeps the migration reversible', 'Lead saves about $75 a week'.",
|
|
129
|
+
"Code identifiers, task ids, PR numbers, links, file paths and version numbers are fine as they are.",
|
|
130
|
+
];
|
|
131
|
+
|
|
132
|
+
export const PLAIN_QUESTION = {
|
|
133
|
+
type: "noul",
|
|
134
|
+
instructions:
|
|
135
|
+
"Is agent_message written like the plain examples in plain_language_rules rather than like the NOT plain ones? Ignore length and tone.",
|
|
136
|
+
criteria: {
|
|
137
|
+
true: "Written like the plain examples: everyday words and numbers with units and direction.",
|
|
138
|
+
false:
|
|
139
|
+
"Written like the NOT plain examples: statistics shorthand, abbreviations, signed or cryptic numbers.",
|
|
140
|
+
},
|
|
141
|
+
};
|
|
142
|
+
|
|
143
|
+
/** Structured output (an outputSchema answer) is for machines; `plain` skips it. */
|
|
144
|
+
export const looksLikeJson = (text: string): boolean => {
|
|
145
|
+
if (!/^[[{]/.test(text)) return false;
|
|
146
|
+
try {
|
|
147
|
+
JSON.parse(text);
|
|
148
|
+
return true;
|
|
149
|
+
} catch {
|
|
150
|
+
return false;
|
|
151
|
+
}
|
|
152
|
+
};
|
|
153
|
+
|
|
99
154
|
/** One `noul` question per stated preference the gate knows how to judge. */
|
|
100
155
|
export const buildQuestions = (prefs: Prefs): Record<string, unknown> => {
|
|
101
156
|
const questions: Record<string, unknown> = {};
|
|
@@ -246,27 +301,41 @@ const SELFTEST_BAD =
|
|
|
246
301
|
"Great question! Let me walk you through what I did here. First I looked at the task, then I reviewed the relevant files, and after careful consideration I decided to go with option B. I hope this helps - let me know if you would like me to elaborate further on any of the above!";
|
|
247
302
|
const SELFTEST_GOOD =
|
|
248
303
|
"Went with option B: it is the only one that keeps the migration reversible. PR #123.";
|
|
304
|
+
// A real dense monitor post a requester had to ask to have rewritten, and that rewrite.
|
|
305
|
+
const SELFTEST_PLAIN_BAD =
|
|
306
|
+
"Prompt cache monitor, day 7 verdict for v2.0.0: partial. Lead first-call read median 12.4k (baseline v1.173.0, n 410) → 19.1k tokens (n 2234; expected 25.0k-40.0k). First-call saving about $75/week on Lead vs v1.173.0 (expected $129; sampling noise ±$24 at 95%, one baseline version, day-to-day drift not measured); session $ change $-167/week; fleet first-call saving $-7/week.";
|
|
307
|
+
const SELFTEST_PLAIN_GOOD =
|
|
308
|
+
"Cheaper, but half what we hoped: Lead saves about $75 a week, not $129. Whole Lead tasks still cost $167 a week more, probably from other releases and heavier work, not this change.";
|
|
309
|
+
|
|
310
|
+
const below = (bad: unknown, good: unknown, threshold: number): boolean =>
|
|
311
|
+
typeof bad === "number" && typeof good === "number" && bad < threshold && good >= threshold;
|
|
249
312
|
|
|
250
313
|
/**
|
|
251
|
-
* Run the judge on
|
|
252
|
-
* known-good (negative control, must score at or above it)
|
|
253
|
-
* under `selftest:last
|
|
254
|
-
* without touching real traffic.
|
|
314
|
+
* Run the judge on known-bad (positive control, must score below threshold) and
|
|
315
|
+
* known-good (negative control, must score at or above it) messages and record
|
|
316
|
+
* them under `selftest:last`: one pair for verbosity, one for `plain`. Proves key,
|
|
317
|
+
* egress and KV from inside the API runtime without touching real traffic.
|
|
255
318
|
*/
|
|
256
319
|
const selfTest = async (swarm: Swarm, cfg: Cfg, signal?: AbortSignal): Promise<void> => {
|
|
257
320
|
const started = Date.now();
|
|
258
321
|
try {
|
|
259
322
|
const transport = resolveTransport(cfg);
|
|
260
|
-
const
|
|
261
|
-
const judge = (message: string) =>
|
|
323
|
+
const judge = (message: string, dim: "verbosity" | "plain") =>
|
|
262
324
|
askJev(
|
|
263
|
-
|
|
264
|
-
|
|
325
|
+
dim === "plain"
|
|
326
|
+
? { plain_language_rules: PLAIN_RULES, agent_message: message }
|
|
327
|
+
: { requester_preferences: SELFTEST_PREFS, agent_message: message },
|
|
328
|
+
{ [dim]: dim === "plain" ? PLAIN_QUESTION : buildQuestions(SELFTEST_PREFS).verbosity },
|
|
265
329
|
transport,
|
|
266
330
|
cfg,
|
|
267
331
|
signal,
|
|
268
|
-
).then((r) => r.scores
|
|
269
|
-
const [bad, good] = await Promise.all([
|
|
332
|
+
).then((r) => r.scores[dim] ?? null);
|
|
333
|
+
const [bad, good, plainBad, plainGood] = await Promise.all([
|
|
334
|
+
judge(SELFTEST_BAD, "verbosity"),
|
|
335
|
+
judge(SELFTEST_GOOD, "verbosity"),
|
|
336
|
+
judge(SELFTEST_PLAIN_BAD, "plain"),
|
|
337
|
+
judge(SELFTEST_PLAIN_GOOD, "plain"),
|
|
338
|
+
]);
|
|
270
339
|
await swarm.kv_set({
|
|
271
340
|
namespace: NS,
|
|
272
341
|
key: "selftest:last",
|
|
@@ -276,11 +345,11 @@ const selfTest = async (swarm: Swarm, cfg: Cfg, signal?: AbortSignal): Promise<v
|
|
|
276
345
|
threshold: cfg.threshold,
|
|
277
346
|
positiveControlBadScore: bad,
|
|
278
347
|
negativeControlGoodScore: good,
|
|
279
|
-
controlsOk:
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
348
|
+
controlsOk: below(bad, good, cfg.threshold),
|
|
349
|
+
plainThreshold: cfg.plainThreshold,
|
|
350
|
+
plainPositiveControlBadScore: plainBad,
|
|
351
|
+
plainNegativeControlGoodScore: plainGood,
|
|
352
|
+
plainControlsOk: below(plainBad, plainGood, cfg.plainThreshold),
|
|
284
353
|
latencyMs: Date.now() - started,
|
|
285
354
|
},
|
|
286
355
|
});
|
|
@@ -302,13 +371,77 @@ const selfTest = async (swarm: Swarm, cfg: Cfg, signal?: AbortSignal): Promise<v
|
|
|
302
371
|
const TASK_SQL = `SELECT t.source, t.slackThreadTs, t.agentId, t.requestedByUserId,
|
|
303
372
|
json_extract(u.metadata, '$.comms') AS comms, u.name,
|
|
304
373
|
substr(t.task, 1, 400) AS askHead,
|
|
305
|
-
substr(t.task, -1400) AS askTail
|
|
374
|
+
substr(t.task, -1400) AS askTail,
|
|
375
|
+
t.creatorAgentId
|
|
306
376
|
FROM agent_tasks t
|
|
307
377
|
LEFT JOIN users u ON u.id = t.requestedByUserId
|
|
308
378
|
WHERE t.id = ?`;
|
|
309
379
|
|
|
310
380
|
const LEAD_SQL = "SELECT isLead FROM agents WHERE id = ?";
|
|
311
381
|
|
|
382
|
+
/** Slack user mentions in agent text: `<@U123>` or `<@U123|name>`. */
|
|
383
|
+
const MENTION_RE = /<@([UW][A-Z0-9]+)(?:\|[^>]*)?>/g;
|
|
384
|
+
const MAX_MENTIONS = 5;
|
|
385
|
+
|
|
386
|
+
/** Mentioned Slack user ids in order of first appearance, capped. */
|
|
387
|
+
export const mentionedSlackIds = (text: string): string[] => {
|
|
388
|
+
const ids: string[] = [];
|
|
389
|
+
for (const m of text.matchAll(MENTION_RE)) {
|
|
390
|
+
if (!ids.includes(m[1]!)) ids.push(m[1]!);
|
|
391
|
+
if (ids.length >= MAX_MENTIONS) break;
|
|
392
|
+
}
|
|
393
|
+
return ids;
|
|
394
|
+
};
|
|
395
|
+
|
|
396
|
+
const addresseeSql = (n: number) => `SELECT x.externalId, u.id, u.name,
|
|
397
|
+
json_extract(u.metadata, '$.comms') AS comms
|
|
398
|
+
FROM user_external_ids x
|
|
399
|
+
JOIN users u ON u.id = x.userId
|
|
400
|
+
WHERE x.kind = 'slack' AND x.externalId IN (${Array(n).fill("?").join(", ")})`;
|
|
401
|
+
|
|
402
|
+
const parsePrefs = (raw: unknown): Prefs | null => {
|
|
403
|
+
if (!raw) return null;
|
|
404
|
+
const parsed = typeof raw === "string" ? JSON.parse(raw) : raw;
|
|
405
|
+
return parsed && typeof parsed === "object" && Object.keys(parsed).length > 0
|
|
406
|
+
? (parsed as Prefs)
|
|
407
|
+
: null;
|
|
408
|
+
};
|
|
409
|
+
|
|
410
|
+
type Audience = { id: string; name: string; prefs: Prefs; via: "mention" | "requester" };
|
|
411
|
+
|
|
412
|
+
/**
|
|
413
|
+
* Whose preferences a message is judged against. In a thread with several
|
|
414
|
+
* people, the message is for whoever it @-mentions: the first mentioned person
|
|
415
|
+
* with stored preferences wins, including the requester only if they have prefs.
|
|
416
|
+
* Without such a mention, the task requester (the
|
|
417
|
+
* person whose message created this task). One SQL read, only when the text
|
|
418
|
+
* mentions someone; no judge call.
|
|
419
|
+
*/
|
|
420
|
+
const resolveAudience = async (
|
|
421
|
+
swarm: Swarm,
|
|
422
|
+
text: string,
|
|
423
|
+
requester: Audience,
|
|
424
|
+
): Promise<Audience> => {
|
|
425
|
+
const ids = mentionedSlackIds(text);
|
|
426
|
+
if (ids.length === 0) return requester;
|
|
427
|
+
const q = (await swarm.db_query({ sql: addresseeSql(ids.length), params: ids })) as Envelope;
|
|
428
|
+
const rows = q?.data?.rows ?? q?.rows ?? [];
|
|
429
|
+
for (const slackId of ids) {
|
|
430
|
+
const row = rows.find((r) => String(cell(r, 0, "externalId")) === slackId);
|
|
431
|
+
if (!row) continue;
|
|
432
|
+
const userId = String(cell(row, 1, "id") ?? "");
|
|
433
|
+
// A requester with stored preferences keeps their own task context.
|
|
434
|
+
if (userId && userId === requester.id) {
|
|
435
|
+
if (Object.keys(requester.prefs).length > 0) return requester;
|
|
436
|
+
continue;
|
|
437
|
+
}
|
|
438
|
+
const prefs = parsePrefs(cell(row, 3, "comms"));
|
|
439
|
+
if (!prefs) continue;
|
|
440
|
+
return { id: userId, name: String(cell(row, 2, "name") ?? userId), prefs, via: "mention" };
|
|
441
|
+
}
|
|
442
|
+
return requester;
|
|
443
|
+
};
|
|
444
|
+
|
|
312
445
|
/**
|
|
313
446
|
* Mirrors the `task.progress.write` rule store-progress enforces
|
|
314
447
|
* (src/rbac/legacy-policy.ts): the caller owns the task, the task is
|
|
@@ -327,29 +460,56 @@ const mayWriteProgress = async (
|
|
|
327
460
|
return Number(cell(row, 0, "isLead") ?? 0) === 1;
|
|
328
461
|
};
|
|
329
462
|
|
|
463
|
+
/** The text a gated call would put in front of a human, and the task it belongs to. */
|
|
464
|
+
const gatedText = (
|
|
465
|
+
tool: string,
|
|
466
|
+
args: Record<string, unknown>,
|
|
467
|
+
sourceTaskId: string | undefined,
|
|
468
|
+
): { text: string; taskId: string } | null => {
|
|
469
|
+
if (tool === "store-progress") {
|
|
470
|
+
if (args.status !== "completed") return null;
|
|
471
|
+
return {
|
|
472
|
+
text: typeof args.output === "string" ? args.output.trim() : "",
|
|
473
|
+
taskId: String(args.taskId ?? sourceTaskId ?? ""),
|
|
474
|
+
};
|
|
475
|
+
}
|
|
476
|
+
if (SLACK_TOOLS.has(tool)) {
|
|
477
|
+
// slack-reply names its task; the other tools post from the caller's task context.
|
|
478
|
+
const named = tool === "slack-reply" ? args.taskId : undefined;
|
|
479
|
+
return {
|
|
480
|
+
text: typeof args.message === "string" ? args.message.trim() : "",
|
|
481
|
+
taskId: String(named ?? sourceTaskId ?? ""),
|
|
482
|
+
};
|
|
483
|
+
}
|
|
484
|
+
return null;
|
|
485
|
+
};
|
|
486
|
+
|
|
330
487
|
const extension: SwarmExtension = (api) => {
|
|
331
488
|
api.on("pre.tool.call", async (event, ctx) => {
|
|
332
|
-
if (event.tool !== "store-progress") return;
|
|
333
|
-
|
|
489
|
+
if (event.tool !== "store-progress" && !SLACK_TOOLS.has(event.tool)) return;
|
|
490
|
+
const origin = event.requestInfo?.callOrigin;
|
|
491
|
+
if (origin === "extension") return;
|
|
334
492
|
|
|
335
493
|
const cfg = cfgOf(ctx.config);
|
|
336
|
-
if (cfg.mode === "off") return;
|
|
494
|
+
if (cfg.mode === "off" && cfg.plainMode === "off") return;
|
|
337
495
|
|
|
338
496
|
const args = (event.args ?? {}) as Record<string, unknown>;
|
|
339
497
|
|
|
340
498
|
// Self-test hatch: a progress call whose text starts with the sentinel runs
|
|
341
499
|
// the controls and records them under `selftest:last`.
|
|
342
|
-
if (
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
500
|
+
if (
|
|
501
|
+
event.tool === "store-progress" &&
|
|
502
|
+
args.status !== "completed" &&
|
|
503
|
+
typeof args.progress === "string" &&
|
|
504
|
+
args.progress.startsWith(SELFTEST)
|
|
505
|
+
) {
|
|
506
|
+
await selfTest(ctx.swarm, cfg, ctx.signal);
|
|
346
507
|
return;
|
|
347
508
|
}
|
|
348
509
|
|
|
349
|
-
const
|
|
350
|
-
if (!
|
|
351
|
-
|
|
352
|
-
const taskId = String(args.taskId ?? event.requestInfo?.sourceTaskId ?? "");
|
|
510
|
+
const gated = gatedText(event.tool, args, event.requestInfo?.sourceTaskId);
|
|
511
|
+
if (!gated?.text) return;
|
|
512
|
+
const { text, taskId } = gated;
|
|
353
513
|
if (!UUID_RE.test(taskId)) return;
|
|
354
514
|
|
|
355
515
|
try {
|
|
@@ -362,33 +522,54 @@ const extension: SwarmExtension = (api) => {
|
|
|
362
522
|
const agentId = String(cell(row, 2, "agentId") ?? "");
|
|
363
523
|
const requesterId = String(cell(row, 3, "requestedByUserId") ?? "");
|
|
364
524
|
const commsRaw = cell(row, 4, "comms");
|
|
365
|
-
const requesterName = String(cell(row, 5, "name") ?? requesterId);
|
|
525
|
+
const requesterName = String(cell(row, 5, "name") ?? (requesterId || "the requester"));
|
|
366
526
|
const threadOpening = String(cell(row, 6, "askHead") ?? "");
|
|
367
527
|
const ask = latestAsk(String(cell(row, 7, "askTail") ?? ""));
|
|
368
528
|
|
|
369
|
-
// Judge only a
|
|
370
|
-
//
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
529
|
+
// Judge only a call the tool itself will accept, so no agent can ship another
|
|
530
|
+
// task's thread and preferences to the external judge. store-progress follows
|
|
531
|
+
// its write rule; the Slack tools follow slack-reply's: the caller is the
|
|
532
|
+
// task's assignee or its creator.
|
|
533
|
+
const callerId = event.requestInfo?.agentId;
|
|
534
|
+
const creatorAgentId = String(cell(row, 8, "creatorAgentId") ?? "");
|
|
535
|
+
const allowed =
|
|
536
|
+
event.tool === "store-progress"
|
|
537
|
+
? await mayWriteProgress(ctx.swarm, callerId, agentId)
|
|
538
|
+
: Boolean(callerId) && (callerId === agentId || callerId === creatorAgentId);
|
|
539
|
+
if (!allowed) return;
|
|
540
|
+
|
|
541
|
+
// A completion reaches a human only on a Slack task; a Slack tool always does.
|
|
542
|
+
if (event.tool === "store-progress" && source !== "slack" && !slackThreadTs) return;
|
|
543
|
+
|
|
544
|
+
const requester: Audience = {
|
|
545
|
+
id: requesterId,
|
|
546
|
+
name: requesterName,
|
|
547
|
+
prefs: parsePrefs(commsRaw) ?? {},
|
|
548
|
+
via: "requester",
|
|
549
|
+
};
|
|
550
|
+
// `plain` alone judges no one's preferences, so it skips the lookup.
|
|
551
|
+
const audience =
|
|
552
|
+
cfg.mode === "off" ? requester : await resolveAudience(ctx.swarm, text, requester);
|
|
553
|
+
const { prefs, name: audienceName } = audience;
|
|
554
|
+
|
|
555
|
+
const questions: Record<string, unknown> = cfg.mode === "off" ? {} : buildQuestions(prefs);
|
|
556
|
+
// Only a store-progress outputSchema answer is machine JSON; a Slack post is
|
|
557
|
+
// always read by a human, JSON or not.
|
|
558
|
+
if (cfg.plainMode !== "off" && (event.tool !== "store-progress" || !looksLikeJson(text)))
|
|
559
|
+
questions.plain = PLAIN_QUESTION;
|
|
380
560
|
if (Object.keys(questions).length === 0) return;
|
|
381
561
|
|
|
382
562
|
const transport = resolveTransport(cfg);
|
|
383
563
|
const { scores, latencyMs } = await askJev(
|
|
384
564
|
{
|
|
385
|
-
role: "You audit an AI agent's
|
|
386
|
-
requester:
|
|
565
|
+
role: "You audit an AI agent's message to the human who requested the work.",
|
|
566
|
+
requester: audienceName,
|
|
387
567
|
requester_preferences: prefs,
|
|
388
568
|
latest_request: ask,
|
|
389
569
|
thread_opening: threadOpening,
|
|
390
570
|
judging_rules: JUDGING_RULES,
|
|
391
|
-
|
|
571
|
+
plain_language_rules: PLAIN_RULES,
|
|
572
|
+
agent_message: text,
|
|
392
573
|
},
|
|
393
574
|
questions,
|
|
394
575
|
transport,
|
|
@@ -396,26 +577,38 @@ const extension: SwarmExtension = (api) => {
|
|
|
396
577
|
ctx.signal,
|
|
397
578
|
);
|
|
398
579
|
|
|
580
|
+
const limitOf = (dim: string) => (dim === "plain" ? cfg.plainThreshold : cfg.threshold);
|
|
399
581
|
const failed = Object.entries(scores)
|
|
400
|
-
.filter(([, v]) => v <
|
|
582
|
+
.filter(([k, v]) => v < limitOf(k))
|
|
401
583
|
.map(([k]) => k);
|
|
584
|
+
// Script SDK calls never reach pre.tool.call today; if one does, record it only.
|
|
585
|
+
const blocking =
|
|
586
|
+
origin === "script-sdk"
|
|
587
|
+
? []
|
|
588
|
+
: failed.filter((d) => (d === "plain" ? cfg.plainMode : cfg.mode) === "enforce");
|
|
402
589
|
|
|
403
590
|
const verdict = {
|
|
404
591
|
at: new Date().toISOString(),
|
|
592
|
+
tool: event.tool,
|
|
405
593
|
taskId,
|
|
406
594
|
agentId,
|
|
407
595
|
source,
|
|
408
596
|
requester: requesterName,
|
|
409
597
|
requesterId,
|
|
598
|
+
judgedFor: audience.via,
|
|
599
|
+
audienceId: audience.id,
|
|
600
|
+
audience: audienceName,
|
|
410
601
|
provider: transport.provider,
|
|
411
602
|
scores,
|
|
412
603
|
failed,
|
|
413
604
|
threshold: cfg.threshold,
|
|
605
|
+
plainThreshold: cfg.plainThreshold,
|
|
414
606
|
mode: cfg.mode,
|
|
607
|
+
plainMode: cfg.plainMode,
|
|
415
608
|
latencyMs,
|
|
416
609
|
preferences: prefs,
|
|
417
610
|
askPreview: ask.slice(0, 400),
|
|
418
|
-
outputPreview:
|
|
611
|
+
outputPreview: text.slice(0, 700),
|
|
419
612
|
};
|
|
420
613
|
|
|
421
614
|
await ctx.swarm.kv_incr({ namespace: NS, key: "total" });
|
|
@@ -435,7 +628,7 @@ const extension: SwarmExtension = (api) => {
|
|
|
435
628
|
// kv_get answers {success, data:{value}}, and {success:false} when absent.
|
|
436
629
|
const prior = (await ctx.swarm.kv_get({ namespace: NS, key: blocksKey })) as Envelope;
|
|
437
630
|
const priorBlocks = Number(prior?.data?.value ?? prior?.value ?? 0) || 0;
|
|
438
|
-
const willBlock =
|
|
631
|
+
const willBlock = blocking.length > 0 && priorBlocks < cfg.maxBlocksPerTask;
|
|
439
632
|
|
|
440
633
|
await ctx.swarm.kv_set({
|
|
441
634
|
namespace: NS,
|
|
@@ -446,7 +639,7 @@ const extension: SwarmExtension = (api) => {
|
|
|
446
639
|
if (!willBlock) {
|
|
447
640
|
await ctx.swarm.kv_incr({
|
|
448
641
|
namespace: NS,
|
|
449
|
-
key:
|
|
642
|
+
key: blocking.length > 0 ? "escaped" : "shadowed",
|
|
450
643
|
});
|
|
451
644
|
return;
|
|
452
645
|
}
|
|
@@ -454,20 +647,25 @@ const extension: SwarmExtension = (api) => {
|
|
|
454
647
|
await ctx.swarm.kv_incr({ namespace: NS, key: "blocked" });
|
|
455
648
|
await ctx.swarm.kv_incr({ namespace: NS, key: blocksKey });
|
|
456
649
|
|
|
457
|
-
const
|
|
458
|
-
.
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
650
|
+
const lines = [
|
|
651
|
+
blocking.includes("plain")
|
|
652
|
+
? `Comms gate: this message is not in plain words for ${audienceName}.`
|
|
653
|
+
: `Comms gate: this output does not match ${audienceName}'s stated preferences.`,
|
|
654
|
+
];
|
|
655
|
+
for (const d of blocking) {
|
|
656
|
+
const score = `${d} (${scores[d]!.toFixed(2)} vs ${limitOf(d)})`;
|
|
657
|
+
lines.push(
|
|
658
|
+
d === "plain"
|
|
659
|
+
? `${score}: a reader outside the swarm cannot follow it on first read.`
|
|
660
|
+
: `${score}: requested ${String(prefs[d] ?? "").slice(0, 220)}`,
|
|
661
|
+
);
|
|
662
|
+
}
|
|
663
|
+
lines.push(
|
|
664
|
+
blocking.includes("plain")
|
|
665
|
+
? `Rewrite the message with the \`comms\` skill (Simpler mode): common words, one idea per sentence, every number with a unit and a direction word, shorthand explained or replaced. Keep every number, name, link and hedge verbatim, then call ${event.tool} again.`
|
|
666
|
+
: `Rewrite the message to match and call ${event.tool} again. Do not change the facts, only how they are said.`,
|
|
470
667
|
);
|
|
668
|
+
return block(lines.join("\n"));
|
|
471
669
|
} catch (err) {
|
|
472
670
|
// Fail open, always.
|
|
473
671
|
try {
|
|
@@ -475,7 +673,12 @@ const extension: SwarmExtension = (api) => {
|
|
|
475
673
|
await ctx.swarm.kv_set({
|
|
476
674
|
namespace: NS,
|
|
477
675
|
key: "error:last",
|
|
478
|
-
value: {
|
|
676
|
+
value: {
|
|
677
|
+
at: new Date().toISOString(),
|
|
678
|
+
tool: event.tool,
|
|
679
|
+
taskId,
|
|
680
|
+
error: String(err).slice(0, 400),
|
|
681
|
+
},
|
|
479
682
|
});
|
|
480
683
|
} catch {
|
|
481
684
|
/* ignore */
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "../manifest.schema.json",
|
|
3
3
|
"name": "comms-adherence-gate",
|
|
4
|
-
"description": "Judges every Slack-facing store-progress completion against the
|
|
5
|
-
"version": "1.
|
|
4
|
+
"description": "Judges every Slack-facing store-progress completion and agent Slack post against the comms preferences of the requester, or of the person it @-mentions, their latest request and a plain-language rubric with Jev (TypeSafe or OpenRouter), and in enforce blocks a failing call once per task",
|
|
5
|
+
"version": "1.7.0",
|
|
6
6
|
"runtime": "api",
|
|
7
7
|
"assets": { "hooks": "hooks.ts" }
|
|
8
8
|
}
|
|
@@ -89,7 +89,7 @@ turso db list # verify
|
|
|
89
89
|
|
|
90
90
|
Do NOT use `turso auth login` — it needs a browser. Always feed the config token in.
|
|
91
91
|
|
|
92
|
-
If `turso db list` returns 401, the platform JWT has expired — refresh `TURSO_API_TOKEN` in swarm config
|
|
92
|
+
If `turso db list` returns 401, the platform JWT has expired — refresh `TURSO_API_TOKEN` in swarm config. If you cannot mint a new one, ask the token owner with a `request-human-input` question of type `config` (`config: { key: "TURSO_API_TOKEN", scope: "global" }`) and share the returned URL. They paste the new token into the dashboard card; you get back only whether it was set, never the value. Never ask for the token in chat.
|
|
93
93
|
|
|
94
94
|
## CLI database operations
|
|
95
95
|
|
|
@@ -132,7 +132,13 @@ Use these shapes as a starting point, then confirm them against the current exec
|
|
|
132
132
|
```
|
|
133
133
|
|
|
134
134
|
A `completed` run means the key works. A `failed` run carries the reason: `<KEY> is not configured` (missing; for `laya` it can name `LAYA_URL` or `LAYA_API_KEY`, or both), or `<KEY> was rejected by <host> (HTTP 401)` (present but refused). To probe `laya`, set `"provider": "laya"` in the ping node. If `create-workflow` says `system-one-decision` is an unregistered executor type, the node is not deployed on this swarm yet: stop and say so.
|
|
135
|
-
4. If the key is missing or was rejected, **do not build the node.** Ask a human for
|
|
135
|
+
4. If the key is missing or was rejected, **do not build the node.** Ask a human for it with a `request-human-input` question of type `config`, one question per key, and share the returned URL in the thread:
|
|
136
|
+
|
|
137
|
+
```json
|
|
138
|
+
{ "id": "typesafe_key", "type": "config", "config": { "key": "TYPESAFE_API_KEY", "scope": "global" } }
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
The person enters the value on the dashboard card, which already shows whether the key is set. A secret goes straight into encrypted config. You get back `{ key, scope, scopeId, isSecret, status, updatedAt }`; non-secret results may also include `value`. A secret value is never returned. For a non-secret value such as `LAYA_URL`, set `"isSecret": false`; add `options` or `defaultValue` when you know the valid values. When `status` is `set`, rerun step 3. Never ask them to paste a key into chat, a task, or a definition, and never write a key into a definition.
|
|
136
142
|
|
|
137
143
|
Afterwards, a `warnings` entry on `create-workflow` or a patch tool means the key went missing since you checked. Treat it as step 4.
|
|
138
144
|
|
|
@@ -217,6 +223,7 @@ The same rules apply to a standalone `request-human-input` call.
|
|
|
217
223
|
|
|
218
224
|
- **Dashboard card.** `title` is plain text. Each question's `label` is its heading and renders as markdown. `description` renders as full markdown and is the only field that shows images, so put drafts, diffs, and logs there.
|
|
219
225
|
- **Code.** Fence code, logs, and JSON with a language (```` ```json ````): the block keeps its lines and scrolls. Fence a prose draft with no language: it wraps. A single newline outside a fence becomes a paragraph break.
|
|
226
|
+
- **Config questions.** A `config` question (`config: { key, scope }`) renders the key, whether it is already set, and a password field for secrets or a select for non-secret `options`. The value is written only when the request resolves approved, and the response holds only the result (`status` `set`, `kept`, or `skipped`), never a secret value. It needs `approvers.policy: "any"`. Use it whenever you need a person to supply a config value or secret.
|
|
220
227
|
- **Images.** Put an `http(s)` URL ending in `.png`, `.jpg`, `.jpeg`, `.gif`, or `.webp` alone on its own line in `description`, or use ``. The link must outlive the request: an expired presigned URL shows an "image unavailable" note.
|
|
221
228
|
- **Slack.** A node's `notifications` post the title, the question labels (each collapsed to one line, capped at 3000 characters in total), a Review button, and the timeout. Descriptions, options, code, and images are not shown, so the title and labels must make sense alone. `request-human-input` posts nothing to Slack: share the returned URL yourself.
|
|
222
229
|
|
|
File without changes
|
|
File without changes
|