bullswarm 0.9.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -0
- package/README.md +56 -1
- package/connectors/_schema.json +28 -2
- package/connectors/claude-code.json +16 -2
- package/connectors/codex.json +14 -2
- package/connectors/command-code.json +18 -2
- package/connectors/echo-worker.mjs +7 -0
- package/connectors/echo.json +1 -1
- package/connectors/grok.json +15 -2
- package/connectors/opencode2.json +14 -2
- package/docs/experiments/2026-08-28-trending-ai-autonomy.md +208 -0
- package/package.json +1 -1
- package/skill/SKILL.md +32 -5
- package/src/cli.js +5 -2
- package/src/lib/agent-events.js +216 -0
- package/src/lib/usage.js +17 -5
- package/src/lib/verify.js +19 -4
- package/src/lib/watch.js +89 -23
- package/src/setup.js +16 -0
- package/src/workflow/cli.js +64 -5
- package/src/workflow/dashboard.js +28 -3
- package/src/workflow/draft.js +4 -2
- package/src/workflow/goal.js +2 -3
- package/src/workflow/runner.js +82 -38
- package/src/workflow/runtime.js +208 -44
- package/src/workflow/short-id.js +2 -0
- package/src/workflow/steering.js +61 -0
- package/src/workflow/tui.js +8 -0
- package/src/workflow/watch-cli.js +183 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,38 @@
|
|
|
1
1
|
# bullswarm changelog
|
|
2
2
|
|
|
3
|
+
## 0.10.0 — battle-tested advisory orchestration and agent activity
|
|
4
|
+
|
|
5
|
+
- Made `maxAgents` and `maxWorkflowSeconds` advisory planning targets instead
|
|
6
|
+
of hard stops. Workflows can exceed them to finish required implementation
|
|
7
|
+
and verification; structural graph-growth limits remain enforced.
|
|
8
|
+
- Removed implicit connector and generated-goal wall-clock timeouts. Delegates
|
|
9
|
+
wait for natural completion unless an operator explicitly supplies a timeout,
|
|
10
|
+
requests cancellation, or a definitive auth/quota failure is observed.
|
|
11
|
+
- Added compatibility migration for generated 0.9.0 goals carrying Bullswarm's
|
|
12
|
+
former 900-second planner/action timeout defaults.
|
|
13
|
+
- Fixed adaptive completion policy, current-action metadata, provider routing
|
|
14
|
+
history, usage aggregation, latest-worker verification, and truthful partial
|
|
15
|
+
token/cost accounting found during the Kipwise battle test.
|
|
16
|
+
- Added connector-owned native JSONL event adapters for Codex, Claude, Grok,
|
|
17
|
+
Command Code, and OpenCode. Workflows now retain and display the latest three
|
|
18
|
+
semantic shell/read/edit/write/response actions for every active agent.
|
|
19
|
+
- Added conservative stall evidence: ten minutes without transport, parsed
|
|
20
|
+
event, or semantic action activity is labeled `suspected_stalled` but never
|
|
21
|
+
causes an automatic kill.
|
|
22
|
+
- Added a low-noise `workflow watch <id>` progress stream with semantic-change
|
|
23
|
+
updates, heartbeats, last-three agent actions, and terminal per-attempt timing.
|
|
24
|
+
- Added optional durable `workflow steer <id> --message ...` guidance delivered
|
|
25
|
+
only at the next planning checkpoint, never injected into an active worker.
|
|
26
|
+
- Captured runtime model IDs declared in provider event streams for more complete
|
|
27
|
+
model/cost attribution, preserved Grok tool kinds across name-less updates,
|
|
28
|
+
and enabled Claude's supported forwarded-subagent text stream.
|
|
29
|
+
- Taught planners to reuse clean full-suite evidence and isolate mutation/pre-fix
|
|
30
|
+
experiments instead of redundantly or concurrently testing a changing tree.
|
|
31
|
+
- Expanded the offline suite to 224 tests. Real bounded CLI probes confirmed
|
|
32
|
+
all five provider event formats, and an exact packaged OpenCode watch smoke
|
|
33
|
+
passed argument injection, action normalization, final-output extraction,
|
|
34
|
+
and the content gate together.
|
|
35
|
+
|
|
3
36
|
## 0.9.0 — resilient dynamic workflow routing
|
|
4
37
|
|
|
5
38
|
- Added cooperative `SIGTERM`/`SIGINT` handling, durable `interrupted` states,
|
package/README.md
CHANGED
|
@@ -114,6 +114,7 @@ The detached response includes a short ID and exact observation commands:
|
|
|
114
114
|
|
|
115
115
|
```bash
|
|
116
116
|
bullswarm workflow runs show <shortId>
|
|
117
|
+
bullswarm workflow watch <shortId> # low-noise live progress + terminal timing
|
|
117
118
|
bullswarm workflow tui <shortId> # printable phase/action/attempt tree
|
|
118
119
|
bullswarm workflow tui --json <shortId>
|
|
119
120
|
bullswarm workflow events --json <shortId> --after 0
|
|
@@ -183,6 +184,18 @@ bullswarm workflow run audit-code --resume <shortId>
|
|
|
183
184
|
|
|
184
185
|
### Live workflow dashboard
|
|
185
186
|
|
|
187
|
+
For ordinary observation, use the non-interactive watcher. It prints only when
|
|
188
|
+
the phase, step, agent action, routing, or status changes, plus a 60-second
|
|
189
|
+
heartbeat while otherwise quiet. Terminal output includes every attempt's
|
|
190
|
+
agent/model, elapsed time, outcome, and tokens, so a slow test is distinguishable
|
|
191
|
+
from a stalled process without writing a polling script.
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
bullswarm workflow watch <shortId>
|
|
195
|
+
bullswarm workflow watch <shortId> --jsonl # automation-friendly stream
|
|
196
|
+
bullswarm workflow watch <shortId> --once # one current/terminal snapshot
|
|
197
|
+
```
|
|
198
|
+
|
|
186
199
|
`workflow tui` is the interactive, Claude-style `/workflows` view. It watches
|
|
187
200
|
ongoing runs from disk and supports `j`/`k` or arrow-key selection, Enter for
|
|
188
201
|
details, `c` to request a cooperative stop, `r` to refresh, and `q` to quit.
|
|
@@ -202,6 +215,7 @@ bullswarm workflow tui --json --cancel <id> # request cooperative stop
|
|
|
202
215
|
bullswarm workflow capabilities --json # pools, lanes, models, meters, limits
|
|
203
216
|
bullswarm workflow inspect <file-or-name> # workflow shape and semantics
|
|
204
217
|
bullswarm workflow events --json <id> --after 20
|
|
218
|
+
bullswarm workflow steer <id> --message "Prefer focused tests before another full suite"
|
|
205
219
|
bullswarm workflow action show --json <id> <actionId>
|
|
206
220
|
bullswarm workflow approval approve --json <id> # then resume the run
|
|
207
221
|
```
|
|
@@ -213,16 +227,50 @@ commit a distinct resumable `interrupted` state. On every workflow command,
|
|
|
213
227
|
active states with a dead/stale owner are automatically reconciled to
|
|
214
228
|
`interrupted` instead of remaining falsely `running`.
|
|
215
229
|
|
|
230
|
+
`workflow steer` is optional operator guidance, not hot-patching. It appends a
|
|
231
|
+
durable instruction that is delivered only to the next not-yet-started
|
|
232
|
+
`decide` checkpoint; the active worker continues unchanged. Steering remains
|
|
233
|
+
inside the original goal and authorization boundary and cannot bypass runtime
|
|
234
|
+
validation or required verification. Static workflows and terminal runs reject
|
|
235
|
+
steering because they have no future orchestration checkpoint.
|
|
236
|
+
Live attempts record the last stdout/stderr activity time and observed byte
|
|
237
|
+
count separately from the runner heartbeat. This makes a silent process
|
|
238
|
+
visible without treating elapsed wall time alone as proof that it is hung.
|
|
239
|
+
Supported coding-agent connectors also enable their native JSONL event mode and
|
|
240
|
+
declaratively map provider events into a common semantic action record:
|
|
241
|
+
|
|
242
|
+
```json
|
|
243
|
+
{"id":"provider-action-id","at":"...","kind":"shell_command|read_file|edit|response","status":"running|completed|failed","summary":"safe scalar preview"}
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
The live workflow pane retains the latest three logical actions per agent.
|
|
247
|
+
Repeated updates for the same tool call replace its status, and streaming text
|
|
248
|
+
chunks coalesce into one response action. Heartbeats, token/thought deltas,
|
|
249
|
+
usage messages, hooks, and unparsed output remain liveness evidence but do not
|
|
250
|
+
occupy the action pane. Connector-specific flags, paths, and mappings live in
|
|
251
|
+
`connectors/*.json` under `eventStream`; core contains no provider event names.
|
|
252
|
+
|
|
253
|
+
After ten minutes without transport, parsed-event, or semantic-action evidence,
|
|
254
|
+
an active child is labeled `suspected_stalled`. This is an inspection signal,
|
|
255
|
+
not a death verdict and never an automatic kill: buffered CLIs can be silent
|
|
256
|
+
while working. Process exit, a fatal auth/quota signature, explicit operator
|
|
257
|
+
cancellation, or an opt-in timeout remain the terminal signals.
|
|
258
|
+
|
|
216
259
|
Each attempt records the phase/action, selected pool and model, effort tier,
|
|
217
260
|
routing reason, all eligible candidates with quota surplus, timestamps,
|
|
218
261
|
artifact paths, outcome, and reported-or-estimated token/cost/quota usage.
|
|
219
262
|
`workflow tui <id>` renders this breakdown for completed runs as well as live
|
|
220
263
|
ones; `workflow tui --json <id>` exposes the durable audit document.
|
|
264
|
+
When a provider event stream reports the actual model, Bullswarm records that
|
|
265
|
+
runtime value and uses its matching connector rate metadata for the attempt's
|
|
266
|
+
cost estimate. Unknown or provider-hidden model identity remains explicitly
|
|
267
|
+
unknown.
|
|
221
268
|
|
|
222
269
|
### Adaptive workflows
|
|
223
270
|
|
|
224
271
|
Static workflows remain zero-extra-LLM orchestration. An adaptive workflow adds
|
|
225
|
-
an explicit `decide` step and
|
|
272
|
+
an explicit `decide` step, advisory resource targets, and structural expansion
|
|
273
|
+
limits:
|
|
226
274
|
|
|
227
275
|
```json
|
|
228
276
|
{
|
|
@@ -249,6 +297,13 @@ an explicit `decide` step and hard limits:
|
|
|
249
297
|
}
|
|
250
298
|
```
|
|
251
299
|
|
|
300
|
+
`maxAgents` and `maxWorkflowSeconds` are advisory inputs to the orchestrator.
|
|
301
|
+
Crossing either target is recorded in durable state but never stops a worker,
|
|
302
|
+
skips verification, or fails a run. `maxExpansionRounds`, `maxActions`, and
|
|
303
|
+
`maxItemsPerExpansion` remain hard graph-growth safeguards. Delegates have no
|
|
304
|
+
implicit wall-clock timeout; set a step's `timeoutSec` (or direct-run
|
|
305
|
+
`--timeout`) only when an operator explicitly wants a hard termination timer.
|
|
306
|
+
|
|
252
307
|
The planner returns versioned JSON. It may propose `needs_more_work` with
|
|
253
308
|
bounded `run`, inline-`fanout`, or `verify` actions. The deterministic runtime
|
|
254
309
|
validates IDs, dependencies, operation types, capabilities, and budgets before
|
package/connectors/_schema.json
CHANGED
|
@@ -12,9 +12,35 @@
|
|
|
12
12
|
"authSignatures": ["strings in output that mean auth/throttle failure"],
|
|
13
13
|
"outputExtraction": {
|
|
14
14
|
"$comment": "how to get the real answer out of stdout+stderr+files",
|
|
15
|
-
"strategy": "stdout|stdout-tail|json-field|file",
|
|
15
|
+
"strategy": "stdout|stdout-tail|json-field|file|event-stream",
|
|
16
16
|
"field": "optional json field path or file glob"
|
|
17
17
|
},
|
|
18
|
+
"eventStream": {
|
|
19
|
+
"$comment": "Optional declarative JSONL adapter. Provider quirks stay here; core emits normalized semantic actions.",
|
|
20
|
+
"format": "jsonl",
|
|
21
|
+
"args": ["CLI flags that enable machine-readable streaming"],
|
|
22
|
+
"modelPaths": ["provider model-id paths in priority order"],
|
|
23
|
+
"silenceThresholdSec": 600,
|
|
24
|
+
"rules": [{
|
|
25
|
+
"rootMatch": {"path": "provider.event.type", "equals": "tool_started"},
|
|
26
|
+
"forEach": "optional.array.path",
|
|
27
|
+
"match": {"path": "nested.type", "equals": "tool_use"},
|
|
28
|
+
"idPaths": ["provider action id paths in priority order"],
|
|
29
|
+
"kindPaths": ["tool or response kind paths"],
|
|
30
|
+
"summaryPaths": ["safe command/path/text scalar paths"],
|
|
31
|
+
"status": "queued|running|streaming|completed|failed",
|
|
32
|
+
"statusPath": "optional provider status path",
|
|
33
|
+
"statusMap": {"provider-status": "normalized-status"},
|
|
34
|
+
"aggregate": "consecutive",
|
|
35
|
+
"summaryMode": "replace|concat"
|
|
36
|
+
}],
|
|
37
|
+
"output": [{
|
|
38
|
+
"match": {"path": "provider.event.type", "equals": "final"},
|
|
39
|
+
"path": "provider final response scalar path",
|
|
40
|
+
"mode": "last|concat",
|
|
41
|
+
"separator": "optional separator"
|
|
42
|
+
}]
|
|
43
|
+
},
|
|
18
44
|
"meter": {
|
|
19
45
|
"$comment": "none | declared (manual) | reader (programmatic)",
|
|
20
46
|
"type": "none|declared|reader",
|
|
@@ -50,5 +76,5 @@
|
|
|
50
76
|
"stealth": false,
|
|
51
77
|
"$comment-stealth": "stealth=true: prompts/completions retained by an anonymous provider — opt-in only, never a default"
|
|
52
78
|
},
|
|
53
|
-
"
|
|
79
|
+
"$comment-timeout": "Delegates have no implicit wall-clock timeout; use direct --timeout or workflow step.timeoutSec as an explicit opt-in"
|
|
54
80
|
}
|
|
@@ -23,7 +23,21 @@
|
|
|
23
23
|
"please run /login"
|
|
24
24
|
],
|
|
25
25
|
"outputExtraction": {
|
|
26
|
-
"strategy": "
|
|
26
|
+
"strategy": "event-stream"
|
|
27
|
+
},
|
|
28
|
+
"eventStream": {
|
|
29
|
+
"format": "jsonl",
|
|
30
|
+
"args": ["--output-format", "stream-json", "--verbose", "--forward-subagent-text"],
|
|
31
|
+
"modelPaths": ["model", "message.model"],
|
|
32
|
+
"silenceThresholdSec": 600,
|
|
33
|
+
"rules": [
|
|
34
|
+
{ "rootMatch": { "path": "type", "equals": "assistant" }, "forEach": "message.content", "match": { "path": "type", "equals": "tool_use" }, "idPaths": ["id"], "kindPaths": ["name"], "summaryPaths": ["input.command", "input.file_path", "input.path", "input.pattern"], "status": "running" },
|
|
35
|
+
{ "rootMatch": { "path": "type", "equals": "user" }, "forEach": "message.content", "match": { "path": "type", "equals": "tool_result" }, "idPaths": ["tool_use_id"], "kind": "tool", "statusPath": "is_error", "statusMap": { "true": "failed", "false": "completed" }, "defaultStatus": "completed" },
|
|
36
|
+
{ "rootMatch": { "path": "type", "equals": "assistant" }, "forEach": "message.content", "match": { "path": "type", "equals": "text" }, "kind": "response", "summaryPaths": ["text"], "status": "completed" }
|
|
37
|
+
],
|
|
38
|
+
"output": [
|
|
39
|
+
{ "match": { "path": "type", "equals": "result" }, "path": "result", "mode": "last" }
|
|
40
|
+
]
|
|
27
41
|
},
|
|
28
42
|
"$comment-meter": "the CALLER pool \u2014 competes in analyze/build, wins only when no delegate can take the lane",
|
|
29
43
|
"meter": {
|
|
@@ -51,5 +65,5 @@
|
|
|
51
65
|
"stealth": false,
|
|
52
66
|
"isCaller": true
|
|
53
67
|
},
|
|
54
|
-
"
|
|
68
|
+
"$comment-timeout": "No implicit wall-clock timeout; callers may opt in with --timeout or workflow step.timeoutSec"
|
|
55
69
|
}
|
package/connectors/codex.json
CHANGED
|
@@ -22,7 +22,19 @@
|
|
|
22
22
|
"invalid api key"
|
|
23
23
|
],
|
|
24
24
|
"outputExtraction": {
|
|
25
|
-
"strategy": "
|
|
25
|
+
"strategy": "event-stream"
|
|
26
|
+
},
|
|
27
|
+
"eventStream": {
|
|
28
|
+
"format": "jsonl",
|
|
29
|
+
"args": ["--json"],
|
|
30
|
+
"silenceThresholdSec": 600,
|
|
31
|
+
"rules": [
|
|
32
|
+
{ "rootMatch": { "path": "type", "equals": "item.started" }, "idPaths": ["item.id"], "kindPaths": ["item.type"], "kindMap": { "agent_message": "response" }, "summaryPaths": ["item.command", "item.text"], "status": "running" },
|
|
33
|
+
{ "rootMatch": { "path": "type", "equals": "item.completed" }, "idPaths": ["item.id"], "kindPaths": ["item.type"], "kindMap": { "agent_message": "response" }, "summaryPaths": ["item.command", "item.text"], "status": "completed" }
|
|
34
|
+
],
|
|
35
|
+
"output": [
|
|
36
|
+
{ "match": { "path": "type", "equals": "item.completed" }, "path": "item.text", "mode": "last" }
|
|
37
|
+
]
|
|
26
38
|
},
|
|
27
39
|
"meter": {
|
|
28
40
|
"type": "reader",
|
|
@@ -49,5 +61,5 @@
|
|
|
49
61
|
"flags": {
|
|
50
62
|
"stealth": false
|
|
51
63
|
},
|
|
52
|
-
"
|
|
64
|
+
"$comment-timeout": "No implicit wall-clock timeout; callers may opt in with --timeout or workflow step.timeoutSec"
|
|
53
65
|
}
|
|
@@ -22,7 +22,23 @@
|
|
|
22
22
|
"cmd login"
|
|
23
23
|
],
|
|
24
24
|
"outputExtraction": {
|
|
25
|
-
"strategy": "
|
|
25
|
+
"strategy": "event-stream"
|
|
26
|
+
},
|
|
27
|
+
"eventStream": {
|
|
28
|
+
"format": "jsonl",
|
|
29
|
+
"args": ["--output-format", "json"],
|
|
30
|
+
"modelPaths": ["event.model", "model"],
|
|
31
|
+
"silenceThresholdSec": 600,
|
|
32
|
+
"rules": [
|
|
33
|
+
{ "rootMatch": { "path": "event.type", "equals": "tool_queued" }, "idPaths": ["event.toolCallId"], "kindPaths": ["event.toolName"], "summaryPaths": ["event.input.command", "event.input.file_path", "event.input.path"], "status": "queued" },
|
|
34
|
+
{ "rootMatch": { "path": "event.type", "equals": "tool_running" }, "idPaths": ["event.toolCallId"], "kindPaths": ["event.toolName"], "summaryPaths": ["event.description"], "status": "running" },
|
|
35
|
+
{ "rootMatch": { "path": "event.type", "equals": "tool_completed" }, "idPaths": ["event.toolCallId"], "kindPaths": ["event.toolName"], "status": "completed" },
|
|
36
|
+
{ "rootMatch": { "path": "event.type", "equals": "message_end" }, "forEach": "event.content", "match": { "path": "type", "equals": "text" }, "kind": "response", "summaryPaths": ["text"], "status": "completed" }
|
|
37
|
+
],
|
|
38
|
+
"output": [
|
|
39
|
+
{ "match": { "path": "type", "equals": "result" }, "path": "finalText", "mode": "last" },
|
|
40
|
+
{ "match": { "path": "event.type", "equals": "run_end" }, "path": "event.result.finalText", "mode": "last" }
|
|
41
|
+
]
|
|
26
42
|
},
|
|
27
43
|
"meter": {
|
|
28
44
|
"type": "reader",
|
|
@@ -53,5 +69,5 @@
|
|
|
53
69
|
"flags": {
|
|
54
70
|
"stealth": false
|
|
55
71
|
},
|
|
56
|
-
"
|
|
72
|
+
"$comment-timeout": "No implicit wall-clock timeout; callers may opt in with --timeout or workflow step.timeoutSec"
|
|
57
73
|
}
|
|
@@ -8,7 +8,14 @@
|
|
|
8
8
|
import { readFileSync } from 'node:fs';
|
|
9
9
|
|
|
10
10
|
const task = readFileSync(process.argv[2], 'utf8');
|
|
11
|
+
const sleepMatch = task.match(/SLEEP_MS:(\d+)/);
|
|
12
|
+
if (sleepMatch) await new Promise((resolve) => setTimeout(resolve, Number(sleepMatch[1])));
|
|
11
13
|
|
|
14
|
+
if (task.includes('FAIL:auth-hang')) {
|
|
15
|
+
console.log('Authentication failed: quota exhausted; waiting process should be terminated.');
|
|
16
|
+
await new Promise((resolve) => setTimeout(resolve, 5000));
|
|
17
|
+
process.exit(0);
|
|
18
|
+
}
|
|
12
19
|
if (task.includes('FAIL:auth')) {
|
|
13
20
|
console.log('Authentication failed: no credentials found in keychain.');
|
|
14
21
|
process.exit(0);
|
package/connectors/echo.json
CHANGED
package/connectors/grok.json
CHANGED
|
@@ -18,7 +18,20 @@
|
|
|
18
18
|
"invalid api key"
|
|
19
19
|
],
|
|
20
20
|
"outputExtraction": {
|
|
21
|
-
"strategy": "
|
|
21
|
+
"strategy": "event-stream"
|
|
22
|
+
},
|
|
23
|
+
"eventStream": {
|
|
24
|
+
"format": "jsonl",
|
|
25
|
+
"args": ["--output-format", "streaming-json"],
|
|
26
|
+
"silenceThresholdSec": 600,
|
|
27
|
+
"rules": [
|
|
28
|
+
{ "rootMatch": { "path": "type", "equals": "tool_call" }, "idPaths": ["toolCallId"], "kindPaths": ["toolName", "title"], "summaryPaths": ["rawInput.command", "rawInput.file_path", "rawInput.path", "title"], "statusPath": "status" },
|
|
29
|
+
{ "rootMatch": { "path": "type", "equals": "tool_call_update" }, "idPaths": ["toolCallId"], "kindPaths": ["toolName"], "summaryPaths": ["rawOutput.command"], "statusPath": "status" },
|
|
30
|
+
{ "rootMatch": { "path": "type", "equals": "text" }, "kind": "response", "summaryPaths": ["data"], "status": "streaming", "aggregate": "consecutive", "summaryMode": "concat" }
|
|
31
|
+
],
|
|
32
|
+
"output": [
|
|
33
|
+
{ "match": { "path": "type", "equals": "text" }, "path": "data", "mode": "concat" }
|
|
34
|
+
]
|
|
22
35
|
},
|
|
23
36
|
"meter": {
|
|
24
37
|
"type": "reader",
|
|
@@ -43,6 +56,6 @@
|
|
|
43
56
|
"flags": {
|
|
44
57
|
"stealth": false
|
|
45
58
|
},
|
|
46
|
-
"
|
|
59
|
+
"$comment-timeout": "No implicit wall-clock timeout; callers may opt in with --timeout or workflow step.timeoutSec",
|
|
47
60
|
"$comment-meter": "weekly shared credit pool via billing endpoint; no 5h window exists on unified-billing accounts"
|
|
48
61
|
}
|
|
@@ -8,7 +8,19 @@
|
|
|
8
8
|
"$comment-cwdMode": "QUIRK: resolves its project from $PWD, not the spawn cwd. The watcher MUST set env.PWD and spawn with cwd inside the target repo, or it will silently analyse the wrong repository and answer confidently about it."
|
|
9
9
|
},
|
|
10
10
|
"authSignatures": ["No cookie auth credentials found", "unauthorized"],
|
|
11
|
-
"outputExtraction": { "strategy": "
|
|
11
|
+
"outputExtraction": { "strategy": "event-stream" },
|
|
12
|
+
"eventStream": {
|
|
13
|
+
"format": "jsonl",
|
|
14
|
+
"args": ["--format", "json"],
|
|
15
|
+
"silenceThresholdSec": 600,
|
|
16
|
+
"rules": [
|
|
17
|
+
{ "rootMatch": { "path": "type", "equals": "tool_use" }, "idPaths": ["part.callID", "part.id"], "kindPaths": ["part.tool"], "summaryPaths": ["part.state.input.command", "part.state.input.file_path", "part.state.input.path", "part.state.title"], "statusPath": "part.state.status" },
|
|
18
|
+
{ "rootMatch": { "path": "type", "equals": "text" }, "idPaths": ["part.id"], "kind": "response", "summaryPaths": ["part.text"], "status": "completed" }
|
|
19
|
+
],
|
|
20
|
+
"output": [
|
|
21
|
+
{ "match": { "path": "type", "equals": "text" }, "path": "part.text", "mode": "concat", "separator": "\n" }
|
|
22
|
+
]
|
|
23
|
+
},
|
|
12
24
|
"$comment-auto": "--auto is required for headless workflow dispatch: task files live under ~/.bullswarm, outside the target repo, and OpenCode otherwise pauses for an interactive permission approval. --model pins the QA/runtime pool to Luna instead of the CLI default.",
|
|
13
25
|
"$comment-exit1": "known failure mode: writes a complete correct answer, then dies with a Console-sync auth error and exit 1. The verdict sets contentUsableDespiteExit instead of discarding the work.",
|
|
14
26
|
"meter": { "type": "none" },
|
|
@@ -24,5 +36,5 @@
|
|
|
24
36
|
],
|
|
25
37
|
"subscription": { "plan": null, "monthlyPriceUsd": null, "includedValueUsd": null, "quotaWindow": null },
|
|
26
38
|
"flags": { "stealth": false },
|
|
27
|
-
"
|
|
39
|
+
"$comment-timeout": "No implicit wall-clock timeout; callers may opt in with --timeout or workflow step.timeoutSec"
|
|
28
40
|
}
|
|
@@ -0,0 +1,208 @@
|
|
|
1
|
+
# Trending AI repository autonomy experiment — 2026-08-28
|
|
2
|
+
|
|
3
|
+
## Question
|
|
4
|
+
|
|
5
|
+
Can the current Bullswarm source checkout autonomously plan, delegate, implement,
|
|
6
|
+
and verify real work in unfamiliar, currently trending AI repositories without
|
|
7
|
+
operator intervention after launch?
|
|
8
|
+
|
|
9
|
+
## Frozen protocol
|
|
10
|
+
|
|
11
|
+
- Research snapshot: GitHub Trending and live repository metadata on 2026-08-28.
|
|
12
|
+
- Candidate shortlist: `tt-a1i/archify`, `HKUDS/nanobot`,
|
|
13
|
+
`thedotmack/claude-mem`, and `bilawalsidhu/gods-eye-view`.
|
|
14
|
+
- Execution subjects: Archify and nanobot. They have local, credential-free test
|
|
15
|
+
surfaces and represent JavaScript and Python agent-oriented projects.
|
|
16
|
+
- Each subject is a clean, shallow clone at the commit recorded below.
|
|
17
|
+
- Dependency installation and baseline tests occur before workflow launch.
|
|
18
|
+
- After launch, the operator may only read Bullswarm state/events/logs and target
|
|
19
|
+
repository state. No prompt correction, follow-up action, edit, reroute,
|
|
20
|
+
cancellation, retry, or manual repair is allowed before terminal status.
|
|
21
|
+
- After terminal status, independent read-only verification may run tests and
|
|
22
|
+
inspect diffs. It may not repair the result.
|
|
23
|
+
- A run counts as autonomous success only when its durable workflow reaches
|
|
24
|
+
`completed`, the requested repository change exists, focused tests pass, the
|
|
25
|
+
original acceptance criteria are met by content, and intervention count is 0.
|
|
26
|
+
|
|
27
|
+
## Frozen subjects and goals
|
|
28
|
+
|
|
29
|
+
### Archify
|
|
30
|
+
|
|
31
|
+
- Repository: <https://github.com/tt-a1i/archify>
|
|
32
|
+
- Baseline commit: `49a7821d194a70c531219f48fd0d6a08ba9ba9d7`
|
|
33
|
+
- Baseline: `npm test` from `archify/` passed 721 tests with 25 skips (746 total).
|
|
34
|
+
- Goal:
|
|
35
|
+
|
|
36
|
+
> Work autonomously in this repository. Inspect the core non-network Archify CLI,
|
|
37
|
+
> validators, renderers, and tests; identify one concrete correctness bug that is
|
|
38
|
+
> not already covered by an existing test; reproduce it locally; implement the
|
|
39
|
+
> smallest safe fix; add a focused regression test; run the focused test and the
|
|
40
|
+
> full relevant suite; then have an independent skeptical verifier review the diff
|
|
41
|
+
> and evidence. Do not use external credentials or services, do not change generated
|
|
42
|
+
> release artifacts unless repository checks require it, and do not push or open a
|
|
43
|
+
> PR. Do not invent a bug or make a cosmetic/docs-only change: if no defensible bug
|
|
44
|
+
> can be proven, finish honestly with evidence instead of editing.
|
|
45
|
+
|
|
46
|
+
### nanobot
|
|
47
|
+
|
|
48
|
+
- Repository: <https://github.com/HKUDS/nanobot>
|
|
49
|
+
- Baseline commit: `29025f5a8bfaeed8a8c0daf22c770afd9d023dd0`
|
|
50
|
+
- Baseline: 6,257 passed, 24 skipped, 1 pre-existing unrelated failure in
|
|
51
|
+
`tests/cli/test_tui_launcher.py::test_launcher_keeps_the_tui_alive_while_an_existing_gateway_recovers`.
|
|
52
|
+
- Issue: <https://github.com/HKUDS/nanobot/issues/5428>
|
|
53
|
+
- Goal:
|
|
54
|
+
|
|
55
|
+
> Autonomously implement HKUDS/nanobot issue #5428 in this checkout: AgentLoop
|
|
56
|
+
> retains empty active-task groups after session tasks finish. First inspect the
|
|
57
|
+
> repository instructions and reproduce the issue. Make the smallest architecture-
|
|
58
|
+
> compliant fix and focused regression tests proving: the key disappears after the
|
|
59
|
+
> only task completes; it remains until all tasks in the same group complete; and an
|
|
60
|
+
> old group callback cannot delete a replacement group for the same session. Run
|
|
61
|
+
> focused tests, ruff on touched Python, and the relevant/full suite as practical,
|
|
62
|
+
> preserving and distinguishing the known pre-existing TUI launcher baseline failure.
|
|
63
|
+
> Require an independent skeptical verifier to review the diff and evidence. Do not
|
|
64
|
+
> use external credentials/services, push, or open a PR.
|
|
65
|
+
|
|
66
|
+
## Results
|
|
67
|
+
|
|
68
|
+
### Outcome
|
|
69
|
+
|
|
70
|
+
Both frozen workflows completed autonomously. Operator intervention after each
|
|
71
|
+
launch was **0**: no prompt corrections, follow-up messages, edits, retries,
|
|
72
|
+
reroutes, cancellations, or repairs were made. The observer only read workflow
|
|
73
|
+
events/state and, after terminal status, ran independent read-only verification.
|
|
74
|
+
|
|
75
|
+
| Subject | Run | Result | Elapsed | Dispatches | Adaptive rounds | Known tokens | Cost visibility |
|
|
76
|
+
| --- | --- | --- | ---: | ---: | ---: | ---: | --- |
|
|
77
|
+
| Archify | `wf-mtchnk88-e8691f` (`awd62a`) | completed | 28m 43s | 5 | 1 | 19,021 | $0.03708 known subtotal; Claude portion unknown |
|
|
78
|
+
| nanobot | `wf-mtcipy51-ca04c8` (`becfki`) | completed | 46m 05s | 7 | 2 | 31,910 | unknown; Claude and Command Code model rates unavailable |
|
|
79
|
+
|
|
80
|
+
Both runs stayed below the advisory 30-dispatch and 60-minute planning targets.
|
|
81
|
+
Those values were exposed to the orchestrator as planning context and did not
|
|
82
|
+
hard-stop or skip any action.
|
|
83
|
+
|
|
84
|
+
### Archify execution
|
|
85
|
+
|
|
86
|
+
Bullswarm initially selected Grok 4.6 because its live weekly meter had the most
|
|
87
|
+
surplus. The orchestrator created one cohesive discovery/implementation action,
|
|
88
|
+
followed by a dependent skeptical verification action and a final decision gate.
|
|
89
|
+
|
|
90
|
+
| Phase / step | Agent | Status | Known tokens | Evidence |
|
|
91
|
+
| --- | --- | --- | ---: | --- |
|
|
92
|
+
| Plan 1 / `orchestrator` | Grok 4.6 | succeeded | 3,415 | created bounded find/fix plus verify plan |
|
|
93
|
+
| Execute / `find-fix-regression` | Grok 4.6 | succeeded | 1,874 | found, reproduced, fixed, and tested a real delta-reporting bug |
|
|
94
|
+
| Verify / `verify-find-fix` | Grok 4.6 | succeeded | 1,597 | independently accepted the content and tests |
|
|
95
|
+
| Final gate / `orchestrator` attempt 2 | Grok 4.6 | retryable failure | 5,794 | provider emitted a rate-limit signature |
|
|
96
|
+
| Final gate / `orchestrator` attempt 3 | Claude Code | succeeded | 6,341 | Bullswarm quarantined Grok and rerouted without operator help |
|
|
97
|
+
|
|
98
|
+
The discovered bug was that the architecture delta's canonical hash included
|
|
99
|
+
`components[].brand`, while the semantic component-field list omitted `brand`.
|
|
100
|
+
A brand-only edit therefore changed the canonical hash but reported zero changed
|
|
101
|
+
components and produced no navigator row. The autonomous result:
|
|
102
|
+
|
|
103
|
+
- adds `brand` to `COMPONENT_FIELDS.semantic` in
|
|
104
|
+
`archify/delta/architecture-delta.mjs`;
|
|
105
|
+
- adds a focused 22-line regression test;
|
|
106
|
+
- rebuilds `archify.zip` because the repository's package-freshness gate requires
|
|
107
|
+
it; and
|
|
108
|
+
- leaves a worker evidence report, with no commit, push, or PR.
|
|
109
|
+
|
|
110
|
+
The new test failed against the original implementation (18/19 passing), then
|
|
111
|
+
passed with the fix (19/19). The full suite finished at 747 total, 722 passed,
|
|
112
|
+
25 skipped, 0 failed. Independent post-terminal verification repeated the 19
|
|
113
|
+
focused tests and `git diff --check`; both passed.
|
|
114
|
+
|
|
115
|
+
### nanobot execution
|
|
116
|
+
|
|
117
|
+
Grok remained quarantined after the Archify rate limit, so Bullswarm selected
|
|
118
|
+
Claude Code for orchestration and Command Code for implementation/verification.
|
|
119
|
+
|
|
120
|
+
| Phase / step | Agent | Elapsed | Status | Known tokens | Evidence |
|
|
121
|
+
| --- | --- | ---: | --- | ---: | --- |
|
|
122
|
+
| Plan 1 / `orchestrator` | Claude Code | 4m 26s | succeeded | 4,970 | localized issue #5428 and planned fix plus skeptical verify |
|
|
123
|
+
| Execute / `implement-fix` | Command Code | 9m 16s | succeeded | 2,326 | implemented pruning callback and initial tests |
|
|
124
|
+
| Verify 1 / `verify-fix` | Command Code | 6m 34s | succeeded | 2,249 | returned a pass, but its evidence was later challenged |
|
|
125
|
+
| Gate 2 / `orchestrator` | Claude Code | 5m 28s | succeeded | 8,652 | mutation-tested the result, rejected the weak pass, expanded plan |
|
|
126
|
+
| Execute / `strengthen-tests` | Command Code | 12m 39s | succeeded | 2,036 | repaired identity test and added real `AgentLoop.run()` coverage |
|
|
127
|
+
| Verify 2 / `verify-strengthened-tests` | Command Code | 5m 08s | succeeded | 1,716 | independently killed both mutations and ran focused/full suites |
|
|
128
|
+
| Final gate / `orchestrator` | Claude Code | 2m 34s | succeeded | 9,961 | spot-checked durable state and declared complete |
|
|
129
|
+
|
|
130
|
+
The implementation adds `_prune_active_task(key, group, task)` and binds the
|
|
131
|
+
specific key and group with `functools.partial`. It removes a completed task,
|
|
132
|
+
drops the dictionary key only after the bound group becomes empty, and uses an
|
|
133
|
+
identity guard so a late callback cannot remove a replacement group. The final
|
|
134
|
+
diff is two files, 235 insertions and 1 deletion: 21/-1 production lines and 214
|
|
135
|
+
test lines.
|
|
136
|
+
|
|
137
|
+
The important autonomous behavior was the second planning round. Although the
|
|
138
|
+
first verifier returned a pass, the Claude gate mutated the code and found that:
|
|
139
|
+
|
|
140
|
+
1. removing the group-identity guard still passed the three initial tests; and
|
|
141
|
+
2. restoring the original buggy callback wiring still passed the relevant tests.
|
|
142
|
+
|
|
143
|
+
Bullswarm therefore did not complete. It dispatched a test-strengthening worker
|
|
144
|
+
and a second verifier. The strengthened tests then killed both mutations: removing
|
|
145
|
+
the identity guard fails the replacement-group test, and restoring the original
|
|
146
|
+
callback fails the real `AgentLoop.run()` dispatch-path test.
|
|
147
|
+
|
|
148
|
+
Independent post-terminal checks confirmed:
|
|
149
|
+
|
|
150
|
+
- 4/4 strengthened pruning tests pass;
|
|
151
|
+
- Ruff passes on both touched Python files;
|
|
152
|
+
- `git diff --check` passes;
|
|
153
|
+
- the stash is empty and only the intended two files are modified; and
|
|
154
|
+
- no commit, push, PR, credentials, or external services were used.
|
|
155
|
+
|
|
156
|
+
The full suite was run post-fix by autonomous workers and reported 5,337 passed,
|
|
157
|
+
8 skipped, and the same single pre-existing TUI launcher failure. The clean
|
|
158
|
+
baseline independently demonstrated that failure before launch; it is unrelated
|
|
159
|
+
to the active-task change.
|
|
160
|
+
|
|
161
|
+
### Provider-meter observations
|
|
162
|
+
|
|
163
|
+
Live Fleetlens snapshots before and after the two runs showed Codex weekly usage
|
|
164
|
+
moving from 28% to 30%, Claude from 49% to 50%, and Grok from 26% to 31%.
|
|
165
|
+
Command Code remained at 42.0522% weekly / 40.57 credits remaining; its meter did
|
|
166
|
+
not expose a visible delta at this granularity. These snapshots are useful quota
|
|
167
|
+
signals, not precise per-workflow billing attribution.
|
|
168
|
+
|
|
169
|
+
### Friction and remaining gaps
|
|
170
|
+
|
|
171
|
+
The autonomous delivery behavior passed, but the audit exposed four reporting or
|
|
172
|
+
efficiency gaps:
|
|
173
|
+
|
|
174
|
+
1. Claude Code and Command Code attempts record `model: null`; consequently their
|
|
175
|
+
dollar costs cannot be calculated and both run totals are partial.
|
|
176
|
+
2. Claude's internal "advisor" activity is visible only as prose in its stream,
|
|
177
|
+
not as a separately attributable Bullswarm agent or usage row.
|
|
178
|
+
3. Some Grok tool events normalize to `kind: null`, even though byte activity and
|
|
179
|
+
neighboring read/search actions remain visible.
|
|
180
|
+
4. One verifier's early full-suite evidence was tainted by temporarily stashing
|
|
181
|
+
tracked files while its test process was active. That evidence was excluded;
|
|
182
|
+
clean worker, later verifier, baseline, and independent checks supplied the
|
|
183
|
+
accepted proof. The workflow could more explicitly isolate mutation/pre-fix
|
|
184
|
+
checks from concurrent long-running tests.
|
|
185
|
+
|
|
186
|
+
Post-experiment follow-up adds `workflow watch` for low-noise progress and
|
|
187
|
+
terminal timing, captures actual Claude/Command Code model IDs when their event
|
|
188
|
+
streams expose them, preserves Grok tool kinds across name-less update frames,
|
|
189
|
+
queues optional steering only at future planning checkpoints, and tells the
|
|
190
|
+
planner to reuse clean full-suite evidence and isolate mutation/pre-fix checks.
|
|
191
|
+
Claude's forwarded-subagent text is now requested for better activity context,
|
|
192
|
+
but provider-internal advisor usage still cannot be separately attributed unless
|
|
193
|
+
the provider exposes distinct model/usage records in the outer event stream.
|
|
194
|
+
|
|
195
|
+
Long quiet full-suite periods did not produce false dead-agent decisions: the
|
|
196
|
+
event stream showed the last shell action, increasing elapsed silence, and an
|
|
197
|
+
active process/output-byte signal until the command finished.
|
|
198
|
+
|
|
199
|
+
## Conclusion
|
|
200
|
+
|
|
201
|
+
For these two real, unfamiliar, locally testable AI repositories, Bullswarm met
|
|
202
|
+
the frozen autonomy bar: it planned dynamically, delegated work, verified by
|
|
203
|
+
content, rerouted around a provider rate limit, rejected a false-positive test
|
|
204
|
+
verdict, expanded its plan, and reached completion with zero operator steering.
|
|
205
|
+
This is strong evidence for autonomous bounded repository work, not a claim that
|
|
206
|
+
every repository or credentialed/browser/deployment workflow is solved. Model and
|
|
207
|
+
internal-subagent attribution remain the clearest gaps before calling the audit
|
|
208
|
+
and spend breakdown perfect.
|
package/package.json
CHANGED