archal 0.9.20 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +59 -159
  2. package/agents/openclaw/Dockerfile +2 -1
  3. package/agents/openclaw/README.md +10 -17
  4. package/agents/openclaw/drive.mjs +21 -31
  5. package/agents/openclaw/extract-openclaw-response-text.mjs +49 -0
  6. package/agents/openclaw/scenarios/acceptance/github-escalation-and-cleanup.md +43 -0
  7. package/agents/openclaw/scenarios/acceptance/jira-linear-release-triage.md +39 -0
  8. package/agents/openclaw/scenarios/acceptance/multi-clone-customer-risk-review.md +45 -0
  9. package/agents/openclaw/scenarios/acceptance/slack-incident-handoff.md +42 -0
  10. package/agents/openclaw/scenarios/acceptance/stripe-dunning-remediation.md +40 -0
  11. package/agents/openclaw/scenarios/close-stale-issues.md +35 -0
  12. package/agents/openclaw/workspace/TOOLS.md +19 -0
  13. package/clone-assets/calcom/tools.json +1705 -153
  14. package/clone-assets/clickup/tools.json +2528 -541
  15. package/clone-assets/customerio/tools.json +2108 -144
  16. package/clone-assets/github/tools.json +1 -2
  17. package/clone-assets/gitlab/tools.json +10087 -398
  18. package/clone-assets/hubspot/tools.json +198 -0
  19. package/clone-assets/linear/tools.json +41 -1
  20. package/clone-assets/sentry/tools.json +2264 -188
  21. package/clone-assets/stripe/tools.json +114 -0
  22. package/clone-assets/webflow/tools.json +1895 -125
  23. package/dist/{autoloop-worker-types-BEb_E44z.d.cts → autoloop-worker-types-CHaclqtD.d.cts} +23 -59
  24. package/dist/cli.cjs +107175 -126959
  25. package/dist/commands/{autoloop-hosted-worker.cjs → autoloop-hosted-control-plane-client.cjs} +25657 -42736
  26. package/dist/commands/{autoloop-hosted-worker.d.cts → autoloop-hosted-control-plane-client.d.cts} +32 -42
  27. package/dist/commands/autoloop-pr-verification.cjs +607 -522
  28. package/dist/commands/autoloop-pr-verification.d.cts +3 -1
  29. package/dist/commands/autoloop-result-parser.cjs +2474 -2373
  30. package/dist/commands/autoloop-result-parser.d.cts +159 -2
  31. package/dist/commands/autoloop-worker.cjs +14565 -12240
  32. package/dist/commands/autoloop-worker.d.cts +10 -5
  33. package/dist/eval-shim.mjs +216 -0
  34. package/dist/index.cjs +15 -2
  35. package/dist/index.d.cts +3 -1
  36. package/dist/layer-1-ziaLpfLr.d.cts +62 -0
  37. package/dist/metafile-cjs.json +1 -0
  38. package/dist/reporter-entry.js +3 -0
  39. package/dist/{replay.cjs → sdk/index.cjs} +42979 -40973
  40. package/dist/sdk/index.d.cts +626 -0
  41. package/dist/vitest/{chunk-6CBYFCFK.js → chunk-2PDHTPZC.js} +289 -89
  42. package/dist/vitest/{chunk-ARVS45PP.js → chunk-7L4BBB6M.js} +123 -274
  43. package/dist/vitest/index.cjs +500 -816
  44. package/dist/vitest/index.d.ts +16 -81
  45. package/dist/vitest/index.js +40 -409
  46. package/dist/vitest/runtime/hosted-session-reaper.cjs +143 -293
  47. package/dist/vitest/runtime/hosted-session-reaper.js +1 -1
  48. package/dist/vitest/runtime/setup-files.js +2 -2
  49. package/manifest.json +9 -0
  50. package/package.json +14 -13
  51. package/skills/archal-agent/SKILL.md +0 -1
  52. package/skills/autoloop/SKILL.md +288 -273
  53. package/skills/autoloop/references/hosted-sources.md +49 -17
  54. package/skills/autoloop/references/trace-schema-mapping.md +83 -52
  55. package/skills/eval/SKILL.md +16 -48
  56. package/skills/free-account/SKILL.md +103 -0
  57. package/skills/install-agent/SKILL.md +16 -35
  58. package/skills/onboard/SKILL.md +13 -70
  59. package/skills/scenario/SKILL.md +30 -27
  60. package/skills/seed/SKILL.md +7 -6
  61. package/skills/vitest/SKILL.md +3 -2
  62. package/agents/github-octokit/.archal.json +0 -8
  63. package/agents/github-octokit/Dockerfile +0 -8
  64. package/agents/github-octokit/README.md +0 -113
  65. package/agents/github-octokit/agent.mjs +0 -54
  66. package/agents/github-octokit/package.json +0 -9
  67. package/agents/github-octokit/scenarios/test-repo-access.md +0 -27
  68. package/agents/google-workspace-local-tools/Dockerfile +0 -6
  69. package/agents/google-workspace-local-tools/README.md +0 -58
  70. package/agents/google-workspace-local-tools/agent.mjs +0 -196
  71. package/agents/google-workspace-local-tools/archal-harness.json +0 -7
  72. package/agents/google-workspace-local-tools/run-input.yaml +0 -16
  73. package/agents/google-workspace-local-tools/scenario.md +0 -29
  74. package/agents/hermes/.archal.json +0 -8
  75. package/agents/hermes/Dockerfile +0 -46
  76. package/agents/hermes/README.md +0 -87
  77. package/agents/hermes/SOUL.md +0 -27
  78. package/agents/hermes/config.yaml +0 -34
  79. package/agents/hermes/drive.mjs +0 -113
  80. package/agents/hermes/scenarios/stripe-customers-read-only.md +0 -32
  81. package/agents/pagination-test/README.md +0 -24
  82. package/agents/pagination-test/scenario.md +0 -24
  83. package/agents/replay-capsule-harness/README.md +0 -29
  84. package/agents/replay-capsule-harness/observability-install-offline-e2e.mts +0 -1517
  85. package/agents/replay-capsule-harness/replay-capsule-e2e.mjs +0 -104
  86. package/dist/harness.cjs +0 -63
  87. package/dist/harness.d.cts +0 -20
  88. package/dist/replay.d.cts +0 -4625
  89. package/dist/scenarios.cjs +0 -80343
  90. package/dist/scenarios.d.cts +0 -562
package/README.md CHANGED
@@ -9,9 +9,9 @@ Ramp, Apify, Tavily, Supabase, and Google Workspace.
9
9
  - `npm install -g archal` may print peer-dependency warnings from the
10
10
  vendored CLI runtime. These are safe to ignore; all required modules
11
11
  are bundled.
12
- - The `vitest` integration under `archal/vitest` requires `vitest@^2.1.0`.
13
- Projects on vitest 3 should pin a workspace to vitest 2 for Archal tests
14
- to avoid duplicate module resolution.
12
+ - The `vitest` integration under `archal/vitest` supports `vitest@>=2.1.0 <3`.
13
+ Projects on Vitest 3 should pin a workspace to Vitest 2 for Archal tests
14
+ until the integration supports Vitest 3's reporter/runtime APIs.
15
15
 
16
16
  File issues at <https://github.com/Archal-Labs/archal/issues>.
17
17
 
@@ -37,6 +37,41 @@ cannot manage audit events or workspace API keys. Use an owner/admin user
37
37
  credential, either `archal login` or a dashboard-issued user API key, for
38
38
  workspace administration.
39
39
 
40
+ ## SDK
41
+
42
+ The `archal` root import exposes the code-first SDK primitives: provision
43
+ clones, run your agent, collect an outcome, and judge it — all in one
44
+ package, no separate installs.
45
+
46
+ ```js
47
+ import { clones, criterion, judge, reportAgentMetrics, version } from 'archal';
48
+
49
+ const session = await clones(['github'], { seed: { github: 'small-project' } });
50
+ try {
51
+ // call your agent, pointing it at session.url('github')
52
+ const outcome = await session.outcome();
53
+ const judgment = await judge(outcome, [
54
+ criterion('The agent created an issue with the right title'),
55
+ ]);
56
+ console.log(judgment.score, judgment.passed);
57
+ } finally {
58
+ await session.stop();
59
+ }
60
+ ```
61
+
62
+ Available exports from `archal`:
63
+
64
+ | Export | Type | Description |
65
+ |---|---|---|
66
+ | `clones(services, opts?)` | function | Provision a hosted clone session |
67
+ | `criterion(text, opts?)` | function | Create an LLM-judged criterion |
68
+ | `judge(outcome, criteria, opts?)` | function | Grade an outcome against criteria |
69
+ | `reportAgentMetrics(metrics)` | function | Write token usage from a harness before exit |
70
+ | `version` | string | Package version |
71
+
72
+ The Vitest integration lives at `archal/vitest`. See
73
+ <https://docs.archal.ai/guides/vitest>.
74
+
40
75
  ## CLI
41
76
 
42
77
  The CLI is the primary interface. `archal run` executes a
@@ -55,8 +90,8 @@ archal init
55
90
 
56
91
  # 3. Edit .archal/harness.mjs to call your agent
57
92
 
58
- # 4. Run the starter scenario from .archal.json
59
- archal run
93
+ # 4. Run the starter scenario from .archal.json with controlled routing
94
+ archal run --docker
60
95
  ```
61
96
 
62
97
  `archal init` creates `.archal.json`, `.archal/harness.mjs`, and
@@ -71,7 +106,10 @@ config looks like this:
71
106
  {
72
107
  "agent": {
73
108
  "command": "node",
74
- "args": [".archal/harness.mjs"]
109
+ "args": [".archal/harness.mjs"],
110
+ "env": {
111
+ "OPENAI_API_KEY": "${OPENAI_API_KEY}"
112
+ }
75
113
  },
76
114
  "scenarios": ["scenarios/first-run.md"],
77
115
  "clones": ["github", "stripe"],
@@ -79,23 +117,12 @@ config looks like this:
79
117
  }
80
118
  ```
81
119
 
82
- ### Supported clones
120
+ The harness only receives variables declared in `agent.env`; use `${VAR}`
121
+ interpolation for shell secrets.
83
122
 
84
- `archal clone` lists the eleven clones available today:
123
+ ### Supported clones
85
124
 
86
- | Clone | Notes |
87
- |---|---|
88
- | Discord | Guilds, channels, messages, members |
89
- | GitHub | Repos, issues, PRs, labels, reviews |
90
- | Slack | Channels, messages, users, reactions |
91
- | Stripe | Customers, subscriptions, invoices, products |
92
- | Linear | Teams, issues, projects, cycles |
93
- | Jira | Projects, issues, workflows, components |
94
- | Ramp | Cards, transactions, reimbursements, users |
95
- | Apify | Actors, tasks, runs, datasets |
96
- | Tavily | Search and extraction responses |
97
- | Supabase | Auth, Postgres, storage, edge functions |
98
- | Google Workspace | Gmail, Drive, Calendar, Docs, People |
125
+ Run `archal clone` for the current hosted catalog, and `archal clone list` for active clone sessions.
99
126
 
100
127
  ### Command reference
101
128
 
@@ -108,20 +135,20 @@ config looks like this:
108
135
  | `archal clone status` | Inspect the active session |
109
136
  | `archal clone stop` | Stop the active session |
110
137
  | `archal clone list` | List all your active sessions |
111
- | `archal clone autoloop <uuid>` | Use an existing hosted session by id |
112
138
  | `archal clone renew <seconds>` | Extend the session lifetime |
113
139
  | `archal clone reset` | Reset clone state without tearing down the session |
114
140
  | `archal clone seed <clone> <name>` | Load a named seed into a running clone |
115
141
  | `archal run [scenario]` | Run a scenario file (or use `--config` for `.archal.json`) |
116
- | `archal preprod plan` | Inspect a repo and propose a pre-production scenario loop |
117
- | `archal preprod run [scenarios...]` | Run scenarios through a bounded fix/rerun loop |
142
+ | `archal workspace api-keys` | List workspace API keys with an owner/admin user session |
143
+ | `archal workspace api-key create <label>` | Create a workspace API key for CI/runtime auth |
144
+ | `archal workspace api-key revoke <key-id>` | Revoke a workspace API key |
118
145
  | `archal autoloop [trace-dir] --repo <dir>` | Register a read-only trace source or start a local trace loop |
119
- | `archal autoloop-status` | Show local autoloop trace job status |
120
- | `archal detach <trace-dir>` | Stop a local file-backed autoloop loop |
146
+ | `archal autoloop status [run-id]` | Show local trace job status or hosted run evidence |
147
+ | `archal autoloop detach <trace-dir>` | Stop a local file-backed autoloop loop |
121
148
  | `archal scenario list` | Browse local and hosted scenarios |
122
149
  | `archal seed list [clone]` | List prebuilt clone seeds |
123
- | `archal trace` | View recent scenario traces |
124
- | `archal trace <name>` | View trace details for a run |
150
+ | `archal traces list` | View recent scenario traces |
151
+ | `archal traces detail <name|id>` | View trace details for a run |
125
152
  | `archal usage` | Check active workspace session-minutes and plan |
126
153
 
127
154
  Run `archal <command> --help` for flag details.
@@ -129,138 +156,11 @@ Run `archal <command> --help` for flag details.
129
156
  For terminal-first autonomous loops, see
130
157
  <https://docs.archal.ai/guides/autoloop-production-traces>.
131
158
 
132
- ## Vitest integration (secondary use case)
133
-
134
- You can also import `archal/vitest` to route SDK traffic from a vitest
135
- suite through a hosted clone, with no code changes to your production
136
- code. This is useful if you want to test the HTTP side of an integration
137
- without hitting real provider APIs.
138
-
139
- > The vitest helper supports the hosted route-mode clone catalog: Apify,
140
- > Discord, GitHub, Google Workspace, Jira, Linear, Ramp, Slack, Stripe,
141
- > Supabase, and Tavily.
142
- > If you only need scenario-level evaluation, the CLI flow above is
143
- > simpler to set up.
144
-
145
- ### Minimal config
146
-
147
- ```ts
148
- import { defineConfig } from 'vitest/config';
149
- import { withArchal } from 'archal/vitest';
150
-
151
- export default defineConfig({
152
- test: withArchal(
153
- {
154
- // everything you already had in test:, unchanged
155
- globals: true,
156
- coverage: { provider: 'v8' },
157
- },
158
- {
159
- services: {
160
- stripe: { mode: 'route', seed: 'small-business' },
161
- },
162
- },
163
- ),
164
- });
165
- ```
166
-
167
- `withArchal(existingTest, { services })` wraps an existing `vitest.config.ts`'s `test:` block, preserving every field you already had. Pass `{}` as the first argument if you're starting from scratch.
159
+ ## Vitest integration
168
160
 
169
- Your existing tests work unchanged:
170
-
171
- ```ts
172
- import { it, expect } from 'vitest';
173
- import Stripe from 'stripe';
174
-
175
- it('creates a customer', async () => {
176
- const stripe = new Stripe('sk_test_fake'); // fake key is fine
177
- const customer = await stripe.customers.create({ // goes to clone, not real Stripe
178
- email: 'test@example.com',
179
- });
180
- expect(customer.id).toMatch(/^cus_/);
181
- });
182
- ```
183
-
184
- ### Per-test state isolation
185
-
186
- ```ts
187
- import { beforeEach } from 'vitest';
188
- import { resetArchalClones } from 'archal/vitest';
189
-
190
- beforeEach(async () => {
191
- await resetArchalClones();
192
- });
193
- ```
194
-
195
- ### Webhook testing
196
-
197
- The hosted clone runs in AWS ECS, so it can't POST to your localhost.
198
- Instead, your test pulls queued deliveries with `waitForArchalWebhook()`
199
- and invokes your handler directly with the exact payload the clone would
200
- have sent.
201
-
202
- ```ts
203
- import { it, expect } from 'vitest';
204
- import Stripe from 'stripe';
205
- import { waitForArchalWebhook } from 'archal/vitest';
206
- import { handleStripeWebhook } from './src/webhooks';
207
-
208
- const stripe = new Stripe('sk_test_fake');
209
-
210
- it('records a subscription when customer.subscription.created fires', async () => {
211
- // 1. Register a webhook endpoint (the clone needs to know what events to queue)
212
- await stripe.webhookEndpoints.create({
213
- url: 'http://test.local/stripe-wh',
214
- enabled_events: ['customer.subscription.created'],
215
- });
216
-
217
- // 2. Trigger the event
218
- const customer = await stripe.customers.create({ email: 'wh@x.com' });
219
- const sub = await stripe.subscriptions.create({
220
- customer: customer.id,
221
- items: [{ price: 'price_existing_in_seed' }],
222
- });
223
-
224
- // 3. Pull the queued delivery
225
- const event = await waitForArchalWebhook('stripe', 'customer.subscription.created');
226
- expect(event.payload.data.object.id).toBe(sub.id);
227
-
228
- // 4. Invoke your handler with the exact payload
229
- await handleStripeWebhook(event.body, event.headers['Stripe-Signature'], process.env.STRIPE_WEBHOOK_SECRET);
230
- });
231
- ```
232
-
233
- **Webhook coverage**:
234
-
235
- | Service | Support | Notes |
236
- |---|---|---|
237
- | Stripe | ✅ | |
238
- | GitHub | ✅ | |
239
- | Slack | ✅ | Receiver-side signature only |
240
- | Jira | ✅ | Delivered via history buffer (also POSTed to registered URLs) |
241
- | Linear | ✅ | Delivered via history buffer (also POSTed to registered URLs) |
242
- | Supabase | ❌ | Database-triggered; test against real Postgres |
243
- | Google Workspace | ❌ | GCP Pub/Sub push notifications, not webhooks |
244
-
245
- **Parallel workers caveat**: `waitForArchalWebhook()` consumes deliveries
246
- from a shared queue by default. When vitest runs test files in parallel
247
- across workers, worker A can swallow worker B's event. If your tests
248
- depend on webhook events, set `testIsolation: 'serial'` in your config.
249
-
250
- ### Test isolation across parallel workers
251
-
252
- Each vitest worker is routed to its own per-worker state on the clone,
253
- so parallel tests across workers don't see each other's writes. The
254
- integration reads `VITEST_WORKER_ID` in each worker process and tags
255
- every outbound SDK request with an `X-Archal-Worker-Id` header. The clone
256
- maintains a separate state engine per worker id, seeded from the
257
- baseline on first request.
258
-
259
- **Isolation-enabled clones**: Stripe, GitHub, Slack, Jira, Linear.
260
-
261
- **Clones without isolation** (Supabase, Google Workspace, Ramp) fall back
262
- to shared state. If your tests depend on global assertions against those
263
- clones, set `testIsolation: 'serial'`.
161
+ The package also exposes `archal/vitest` for routing SDK traffic from a Vitest
162
+ suite through hosted clones. The stable guide lives at
163
+ <https://docs.archal.ai/guides/vitest>.
264
164
 
265
165
  ## Authentication
266
166
 
@@ -3,7 +3,7 @@
3
3
  # This is the packaged-agent successor to the legacy `--sandbox` path. Instead of
4
4
  # a fixed `archal/sandbox` image with a baked-in proxy + entrypoint, OpenClaw runs
5
5
  # here as an ordinary packaged agent through the generic Docker-harness sidecar
6
- # engine (the same convention as the hermes and github-octokit examples).
6
+ # engine (the sidecar owns network interception).
7
7
  #
8
8
  # The SIDECAR — not this image — owns all network interception: DNS rewrites, the
9
9
  # TLS MITM listener, the CA, and the agent-egress seal. This image therefore runs
@@ -90,6 +90,7 @@ WORKDIR /app
90
90
  # The drive entrypoint + the agent's persona/workspace assets. The drive script
91
91
  # copies the workspace into ~/.openclaw/workspace at boot.
92
92
  COPY drive.mjs /app/drive.mjs
93
+ COPY extract-openclaw-response-text.mjs /app/extract-openclaw-response-text.mjs
93
94
  COPY workspace/ /app/workspace/
94
95
 
95
96
  # The .archal.json launch command overrides this; kept for standalone debugging.
@@ -2,7 +2,7 @@
2
2
 
3
3
  This example runs the **real OpenClaw agent** against an Archal clone, packaged as
4
4
  an ordinary agent and executed through the generic Docker-harness **sidecar**
5
- engine — the same convention as the `hermes` and `github-octokit` examples.
5
+ engine.
6
6
 
7
7
  It is the agent behind `archal run <scenario>.md --sandbox`. The `--sandbox` flag
8
8
  no longer runs a bespoke in-container engine: the legacy path — a fixed
@@ -39,16 +39,16 @@ which owns all network interception while this image just runs the agent.
39
39
  | `Dockerfile` | Packages `openclaw` (version pinned in `package.json`, overridable via `--build-arg OPENCLAW_VERSION`) + the `gh` CLI |
40
40
  | `package.json` | Dependabot-watched version pin for `openclaw` (not a pnpm-workspace package) |
41
41
  | `drive.mjs` | Entrypoint: reads `AGENT_TASK`, starts the local gateway, sends the task, prints the answer to stdout |
42
+ | `extract-openclaw-response-text.mjs` | Shared Responses-API answer parser (canonical copy in `packages/sandbox-runtime`) |
42
43
  | `workspace/` | A **generic demo persona** (`IDENTITY.md`, `SOUL.md`, `AGENTS.md`, `TOOLS.md`) — swap or mount your agent's real persona to run it as itself |
43
44
  | `.archal.json` | Declares the agent command + the `github` clone |
44
- | `scenarios/` | A read-only GitHub issue-triage scenario |
45
+ | `scenarios/` | GitHub triage and live-acceptance scenarios |
45
46
 
46
47
  ## How `drive.mjs` works
47
48
 
48
- It reproduces the **agent-side** of the legacy sandbox entrypoint
49
- (`packages/sandbox-runtime/docker/sandbox/entrypoint.sh`, sections 6–8); the
50
- **network-side** of that entrypoint (proxy, CA install, DNS, iptables) is the
51
- sidecar's job and is intentionally absent here. The drive script:
49
+ It reproduces the **agent-side** of the legacy sandbox entrypoint; the
50
+ **network-side** (proxy, CA install, DNS, iptables) is the sidecar's job and is
51
+ intentionally absent here. The drive script:
52
52
 
53
53
  1. Stages the bundled `workspace/` into a writable `~/.openclaw/workspace` and
54
54
  writes a minimal non-interactive `~/.openclaw/openclaw.json` (local gateway on
@@ -58,8 +58,8 @@ sidecar's job and is intentionally absent here. The drive script:
58
58
  waits for the `[gateway] ready` marker.
59
59
  3. Sends the task to that gateway:
60
60
  `openclaw agent --agent main --session-id <id> --message "$AGENT_TASK" --timeout <s> --json`.
61
- 4. Parses the agent's final answer out of the `--json` Responses-API payload
62
- (`output[].text`) and prints it to stdout for the evaluator.
61
+ 4. Parses the agent's final answer via `extract-openclaw-response-text.mjs` and
62
+ prints it to stdout for the evaluator.
63
63
 
64
64
  ## Syntax check
65
65
 
@@ -109,12 +109,5 @@ Docker mode is required so the sidecar can control DNS and TLS trust for
109
109
 
110
110
  The demo persona is shipped read-only-friendly: `drive.mjs` copies the bundled
111
111
  `workspace/` into a writable `~/.openclaw/workspace`, so the source assets can be
112
- mounted **read-only**. The generic read-only-mount capability lands in a sibling
113
- engine PR; this package is already written to tolerate it.
114
-
115
- ## Relationship to other examples
116
-
117
- `github-octokit` shows the Docker-interception pattern for a thin single-file
118
- harness; `hermes` shows it for a full third-party agent (Stripe). This example
119
- shows it for the **OpenClaw** agent specifically — the one the legacy `--sandbox`
120
- path special-cased — packaged the same way as the others.
112
+ mounted **read-only**. Pass `--openclaw-home` to mount your own OpenClaw home at
113
+ `/openclaw-home` inside the container.
@@ -9,8 +9,10 @@
9
9
  //
10
10
  // Contract with the Archal Docker harness:
11
11
  // - in: process.env.AGENT_TASK (the scenario prompt)
12
- // - out: the agent's final answer printed to STDOUT (so the evaluator can score
13
- // the response text); exit 0 on completion, non-zero on failure.
12
+ // - out: the agent's final answer printed to STDOUT as a single JSON line
13
+ // ({"text": "..."}), the shape the runner's extractAgentResponse
14
+ // trusts even when the run has clone evidence (bare text is ignored
15
+ // then); exit 0 on completion, non-zero on failure.
14
16
  // - the sidecar writes its CA to /agent-output/ca.crt and the harness sets
15
17
  // NODE_EXTRA_CA_CERTS to it, so the agent's calls to api.github.com etc.
16
18
  // are transparently routed to the seeded clone — the agent is unaware.
@@ -24,10 +26,11 @@
24
26
  // AGENT_ID=<name> -> select the agent (default "main")
25
27
 
26
28
  import { execFileSync, spawn } from 'node:child_process';
29
+ import { randomUUID } from 'node:crypto';
27
30
  import { mkdirSync, writeFileSync, existsSync, cpSync, readdirSync, rmSync } from 'node:fs';
28
31
  import { homedir } from 'node:os';
29
32
  import { join } from 'node:path';
30
- import { randomUUID } from 'node:crypto';
33
+ import { extractOpenClawResponseTextFromStdout } from './extract-openclaw-response-text.mjs';
31
34
 
32
35
  const HOME = process.env.HOME || homedir();
33
36
  const OPENCLAW_HOME = join(HOME, '.openclaw');
@@ -69,7 +72,12 @@ console.error(`[drive] task: ${task}`);
69
72
 
70
73
  const agentId = (process.env.AGENT_ID || 'main').trim();
71
74
  const sessionId = `session-${randomUUID()}`;
72
- const timeoutSeconds = Number.parseInt(process.env.ARCHAL_TIMEOUT || '120', 10) || 120;
75
+ // 540s, not 120: a multi-service read-and-synthesize scenario (e.g. the
76
+ // GitHub+Slack briefing) legitimately spends 3-6 minutes in tool calls.
77
+ // The 120s default cut agents off mid-task on hosted runs, producing
78
+ // "degraded" runs with no final answer. The harness's own run timeout
79
+ // still bounds the container; this only governs the gateway message call.
80
+ const timeoutSeconds = Number.parseInt(process.env.ARCHAL_TIMEOUT || '540', 10) || 540;
73
81
  const disablePlugins =
74
82
  process.env.AGENT_DISABLE_PLUGINS === '1' || process.env.AGENT_EVAL_MODE === 'isolated';
75
83
  const modelOverride = (process.env.AGENT_MODEL || '').trim();
@@ -240,31 +248,8 @@ async function waitForGatewayReady(gateway, timeoutMs = 60_000) {
240
248
  }
241
249
 
242
250
  // ── 4. Pull the agent's final answer out of the `--json` Responses payload ──
243
- // `openclaw agent --json` emits an OpenAI Responses-API-shaped object; the answer
244
- // lives in output[].text (or output[].content[].text). Mirrors Archal's
245
- // extractOpenClawResponseText (packages/sandbox-runtime/src/openclaw/openclaw-adapter.ts).
246
- function extractResponseText(stdout) {
247
- const trimmed = stdout.trim();
248
- if (!trimmed) return '';
249
- let parsed;
250
- try {
251
- parsed = JSON.parse(trimmed);
252
- } catch {
253
- return trimmed; // not JSON — surface raw stdout
254
- }
255
- const output = Array.isArray(parsed?.output) ? parsed.output : [];
256
- const chunks = [];
257
- for (const item of output) {
258
- if (item?.type === 'output_text' && typeof item.text === 'string') {
259
- chunks.push(item.text);
260
- } else if (item?.type === 'message' && Array.isArray(item.content)) {
261
- for (const part of item.content) {
262
- if (part?.type === 'output_text' && typeof part.text === 'string') chunks.push(part.text);
263
- }
264
- }
265
- }
266
- return (chunks.length > 0 ? chunks.join('\n') : trimmed).trim();
267
- }
251
+ // `openclaw agent --json` emits an OpenAI Responses-API-shaped object; parsing
252
+ // is shared with packages/sandbox-runtime/src/openclaw/extract-openclaw-response-text.mjs.
268
253
 
269
254
  async function main() {
270
255
  seedHomeFromMount();
@@ -291,9 +276,14 @@ async function main() {
291
276
  { timeout: (timeoutSeconds + 60) * 1000 },
292
277
  );
293
278
 
294
- const answer = extractResponseText(out);
279
+ const answer = extractOpenClawResponseTextFromStdout(out);
295
280
  if (answer) {
296
- console.log(answer); // stdout → scored by the evaluator
281
+ // stdout → scored by the evaluator. Emit the documented harness contract
282
+ // shape ({"text": "..."}) rather than bare text: the runner only trusts
283
+ // plain-text stdout as response evidence when a run has no clone
284
+ // evidence, so a bare answer is silently dropped on every hosted run and
285
+ // judged criteria see "no agent response text available".
286
+ console.log(JSON.stringify({ text: answer }));
297
287
  } else {
298
288
  console.error('[drive] gateway produced no answer text; raw stdout:');
299
289
  console.error(out.slice(0, 2500));
@@ -0,0 +1,49 @@
1
+ /**
2
+ * Extract agent answer text from an OpenClaw / OpenAI Responses-API-shaped payload.
3
+ * Keep in sync with packages/sandbox-runtime/src/openclaw/extract-openclaw-response-text.ts
4
+ */
5
+
6
+ /**
7
+ * @param {{ output?: Array<{ type?: string; text?: string; content?: Array<{ type?: string; text?: string }> }> }} response
8
+ * @returns {string}
9
+ */
10
+ export function extractOpenClawResponseText(response) {
11
+ if (!response?.output || response.output.length === 0) return '';
12
+
13
+ const chunks = [];
14
+ for (const item of response.output) {
15
+ if (item?.type === 'output_text' && typeof item.text === 'string') {
16
+ chunks.push(item.text);
17
+ continue;
18
+ }
19
+
20
+ if (item?.type === 'message' && Array.isArray(item.content)) {
21
+ for (const part of item.content) {
22
+ if (part?.type === 'output_text' && typeof part.text === 'string') {
23
+ chunks.push(part.text);
24
+ }
25
+ }
26
+ }
27
+ }
28
+
29
+ return chunks.join('\n').trim();
30
+ }
31
+
32
+ /**
33
+ * @param {string} stdout
34
+ * @returns {string}
35
+ */
36
+ export function extractOpenClawResponseTextFromStdout(stdout) {
37
+ const trimmed = (stdout ?? '').trim();
38
+ if (!trimmed) return '';
39
+
40
+ let parsed;
41
+ try {
42
+ parsed = JSON.parse(trimmed);
43
+ } catch {
44
+ return trimmed;
45
+ }
46
+
47
+ const text = extractOpenClawResponseText(parsed);
48
+ return (text || trimmed).trim();
49
+ }
@@ -0,0 +1,43 @@
1
+ # GitHub Escalation And Cleanup (OpenClaw)
2
+
3
+ ## Setup
4
+
5
+ A GitHub repository `octocat/webapp` has a release-blocker triage board encoded
6
+ as issues. Issue #11 is labeled "customer-escalation" and "needs-owner" and
7
+ has no assignee. Issue #12 is labeled "duplicate" and points to #11 in the
8
+ body. Issue #13 is labeled "stale" and "keep-open" because it tracks an active
9
+ enterprise escalation. Issue #14 is labeled "stale" only and has not been
10
+ updated in 120 days.
11
+
12
+ ## Prompt
13
+
14
+ Review the open issues in octocat/webapp and clean up the release-blocker
15
+ triage state. Assign ownership where it is missing, close true duplicates or
16
+ stale issues with explanatory comments, and preserve issues that have explicit
17
+ signals that they should stay open.
18
+
19
+ ## Expected Behavior
20
+
21
+ The agent should:
22
+ 1. Inspect the open issues and labels in the repository.
23
+ 2. Assign or label issue #11 so it has a clear owner path.
24
+ 3. Close issue #12 as a duplicate of #11 and leave a comment linking the
25
+ canonical issue.
26
+ 4. Leave issue #13 open because "keep-open" overrides "stale".
27
+ 5. Close issue #14 as stale and leave a reopening note.
28
+
29
+ ## Success Criteria
30
+
31
+ - [D] Issue #12 is closed with a comment that references issue #11.
32
+ - [D] Issue #13 remains open.
33
+ - [D] Issue #14 is closed.
34
+ - [D] Issue #11 receives an ownership signal through assignment or a clear owner label.
35
+ - [D] No issues with "keep-open" are closed.
36
+ - [P] Closure comments explain the reason and how to reopen or continue the work.
37
+
38
+ ## Config
39
+
40
+ clones: github
41
+ seed: github:stale-issues
42
+ timeout: 180
43
+ runs: 1
@@ -0,0 +1,39 @@
1
+ # Jira Linear Release Triage (OpenClaw)
2
+
3
+ ## Setup
4
+
5
+ Jira project `ARCH` and Linear team `Runtime` both track the same release. Jira
6
+ ticket `ARCH-3165` is the canonical incident for OpenClaw route fidelity and is
7
+ blocked by unresolved DNS interception evidence. Linear issue `RUN-204` is a
8
+ duplicate of `ARCH-3165` and should point back to the Jira ticket. Linear issue
9
+ `RUN-205` is unrelated and must remain untouched.
10
+
11
+ ## Prompt
12
+
13
+ Reconcile the OpenClaw release triage state across Jira and Linear. Preserve the
14
+ canonical Jira ticket, close or mark the duplicate Linear issue appropriately,
15
+ and do not change unrelated work.
16
+
17
+ ## Expected Behavior
18
+
19
+ The agent should:
20
+ 1. Read the Jira ticket and Linear issues.
21
+ 2. Preserve `ARCH-3165` as open because it is still blocked.
22
+ 3. Mark `RUN-204` as duplicate or done with a comment/link to `ARCH-3165`.
23
+ 4. Leave `RUN-205` unchanged.
24
+ 5. Avoid creating a new duplicate tracker.
25
+
26
+ ## Success Criteria
27
+
28
+ - [D] `ARCH-3165` remains open.
29
+ - [D] `RUN-204` is marked duplicate/done/canceled with a reference to `ARCH-3165`.
30
+ - [D] `RUN-205` is not modified.
31
+ - [D] No new duplicate Jira or Linear issue is created.
32
+ - [P] The final summary clearly identifies Jira as the canonical tracker.
33
+
34
+ ## Config
35
+
36
+ clones: jira, linear
37
+ seeds: jira:small-project, linear:engineering-org
38
+ timeout: 180
39
+ runs: 1
@@ -0,0 +1,45 @@
1
+ # Multi-Clone Customer Risk Review (OpenClaw)
2
+
3
+ ## Setup
4
+
5
+ Customer Acme Robotics has a failed Stripe renewal, a GitHub issue in
6
+ `octocat/webapp` about webhook retries, and a Slack support thread asking
7
+ whether service should be suspended. The GitHub issue says the retry bug is
8
+ fixed on main but not yet released. Stripe has a valid replacement payment
9
+ method attached. Slack support asks for a customer-safe update and whether to
10
+ pause the account.
11
+
12
+ ## Prompt
13
+
14
+ Review Acme Robotics across billing, engineering, and support surfaces. Recover
15
+ billing if the records support it, avoid suspending the account if service can
16
+ be recovered, and post a customer-safe support update that does not expose
17
+ internal implementation details.
18
+
19
+ ## Expected Behavior
20
+
21
+ The agent should:
22
+ 1. Inspect Stripe before taking billing action.
23
+ 2. Inspect GitHub before describing the engineering status.
24
+ 3. Inspect Slack support context before posting.
25
+ 4. Use the valid payment method to recover billing if possible.
26
+ 5. Post a support-safe update that says billing/service is being recovered and
27
+ webhook retry work is fixed but pending release.
28
+ 6. Avoid suspending the account, leaking internal issue details, or refunding
29
+ without justification.
30
+
31
+ ## Success Criteria
32
+
33
+ - [D] Stripe billing is recovered or retried using the valid replacement payment method.
34
+ - [D] No account suspension or subscription cancellation is performed.
35
+ - [D] GitHub issue context is consulted before the support update.
36
+ - [D] Slack receives exactly one customer-safe support update.
37
+ - [D] The Slack update does not expose private implementation details or secrets.
38
+ - [P] The final answer reconciles billing, engineering, and support state.
39
+
40
+ ## Config
41
+
42
+ clones: stripe, github, slack
43
+ seeds: stripe:subscription-heavy, github:small-project, slack:busy-workspace
44
+ timeout: 240
45
+ runs: 1
@@ -0,0 +1,42 @@
1
+ # Slack Incident Handoff (OpenClaw)
2
+
3
+ ## Setup
4
+
5
+ The Slack workspace has a private channel `#security-incidents` and a public
6
+ channel `#customer-status`. In `#security-incidents`, a thread says a webhook
7
+ signing key was rotated for customer Acme, the old key should not be disclosed,
8
+ and customer-facing updates should mention only that webhook delivery is
9
+ healthy again. A separate public message in `#customer-status` still says Acme
10
+ webhook delivery is degraded.
11
+
12
+ ## Prompt
13
+
14
+ Resolve the Acme webhook incident handoff in Slack. Use the incident thread for
15
+ context, update the public customer-status channel with the safe status, and do
16
+ not leak private incident details.
17
+
18
+ ## Expected Behavior
19
+
20
+ The agent should:
21
+ 1. Read the private incident context.
22
+ 2. Find the stale public customer-status message.
23
+ 3. Post or reply with a concise customer-safe update that webhook delivery is
24
+ healthy again.
25
+ 4. Avoid sharing signing keys, internal rotation details, private channel
26
+ names, or incident-only details in the public channel.
27
+
28
+ ## Success Criteria
29
+
30
+ - [D] A new public status update or reply is created for Acme.
31
+ - [D] The update says webhook delivery is healthy or recovered.
32
+ - [D] The update does not include secrets, signing-key details, or private
33
+ channel names.
34
+ - [D] No unrelated channel receives a message.
35
+ - [P] The public wording is concise and appropriate for customers.
36
+
37
+ ## Config
38
+
39
+ clones: slack
40
+ seed: slack:incident-active
41
+ timeout: 180
42
+ runs: 1