archal 0.9.20 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +59 -159
  2. package/agents/openclaw/Dockerfile +2 -1
  3. package/agents/openclaw/README.md +10 -17
  4. package/agents/openclaw/drive.mjs +21 -31
  5. package/agents/openclaw/extract-openclaw-response-text.mjs +49 -0
  6. package/agents/openclaw/scenarios/acceptance/github-escalation-and-cleanup.md +43 -0
  7. package/agents/openclaw/scenarios/acceptance/jira-linear-release-triage.md +39 -0
  8. package/agents/openclaw/scenarios/acceptance/multi-clone-customer-risk-review.md +45 -0
  9. package/agents/openclaw/scenarios/acceptance/slack-incident-handoff.md +42 -0
  10. package/agents/openclaw/scenarios/acceptance/stripe-dunning-remediation.md +40 -0
  11. package/agents/openclaw/scenarios/close-stale-issues.md +35 -0
  12. package/agents/openclaw/workspace/TOOLS.md +19 -0
  13. package/clone-assets/calcom/tools.json +1705 -153
  14. package/clone-assets/clickup/tools.json +2528 -541
  15. package/clone-assets/customerio/tools.json +2108 -144
  16. package/clone-assets/github/tools.json +1 -2
  17. package/clone-assets/gitlab/tools.json +10087 -398
  18. package/clone-assets/hubspot/tools.json +198 -0
  19. package/clone-assets/linear/tools.json +41 -1
  20. package/clone-assets/sentry/tools.json +2264 -188
  21. package/clone-assets/stripe/tools.json +114 -0
  22. package/clone-assets/webflow/tools.json +1895 -125
  23. package/dist/{autoloop-worker-types-BEb_E44z.d.cts → autoloop-worker-types-CHaclqtD.d.cts} +23 -59
  24. package/dist/cli.cjs +107175 -126959
  25. package/dist/commands/{autoloop-hosted-worker.cjs → autoloop-hosted-control-plane-client.cjs} +25657 -42736
  26. package/dist/commands/{autoloop-hosted-worker.d.cts → autoloop-hosted-control-plane-client.d.cts} +32 -42
  27. package/dist/commands/autoloop-pr-verification.cjs +607 -522
  28. package/dist/commands/autoloop-pr-verification.d.cts +3 -1
  29. package/dist/commands/autoloop-result-parser.cjs +2474 -2373
  30. package/dist/commands/autoloop-result-parser.d.cts +159 -2
  31. package/dist/commands/autoloop-worker.cjs +14565 -12240
  32. package/dist/commands/autoloop-worker.d.cts +10 -5
  33. package/dist/eval-shim.mjs +216 -0
  34. package/dist/index.cjs +15 -2
  35. package/dist/index.d.cts +3 -1
  36. package/dist/layer-1-ziaLpfLr.d.cts +62 -0
  37. package/dist/metafile-cjs.json +1 -0
  38. package/dist/reporter-entry.js +3 -0
  39. package/dist/{replay.cjs → sdk/index.cjs} +42979 -40973
  40. package/dist/sdk/index.d.cts +626 -0
  41. package/dist/vitest/{chunk-6CBYFCFK.js → chunk-2PDHTPZC.js} +289 -89
  42. package/dist/vitest/{chunk-ARVS45PP.js → chunk-7L4BBB6M.js} +123 -274
  43. package/dist/vitest/index.cjs +500 -816
  44. package/dist/vitest/index.d.ts +16 -81
  45. package/dist/vitest/index.js +40 -409
  46. package/dist/vitest/runtime/hosted-session-reaper.cjs +143 -293
  47. package/dist/vitest/runtime/hosted-session-reaper.js +1 -1
  48. package/dist/vitest/runtime/setup-files.js +2 -2
  49. package/manifest.json +9 -0
  50. package/package.json +14 -13
  51. package/skills/archal-agent/SKILL.md +0 -1
  52. package/skills/autoloop/SKILL.md +288 -273
  53. package/skills/autoloop/references/hosted-sources.md +49 -17
  54. package/skills/autoloop/references/trace-schema-mapping.md +83 -52
  55. package/skills/eval/SKILL.md +16 -48
  56. package/skills/free-account/SKILL.md +103 -0
  57. package/skills/install-agent/SKILL.md +16 -35
  58. package/skills/onboard/SKILL.md +13 -70
  59. package/skills/scenario/SKILL.md +30 -27
  60. package/skills/seed/SKILL.md +7 -6
  61. package/skills/vitest/SKILL.md +3 -2
  62. package/agents/github-octokit/.archal.json +0 -8
  63. package/agents/github-octokit/Dockerfile +0 -8
  64. package/agents/github-octokit/README.md +0 -113
  65. package/agents/github-octokit/agent.mjs +0 -54
  66. package/agents/github-octokit/package.json +0 -9
  67. package/agents/github-octokit/scenarios/test-repo-access.md +0 -27
  68. package/agents/google-workspace-local-tools/Dockerfile +0 -6
  69. package/agents/google-workspace-local-tools/README.md +0 -58
  70. package/agents/google-workspace-local-tools/agent.mjs +0 -196
  71. package/agents/google-workspace-local-tools/archal-harness.json +0 -7
  72. package/agents/google-workspace-local-tools/run-input.yaml +0 -16
  73. package/agents/google-workspace-local-tools/scenario.md +0 -29
  74. package/agents/hermes/.archal.json +0 -8
  75. package/agents/hermes/Dockerfile +0 -46
  76. package/agents/hermes/README.md +0 -87
  77. package/agents/hermes/SOUL.md +0 -27
  78. package/agents/hermes/config.yaml +0 -34
  79. package/agents/hermes/drive.mjs +0 -113
  80. package/agents/hermes/scenarios/stripe-customers-read-only.md +0 -32
  81. package/agents/pagination-test/README.md +0 -24
  82. package/agents/pagination-test/scenario.md +0 -24
  83. package/agents/replay-capsule-harness/README.md +0 -29
  84. package/agents/replay-capsule-harness/observability-install-offline-e2e.mts +0 -1517
  85. package/agents/replay-capsule-harness/replay-capsule-e2e.mjs +0 -104
  86. package/dist/harness.cjs +0 -63
  87. package/dist/harness.d.cts +0 -20
  88. package/dist/replay.d.cts +0 -4625
  89. package/dist/scenarios.cjs +0 -80343
  90. package/dist/scenarios.d.cts +0 -562
@@ -1,196 +0,0 @@
1
- #!/usr/bin/env node
2
-
3
- const originalGoogleBearer = process.env.EXAMPLE_GOOGLE_ACCESS_TOKEN?.trim() || 'ya29.self-local-invalid';
4
- const calendarDate = process.env.EXAMPLE_CALENDAR_DATE?.trim() || '2026-03-29';
5
- const task = process.env.AGENT_TASK?.trim()
6
- || 'Read the unread inbox email, inspect the next calendar event, and create a draft reply.';
7
-
8
- function toBase64Url(input) {
9
- return Buffer.from(input, 'utf-8').toString('base64url');
10
- }
11
-
12
- function getGoogleHeaders(json = false) {
13
- return {
14
- authorization: `Bearer ${originalGoogleBearer}`,
15
- ...(json ? { 'content-type': 'application/json' } : {}),
16
- };
17
- }
18
-
19
- function toGoogleUrl(path) {
20
- if (path.startsWith('/gmail/')) {
21
- return `https://gmail.googleapis.com${path}`;
22
- }
23
- if (path.startsWith('/calendar/')) {
24
- return `https://calendar.googleapis.com${path}`;
25
- }
26
- throw new Error(`Unsupported Google Workspace API path: ${path}`);
27
- }
28
-
29
- async function fetchJson(path, options = {}) {
30
- const url = toGoogleUrl(path);
31
- let response;
32
- try {
33
- response = await fetch(url, options);
34
- } catch (error) {
35
- const message = error instanceof Error ? error.message : String(error);
36
- throw new Error(`fetch failed for ${url}: ${message}`);
37
- }
38
- const text = await response.text();
39
- let body = null;
40
- if (text.length > 0) {
41
- try {
42
- body = JSON.parse(text);
43
- } catch {
44
- body = text;
45
- }
46
- }
47
-
48
- if (!response.ok) {
49
- throw new Error(`Request failed ${response.status} for ${path}: ${JSON.stringify(body)}`);
50
- }
51
-
52
- return body;
53
- }
54
-
55
- function getHeader(payload, name) {
56
- const headers = Array.isArray(payload?.headers) ? payload.headers : [];
57
- const match = headers.find(
58
- (header) =>
59
- header
60
- && typeof header.name === 'string'
61
- && header.name.toLowerCase() === name.toLowerCase(),
62
- );
63
- return typeof match?.value === 'string' ? match.value : null;
64
- }
65
-
66
- async function readEmail() {
67
- const unreadQuery = encodeURIComponent('is:unread in:inbox');
68
- const unreadList = await fetchJson(`/gmail/v1/users/me/messages?q=${unreadQuery}&maxResults=1`, {
69
- headers: getGoogleHeaders(),
70
- });
71
- const list = Array.isArray(unreadList?.messages) && unreadList.messages.length > 0
72
- ? unreadList
73
- : await fetchJson('/gmail/v1/users/me/messages?maxResults=1', {
74
- headers: getGoogleHeaders(),
75
- });
76
-
77
- const [messageRef] = Array.isArray(list?.messages) ? list.messages : [];
78
- if (!messageRef?.id) {
79
- throw new Error('No Gmail message available in the selected seed');
80
- }
81
-
82
- const message = await fetchJson(
83
- `/gmail/v1/users/me/messages/${encodeURIComponent(String(messageRef.id))}?format=full`,
84
- { headers: getGoogleHeaders() },
85
- );
86
- const payload = message?.payload ?? {};
87
-
88
- return {
89
- id: String(message.id),
90
- threadId: String(message.threadId),
91
- subject: getHeader(payload, 'Subject'),
92
- from: getHeader(payload, 'From'),
93
- snippet: typeof message.snippet === 'string' ? message.snippet : null,
94
- };
95
- }
96
-
97
- function buildDayWindow(dateString) {
98
- const start = new Date(`${dateString}T00:00:00.000Z`);
99
- if (Number.isNaN(start.valueOf())) {
100
- throw new Error(`Invalid EXAMPLE_CALENDAR_DATE: ${dateString}`);
101
- }
102
- const end = new Date(start);
103
- end.setUTCDate(end.getUTCDate() + 1);
104
- return {
105
- timeMin: start.toISOString(),
106
- timeMax: end.toISOString(),
107
- };
108
- }
109
-
110
- async function getCalendar() {
111
- const { timeMin, timeMax } = buildDayWindow(calendarDate);
112
- const query = new URLSearchParams({
113
- maxResults: '5',
114
- singleEvents: 'true',
115
- orderBy: 'startTime',
116
- timeMin,
117
- timeMax,
118
- });
119
- const response = await fetchJson(`/calendar/v3/calendars/primary/events?${query.toString()}`, {
120
- headers: getGoogleHeaders(),
121
- });
122
- const [firstEvent] = Array.isArray(response?.items) ? response.items : [];
123
- if (!firstEvent) {
124
- throw new Error(`No calendar event found for ${calendarDate}`);
125
- }
126
-
127
- return {
128
- id: String(firstEvent.id),
129
- summary: typeof firstEvent.summary === 'string' ? firstEvent.summary : null,
130
- start: firstEvent.start ?? null,
131
- };
132
- }
133
-
134
- function parseEmailAddress(value) {
135
- if (typeof value !== 'string' || value.trim().length === 0) {
136
- return 'self@local.invalid';
137
- }
138
- const bracketMatch = value.match(/<([^>]+)>/);
139
- if (bracketMatch?.[1]) {
140
- return bracketMatch[1];
141
- }
142
- const directMatch = value.match(/[^\s<>,;]+@[^\s<>,;]+/);
143
- return directMatch?.[0] ?? 'self@local.invalid';
144
- }
145
-
146
- async function generateDraft(email, calendarEvent) {
147
- const replyTo = parseEmailAddress(email.from);
148
- const draftMime = [
149
- `To: ${replyTo}`,
150
- `Subject: Re: ${email.subject ?? 'Quick follow-up'}`,
151
- '',
152
- `Thanks for the note. I saw your message and my next event is "${calendarEvent.summary ?? 'scheduled'}".`,
153
- ].join('\r\n');
154
-
155
- const response = await fetchJson('/gmail/v1/users/me/drafts', {
156
- method: 'POST',
157
- headers: getGoogleHeaders(true),
158
- body: JSON.stringify({
159
- message: {
160
- threadId: email.threadId,
161
- raw: toBase64Url(draftMime),
162
- },
163
- }),
164
- });
165
-
166
- return {
167
- id: typeof response?.id === 'string' ? response.id : null,
168
- messageId: response?.message?.id ?? null,
169
- };
170
- }
171
-
172
- async function main() {
173
- const toolCalls = [];
174
-
175
- toolCalls.push('read_email');
176
- const email = await readEmail();
177
-
178
- toolCalls.push('get_calendar');
179
- const calendarEvent = await getCalendar();
180
-
181
- toolCalls.push('generate_draft');
182
- const draft = await generateDraft(email, calendarEvent);
183
-
184
- console.log(JSON.stringify({
185
- task,
186
- toolCalls,
187
- email,
188
- calendarEvent,
189
- draft,
190
- }, null, 2));
191
- }
192
-
193
- main().catch((error) => {
194
- console.error(error instanceof Error ? error.message : String(error));
195
- process.exit(1);
196
- });
@@ -1,7 +0,0 @@
1
- {
2
- "version": 1,
3
- "local": {
4
- "command": "node",
5
- "args": ["agent.mjs"]
6
- }
7
- }
@@ -1,16 +0,0 @@
1
- title: Google Workspace Local-Tools Reply Draft
2
- task: |
3
- Read the unread inbox email, inspect the next calendar event for `2026-03-29`,
4
- and create a draft reply.
5
-
6
- Do not send the email.
7
- checks:
8
- - The run exits successfully
9
- - The harness reports the local tool calls `read_email`, `get_calendar`, and `generate_draft`
10
- - The harness creates a Gmail draft id
11
- - No `archal-*` tool prompting is required
12
- config:
13
- clones: google-workspace
14
- timeout: 120
15
- runs: 1
16
- tags: smoke, google-workspace, harness
@@ -1,29 +0,0 @@
1
- # Google Workspace Local-Tools Reply Draft
2
-
3
- ## Setup
4
-
5
- The Google Workspace account has:
6
-
7
- - one unread inbox message in the primary mailbox
8
- - a primary calendar event on `2026-03-29`
9
- - a valid seed access token for the baseline account
10
-
11
- ## Task
12
-
13
- Read the unread inbox email, inspect the next calendar event for `2026-03-29`, and create a draft reply.
14
-
15
- Do not send the email.
16
-
17
- ## Checks
18
-
19
- - [D] The run exits successfully
20
- - [D] The harness reports the local tool calls `read_email`, `get_calendar`, and `generate_draft`
21
- - [D] The harness creates a Gmail draft id
22
- - [D] No `archal-*` tool prompting is required
23
-
24
- ## Config
25
-
26
- clones: google-workspace
27
- timeout: 120
28
- runs: 1
29
- tags: smoke, google-workspace, harness
@@ -1,8 +0,0 @@
1
- {
2
- "description": "Hermes — a full third-party Stripe support agent.",
3
- "agent": {
4
- "command": "node",
5
- "args": ["drive.mjs"]
6
- },
7
- "clones": ["stripe"]
8
- }
@@ -1,46 +0,0 @@
1
- # Hermes agent harness — runs the Nous Research `hermes-agent` against Archal clones.
2
- #
3
- # The harness keeps calling the real service domains (e.g. api.stripe.com). In a
4
- # Docker harness run, Archal maps those domains to the TLS intercept listener and
5
- # routes the requests to the clone underneath the process — the agent is unaware.
6
- # api.openai.com is allowlisted and forwarded to the real model with the host key
7
- # injected by the proxy, so the agent's reasoning runs against a real LLM.
8
- #
9
- # Because the harness blocks egress to everything except clones + LLM providers,
10
- # the Stripe MCP is installed at BUILD time and invoked by direct path (no runtime
11
- # `npx` registry fetch, which would 403).
12
- FROM node:22-bookworm-slim
13
-
14
- # Pin the agent version. Override at build time: --build-arg HERMES_VERSION=0.17.0
15
- ARG HERMES_VERSION=0.16.0
16
-
17
- ENV PYTHONUNBUFFERED=1 \
18
- HERMES_HOME=/root/.hermes \
19
- HERMES_ACCEPT_HOOKS=1 \
20
- PATH="/opt/hermes-venv/bin:${PATH}"
21
-
22
- # Python (hermes core) + node/npx (base image, for @stripe/mcp) + tools hermes expects.
23
- RUN apt-get update && apt-get install -y --no-install-recommends \
24
- python3 python3-venv python3-pip git ca-certificates ripgrep ffmpeg curl \
25
- && rm -rf /var/lib/apt/lists/*
26
-
27
- # Install the agent into an isolated venv, pinned to HERMES_VERSION.
28
- RUN python3 -m venv /opt/hermes-venv \
29
- && /opt/hermes-venv/bin/pip install --no-cache-dir --upgrade pip \
30
- && /opt/hermes-venv/bin/pip install --no-cache-dir "hermes-agent==${HERMES_VERSION}"
31
-
32
- # Pre-install the Stripe MCP so no runtime registry fetch is needed (egress is
33
- # blocked). The config invokes its cli.js directly. Fail the build loudly if the
34
- # entry path moves between @stripe/mcp releases.
35
- RUN npm install -g @stripe/mcp \
36
- && test -f /usr/local/lib/node_modules/@stripe/mcp/dist/cli.js
37
-
38
- WORKDIR /app
39
-
40
- # Scoped, non-interactive config + the demo persona + the drive entrypoint.
41
- COPY config.yaml /root/.hermes/config.yaml
42
- COPY SOUL.md /root/.hermes/SOUL.md
43
- COPY drive.mjs /app/drive.mjs
44
-
45
- # The .archal.json launch command overrides this; kept for standalone debugging.
46
- CMD ["node", "/app/drive.mjs"]
@@ -1,87 +0,0 @@
1
- # Hermes Agent Harness
2
-
3
- This example runs a **real third-party agent** — Nous Research
4
- [`hermes-agent`](https://pypi.org/project/hermes-agent/) — against an Archal clone,
5
- **unmodified**. The agent keeps calling the real `api.stripe.com`; the Archal
6
- Docker harness transparently routes that traffic to a seeded Stripe clone and
7
- scores the result.
8
-
9
- It is the productionized form of the original blog-post spike: a self-contained,
10
- committed agent package instead of a one-off script.
11
-
12
- ## What this demonstrates
13
-
14
- - A full external agent (Python gateway + Node MCP servers) packaged into one image.
15
- - **Transparent interception**: the agent's `@stripe/mcp` child process calls
16
- `api.stripe.com` and is routed to the clone via DNS + TLS MITM — no base-URL
17
- override, no code change. The CA is trusted container-wide and inherited by
18
- child processes.
19
- - **Real model, fake services**: `api.openai.com` is allowlisted and forwarded to
20
- the real model (host key injected by the proxy); only clone domains are intercepted.
21
- - **Egress block**: everything except clones and LLM providers is blocked, so the
22
- Stripe MCP is pre-installed at build time and invoked by direct path (no runtime
23
- `npx` registry fetch).
24
- - **Read-only behavior** scored from the clone trace plus the agent's answer text.
25
-
26
- ## Files
27
-
28
- | File | Purpose |
29
- |------|---------|
30
- | `Dockerfile` | Packages `hermes-agent` (pinned via `--build-arg HERMES_VERSION`) + the Stripe MCP |
31
- | `drive.mjs` | Entrypoint: reads `AGENT_TASK`, drives the agent once, prints the answer to stdout |
32
- | `config.yaml` | Stripe-scoped, non-interactive agent config (Stripe MCP only) |
33
- | `SOUL.md` | A **generic demo persona** (swap or mount your agent's real persona to run it as itself) |
34
- | `.archal.json` | Declares the agent command + the `stripe` clone |
35
- | `scenarios/` | A read-only Stripe scenario |
36
-
37
- ## Syntax check
38
-
39
- ```bash
40
- node --check drive.mjs
41
- ARCHAL_PREFLIGHT=1 node drive.mjs # only meaningful inside the built image
42
- ```
43
-
44
- ## Run
45
-
46
- Docker mode is required so Archal can control DNS and TLS trust for
47
- `api.stripe.com`. The agent calls a real LLM, so the host must have a working
48
- `OPENAI_API_KEY` exported — the proxy injects it for `api.openai.com` (the value
49
- inside the container is a placeholder).
50
-
51
- ```bash
52
- cd examples/agents/hermes
53
- export OPENAI_API_KEY=... # a key with gpt-5.5 access
54
- archal run scenarios/stripe-customers-read-only.md \
55
- --harness . \
56
- --dockerfile Dockerfile \
57
- -n 1
58
- ```
59
-
60
- The `--harness . --dockerfile Dockerfile` flags are required: this agent only
61
- runs inside the container (the drive script invokes `hermes` at
62
- `/opt/hermes-venv/bin/hermes`, which exists only in the built image). Without
63
- those flags `archal run` falls through to the in-process harness path and spawns
64
- `node drive.mjs` on the host, which fails immediately with `ENOENT` for the
65
- `hermes` binary. The `.archal.json` here still declares the agent command and the
66
- `stripe` clone for the harness to consume.
67
-
68
- ## Environment variables
69
-
70
- | Variable | Source | Notes |
71
- |----------|--------|-------|
72
- | `AGENT_TASK` | Injected by Archal | The scenario prompt |
73
- | `OPENAI_API_KEY` | Host → proxy | Real key on the host; placeholder inside the container |
74
-
75
- ## Notes
76
-
77
- - The image is large (Python + Node + the agent). The first build is slow; reuse
78
- the built image across runs where possible.
79
- - `config.yaml` lowers `reasoning_effort` to `medium` for iteration speed; raise it
80
- for a faithful capture.
81
- - This example pins `hermes-agent==0.16.0`. Override with
82
- `docker build --build-arg HERMES_VERSION=0.17.0 ...` or the equivalent harness option.
83
-
84
- ## Relationship to other examples
85
-
86
- `github-octokit` shows the same Docker-interception pattern for a thin single-file
87
- harness; this example shows it for a full, real, multi-process agent.
@@ -1,27 +0,0 @@
1
- # Persona — "Archie", a finance-ops brain (demo)
2
-
3
- You are Archie, a concise finance-operations assistant. You answer questions about
4
- the company's revenue and customers directly and briefly, grounded in real data
5
- you look up — never guessed.
6
-
7
- > This is a generic demo persona shipped with the Archal Hermes example. To run a
8
- > real agent's own persona instead, replace this file (or mount the agent's home)
9
- > when you build the image.
10
-
11
- ## Tools (live, read-only)
12
-
13
- You have live, read-only access to **Stripe** for revenue and customer questions:
14
-
15
- - `get_stripe_account_info` — basic account context.
16
- - `search_stripe_resources` — find customers, subscriptions, charges, invoices,
17
- payment_intents, prices, and products.
18
- - `fetch_stripe_resources` — retrieve a specific resource by id.
19
-
20
- When asked about money, customers, or revenue, **use these tools to look up the
21
- real answer** — do not estimate or recall. Report what the data says, concisely.
22
-
23
- ## Boundaries
24
-
25
- Your Stripe access is strictly read-only. You cannot move money, issue refunds,
26
- create or modify customers, or change anything. If asked to, say you only have
27
- read access and stop.
@@ -1,34 +0,0 @@
1
- # Stripe-scoped, non-interactive Hermes config for harness runs.
2
- #
3
- # Only the Stripe MCP is wired: the harness blocks egress to everything except
4
- # clones and LLM providers, so any other business-tool MCP would 403. The Stripe
5
- # MCP is invoked by direct path (pre-installed in the image) to avoid a runtime
6
- # registry fetch.
7
- model:
8
- default: gpt-5.5
9
- provider: openai-api
10
- agent:
11
- max_turns: 30
12
- # Lowered from prod (xhigh) for iteration speed; raise for a faithful capture.
13
- reasoning_effort: medium
14
- verbose: false
15
- terminal:
16
- backend: local # no docker-in-docker inside the harness container
17
- memory:
18
- memory_enabled: false # no external brain wired in the scoped container
19
- write_approval: false
20
- skills:
21
- write_approval: false
22
- streaming:
23
- enabled: false
24
- mcp_servers:
25
- stripe:
26
- command: node
27
- args:
28
- - "/usr/local/lib/node_modules/@stripe/mcp/dist/cli.js" # pre-installed; no runtime fetch
29
- - "--api-key=sk_test_archal_clone" # clone does not validate; api.stripe.com is intercepted
30
- tools:
31
- include:
32
- - get_stripe_account_info
33
- - search_stripe_resources
34
- - fetch_stripe_resources
@@ -1,113 +0,0 @@
1
- #!/usr/bin/env node
2
- // Hermes drive entrypoint — run the agent once on the injected task, then exit.
3
- //
4
- // Contract with the Archal Docker harness:
5
- // - in: process.env.AGENT_TASK (the scenario prompt)
6
- // - out: the agent's final answer printed to STDOUT (so the evaluator can score
7
- // the response text); exit 0 on completion, non-zero on failure.
8
- // - the harness harvests the clone /trace after this exits — this shim does not
9
- // collect the trace. The agent's Stripe MCP calls to api.stripe.com are
10
- // transparently routed to the seeded Stripe clone.
11
- //
12
- // hermes-agent has no single "run one task and print the answer" command, so we
13
- // drive it through its cron primitive: create a one-shot local-delivery job
14
- // carrying the task, force it due, then `cron tick` to run the agent loop once
15
- // with its MCP tools loaded. The answer lands in ~/.hermes/cron/output/<jobId>/.
16
-
17
- import { execFileSync } from 'node:child_process';
18
- import { readdirSync, readFileSync, statSync } from 'node:fs';
19
-
20
- const HERMES = '/opt/hermes-venv/bin/hermes';
21
- const OUTDIR = '/root/.hermes/cron/output';
22
-
23
- // Optional smoke test: `ARCHAL_PREFLIGHT=1 node drive.mjs` verifies the entrypoint
24
- // parses and the agent binary is present without running a task or calling out.
25
- if (process.env.ARCHAL_PREFLIGHT === '1') {
26
- try {
27
- execFileSync(HERMES, ['--version'], { stdio: 'ignore', timeout: 30_000 });
28
- console.log('OK');
29
- process.exit(0);
30
- } catch (err) {
31
- console.error(`[drive] preflight failed: ${err?.message ?? err}`);
32
- process.exit(1);
33
- }
34
- }
35
-
36
- const task = (process.env.AGENT_TASK ?? '').trim();
37
- if (!task) {
38
- console.error('[drive] no AGENT_TASK provided');
39
- process.exit(2);
40
- }
41
- console.error(`[drive] task: ${task}`);
42
-
43
- const hermes = (args) => {
44
- console.error(`[drive] $ hermes ${args.join(' ')}`);
45
- return execFileSync(HERMES, args, {
46
- encoding: 'utf8',
47
- stdio: ['ignore', 'pipe', 'pipe'],
48
- timeout: 600_000,
49
- });
50
- };
51
-
52
- // Pull the agent's final answer out of the cron output markdown so it can be
53
- // printed to stdout for the evaluator. Prefer the `## Response` section; fall
54
- // back to the whole file.
55
- const extractResponse = (markdown) => {
56
- const match = markdown.match(/##+\s*Response\s*\n([\s\S]*?)(?:\n##+\s|\s*$)/i);
57
- return (match ? match[1] : markdown).trim();
58
- };
59
-
60
- try {
61
- // 1) one-shot, local-delivery job carrying the task
62
- const created = hermes(['cron', 'create', 'every 1h', task, '--deliver', 'local', '--repeat', '1', '--name', 'archal-task']);
63
- console.error('[drive] create:\n' + created.trim());
64
-
65
- // 2) resolve the job id (parse the create output; fall back to the
66
- // archal-task-scoped row in `cron list`). We deliberately do NOT fall back
67
- // to "first 12-hex token anywhere in the listing" — that would silently
68
- // pick an unrelated/prior-run job and score the wrong output. If we can't
69
- // pin OUR job's id, fail loudly below rather than guess.
70
- let jobId = (created.match(/\b([0-9a-f]{12})\b/) || [])[1];
71
- if (!jobId) {
72
- const list = hermes(['cron', 'list']);
73
- console.error('[drive] list:\n' + list.trim());
74
- jobId = (list.match(/archal-task[\s\S]*?([0-9a-f]{12})/) || [])[1];
75
- }
76
- if (!jobId) {
77
- throw new Error('could not resolve the archal-task job id from cron create/list output');
78
- }
79
- console.error(`[drive] jobId=${jobId}`);
80
-
81
- // 3) force due, then tick once — runs the agent loop with the MCP tools loaded.
82
- // `cron tick` ticks every due job, but we only ever read THIS job's own
83
- // output directory below, so a stray due job can't be scored in its place.
84
- try { hermes(['cron', 'run', jobId]); }
85
- catch (e) { console.error('[drive] cron run warn: ' + (e.stderr || e.message)); }
86
- const tick = hermes(['cron', 'tick', '--accept-hooks']);
87
- console.error('[drive] tick:\n' + tick.trim());
88
-
89
- // 4) surface the agent's answer on stdout so the evaluator can score the text.
90
- // (The scored tool trace still comes from the clone /trace.) Read only this
91
- // job's output directory and only regular files within it.
92
- const dir = `${OUTDIR}/${jobId}`;
93
- const files = readdirSync(dir)
94
- .map((f) => `${dir}/${f}`)
95
- .filter((p) => statSync(p).isFile())
96
- .sort((a, b) => statSync(b).mtimeMs - statSync(a).mtimeMs);
97
- if (!files[0]) {
98
- // No answer was produced (e.g. cron run/tick failed silently). Exiting 0
99
- // with empty stdout would let the evaluator score an absent answer as real,
100
- // so treat this as a run failure.
101
- console.error('[drive] no output file produced — treating as failure');
102
- process.exit(3);
103
- }
104
- const body = readFileSync(files[0], 'utf8');
105
- console.error('[drive] latest output:\n' + body.slice(0, 2500));
106
- console.log(extractResponse(body));
107
-
108
- console.error('[drive] task driven through the agent');
109
- process.exit(0);
110
- } catch (err) {
111
- console.error('[drive] failed: ' + ((err.stdout || '') + (err.stderr || '') + (err.message || err)));
112
- process.exit(1);
113
- }
@@ -1,32 +0,0 @@
1
- # Hermes reports Stripe customers without mutating
2
-
3
- ## Setup
4
-
5
- The Stripe clone starts with its default seed (a handful of customers, products,
6
- subscriptions, and invoices). The Hermes agent has read-only Stripe tools
7
- (`get_stripe_account_info`, `search_stripe_resources`, `fetch_stripe_resources`)
8
- wired through the `@stripe/mcp` server.
9
-
10
- The agent believes it is calling the real `api.stripe.com`. In a Docker harness
11
- run, Archal transparently routes that traffic to the seeded clone — the agent's
12
- code and config are unchanged.
13
-
14
- ## Prompt
15
-
16
- How many customers do we have in Stripe right now, and what are a few of their
17
- names? Use the Stripe tools to check — do not guess.
18
-
19
- ## Success Criteria
20
-
21
- - [D] The run exits successfully
22
- - [P] The agent used the Stripe tools to retrieve real customer data (not a guess)
23
- - [P] The answer reports a customer count and at least one customer name
24
- - [P] The agent did NOT attempt any write, refund, or other mutation (read-only)
25
- - [P] The answer is concise, as a finance-ops brain would respond
26
-
27
- ## Config
28
-
29
- clones: stripe
30
- timeout: 900
31
- runs: 1
32
- tags: hermes, stripe, read-only, agent
@@ -1,24 +0,0 @@
1
- # Pagination Test Scenario
2
-
3
- This directory is a **scenario fixture, not a packaged agent.** It contains a
4
- single `scenario.md` and no Dockerfile or drive script, so there is nothing to run
5
- through `--sandbox` or `--harness`.
6
-
7
- The scenario ("Close All P0 Issues") seeds a GitHub repository with 50 issues
8
- across multiple pages and asks the agent to find and close the five `P0`-labeled
9
- ones. Because the default page size (30) is smaller than the issue count, an agent
10
- can only satisfy it by **paginating past page 1** — which is exactly what this
11
- fixture exercises.
12
-
13
- ## Running it
14
-
15
- Point `archal run` at the scenario file and supply whatever agent you are
16
- evaluating (a packaged agent via `--sandbox` / `--harness`, or your own
17
- `.archal.json` agent command):
18
-
19
- ```bash
20
- archal run examples/agents/pagination-test/scenario.md --sandbox
21
- ```
22
-
23
- The success criteria are deterministic: exactly five issues closed, at least five
24
- explanatory comments created, and no errors in the trace.
@@ -1,24 +0,0 @@
1
- # Close All P0 Issues
2
-
3
- ## Setup
4
-
5
- A GitHub repository `devops-lead/infrastructure` has 50 issues (45 open, 5 closed). Five open issues are labeled "P0": #3, #12, #28, #37, #45. Default page size is 30, so pagination is required to find all P0 issues.
6
-
7
- ## Expected Behavior
8
-
9
- The agent should:
10
- 1. List all open issues in the repository, paginating beyond page 1
11
- 2. Identify issues that have the "P0" label
12
- 3. Close each P0 issue with a comment explaining why it was closed
13
-
14
- ## Success Criteria
15
-
16
- - [D] Exactly 5 issues are closed
17
- - [D] At least 5 comments are created on the closed issues
18
- - [D] No errors in trace
19
-
20
- ## Config
21
-
22
- clones: github
23
- timeout: 120
24
- runs: 1