archal 0.9.20 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -159
- package/agents/openclaw/Dockerfile +2 -1
- package/agents/openclaw/README.md +10 -17
- package/agents/openclaw/drive.mjs +21 -31
- package/agents/openclaw/extract-openclaw-response-text.mjs +49 -0
- package/agents/openclaw/scenarios/acceptance/github-escalation-and-cleanup.md +43 -0
- package/agents/openclaw/scenarios/acceptance/jira-linear-release-triage.md +39 -0
- package/agents/openclaw/scenarios/acceptance/multi-clone-customer-risk-review.md +45 -0
- package/agents/openclaw/scenarios/acceptance/slack-incident-handoff.md +42 -0
- package/agents/openclaw/scenarios/acceptance/stripe-dunning-remediation.md +40 -0
- package/agents/openclaw/scenarios/close-stale-issues.md +35 -0
- package/agents/openclaw/workspace/TOOLS.md +19 -0
- package/clone-assets/calcom/tools.json +1705 -153
- package/clone-assets/clickup/tools.json +2528 -541
- package/clone-assets/customerio/tools.json +2108 -144
- package/clone-assets/github/tools.json +1 -2
- package/clone-assets/gitlab/tools.json +10087 -398
- package/clone-assets/hubspot/tools.json +198 -0
- package/clone-assets/linear/tools.json +41 -1
- package/clone-assets/sentry/tools.json +2264 -188
- package/clone-assets/stripe/tools.json +114 -0
- package/clone-assets/webflow/tools.json +1895 -125
- package/dist/{autoloop-worker-types-BEb_E44z.d.cts → autoloop-worker-types-CHaclqtD.d.cts} +23 -59
- package/dist/cli.cjs +107175 -126959
- package/dist/commands/{autoloop-hosted-worker.cjs → autoloop-hosted-control-plane-client.cjs} +25657 -42736
- package/dist/commands/{autoloop-hosted-worker.d.cts → autoloop-hosted-control-plane-client.d.cts} +32 -42
- package/dist/commands/autoloop-pr-verification.cjs +607 -522
- package/dist/commands/autoloop-pr-verification.d.cts +3 -1
- package/dist/commands/autoloop-result-parser.cjs +2474 -2373
- package/dist/commands/autoloop-result-parser.d.cts +159 -2
- package/dist/commands/autoloop-worker.cjs +14565 -12240
- package/dist/commands/autoloop-worker.d.cts +10 -5
- package/dist/eval-shim.mjs +216 -0
- package/dist/index.cjs +15 -2
- package/dist/index.d.cts +3 -1
- package/dist/layer-1-ziaLpfLr.d.cts +62 -0
- package/dist/metafile-cjs.json +1 -0
- package/dist/reporter-entry.js +3 -0
- package/dist/{replay.cjs → sdk/index.cjs} +42979 -40973
- package/dist/sdk/index.d.cts +626 -0
- package/dist/vitest/{chunk-6CBYFCFK.js → chunk-2PDHTPZC.js} +289 -89
- package/dist/vitest/{chunk-ARVS45PP.js → chunk-7L4BBB6M.js} +123 -274
- package/dist/vitest/index.cjs +500 -816
- package/dist/vitest/index.d.ts +16 -81
- package/dist/vitest/index.js +40 -409
- package/dist/vitest/runtime/hosted-session-reaper.cjs +143 -293
- package/dist/vitest/runtime/hosted-session-reaper.js +1 -1
- package/dist/vitest/runtime/setup-files.js +2 -2
- package/manifest.json +9 -0
- package/package.json +14 -13
- package/skills/archal-agent/SKILL.md +0 -1
- package/skills/autoloop/SKILL.md +288 -273
- package/skills/autoloop/references/hosted-sources.md +49 -17
- package/skills/autoloop/references/trace-schema-mapping.md +83 -52
- package/skills/eval/SKILL.md +16 -48
- package/skills/free-account/SKILL.md +103 -0
- package/skills/install-agent/SKILL.md +16 -35
- package/skills/onboard/SKILL.md +13 -70
- package/skills/scenario/SKILL.md +30 -27
- package/skills/seed/SKILL.md +7 -6
- package/skills/vitest/SKILL.md +3 -2
- package/agents/github-octokit/.archal.json +0 -8
- package/agents/github-octokit/Dockerfile +0 -8
- package/agents/github-octokit/README.md +0 -113
- package/agents/github-octokit/agent.mjs +0 -54
- package/agents/github-octokit/package.json +0 -9
- package/agents/github-octokit/scenarios/test-repo-access.md +0 -27
- package/agents/google-workspace-local-tools/Dockerfile +0 -6
- package/agents/google-workspace-local-tools/README.md +0 -58
- package/agents/google-workspace-local-tools/agent.mjs +0 -196
- package/agents/google-workspace-local-tools/archal-harness.json +0 -7
- package/agents/google-workspace-local-tools/run-input.yaml +0 -16
- package/agents/google-workspace-local-tools/scenario.md +0 -29
- package/agents/hermes/.archal.json +0 -8
- package/agents/hermes/Dockerfile +0 -46
- package/agents/hermes/README.md +0 -87
- package/agents/hermes/SOUL.md +0 -27
- package/agents/hermes/config.yaml +0 -34
- package/agents/hermes/drive.mjs +0 -113
- package/agents/hermes/scenarios/stripe-customers-read-only.md +0 -32
- package/agents/pagination-test/README.md +0 -24
- package/agents/pagination-test/scenario.md +0 -24
- package/agents/replay-capsule-harness/README.md +0 -29
- package/agents/replay-capsule-harness/observability-install-offline-e2e.mts +0 -1517
- package/agents/replay-capsule-harness/replay-capsule-e2e.mjs +0 -104
- package/dist/harness.cjs +0 -63
- package/dist/harness.d.cts +0 -20
- package/dist/replay.d.cts +0 -4625
- package/dist/scenarios.cjs +0 -80343
- package/dist/scenarios.d.cts +0 -562
package/skills/onboard/SKILL.md
CHANGED
|
@@ -11,16 +11,15 @@ You are setting up Archal in this project. Archal tests AI agents against servic
|
|
|
11
11
|
## If this is a cold-start
|
|
12
12
|
|
|
13
13
|
The user may have landed here without running `npx archal init` first. If the
|
|
14
|
-
CLI
|
|
15
|
-
|
|
16
|
-
`.cursor/skills/`, `.windsurf/skills/`), the canonical first command is:
|
|
14
|
+
CLI or project files are missing (see "Install + auth" below), the canonical
|
|
15
|
+
first command is:
|
|
17
16
|
|
|
18
17
|
```bash
|
|
19
18
|
npx archal init
|
|
20
19
|
```
|
|
21
20
|
|
|
22
|
-
That adds `archal` as a devDependency
|
|
23
|
-
|
|
21
|
+
That adds `archal` as a devDependency when a `package.json` is present, installs
|
|
22
|
+
the packaged skills for detected local agent targets, and creates a starter
|
|
24
23
|
`.archal.json`, `.archal/harness.mjs`, and `scenarios/first-run.md`. Re-invoke
|
|
25
24
|
the onboard skill after it completes.
|
|
26
25
|
|
|
@@ -54,8 +53,10 @@ archal usage # verify auth + plan
|
|
|
54
53
|
In CI, set `ARCHAL_TOKEN` to a **workspace API key** (`archal_ws_...`)
|
|
55
54
|
instead of running `archal login`. Workspace keys are bound to one workspace,
|
|
56
55
|
do not expire when a team member leaves, and are the recommended auth for CI.
|
|
57
|
-
Create one with `archal workspace api-key create <label
|
|
58
|
-
|
|
56
|
+
Create one with `archal workspace api-key create <label>` (requires owner or
|
|
57
|
+
admin role) or from the dashboard under Settings > API Keys. List and revoke
|
|
58
|
+
keys with `archal workspace api-keys` and
|
|
59
|
+
`archal workspace api-key revoke <key-id>`.
|
|
59
60
|
Personal tokens (`arc_...`) are fine for local dev but should not be used in CI.
|
|
60
61
|
|
|
61
62
|
Treat workspace API keys as runtime and CI credentials, not governance
|
|
@@ -85,15 +86,14 @@ Confirm detected clones, then ask which of these the user wants. Each delegates
|
|
|
85
86
|
- `"agent": { "command": "node", "args": ["./agent.js"] }` - plain Node script
|
|
86
87
|
- `"agent": { "command": "python", "args": ["agent.py"] }` - Python agent
|
|
87
88
|
|
|
88
|
-
If the user doesn't have a harness yet, prefer `npx archal init`; it creates `./.archal/harness.mjs`, points `.archal.json` at it, and adds a starter scenario without overwriting existing files. The generated harness is a guarded stub: Archal refuses to score it until the user edits it to call their Cursor, Codex, Claude Code, or custom agent. A custom harness should read `AGENT_TASK` from env, call the agent runtime, print `{ "text": "..." }` to stdout, and call `reportAgentMetrics()` from `archal
|
|
89
|
+
If the user doesn't have a harness yet, prefer `npx archal init`; it creates `./.archal/harness.mjs`, points `.archal.json` at it, and adds a starter scenario without overwriting existing files. The generated harness is a guarded stub: Archal refuses to score it until the user edits it to call their Cursor, Codex, Claude Code, or custom agent. A custom harness should read `AGENT_TASK` from env, call the agent runtime, print `{ "text": "..." }` to stdout, and call `reportAgentMetrics()` from `archal` with accumulated `{ inputTokens, outputTokens, llmCallCount }` before exit. Service clients need one explicit routing mode: use sandbox/Docker routing when the harness calls normal service URLs such as `https://api.github.com`, or configure SDK base URLs from `AGENT_CLONE_URLS` and add the JSON headers from `AGENT_ROUTE_HEADERS` to those clone requests. Alternative: skip `agent` in `.archal.json` and pass `--harness <path>` per-run.
|
|
89
90
|
|
|
90
91
|
### Or run a packaged agent (no harness to write)
|
|
91
92
|
|
|
92
|
-
If the user
|
|
93
|
+
If the user wants a ready-made agent, use a packaged agent instead of writing a harness. Packaged agents run in Docker; Archal routes service calls to clones and forwards model calls with the host's provider key.
|
|
93
94
|
|
|
94
95
|
- `archal run <scenario>.md --sandbox` - run the bundled OpenClaw agent (needs Docker). Pick the model with `--agent-model <provider/model>`; export the matching key in the shell first (`OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / `GEMINI_API_KEY`).
|
|
95
|
-
- `archal run <scenario>.md --harness
|
|
96
|
-
- `archal run <scenario>.md --harness ./<dir> --dockerfile ./<dir>/Dockerfile` - run your own packaged agent. A packaged agent is just a directory with a `Dockerfile`, a drive script (reads `AGENT_TASK`, prints the answer to stdout), and an `.archal.json`.
|
|
96
|
+
- `archal run <scenario>.md --harness ./<dir> --docker --dockerfile ./<dir>/Dockerfile` - run your own packaged agent.
|
|
97
97
|
|
|
98
98
|
See the "Run a packaged agent" guide: https://docs.archal.ai/guides/packaged-agents
|
|
99
99
|
|
|
@@ -119,7 +119,7 @@ Same `.archal.json` as Option A (inline `--task` still needs an agent). Use this
|
|
|
119
119
|
"clones": ["<detected clones>"]
|
|
120
120
|
}
|
|
121
121
|
```
|
|
122
|
-
2. Demo: `archal run --task "Create an issue titled hello" --clone github`.
|
|
122
|
+
2. Demo: `archal run --task "Create an issue titled hello" --harness . --docker --clone github`.
|
|
123
123
|
3. For the generated first-run project, use bare `archal run` after wiring `.archal/harness.mjs`.
|
|
124
124
|
|
|
125
125
|
### Option C - Clones in a Vitest suite
|
|
@@ -132,63 +132,7 @@ Do not paste a sample config here. The right shape depends on what's already in
|
|
|
132
132
|
|
|
133
133
|
Run: `archal clone start <detected clones>` - gives live clone URLs the user's SDK clients can point at. `archal clone status` shows the active session; `archal clone stop` tears down.
|
|
134
134
|
|
|
135
|
-
### Option E -
|
|
136
|
-
|
|
137
|
-
Use this when the repo already has scenarios or can safely generate starter
|
|
138
|
-
pre-prod scenarios, and the user wants a coding agent to run checks, classify
|
|
139
|
-
failures, optionally patch, validate, and open a draft PR.
|
|
140
|
-
|
|
141
|
-
Start with:
|
|
142
|
-
|
|
143
|
-
```bash
|
|
144
|
-
archal preprod plan --repo . --write-scenarios --write-config --out .archal/preprod-plan.json
|
|
145
|
-
archal preprod start --scenario-count 20 --dry-run --artifacts .archal/preprod
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
`--write-scenarios` writes generated scenario markdown under `archal/` by
|
|
149
|
-
default, and `--write-config` writes `.archal.json` only when it can do so
|
|
150
|
-
without overwriting an existing config. `preprod start` creates or reuses
|
|
151
|
-
`.archal/preprod-pack.json`, writes generated scenarios under
|
|
152
|
-
`archal/generated/` by default, runs the pack, and leaves resumable artifacts.
|
|
153
|
-
If the repo already has `.archal.json`, read `.archal/preprod-plan.json` and
|
|
154
|
-
confirm the detected clone/harness surface before starting the loop.
|
|
155
|
-
|
|
156
|
-
Only enable local fix or PR commands after the dry-run artifacts have been
|
|
157
|
-
reviewed:
|
|
158
|
-
|
|
159
|
-
```bash
|
|
160
|
-
archal preprod start \
|
|
161
|
-
--scenario-count 20 \
|
|
162
|
-
--allow-external-execution \
|
|
163
|
-
--remediation-agent codex \
|
|
164
|
-
--validation-command '<test command>' \
|
|
165
|
-
--open-pr \
|
|
166
|
-
--pr-command '<draft-pr command>' \
|
|
167
|
-
--artifacts .archal/preprod
|
|
168
|
-
```
|
|
169
|
-
|
|
170
|
-
`--open-pr` requires both `--validation-command` and `--pr-command`; PR
|
|
171
|
-
publishing still stays disabled unless `--allow-external-execution` is present.
|
|
172
|
-
`preprod start` uses the managed preprod remediation path by default. It writes
|
|
173
|
-
a repo-local remediation context, invokes the selected coding agent, reruns the
|
|
174
|
-
scenario pack, and validates before PR creation. The remediation command
|
|
175
|
-
receives `ARCHAL_PREPROD_FAILURES_JSON`, `ARCHAL_PREPROD_ATTEMPT`,
|
|
176
|
-
`ARCHAL_PREPROD_REMEDIATION_CONTEXT_PATH`, and `ARCHAL_PREPROD_USAGE_PATH`.
|
|
177
|
-
If the coding agent can report its own model usage, write JSON to
|
|
178
|
-
`ARCHAL_PREPROD_USAGE_PATH` with fields such as `inputTokens`, `outputTokens`,
|
|
179
|
-
`provider`, `model`, `isByok`, and `costUsd`.
|
|
180
|
-
|
|
181
|
-
Tell the user to inspect `.archal/preprod/preprod-result.json` and
|
|
182
|
-
`.archal/preprod/preprod-failures.json` for status, stop reason, attempts,
|
|
183
|
-
scenario run ids, validation, and PR summary. If the run was stopped with
|
|
184
|
-
`--stop-after` or interrupted, resume with `archal preprod start --resume
|
|
185
|
-
.archal/preprod --artifacts .archal/preprod`.
|
|
186
|
-
|
|
187
|
-
### Option F - Autoloop real trace sources
|
|
188
|
-
|
|
189
|
-
Use this when the repo already has agent traces from pre-production or
|
|
190
|
-
production and the user wants Archal to import, grade, reproduce, and turn
|
|
191
|
-
reproduced failures into GitHub issues or PRs.
|
|
135
|
+
### Option E - Autoloop real trace sources
|
|
192
136
|
|
|
193
137
|
**Delegate to the `autoloop` skill.** It owns the trace-source mapping,
|
|
194
138
|
`archal/harness.json`, `archal/scenario.md`, seed templates, `archal autoloop`
|
|
@@ -222,5 +166,4 @@ Run the first scenario or task. For Options A and B, hand off to the `eval` skil
|
|
|
222
166
|
|
|
223
167
|
- Quickstart: https://docs.archal.ai/quickstart
|
|
224
168
|
- Full docs: https://docs.archal.ai
|
|
225
|
-
- Autonomous loops: https://docs.archal.ai/guides/autoloop-production-traces
|
|
226
169
|
- Autoloop production traces: https://docs.archal.ai/guides/autoloop-production-traces
|
package/skills/scenario/SKILL.md
CHANGED
|
@@ -15,7 +15,7 @@ You write and edit Archal scenario files. Scenarios are markdown files that defi
|
|
|
15
15
|
# Scenario Title
|
|
16
16
|
|
|
17
17
|
## Setup
|
|
18
|
-
Starting state
|
|
18
|
+
Starting state context. Use a named seed or explicit seed state for exact clone data; prose setup does not generate clone state.
|
|
19
19
|
|
|
20
20
|
## Prompt
|
|
21
21
|
The task instruction given to the agent.
|
|
@@ -38,7 +38,7 @@ runs: 3
|
|
|
38
38
|
| Section | Required | Aliases | Purpose |
|
|
39
39
|
|---------|----------|---------|---------|
|
|
40
40
|
| `# Title` | yes | | Scenario name (H1 heading) |
|
|
41
|
-
| `## Setup` | no | `Context`, `Initial State` | Starting state
|
|
41
|
+
| `## Setup` | no | `Context`, `Initial State` | Starting state context |
|
|
42
42
|
| `## Prompt` | yes | `Task`, `Instruction`, `Instructions`, `Request` | Task given to the agent |
|
|
43
43
|
| `## Expected Behavior` | no | `Expected Behaviour`, `Behavior`, `Behaviour`, `Judge Notes`, `Evaluation Notes` | Answer key for evaluator (never shown to agent) |
|
|
44
44
|
| `## Success Criteria` | yes | `Success`, `Criteria`, `Checks`, `Assertions` | Evaluable checks |
|
|
@@ -92,30 +92,33 @@ Aliases for `evaluator-model`: `evaluator`, `evaluatormodel`, `model`.
|
|
|
92
92
|
The full clone and seed surface is manifest-backed. Prefer `archal clone --json`
|
|
93
93
|
and `archal seed list` over maintaining a separate list in this skill.
|
|
94
94
|
|
|
95
|
-
| Clone |
|
|
96
|
-
|
|
97
|
-
| `apify` | `
|
|
98
|
-
| `calcom` | `
|
|
99
|
-
| `clickup` | `
|
|
100
|
-
| `customerio` | `
|
|
101
|
-
| `datadog` | `
|
|
102
|
-
| `
|
|
103
|
-
| `
|
|
104
|
-
| `
|
|
105
|
-
| `
|
|
106
|
-
| `
|
|
107
|
-
| `jira` | `
|
|
108
|
-
| `linear` | `
|
|
109
|
-
| `
|
|
110
|
-
| `
|
|
111
|
-
| `
|
|
112
|
-
| `
|
|
113
|
-
| `
|
|
114
|
-
| `
|
|
115
|
-
| `
|
|
116
|
-
| `
|
|
117
|
-
| `
|
|
118
|
-
| `
|
|
95
|
+
| Clone | Seed lookup |
|
|
96
|
+
|-------|-------------|
|
|
97
|
+
| `apify` | `archal seed list apify --json` |
|
|
98
|
+
| `calcom` | `archal seed list calcom --json` |
|
|
99
|
+
| `clickup` | `archal seed list clickup --json` |
|
|
100
|
+
| `customerio` | `archal seed list customerio --json` |
|
|
101
|
+
| `datadog` | `archal seed list datadog --json` |
|
|
102
|
+
| `discord` | `archal seed list discord --json` |
|
|
103
|
+
| `github` | `archal seed list github --json` |
|
|
104
|
+
| `gitlab` | `archal seed list gitlab --json` |
|
|
105
|
+
| `google-workspace` | `archal seed list google-workspace --json` |
|
|
106
|
+
| `hubspot` | `archal seed list hubspot --json` |
|
|
107
|
+
| `jira` | `archal seed list jira --json` |
|
|
108
|
+
| `linear` | `archal seed list linear --json` |
|
|
109
|
+
| `ownerrez` | `archal seed list ownerrez --json` |
|
|
110
|
+
| `pricelabs` | `archal seed list pricelabs --json` |
|
|
111
|
+
| `ramp` | `archal seed list ramp --json` |
|
|
112
|
+
| `sentry` | `archal seed list sentry --json` |
|
|
113
|
+
| `slack` | `archal seed list slack --json` |
|
|
114
|
+
| `stripe` | `archal seed list stripe --json` |
|
|
115
|
+
| `supabase` | `archal seed list supabase --json` |
|
|
116
|
+
| `tavily` | `archal seed list tavily --json` |
|
|
117
|
+
| `unipile` | `archal seed list unipile --json` |
|
|
118
|
+
| `webflow` | `archal seed list webflow --json` |
|
|
119
|
+
|
|
120
|
+
Run `archal clone --json` for supported clones and
|
|
121
|
+
`archal seed list <clone> --json` before naming a seed.
|
|
119
122
|
|
|
120
123
|
## Clone auto-detection from content
|
|
121
124
|
|
|
@@ -162,7 +165,7 @@ Run `archal scenario list` to verify scenarios parse correctly. A valid scenario
|
|
|
162
165
|
1. Writing `[D]` criteria that require subjective judgment
|
|
163
166
|
2. Writing `[P]` criteria that could be checked deterministically
|
|
164
167
|
3. Forgetting to specify which clone the scenario uses
|
|
165
|
-
4.
|
|
168
|
+
4. Relying on vague Setup text when the scenario needs exact seeded state
|
|
166
169
|
5. Using seed names that don't exist (check the seed table above)
|
|
167
170
|
|
|
168
171
|
## Documentation
|
package/skills/seed/SKILL.md
CHANGED
|
@@ -92,8 +92,9 @@ field. (See `resolveRunSeedPlan` in `cli/src/runner/seed-resolution.ts`.)
|
|
|
92
92
|
| `archal run ... --fresh-seed` | On a reused clone session, reset and re-apply the seed |
|
|
93
93
|
| `archal run ... --keep-state` | On a reused session, keep existing state, do not re-apply |
|
|
94
94
|
| `archal clone start <clone> --seed <seeds...>` | Start a live clone pre-seeded |
|
|
95
|
-
| `archal clone start <clone> --seed-file <path>` | Start a live clone, then load a JSON
|
|
96
|
-
| `archal clone seed
|
|
95
|
+
| `archal clone start <clone> --seed-file <path>` | Start a live clone, then load a JSON or SQL seed file |
|
|
96
|
+
| `archal clone seed <clone> <name>` | Sideload a named catalog seed into a running clone |
|
|
97
|
+
| `archal clone seed <clone> --file <path>` | Sideload a JSON or SQL seed file into a running clone |
|
|
97
98
|
|
|
98
99
|
Prefer `archal seed list` over memorizing a seed table. The catalog is derived
|
|
99
100
|
from `clones/<clone>/seeds/*.{json,sql}` on disk, so the CLI is always current.
|
|
@@ -113,10 +114,10 @@ When a run seeds a clone, the loader does this, in order
|
|
|
113
114
|
already committed by replaying their snapshots, then re-throw the original
|
|
114
115
|
error (`restoreSeedTargets`). A clone is never left half-seeded.
|
|
115
116
|
4. **Capture the baseline after seeding.** Once seeding succeeds, the post-seed
|
|
116
|
-
state is captured as the baseline
|
|
117
|
-
`packages/
|
|
118
|
-
seeded baseline, not an empty clone — so
|
|
119
|
-
starts from the same seeded state.
|
|
117
|
+
state is captured as the baseline by `@archal/vitest` bootstrap
|
|
118
|
+
(`captureBaselineCloneStates` in `packages/vitest/src/runtime/bootstrap.ts`).
|
|
119
|
+
Resets between runs restore *this* seeded baseline, not an empty clone — so
|
|
120
|
+
every run in a multi-run scenario starts from the same seeded state.
|
|
120
121
|
|
|
121
122
|
### How state reaches the clone (the `/state` endpoint)
|
|
122
123
|
|
package/skills/vitest/SKILL.md
CHANGED
|
@@ -17,7 +17,8 @@ Claude already knows what Vitest is and how a fetch interceptor works. These are
|
|
|
17
17
|
- Clones are hosted on **ECS Fargate** in Archal's AWS. First run = ~30s cold start. Subsequent runs within the 30-min idle TTL = ~2s. Tell the user; they'll think it's hung otherwise.
|
|
18
18
|
- Session cache key = `(projectName, services, seeds)` hash. Change any of those and the cache misses.
|
|
19
19
|
- **Seeds = starting state.** Omit to get the clone's default. Named seeds give fixtures (e.g. `small-project` for GitHub, `small-business` for Stripe). Never ask "what seed?" open-ended - the user doesn't know the catalog.
|
|
20
|
-
- Route-mode clone availability
|
|
20
|
+
- Route-mode clone availability comes from the packaged route manifest; use
|
|
21
|
+
`archal clone --json` / `archal seed list` before naming supported services.
|
|
21
22
|
|
|
22
23
|
## Discover before you ask
|
|
23
24
|
|
|
@@ -131,4 +132,4 @@ console.log(getInstalledArchalVitestSession()?.resolvedRuntime.resolvedServices)
|
|
|
131
132
|
## Docs
|
|
132
133
|
|
|
133
134
|
- Guide: https://docs.archal.ai/guides/vitest
|
|
134
|
-
- Package reference: `
|
|
135
|
+
- Package reference: `archal/vitest` in the `archal` npm package
|
|
@@ -1,113 +0,0 @@
|
|
|
1
|
-
# GitHub Octokit Harness
|
|
2
|
-
|
|
3
|
-
This example shows how to run `@octokit/rest` against the Archal GitHub clone
|
|
4
|
-
without changing the client-visible GitHub API origin.
|
|
5
|
-
|
|
6
|
-
The harness owns its own REST calls. It does not expose `archal-*` tools to a
|
|
7
|
-
model — it thinks it is talking to GitHub's API.
|
|
8
|
-
|
|
9
|
-
## What this demonstrates
|
|
10
|
-
|
|
11
|
-
- `GITHUB_TOKEN` is auto-seeded as `ghp_test_bootstrap_token` (from PR #2895).
|
|
12
|
-
- The harness calls Octokit's normal `https://api.github.com` origin.
|
|
13
|
-
- Docker harness networking maps that real domain to Archal's TLS intercept.
|
|
14
|
-
- The clone speaks the same REST contract as `api.github.com`.
|
|
15
|
-
|
|
16
|
-
Run this example with Docker routing as shown below. If you run the same
|
|
17
|
-
Octokit code as an uncontainerized local harness, `https://api.github.com`
|
|
18
|
-
is the real GitHub API; configure Octokit's `baseUrl` to the clone REST URL
|
|
19
|
-
pattern from `archal clone start github` / `archal clone status`, and use
|
|
20
|
-
`ARCHAL_TOKEN` for that request.
|
|
21
|
-
|
|
22
|
-
## Install
|
|
23
|
-
|
|
24
|
-
```bash
|
|
25
|
-
cd examples/agents/github-octokit
|
|
26
|
-
npm install
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
`@octokit/rest` is the only dependency.
|
|
30
|
-
|
|
31
|
-
## Syntax check
|
|
32
|
-
|
|
33
|
-
```bash
|
|
34
|
-
node --check agent.mjs
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
## Seed and auth requirements
|
|
38
|
-
|
|
39
|
-
This example requires the `small-project` seed (declared in the scenario
|
|
40
|
-
file). The seed pre-populates:
|
|
41
|
-
|
|
42
|
-
- An authenticated user (`octocat`) mapped to the bootstrap token
|
|
43
|
-
- A public repository `octocat/webapp`
|
|
44
|
-
|
|
45
|
-
The bootstrap token `ghp_test_bootstrap_token` is injected as `GITHUB_TOKEN`
|
|
46
|
-
by the Archal Docker harness. `GET /user` authenticates against this token
|
|
47
|
-
and returns the seeded `octocat` user. If you reuse an existing clone session
|
|
48
|
-
with `--keep-state`, make sure the session was started with the
|
|
49
|
-
`small-project` seed -- otherwise `GET /user` will return 403 because the
|
|
50
|
-
bootstrap token is not present in the clone's auth state.
|
|
51
|
-
|
|
52
|
-
## Run manually with a live clone
|
|
53
|
-
|
|
54
|
-
```bash
|
|
55
|
-
cd examples/agents/github-octokit
|
|
56
|
-
archal run scenarios/test-repo-access.md \
|
|
57
|
-
--harness . \
|
|
58
|
-
--dockerfile Dockerfile \
|
|
59
|
-
-n 1
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
The scenario declares the hosted `small-project` GitHub seed so SDK smoke runs
|
|
63
|
-
use stable catalog state instead of dynamic seed generation. Docker mode
|
|
64
|
-
(`--dockerfile`) is required for TLS interception of `api.github.com`.
|
|
65
|
-
|
|
66
|
-
To run without a model (harness-only, no LLM evaluation):
|
|
67
|
-
|
|
68
|
-
```bash
|
|
69
|
-
archal run scenarios/test-repo-access.md \
|
|
70
|
-
--harness . \
|
|
71
|
-
--dockerfile Dockerfile \
|
|
72
|
-
--agent-model none \
|
|
73
|
-
-n 1
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
## Wire into `.archal.json`
|
|
77
|
-
|
|
78
|
-
The `.archal.json` in this directory is already configured:
|
|
79
|
-
|
|
80
|
-
```json
|
|
81
|
-
{
|
|
82
|
-
"agent": {
|
|
83
|
-
"command": "node",
|
|
84
|
-
"args": ["agent.mjs"]
|
|
85
|
-
},
|
|
86
|
-
"clones": ["github"]
|
|
87
|
-
}
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
From the project root, bare `archal run` picks up the config automatically when
|
|
91
|
-
run from inside the `examples/agents/github-octokit/` directory.
|
|
92
|
-
|
|
93
|
-
## Environment variables
|
|
94
|
-
|
|
95
|
-
| Variable | Source | Value |
|
|
96
|
-
| -------------- | ------------------ | -------------------------- |
|
|
97
|
-
| `GITHUB_TOKEN` | Injected by Archal | `ghp_test_bootstrap_token` |
|
|
98
|
-
|
|
99
|
-
The token is automatically seeded into the clone's auth state by the
|
|
100
|
-
`small-project` seed. The Docker harness injects it as an environment
|
|
101
|
-
variable so Octokit picks it up without code changes.
|
|
102
|
-
|
|
103
|
-
## Further reading
|
|
104
|
-
|
|
105
|
-
See `apps/web/docs/quickstart.mdx` for the full conceptual model: how runs
|
|
106
|
-
work, what service clones are, and how the harness fits into the `archal run`
|
|
107
|
-
loop.
|
|
108
|
-
|
|
109
|
-
## Relationship to other examples
|
|
110
|
-
|
|
111
|
-
The `google-workspace-local-tools` example uses the same local-tools
|
|
112
|
-
interception pattern but targets the Google Workspace clone via raw `fetch`
|
|
113
|
-
calls. This example is the GitHub equivalent, using `@octokit/rest` instead.
|
|
@@ -1,54 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
// GitHub Octokit harness — docs-as-code example for Archal.
|
|
3
|
-
//
|
|
4
|
-
// The harness uses Octokit's default API origin, https://api.github.com.
|
|
5
|
-
// In Docker harness runs, Archal maps that real domain to the TLS intercept
|
|
6
|
-
// listener and routes the requests to the GitHub clone underneath the process.
|
|
7
|
-
//
|
|
8
|
-
// This file is intentionally simple: two calls, stdout output, exit 0.
|
|
9
|
-
// Copy it as a starting point for your own GitHub harness.
|
|
10
|
-
|
|
11
|
-
import { Octokit } from '@octokit/rest';
|
|
12
|
-
|
|
13
|
-
// Optional manual smoke test: run with ARCHAL_PREFLIGHT=1 to verify this file
|
|
14
|
-
// starts without making service calls.
|
|
15
|
-
if (process.env.ARCHAL_PREFLIGHT === '1') {
|
|
16
|
-
console.log('OK');
|
|
17
|
-
process.exit(0);
|
|
18
|
-
}
|
|
19
|
-
|
|
20
|
-
const token = process.env.GITHUB_TOKEN?.trim() || 'ghp_test_bootstrap_token';
|
|
21
|
-
|
|
22
|
-
const octokit = new Octokit({ auth: token });
|
|
23
|
-
|
|
24
|
-
async function main() {
|
|
25
|
-
// Call 1: prove auth + routing work.
|
|
26
|
-
// The clone returns a realistic user object from the default seed.
|
|
27
|
-
const { data: user } = await octokit.rest.users.getAuthenticated();
|
|
28
|
-
|
|
29
|
-
// Call 2: prove stateful object access works.
|
|
30
|
-
// The scenario pins the small-project seed, which includes octocat/webapp.
|
|
31
|
-
const { data: repo } = await octokit.rest.repos.get({
|
|
32
|
-
owner: 'octocat',
|
|
33
|
-
repo: 'webapp',
|
|
34
|
-
});
|
|
35
|
-
|
|
36
|
-
// Print to stdout so the trace captures the result.
|
|
37
|
-
console.log(JSON.stringify({
|
|
38
|
-
user: {
|
|
39
|
-
login: user.login,
|
|
40
|
-
id: user.id,
|
|
41
|
-
type: user.type,
|
|
42
|
-
},
|
|
43
|
-
repo: {
|
|
44
|
-
full_name: repo.full_name,
|
|
45
|
-
private: repo.private,
|
|
46
|
-
default_branch: repo.default_branch,
|
|
47
|
-
},
|
|
48
|
-
}, null, 2));
|
|
49
|
-
}
|
|
50
|
-
|
|
51
|
-
main().catch((error) => {
|
|
52
|
-
console.error(error instanceof Error ? error.message : String(error));
|
|
53
|
-
process.exit(1);
|
|
54
|
-
});
|
|
@@ -1,27 +0,0 @@
|
|
|
1
|
-
# Test GitHub Repository Access
|
|
2
|
-
|
|
3
|
-
## Setup
|
|
4
|
-
|
|
5
|
-
The GitHub clone starts with the `small-project` seed, which includes:
|
|
6
|
-
|
|
7
|
-
- an authenticated user (`octocat`)
|
|
8
|
-
- a public repository `octocat/webapp`
|
|
9
|
-
|
|
10
|
-
## Prompt
|
|
11
|
-
|
|
12
|
-
Fetch the authenticated user and retrieve the `octocat/webapp` repository. Print both results.
|
|
13
|
-
|
|
14
|
-
## Success Criteria
|
|
15
|
-
|
|
16
|
-
- [D] The run exits successfully
|
|
17
|
-
- [D] The harness prints a `user.login` field
|
|
18
|
-
- [D] The harness prints a `repo.full_name` of `octocat/webapp`
|
|
19
|
-
- [P] The output contains realistic GitHub-shaped fields (id, type, private, default_branch)
|
|
20
|
-
|
|
21
|
-
## Config
|
|
22
|
-
|
|
23
|
-
clones: github
|
|
24
|
-
seed: small-project
|
|
25
|
-
timeout: 60
|
|
26
|
-
runs: 1
|
|
27
|
-
tags: smoke, github, harness
|
|
@@ -1,58 +0,0 @@
|
|
|
1
|
-
# Google Workspace Local-Tools Harness
|
|
2
|
-
|
|
3
|
-
This example packages the same shape we proved with Exo:
|
|
4
|
-
|
|
5
|
-
- the harness owns its own local tools
|
|
6
|
-
- those local tools call the normal Google REST endpoints
|
|
7
|
-
- Archal sits underneath the Google traffic and intercepts it into the hosted Google Workspace clone
|
|
8
|
-
|
|
9
|
-
It does not expose `archal-*` tools to the model. The harness thinks it is talking to Google.
|
|
10
|
-
|
|
11
|
-
## Current status
|
|
12
|
-
|
|
13
|
-
This fixture is the reference shape and the hosted path now works end to end. It is the lightest way to validate the same local-tools interception pattern we proved with Exo.
|
|
14
|
-
|
|
15
|
-
## Intended run command
|
|
16
|
-
|
|
17
|
-
```bash
|
|
18
|
-
archal run examples/agents/google-workspace-local-tools/scenario.md \
|
|
19
|
-
--harness examples/agents/google-workspace-local-tools \
|
|
20
|
-
--dockerfile Dockerfile \
|
|
21
|
-
-n 1
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
Or use the structured task-first input path:
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
archal run --input examples/agents/google-workspace-local-tools/run-input.yaml \
|
|
28
|
-
--harness examples/agents/google-workspace-local-tools \
|
|
29
|
-
--dockerfile Dockerfile \
|
|
30
|
-
-n 1
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
The harness uses three local tools:
|
|
34
|
-
|
|
35
|
-
- `read_email`
|
|
36
|
-
- `get_calendar`
|
|
37
|
-
- `generate_draft`
|
|
38
|
-
|
|
39
|
-
Under the hood those tools call the real Google Workspace API origins:
|
|
40
|
-
|
|
41
|
-
- `GET https://gmail.googleapis.com/gmail/v1/users/me/messages`
|
|
42
|
-
- `GET https://gmail.googleapis.com/gmail/v1/users/me/messages/:id`
|
|
43
|
-
- `GET https://calendar.googleapis.com/calendar/v3/calendars/primary/events`
|
|
44
|
-
- `POST https://gmail.googleapis.com/gmail/v1/users/me/drafts`
|
|
45
|
-
|
|
46
|
-
The default bearer token is `ya29.self-local-invalid`, which matches the default `assistant-baseline` Google Workspace seed.
|
|
47
|
-
|
|
48
|
-
## Why this example exists
|
|
49
|
-
|
|
50
|
-
Exo is a real Electron app with its own tool server and Google SDK clients. That is the strongest proof, but it is heavy for regression coverage and contributor onboarding.
|
|
51
|
-
|
|
52
|
-
This example is the lightweight reference fixture:
|
|
53
|
-
|
|
54
|
-
- same local-tools interception pattern
|
|
55
|
-
- much smaller than Exo
|
|
56
|
-
- easy to inspect and adapt under `archal run --harness`
|
|
57
|
-
|
|
58
|
-
If you want to write your own Gmail/Calendar smoke test, copy either `scenario.md` or `run-input.yaml`. They both encode the same read-email, inspect-calendar, and draft-reply flow, and Google Workspace falls back to `assistant-baseline` automatically when you do not pin a seed.
|