@osfactory/har 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -9
- package/dist/index.js +1469 -460
- package/dist/templates/agent-skills/har-maintain.md +2 -0
- package/dist/templates/agent-skills/setup-har.md +13 -2
- package/dist/templates/har-boilerplate/CLAUDE.agent.md +4 -2
- package/dist/templates/har-boilerplate/STAGES.md +91 -0
- package/dist/templates/har-boilerplate/agent-slot.sh +46 -6
- package/dist/templates/har-boilerplate/launch.sh +2 -1
- package/dist/templates/har-boilerplate/stages/README.sh +7 -5
- package/dist/templates/har-boilerplate/verify.sh +9 -2
- package/dist/templates/har-boilerplate-cli/CLAUDE.agent.md +3 -1
- package/dist/templates/har-boilerplate-cli/STAGES.md +91 -0
- package/dist/templates/har-boilerplate-cli/agent-slot.sh +49 -9
- package/dist/templates/har-boilerplate-cli/stages/README.sh +7 -5
- package/dist/templates/har-boilerplate-cli/verify.sh +9 -2
- package/dist/templates/har-boilerplate-ios/CLAUDE.agent.md +2 -0
- package/dist/templates/har-boilerplate-ios/STAGES.md +91 -0
- package/dist/templates/har-boilerplate-ios/agent-slot.sh +35 -6
- package/dist/templates/har-boilerplate-ios/stages/README.sh +6 -7
- package/dist/templates/har-boilerplate-ios/verify.sh +9 -3
- package/dist/templates/stage-templates/custom-stage-skeleton.sh +65 -0
- package/dist/templates/stage-templates/playwright/template.manifest.json +9 -1
- package/dist/templates/stage-templates/rocketsim/template.manifest.json +2 -1
- package/package.json +2 -2
|
@@ -14,6 +14,8 @@ This compares the harness against the current templates and repository, refreshe
|
|
|
14
14
|
|
|
15
15
|
Read `.har/ADAPT-PROMPT.md` and execute its instructions now, in this session: reconcile `.har/` scripts (`launch.sh`, `verify.sh`, `setup-infra.sh`, `harness.env`, `stages.json`) and `AGENT.md` with the repository's current stack, ports, and commands. Preserve intentional project-specific customizations — fix drift, don't blindly reset.
|
|
16
16
|
|
|
17
|
+
Also check stage drift: compare the repository's current check commands (package.json scripts, Makefile, CI) against the stages registered in `.har/stages.json`. Register missing checks (`har env add-stage <id> --custom --command "..." --verification`), and remove or fix stages whose commands no longer exist. `.har/STAGES.md` documents the contract.
|
|
18
|
+
|
|
17
19
|
## 3. Finalize and prove it works
|
|
18
20
|
|
|
19
21
|
```bash
|
|
@@ -38,7 +38,18 @@ If `.har/` already exists, stop and suggest `/har-maintain` instead.
|
|
|
38
38
|
|
|
39
39
|
`har env init` prints an adaptation prompt and writes it to `.har/ADAPT-PROMPT.md`. Read that file and **execute its instructions yourself, now, in this session** — tailor `.har/` scripts (`launch.sh`, `verify.sh`, `setup-infra.sh`, `harness.env`, `stages.json`) and `AGENT.md` to this repository's real stack, ports, and commands. Do not use `--auto` and do not ask the user to paste anything.
|
|
40
40
|
|
|
41
|
-
## 5.
|
|
41
|
+
## 5. Register the project's checks as stages
|
|
42
|
+
|
|
43
|
+
Convert the repository's real check commands (test, lint, typecheck, whatever CI runs) into registered stages so they run in `verify --full` and are visible to every agent. Read `.har/STAGES.md` for the contract, then:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
har env add-stage unit-tests --custom --kind test --command "npm test" --verification
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
- Use `--command` for one-liner checks; use `--script` when a check needs the slot's env, ports, or artifacts (then implement the scaffolded `.har/stages/<id>.sh`).
|
|
50
|
+
- Rich integrations ship as templates: `har env add-stage --list`, then e.g. `har env add-stage playwright` (web) or `har env add-stage rocketsim` (iOS).
|
|
51
|
+
|
|
52
|
+
## 6. Prove the harness works
|
|
42
53
|
|
|
43
54
|
```bash
|
|
44
55
|
har env launch 1
|
|
@@ -47,7 +58,7 @@ har env verify 1
|
|
|
47
58
|
|
|
48
59
|
Fix the harness scripts until both pass. Then tear down or keep the slot as the user prefers (`har env teardown 1` keeps the branch).
|
|
49
60
|
|
|
50
|
-
##
|
|
61
|
+
## 7. Commit
|
|
51
62
|
|
|
52
63
|
After the user confirms, commit the harness:
|
|
53
64
|
|
|
@@ -39,13 +39,15 @@ is alive; it does not automatically mean an agent can use the app.
|
|
|
39
39
|
|
|
40
40
|
- [ ] Full verification returns `"status": "pass"` (`har env verify ${AGENT_ID} --full`, MCP `har_run_verification` with `full: true`, or `./.har/verify.sh ${AGENT_ID} --full`)
|
|
41
41
|
- [ ] The slot is agent-usable for this repo's documented smoke workflow, not only health-check green
|
|
42
|
-
- [ ]
|
|
42
|
+
- [ ] Full verify runs every registered stage in `stages.json` `verificationStages` (Playwright, custom checks, …) — when `stages/browser-e2e.sh` exists, adapt specs under `tests/` for UI changes
|
|
43
43
|
- [ ] New behavior has automated test coverage (unit and/or browser as appropriate)
|
|
44
44
|
- [ ] Changes committed **in the session worktree** with a clear message
|
|
45
45
|
- [ ] The user got the preview URLs to test the app themselves
|
|
46
46
|
- [ ] Finish with `har env complete ${AGENT_ID}` (or MCP `har_complete_environment`) — records the validation and tears down while **keeping the session branch** for the user to push / open a PR
|
|
47
47
|
|
|
48
|
-
Quick loop during development: MCP `har_run_verification`, `har env verify ${AGENT_ID}`, or `./.har/verify.sh ${AGENT_ID}` (
|
|
48
|
+
Quick loop during development: MCP `har_run_verification`, `har env verify ${AGENT_ID}`, or `./.har/verify.sh ${AGENT_ID}` (smoke + health only; `--full` adds the registered verification stages).
|
|
49
|
+
|
|
50
|
+
Stages are the harness's single vocabulary for checks: templates and custom stages compile to generic kinds in `.har/stages.json`, and you interact with them only through the registry (`har_run_stage`, `verify`), never stack-specific tooling. Authoring guide: `.har/STAGES.md`.
|
|
49
51
|
|
|
50
52
|
## Project commands
|
|
51
53
|
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# HAR Stages — authoring guide
|
|
2
|
+
|
|
3
|
+
Stages are the harness's single vocabulary for runnable checks and lifecycle
|
|
4
|
+
actions. Everything — shipped templates (`playwright`, `rocketsim`), your
|
|
5
|
+
project's test/lint commands, bespoke validation scripts — registers in
|
|
6
|
+
`.har/stages.json` with the same schema, and agents interact with stages only
|
|
7
|
+
through that registry (CLI `har env verify`, MCP `har_run_stage` /
|
|
8
|
+
`har_run_verification`), never through stack-specific tooling.
|
|
9
|
+
|
|
10
|
+
## The registry: `.har/stages.json`
|
|
11
|
+
|
|
12
|
+
```jsonc
|
|
13
|
+
{
|
|
14
|
+
"verificationStages": ["typecheck", "unit-tests", "api-health", "browser-e2e"],
|
|
15
|
+
"stages": [
|
|
16
|
+
{
|
|
17
|
+
"id": "browser-e2e", // stable, shell-friendly slug
|
|
18
|
+
"kind": "test", // setup | launch | verify | test | inspect | reset | teardown | custom
|
|
19
|
+
"description": "Playwright E2E",
|
|
20
|
+
"script": "stages/browser-e2e.sh", // relative to .har/ — OR use "command"
|
|
21
|
+
"requiresAgentId": true, // default true for test/verify/custom kinds
|
|
22
|
+
"artifacts": [{ "path": ".har/artifacts/browser-e2e", "kind": "directory" }]
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "unit-tests-fast",
|
|
26
|
+
"kind": "test",
|
|
27
|
+
"command": "npm test -- --agent {agentId}" // {agentId} is substituted at run time
|
|
28
|
+
}
|
|
29
|
+
]
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Optional stage fields: `cwd` (working directory), `env` (extra env vars),
|
|
34
|
+
`group`, `acceptsArgs` (extra CLI args the stage accepts, e.g. `["--full"]`).
|
|
35
|
+
|
|
36
|
+
## Two ways to define a stage
|
|
37
|
+
|
|
38
|
+
**Command stages** — the default for simple checks (`npm test`, `swiftlint`,
|
|
39
|
+
`make check`). One JSON entry, zero files:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
har env add-stage unit-tests-fast --custom --kind test --command "npm test" --verification
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Script stages** — for anything that needs the slot's env, ports, or
|
|
46
|
+
artifacts. Scaffold a contract-compliant skeleton:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
har env add-stage db-integrity --custom --script --description "Check DB invariants"
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
then implement the TODO block in `.har/stages/db-integrity.sh`.
|
|
53
|
+
|
|
54
|
+
## The stage script contract
|
|
55
|
+
|
|
56
|
+
Every script under `.har/stages/` must:
|
|
57
|
+
|
|
58
|
+
1. Source `harness.env` and `agent-slot.sh` from `.har/`.
|
|
59
|
+
2. Take the agent slot id as `$1` (validate with `validate_agent_id`); extra
|
|
60
|
+
args may follow.
|
|
61
|
+
3. Load the slot env via `resolve_agent_env_file` and run checks from the
|
|
62
|
+
agent's work dir (`resolve_agent_work_dir`).
|
|
63
|
+
4. Write artifacts (reports, screenshots, logs) under `.har/artifacts/<id>/`.
|
|
64
|
+
5. Print **only** the normalized JSON result object on stdout
|
|
65
|
+
(`status`, `stageId`, `agent_id`, `total_ms`, …); log progress to stderr.
|
|
66
|
+
6. Exit with the real status code (0 = pass).
|
|
67
|
+
|
|
68
|
+
The scaffolded skeleton implements all of this — replace its TODO block.
|
|
69
|
+
|
|
70
|
+
## Verification membership
|
|
71
|
+
|
|
72
|
+
Listing a stage id in `verificationStages` is what includes it in
|
|
73
|
+
`har env verify <id> --full`. Ids that match a registered stage run via their
|
|
74
|
+
script/command; ids without a registry entry (e.g. `typecheck`, `api-health`)
|
|
75
|
+
are inline steps owned by `.har/verify.sh`. Lifecycle kinds
|
|
76
|
+
(`setup`/`launch`/`reset`/`teardown`/`inspect`) and `verify` itself never run
|
|
77
|
+
as part of verification, even if listed.
|
|
78
|
+
|
|
79
|
+
## Commit gate
|
|
80
|
+
|
|
81
|
+
The registry also holds the optional `commitGate` config (installed via
|
|
82
|
+
`har hooks install`): `{ "commitGate": { "mode": "block" | "warn", "scope":
|
|
83
|
+
"worktrees" | "all" } }` controls whether unverified change batches may be
|
|
84
|
+
committed.
|
|
85
|
+
|
|
86
|
+
## Shipped stage templates
|
|
87
|
+
|
|
88
|
+
`har env add-stage --list` shows available templates; `har env add-stage
|
|
89
|
+
playwright` (web) or `har env add-stage rocketsim` (iOS) installs one. A
|
|
90
|
+
template is just packaging: it copies files, merges `package.json` fragments,
|
|
91
|
+
and registers stages through the exact same registry as `--custom`.
|
|
@@ -273,6 +273,17 @@ har_regenerate_agent_env_file() {
|
|
|
273
273
|
REPO_ROOT="$work_dir" \
|
|
274
274
|
envsubst '${AGENT_ID} ${API_PORT} ${FE_PORT} ${DEBUG_PORT} ${DB_PORT} ${MINIO_PORT} ${BROWSER_PORT} ${REPO_ROOT}' \
|
|
275
275
|
< "$template" > "$env_file"
|
|
276
|
+
if command -v har >/dev/null 2>&1; then
|
|
277
|
+
har telemetry write-env \
|
|
278
|
+
--agent-id "$agent_id" \
|
|
279
|
+
--repo "${HARNESS_ROOT:-$REPO_ROOT}" \
|
|
280
|
+
--env-file "$env_file" \
|
|
281
|
+
--work-dir "$work_dir" \
|
|
282
|
+
${SLOT_BRANCH:+--branch "$SLOT_BRANCH"} \
|
|
283
|
+
${SLOT_SUFFIX:+--suffix "$SLOT_SUFFIX"} \
|
|
284
|
+
${SLOT_PURPOSE:+--purpose "$SLOT_PURPOSE"} \
|
|
285
|
+
>/dev/null 2>&1 || true
|
|
286
|
+
fi
|
|
276
287
|
}
|
|
277
288
|
|
|
278
289
|
# har_launch_preflight <agent_id> <force> <replace> [resume]
|
|
@@ -662,14 +673,43 @@ escape_step_output() {
|
|
|
662
673
|
printf '%s' "$1" | node -e "let d='';process.stdin.on('data',c=>d+=c);process.stdin.on('end',()=>{const s=d.trim().split('\n').slice(0,50).join('\n');process.stdout.write(JSON.stringify(s))})" 2>/dev/null || echo '""'
|
|
663
674
|
}
|
|
664
675
|
|
|
665
|
-
#
|
|
666
|
-
|
|
676
|
+
# Emit "<id>\t<command>" for each registered stage listed in stages.json
|
|
677
|
+
# verificationStages (used by verify --full). Ids without a matching registered
|
|
678
|
+
# stage are inline verify.sh steps and skipped here; lifecycle stages
|
|
679
|
+
# (setup/launch/reset/teardown/inspect) and the verify stage itself never run.
|
|
680
|
+
# Authoring contract: .har/STAGES.md
|
|
681
|
+
list_registered_verification_stage_commands() {
|
|
667
682
|
local script_dir="$1"
|
|
668
683
|
local agent_id="$2"
|
|
669
|
-
local
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
684
|
+
local registry="$script_dir/stages.json"
|
|
685
|
+
[ -f "$registry" ] || return 0
|
|
686
|
+
HAR_STAGE_REGISTRY="$registry" HAR_SCRIPT_DIR="$script_dir" HAR_AGENT_ID="$agent_id" node <<'NODE' 2>/dev/null || true
|
|
687
|
+
const fs = require('fs');
|
|
688
|
+
const { HAR_STAGE_REGISTRY, HAR_SCRIPT_DIR, HAR_AGENT_ID } = process.env;
|
|
689
|
+
const shq = (s) => "'" + String(s).replace(/'/g, "'\\''") + "'";
|
|
690
|
+
let reg;
|
|
691
|
+
try { reg = JSON.parse(fs.readFileSync(HAR_STAGE_REGISTRY, 'utf8')); } catch { process.exit(0); }
|
|
692
|
+
const ids = Array.isArray(reg.verificationStages) ? reg.verificationStages : [];
|
|
693
|
+
const stages = Array.isArray(reg.stages) ? reg.stages : [];
|
|
694
|
+
const runnable = new Set(['test', 'custom']);
|
|
695
|
+
for (const id of ids) {
|
|
696
|
+
const stage = stages.find((s) => s && s.id === id);
|
|
697
|
+
if (!stage || stage.id === 'verify' || !runnable.has(stage.kind)) continue;
|
|
698
|
+
const needsAgent = stage.requiresAgentId !== false;
|
|
699
|
+
let cmd;
|
|
700
|
+
if (stage.script) {
|
|
701
|
+
cmd = shq(HAR_SCRIPT_DIR + '/' + stage.script) + (needsAgent ? ' ' + shq(HAR_AGENT_ID) : '');
|
|
702
|
+
} else if (stage.command) {
|
|
703
|
+
cmd = stage.command.split('{agentId}').join(HAR_AGENT_ID);
|
|
704
|
+
} else {
|
|
705
|
+
continue;
|
|
706
|
+
}
|
|
707
|
+
if (stage.cwd) cmd = 'cd ' + shq(stage.cwd) + ' && ' + cmd;
|
|
708
|
+
const env = stage.env && typeof stage.env === 'object' ? stage.env : {};
|
|
709
|
+
const prefix = Object.entries(env).map(([k, v]) => k + '=' + shq(v)).join(' ');
|
|
710
|
+
process.stdout.write(id + '\t' + (prefix ? prefix + ' ' : '') + cmd + '\n');
|
|
711
|
+
}
|
|
712
|
+
NODE
|
|
673
713
|
}
|
|
674
714
|
|
|
675
715
|
# Optional project-owned "agent usable" smoke beyond health.
|
|
@@ -304,7 +304,8 @@ if [ "$USE_CLAUDE" = true ]; then
|
|
|
304
304
|
if tmux has-session -t "$TMUX_SESSION" 2>/dev/null; then
|
|
305
305
|
tmux kill-session -t "$TMUX_SESSION"
|
|
306
306
|
fi
|
|
307
|
-
tmux new-session -d -s "$TMUX_SESSION" -c "$WORK_DIR"
|
|
307
|
+
tmux new-session -d -s "$TMUX_SESSION" -c "$WORK_DIR" \
|
|
308
|
+
"set -a; [ -f '$ENV_FILE' ] && . '$ENV_FILE'; set +a; exec claude"
|
|
308
309
|
log "Claude session: tmux attach -t $TMUX_SESSION"
|
|
309
310
|
fi
|
|
310
311
|
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
#!/usr/bin/env bash
|
|
2
|
-
#
|
|
3
|
-
# Example: ./.har/stages/browser-e2e.sh <agent-id>
|
|
2
|
+
# Per-stage scripts live here (e.g. ./.har/stages/browser-e2e.sh <agent-id>).
|
|
4
3
|
#
|
|
5
|
-
#
|
|
6
|
-
#
|
|
4
|
+
# Read .har/STAGES.md for the authoring guide: the stage script contract,
|
|
5
|
+
# command vs script stages, and how verificationStages controls verify --full.
|
|
6
|
+
#
|
|
7
|
+
# Scaffold a new one with:
|
|
8
|
+
# har env add-stage <id> --custom --script
|
|
7
9
|
|
|
8
|
-
echo "Add stage scripts under .har/stages/ and register them in stages.json." >&2
|
|
10
|
+
echo "Add stage scripts under .har/stages/ and register them in stages.json — see .har/STAGES.md." >&2
|
|
9
11
|
exit 1
|
|
@@ -5,7 +5,8 @@
|
|
|
5
5
|
# Usage: ./.har/verify.sh <agent-id> [--full]
|
|
6
6
|
#
|
|
7
7
|
# Quick (default): ecosystem smoke + health only
|
|
8
|
-
# Full (--full): + conventional tests, lint,
|
|
8
|
+
# Full (--full): + conventional tests, lint, readiness, and every registered
|
|
9
|
+
# stage in stages.json verificationStages (see .har/STAGES.md)
|
|
9
10
|
# Stock steps are examples. Replace them during adaptation to match this repo.
|
|
10
11
|
set -euo pipefail
|
|
11
12
|
|
|
@@ -229,7 +230,13 @@ run_http_step "api-health" "http://localhost:${API_PORT}${HARNESS_HEALTH_CHECK_P
|
|
|
229
230
|
if [ -n "$FULL" ]; then
|
|
230
231
|
run_full_checks
|
|
231
232
|
run_step "readiness" "run_readiness_if_configured \"$AGENT_ID\"" || true
|
|
232
|
-
|
|
233
|
+
# Registered verification stages from .har/stages.json (see .har/STAGES.md).
|
|
234
|
+
# Every stage listed in verificationStages with a registered script/command
|
|
235
|
+
# runs here -- stage templates and custom stages alike.
|
|
236
|
+
while IFS=$'\t' read -r STAGE_ID STAGE_CMD; do
|
|
237
|
+
[ -n "$STAGE_ID" ] || continue
|
|
238
|
+
run_step "$STAGE_ID" "$STAGE_CMD" || true
|
|
239
|
+
done < <(list_registered_verification_stage_commands "$SCRIPT_DIR" "$AGENT_ID")
|
|
233
240
|
fi
|
|
234
241
|
|
|
235
242
|
# ── Output results ────────────────────────────────────────────────────────────
|
|
@@ -26,13 +26,15 @@ workflow, document the required credentials/default data and wire a smoke into
|
|
|
26
26
|
|
|
27
27
|
- [ ] Full verification returns `"status": "pass"` (`har env verify ${AGENT_ID} --full`, MCP `har_run_verification` with `full: true`, or `./.har/verify.sh ${AGENT_ID} --full`)
|
|
28
28
|
- [ ] The slot is agent-usable for this repo's documented smoke workflow when runtime services are involved
|
|
29
|
-
- [ ]
|
|
29
|
+
- [ ] Full verify runs every registered stage in `stages.json` `verificationStages` (Playwright, custom checks, …) — when `stages/browser-e2e.sh` exists, adapt specs under `tests/` for UI changes
|
|
30
30
|
- [ ] New behavior has automated test coverage
|
|
31
31
|
- [ ] Changes committed **in the session worktree** with a clear message
|
|
32
32
|
- [ ] Finish with `har env complete ${AGENT_ID}` (or MCP `har_complete_environment`) — records the validation and tears down while **keeping the session branch** for the user to push / open a PR
|
|
33
33
|
|
|
34
34
|
Quick loop: MCP `har_run_verification`, `har env verify ${AGENT_ID}`, or `./.har/verify.sh ${AGENT_ID}`
|
|
35
35
|
|
|
36
|
+
Stages are the harness's single vocabulary for checks: templates and custom stages compile to generic kinds in `.har/stages.json`, and you interact with them only through the registry (`har_run_stage`, `verify`), never stack-specific tooling. Authoring guide: `.har/STAGES.md`.
|
|
37
|
+
|
|
36
38
|
## Project commands
|
|
37
39
|
|
|
38
40
|
```bash
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# HAR Stages — authoring guide
|
|
2
|
+
|
|
3
|
+
Stages are the harness's single vocabulary for runnable checks and lifecycle
|
|
4
|
+
actions. Everything — shipped templates (`playwright`, `rocketsim`), your
|
|
5
|
+
project's test/lint commands, bespoke validation scripts — registers in
|
|
6
|
+
`.har/stages.json` with the same schema, and agents interact with stages only
|
|
7
|
+
through that registry (CLI `har env verify`, MCP `har_run_stage` /
|
|
8
|
+
`har_run_verification`), never through stack-specific tooling.
|
|
9
|
+
|
|
10
|
+
## The registry: `.har/stages.json`
|
|
11
|
+
|
|
12
|
+
```jsonc
|
|
13
|
+
{
|
|
14
|
+
"verificationStages": ["typecheck", "unit-tests", "api-health", "browser-e2e"],
|
|
15
|
+
"stages": [
|
|
16
|
+
{
|
|
17
|
+
"id": "browser-e2e", // stable, shell-friendly slug
|
|
18
|
+
"kind": "test", // setup | launch | verify | test | inspect | reset | teardown | custom
|
|
19
|
+
"description": "Playwright E2E",
|
|
20
|
+
"script": "stages/browser-e2e.sh", // relative to .har/ — OR use "command"
|
|
21
|
+
"requiresAgentId": true, // default true for test/verify/custom kinds
|
|
22
|
+
"artifacts": [{ "path": ".har/artifacts/browser-e2e", "kind": "directory" }]
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "unit-tests-fast",
|
|
26
|
+
"kind": "test",
|
|
27
|
+
"command": "npm test -- --agent {agentId}" // {agentId} is substituted at run time
|
|
28
|
+
}
|
|
29
|
+
]
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Optional stage fields: `cwd` (working directory), `env` (extra env vars),
|
|
34
|
+
`group`, `acceptsArgs` (extra CLI args the stage accepts, e.g. `["--full"]`).
|
|
35
|
+
|
|
36
|
+
## Two ways to define a stage
|
|
37
|
+
|
|
38
|
+
**Command stages** — the default for simple checks (`npm test`, `swiftlint`,
|
|
39
|
+
`make check`). One JSON entry, zero files:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
har env add-stage unit-tests-fast --custom --kind test --command "npm test" --verification
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Script stages** — for anything that needs the slot's env, ports, or
|
|
46
|
+
artifacts. Scaffold a contract-compliant skeleton:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
har env add-stage db-integrity --custom --script --description "Check DB invariants"
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
then implement the TODO block in `.har/stages/db-integrity.sh`.
|
|
53
|
+
|
|
54
|
+
## The stage script contract
|
|
55
|
+
|
|
56
|
+
Every script under `.har/stages/` must:
|
|
57
|
+
|
|
58
|
+
1. Source `harness.env` and `agent-slot.sh` from `.har/`.
|
|
59
|
+
2. Take the agent slot id as `$1` (validate with `validate_agent_id`); extra
|
|
60
|
+
args may follow.
|
|
61
|
+
3. Load the slot env via `resolve_agent_env_file` and run checks from the
|
|
62
|
+
agent's work dir (`resolve_agent_work_dir`).
|
|
63
|
+
4. Write artifacts (reports, screenshots, logs) under `.har/artifacts/<id>/`.
|
|
64
|
+
5. Print **only** the normalized JSON result object on stdout
|
|
65
|
+
(`status`, `stageId`, `agent_id`, `total_ms`, …); log progress to stderr.
|
|
66
|
+
6. Exit with the real status code (0 = pass).
|
|
67
|
+
|
|
68
|
+
The scaffolded skeleton implements all of this — replace its TODO block.
|
|
69
|
+
|
|
70
|
+
## Verification membership
|
|
71
|
+
|
|
72
|
+
Listing a stage id in `verificationStages` is what includes it in
|
|
73
|
+
`har env verify <id> --full`. Ids that match a registered stage run via their
|
|
74
|
+
script/command; ids without a registry entry (e.g. `typecheck`, `api-health`)
|
|
75
|
+
are inline steps owned by `.har/verify.sh`. Lifecycle kinds
|
|
76
|
+
(`setup`/`launch`/`reset`/`teardown`/`inspect`) and `verify` itself never run
|
|
77
|
+
as part of verification, even if listed.
|
|
78
|
+
|
|
79
|
+
## Commit gate
|
|
80
|
+
|
|
81
|
+
The registry also holds the optional `commitGate` config (installed via
|
|
82
|
+
`har hooks install`): `{ "commitGate": { "mode": "block" | "warn", "scope":
|
|
83
|
+
"worktrees" | "all" } }` controls whether unverified change batches may be
|
|
84
|
+
committed.
|
|
85
|
+
|
|
86
|
+
## Shipped stage templates
|
|
87
|
+
|
|
88
|
+
`har env add-stage --list` shows available templates; `har env add-stage
|
|
89
|
+
playwright` (web) or `har env add-stage rocketsim` (iOS) installs one. A
|
|
90
|
+
template is just packaging: it copies files, merges `package.json` fragments,
|
|
91
|
+
and registers stages through the exact same registry as `--custom`.
|
|
@@ -248,15 +248,26 @@ har_regenerate_agent_env_file() {
|
|
|
248
248
|
REPO_ROOT="$work_dir" \
|
|
249
249
|
envsubst '${AGENT_ID} ${API_PORT} ${FE_PORT} ${DEBUG_PORT} ${DB_PORT} ${MINIO_PORT} ${BROWSER_PORT} ${REPO_ROOT}' \
|
|
250
250
|
< "$template" > "$env_file"
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
cat > "$env_file" <<EOF
|
|
251
|
+
else
|
|
252
|
+
cat > "$env_file" <<EOF
|
|
254
253
|
# Agent environment — generated by launch.sh
|
|
255
254
|
AGENT_ID=${agent_id}
|
|
256
255
|
REPO_ROOT=${work_dir}
|
|
257
256
|
WORKTREE_DIR=${worktree_dir}
|
|
258
257
|
NODE_ENV=test
|
|
259
258
|
EOF
|
|
259
|
+
fi
|
|
260
|
+
if command -v har >/dev/null 2>&1; then
|
|
261
|
+
har telemetry write-env \
|
|
262
|
+
--agent-id "$agent_id" \
|
|
263
|
+
--repo "${HARNESS_ROOT:-$REPO_ROOT}" \
|
|
264
|
+
--env-file "$env_file" \
|
|
265
|
+
--work-dir "$work_dir" \
|
|
266
|
+
${SLOT_BRANCH:+--branch "$SLOT_BRANCH"} \
|
|
267
|
+
${SLOT_SUFFIX:+--suffix "$SLOT_SUFFIX"} \
|
|
268
|
+
${SLOT_PURPOSE:+--purpose "$SLOT_PURPOSE"} \
|
|
269
|
+
>/dev/null 2>&1 || true
|
|
270
|
+
fi
|
|
260
271
|
}
|
|
261
272
|
|
|
262
273
|
har_launch_preflight() {
|
|
@@ -642,14 +653,43 @@ escape_step_output() {
|
|
|
642
653
|
printf '%s' "$1" | node -e "let d='';process.stdin.on('data',c=>d+=c);process.stdin.on('end',()=>{const s=d.trim().split('\n').slice(0,50).join('\n');process.stdout.write(JSON.stringify(s))})" 2>/dev/null || echo '""'
|
|
643
654
|
}
|
|
644
655
|
|
|
645
|
-
#
|
|
646
|
-
|
|
656
|
+
# Emit "<id>\t<command>" for each registered stage listed in stages.json
|
|
657
|
+
# verificationStages (used by verify --full). Ids without a matching registered
|
|
658
|
+
# stage are inline verify.sh steps and skipped here; lifecycle stages
|
|
659
|
+
# (setup/launch/reset/teardown/inspect) and the verify stage itself never run.
|
|
660
|
+
# Authoring contract: .har/STAGES.md
|
|
661
|
+
list_registered_verification_stage_commands() {
|
|
647
662
|
local script_dir="$1"
|
|
648
663
|
local agent_id="$2"
|
|
649
|
-
local
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
|
|
664
|
+
local registry="$script_dir/stages.json"
|
|
665
|
+
[ -f "$registry" ] || return 0
|
|
666
|
+
HAR_STAGE_REGISTRY="$registry" HAR_SCRIPT_DIR="$script_dir" HAR_AGENT_ID="$agent_id" node <<'NODE' 2>/dev/null || true
|
|
667
|
+
const fs = require('fs');
|
|
668
|
+
const { HAR_STAGE_REGISTRY, HAR_SCRIPT_DIR, HAR_AGENT_ID } = process.env;
|
|
669
|
+
const shq = (s) => "'" + String(s).replace(/'/g, "'\\''") + "'";
|
|
670
|
+
let reg;
|
|
671
|
+
try { reg = JSON.parse(fs.readFileSync(HAR_STAGE_REGISTRY, 'utf8')); } catch { process.exit(0); }
|
|
672
|
+
const ids = Array.isArray(reg.verificationStages) ? reg.verificationStages : [];
|
|
673
|
+
const stages = Array.isArray(reg.stages) ? reg.stages : [];
|
|
674
|
+
const runnable = new Set(['test', 'custom']);
|
|
675
|
+
for (const id of ids) {
|
|
676
|
+
const stage = stages.find((s) => s && s.id === id);
|
|
677
|
+
if (!stage || stage.id === 'verify' || !runnable.has(stage.kind)) continue;
|
|
678
|
+
const needsAgent = stage.requiresAgentId !== false;
|
|
679
|
+
let cmd;
|
|
680
|
+
if (stage.script) {
|
|
681
|
+
cmd = shq(HAR_SCRIPT_DIR + '/' + stage.script) + (needsAgent ? ' ' + shq(HAR_AGENT_ID) : '');
|
|
682
|
+
} else if (stage.command) {
|
|
683
|
+
cmd = stage.command.split('{agentId}').join(HAR_AGENT_ID);
|
|
684
|
+
} else {
|
|
685
|
+
continue;
|
|
686
|
+
}
|
|
687
|
+
if (stage.cwd) cmd = 'cd ' + shq(stage.cwd) + ' && ' + cmd;
|
|
688
|
+
const env = stage.env && typeof stage.env === 'object' ? stage.env : {};
|
|
689
|
+
const prefix = Object.entries(env).map(([k, v]) => k + '=' + shq(v)).join(' ');
|
|
690
|
+
process.stdout.write(id + '\t' + (prefix ? prefix + ' ' : '') + cmd + '\n');
|
|
691
|
+
}
|
|
692
|
+
NODE
|
|
653
693
|
}
|
|
654
694
|
|
|
655
695
|
# Optional project-owned "agent usable" smoke beyond health.
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
#!/usr/bin/env bash
|
|
2
|
-
#
|
|
3
|
-
# Example: ./.har/stages/browser-e2e.sh <agent-id>
|
|
2
|
+
# Per-stage scripts live here (e.g. ./.har/stages/browser-e2e.sh <agent-id>).
|
|
4
3
|
#
|
|
5
|
-
#
|
|
6
|
-
#
|
|
4
|
+
# Read .har/STAGES.md for the authoring guide: the stage script contract,
|
|
5
|
+
# command vs script stages, and how verificationStages controls verify --full.
|
|
6
|
+
#
|
|
7
|
+
# Scaffold a new one with:
|
|
8
|
+
# har env add-stage <id> --custom --script
|
|
7
9
|
|
|
8
|
-
echo "Add stage scripts under .har/stages/ and register them in stages.json." >&2
|
|
10
|
+
echo "Add stage scripts under .har/stages/ and register them in stages.json — see .har/STAGES.md." >&2
|
|
9
11
|
exit 1
|
|
@@ -5,7 +5,8 @@
|
|
|
5
5
|
# Usage: ./.har/verify.sh <agent-id> [--full]
|
|
6
6
|
#
|
|
7
7
|
# Quick (default): ecosystem smoke — compile/import/build only
|
|
8
|
-
# Full (--full): + conventional tests, lint,
|
|
8
|
+
# Full (--full): + conventional tests, lint, readiness, and every registered
|
|
9
|
+
# stage in stages.json verificationStages (see .har/STAGES.md)
|
|
9
10
|
# Stock steps are examples. Replace them during adaptation to match this repo.
|
|
10
11
|
set -euo pipefail
|
|
11
12
|
|
|
@@ -185,7 +186,13 @@ run_quick_smoke || { [ -z "$FULL" ] && true; }
|
|
|
185
186
|
if [ -n "$FULL" ]; then
|
|
186
187
|
run_full_checks
|
|
187
188
|
run_step "readiness" "run_readiness_if_configured \"$AGENT_ID\"" || true
|
|
188
|
-
|
|
189
|
+
# Registered verification stages from .har/stages.json (see .har/STAGES.md).
|
|
190
|
+
# Every stage listed in verificationStages with a registered script/command
|
|
191
|
+
# runs here -- stage templates and custom stages alike.
|
|
192
|
+
while IFS=$'\t' read -r STAGE_ID STAGE_CMD; do
|
|
193
|
+
[ -n "$STAGE_ID" ] || continue
|
|
194
|
+
run_step "$STAGE_ID" "$STAGE_CMD" || true
|
|
195
|
+
done < <(list_registered_verification_stage_commands "$SCRIPT_DIR" "$AGENT_ID")
|
|
189
196
|
fi
|
|
190
197
|
|
|
191
198
|
END_TOTAL=$(now_ms)
|
|
@@ -35,6 +35,8 @@ seeded data, or a simulator flow, document it here and wire the smoke into
|
|
|
35
35
|
|
|
36
36
|
Quick loop: MCP `har_run_verification`, `har env verify ${AGENT_ID}`, or `./.har/verify.sh ${AGENT_ID}`
|
|
37
37
|
|
|
38
|
+
Stages are the harness's single vocabulary for checks: templates and custom stages compile to generic kinds in `.har/stages.json`, and you interact with them only through the registry (`har_run_stage`, `verify`), never stack-specific tooling. Authoring guide: `.har/STAGES.md`.
|
|
39
|
+
|
|
38
40
|
## Project commands
|
|
39
41
|
|
|
40
42
|
```bash
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# HAR Stages — authoring guide
|
|
2
|
+
|
|
3
|
+
Stages are the harness's single vocabulary for runnable checks and lifecycle
|
|
4
|
+
actions. Everything — shipped templates (`playwright`, `rocketsim`), your
|
|
5
|
+
project's test/lint commands, bespoke validation scripts — registers in
|
|
6
|
+
`.har/stages.json` with the same schema, and agents interact with stages only
|
|
7
|
+
through that registry (CLI `har env verify`, MCP `har_run_stage` /
|
|
8
|
+
`har_run_verification`), never through stack-specific tooling.
|
|
9
|
+
|
|
10
|
+
## The registry: `.har/stages.json`
|
|
11
|
+
|
|
12
|
+
```jsonc
|
|
13
|
+
{
|
|
14
|
+
"verificationStages": ["typecheck", "unit-tests", "api-health", "browser-e2e"],
|
|
15
|
+
"stages": [
|
|
16
|
+
{
|
|
17
|
+
"id": "browser-e2e", // stable, shell-friendly slug
|
|
18
|
+
"kind": "test", // setup | launch | verify | test | inspect | reset | teardown | custom
|
|
19
|
+
"description": "Playwright E2E",
|
|
20
|
+
"script": "stages/browser-e2e.sh", // relative to .har/ — OR use "command"
|
|
21
|
+
"requiresAgentId": true, // default true for test/verify/custom kinds
|
|
22
|
+
"artifacts": [{ "path": ".har/artifacts/browser-e2e", "kind": "directory" }]
|
|
23
|
+
},
|
|
24
|
+
{
|
|
25
|
+
"id": "unit-tests-fast",
|
|
26
|
+
"kind": "test",
|
|
27
|
+
"command": "npm test -- --agent {agentId}" // {agentId} is substituted at run time
|
|
28
|
+
}
|
|
29
|
+
]
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Optional stage fields: `cwd` (working directory), `env` (extra env vars),
|
|
34
|
+
`group`, `acceptsArgs` (extra CLI args the stage accepts, e.g. `["--full"]`).
|
|
35
|
+
|
|
36
|
+
## Two ways to define a stage
|
|
37
|
+
|
|
38
|
+
**Command stages** — the default for simple checks (`npm test`, `swiftlint`,
|
|
39
|
+
`make check`). One JSON entry, zero files:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
har env add-stage unit-tests-fast --custom --kind test --command "npm test" --verification
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Script stages** — for anything that needs the slot's env, ports, or
|
|
46
|
+
artifacts. Scaffold a contract-compliant skeleton:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
har env add-stage db-integrity --custom --script --description "Check DB invariants"
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
then implement the TODO block in `.har/stages/db-integrity.sh`.
|
|
53
|
+
|
|
54
|
+
## The stage script contract
|
|
55
|
+
|
|
56
|
+
Every script under `.har/stages/` must:
|
|
57
|
+
|
|
58
|
+
1. Source `harness.env` and `agent-slot.sh` from `.har/`.
|
|
59
|
+
2. Take the agent slot id as `$1` (validate with `validate_agent_id`); extra
|
|
60
|
+
args may follow.
|
|
61
|
+
3. Load the slot env via `resolve_agent_env_file` and run checks from the
|
|
62
|
+
agent's work dir (`resolve_agent_work_dir`).
|
|
63
|
+
4. Write artifacts (reports, screenshots, logs) under `.har/artifacts/<id>/`.
|
|
64
|
+
5. Print **only** the normalized JSON result object on stdout
|
|
65
|
+
(`status`, `stageId`, `agent_id`, `total_ms`, …); log progress to stderr.
|
|
66
|
+
6. Exit with the real status code (0 = pass).
|
|
67
|
+
|
|
68
|
+
The scaffolded skeleton implements all of this — replace its TODO block.
|
|
69
|
+
|
|
70
|
+
## Verification membership
|
|
71
|
+
|
|
72
|
+
Listing a stage id in `verificationStages` is what includes it in
|
|
73
|
+
`har env verify <id> --full`. Ids that match a registered stage run via their
|
|
74
|
+
script/command; ids without a registry entry (e.g. `typecheck`, `api-health`)
|
|
75
|
+
are inline steps owned by `.har/verify.sh`. Lifecycle kinds
|
|
76
|
+
(`setup`/`launch`/`reset`/`teardown`/`inspect`) and `verify` itself never run
|
|
77
|
+
as part of verification, even if listed.
|
|
78
|
+
|
|
79
|
+
## Commit gate
|
|
80
|
+
|
|
81
|
+
The registry also holds the optional `commitGate` config (installed via
|
|
82
|
+
`har hooks install`): `{ "commitGate": { "mode": "block" | "warn", "scope":
|
|
83
|
+
"worktrees" | "all" } }` controls whether unverified change batches may be
|
|
84
|
+
committed.
|
|
85
|
+
|
|
86
|
+
## Shipped stage templates
|
|
87
|
+
|
|
88
|
+
`har env add-stage --list` shows available templates; `har env add-stage
|
|
89
|
+
playwright` (web) or `har env add-stage rocketsim` (iOS) installs one. A
|
|
90
|
+
template is just packaging: it copies files, merges `package.json` fragments,
|
|
91
|
+
and registers stages through the exact same registry as `--custom`.
|