@kirrosh/zond 0.26.1 → 0.27.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +65 -0
- package/README.md +39 -22
- package/bin/zond.mjs +23 -0
- package/package.json +36 -23
- package/scripts/npm/postinstall.mjs +76 -0
- package/src/CLAUDE.md +0 -112
- package/src/bun-types.d.ts +0 -5
- package/src/cli/argv.ts +0 -122
- package/src/cli/commands/add-api.ts +0 -146
- package/src/cli/commands/api/annotate/idempotency.ts +0 -59
- package/src/cli/commands/api/annotate/index.ts +0 -880
- package/src/cli/commands/api/annotate/lifecycle.ts +0 -74
- package/src/cli/commands/api/annotate/overlay.ts +0 -206
- package/src/cli/commands/api/annotate/pagination.ts +0 -64
- package/src/cli/commands/api/annotate/prompts.ts +0 -220
- package/src/cli/commands/api/annotate/readback.ts +0 -58
- package/src/cli/commands/api/annotate/resources.ts +0 -91
- package/src/cli/commands/api/annotate/seed-bodies.ts +0 -61
- package/src/cli/commands/audit.ts +0 -786
- package/src/cli/commands/catalog.ts +0 -97
- package/src/cli/commands/check.ts +0 -361
- package/src/cli/commands/checks.ts +0 -1072
- package/src/cli/commands/ci-init.ts +0 -223
- package/src/cli/commands/clean.ts +0 -212
- package/src/cli/commands/cleanup.ts +0 -236
- package/src/cli/commands/completions.ts +0 -192
- package/src/cli/commands/coverage.ts +0 -872
- package/src/cli/commands/db.ts +0 -611
- package/src/cli/commands/describe.ts +0 -120
- package/src/cli/commands/discover.ts +0 -1356
- package/src/cli/commands/doctor.ts +0 -661
- package/src/cli/commands/fixtures.ts +0 -402
- package/src/cli/commands/generate.ts +0 -577
- package/src/cli/commands/init/agents-md.ts +0 -61
- package/src/cli/commands/init/bootstrap.ts +0 -111
- package/src/cli/commands/init/index.ts +0 -244
- package/src/cli/commands/init/skills.ts +0 -141
- package/src/cli/commands/init/templates/agents.md +0 -86
- package/src/cli/commands/init/templates/markdown.d.ts +0 -4
- package/src/cli/commands/init/templates/skills/warm-up-target.md +0 -122
- package/src/cli/commands/init/templates/skills/zond-checks.md +0 -621
- package/src/cli/commands/init/templates/skills/zond-seed.md +0 -114
- package/src/cli/commands/init/templates/skills/zond-triage.md +0 -272
- package/src/cli/commands/init/templates/skills/zond.md +0 -861
- package/src/cli/commands/init/templates/zond-config.yml +0 -14
- package/src/cli/commands/prepare-fixtures.ts +0 -97
- package/src/cli/commands/probe/_seed-bodies.ts +0 -52
- package/src/cli/commands/probe/mass-assignment.ts +0 -594
- package/src/cli/commands/probe/security.ts +0 -537
- package/src/cli/commands/probe/static.ts +0 -255
- package/src/cli/commands/probe/webhooks.ts +0 -163
- package/src/cli/commands/probe.ts +0 -535
- package/src/cli/commands/reference.ts +0 -87
- package/src/cli/commands/refresh-api.ts +0 -227
- package/src/cli/commands/remove-api.ts +0 -150
- package/src/cli/commands/report-bundle.ts +0 -310
- package/src/cli/commands/report.ts +0 -241
- package/src/cli/commands/request.ts +0 -548
- package/src/cli/commands/run.ts +0 -1162
- package/src/cli/commands/schema-from-runs.ts +0 -128
- package/src/cli/commands/secrets.ts +0 -133
- package/src/cli/commands/session.ts +0 -244
- package/src/cli/commands/use.ts +0 -74
- package/src/cli/index.ts +0 -55
- package/src/cli/json-envelope.ts +0 -108
- package/src/cli/json-schemas.ts +0 -314
- package/src/cli/output.ts +0 -40
- package/src/cli/program.ts +0 -219
- package/src/cli/resolve.ts +0 -105
- package/src/cli/runtime.ts +0 -7
- package/src/cli/safe-live.ts +0 -24
- package/src/cli/status-filter.ts +0 -114
- package/src/cli/util/api-context.ts +0 -85
- package/src/cli/version.ts +0 -8
- package/src/core/audit/persist.ts +0 -183
- package/src/core/checks/budget.ts +0 -59
- package/src/core/checks/checks/_crud-helpers.ts +0 -133
- package/src/core/checks/checks/_negative_mutator.ts +0 -133
- package/src/core/checks/checks/_readback-helpers.ts +0 -133
- package/src/core/checks/checks/content_type_conformance.ts +0 -39
- package/src/core/checks/checks/cross_call_references.ts +0 -147
- package/src/core/checks/checks/cursor_boundary_fuzzing.ts +0 -219
- package/src/core/checks/checks/ensure_resource_availability.ts +0 -62
- package/src/core/checks/checks/idempotency_replay.ts +0 -242
- package/src/core/checks/checks/ignored_auth.ts +0 -254
- package/src/core/checks/checks/index.ts +0 -68
- package/src/core/checks/checks/lifecycle_transitions.ts +0 -416
- package/src/core/checks/checks/missing_required_header.ts +0 -40
- package/src/core/checks/checks/negative_data_rejection.ts +0 -148
- package/src/core/checks/checks/not_a_server_error.ts +0 -35
- package/src/core/checks/checks/open_cors_on_sensitive.ts +0 -160
- package/src/core/checks/checks/pagination_invariants.ts +0 -419
- package/src/core/checks/checks/positive_data_acceptance.ts +0 -33
- package/src/core/checks/checks/rate_limit_headers_absent.ts +0 -77
- package/src/core/checks/checks/response_headers_conformance.ts +0 -74
- package/src/core/checks/checks/response_schema_conformance.ts +0 -30
- package/src/core/checks/checks/status_code_conformance.ts +0 -132
- package/src/core/checks/checks/unsupported_method.ts +0 -63
- package/src/core/checks/checks/use_after_free.ts +0 -78
- package/src/core/checks/index.ts +0 -30
- package/src/core/checks/mode.ts +0 -82
- package/src/core/checks/recommended-action.ts +0 -68
- package/src/core/checks/registry.ts +0 -78
- package/src/core/checks/runner.ts +0 -1461
- package/src/core/checks/sarif.ts +0 -230
- package/src/core/checks/spec-findings.ts +0 -308
- package/src/core/checks/stateful.ts +0 -121
- package/src/core/checks/types.ts +0 -305
- package/src/core/checks/zond-extensions.ts +0 -73
- package/src/core/classifier/recommended-action.ts +0 -251
- package/src/core/context/current.ts +0 -51
- package/src/core/context/session.ts +0 -78
- package/src/core/coverage/loader.ts +0 -216
- package/src/core/coverage/reasons.ts +0 -300
- package/src/core/diagnostics/db-analysis.ts +0 -666
- package/src/core/diagnostics/failure-class.ts +0 -140
- package/src/core/diagnostics/failure-hints.ts +0 -112
- package/src/core/diagnostics/spec-pointer.ts +0 -99
- package/src/core/diagnostics/suggested-fixes.ts +0 -155
- package/src/core/exporter/case-study/index.ts +0 -270
- package/src/core/exporter/curl.ts +0 -40
- package/src/core/exporter/exporter.ts +0 -48
- package/src/core/exporter/html-report/escape.ts +0 -24
- package/src/core/exporter/html-report/index.ts +0 -479
- package/src/core/exporter/html-report/script.ts +0 -100
- package/src/core/exporter/html-report/styles.ts +0 -408
- package/src/core/generator/catalog-builder.ts +0 -179
- package/src/core/generator/chunker.ts +0 -78
- package/src/core/generator/coverage-phase.ts +0 -0
- package/src/core/generator/coverage-scanner.ts +0 -87
- package/src/core/generator/data-factory.ts +0 -759
- package/src/core/generator/describe.ts +0 -302
- package/src/core/generator/endpoint-warnings.ts +0 -52
- package/src/core/generator/fixtures-builder.ts +0 -332
- package/src/core/generator/index.ts +0 -14
- package/src/core/generator/openapi-reader.ts +0 -297
- package/src/core/generator/path-param-disambig.ts +0 -140
- package/src/core/generator/resources-builder.ts +0 -898
- package/src/core/generator/schema-utils.ts +0 -112
- package/src/core/generator/serializer.ts +0 -343
- package/src/core/generator/suite-generator.ts +0 -1301
- package/src/core/generator/types.ts +0 -63
- package/src/core/identity/identity-file.ts +0 -0
- package/src/core/lint/affects.ts +0 -28
- package/src/core/lint/config.ts +0 -96
- package/src/core/lint/format.ts +0 -42
- package/src/core/lint/index.ts +0 -94
- package/src/core/lint/reporter.ts +0 -128
- package/src/core/lint/rules/consistency.ts +0 -158
- package/src/core/lint/rules/heuristics.ts +0 -97
- package/src/core/lint/rules/strictness.ts +0 -109
- package/src/core/lint/types.ts +0 -96
- package/src/core/lint/walker.ts +0 -248
- package/src/core/meta/meta-store.ts +0 -11
- package/src/core/output/README.md +0 -73
- package/src/core/output/index.ts +0 -13
- package/src/core/output/run.ts +0 -91
- package/src/core/output/types.ts +0 -122
- package/src/core/parser/dynamic-values.ts +0 -160
- package/src/core/parser/env-interpolation.ts +0 -104
- package/src/core/parser/filter.ts +0 -97
- package/src/core/parser/schema.ts +0 -359
- package/src/core/parser/types.ts +0 -117
- package/src/core/parser/variables.ts +0 -0
- package/src/core/parser/yaml-parser.ts +0 -181
- package/src/core/probe/bootstrap.ts +0 -34
- package/src/core/probe/dry-run-envelope.ts +0 -61
- package/src/core/probe/mass-assignment/classify.ts +0 -175
- package/src/core/probe/mass-assignment/cleanup.ts +0 -52
- package/src/core/probe/mass-assignment/digest.ts +0 -114
- package/src/core/probe/mass-assignment/orchestrator.ts +0 -459
- package/src/core/probe/mass-assignment/regression.ts +0 -141
- package/src/core/probe/mass-assignment/suspects.ts +0 -92
- package/src/core/probe/mass-assignment/types.ts +0 -135
- package/src/core/probe/mass-assignment-probe-class.ts +0 -198
- package/src/core/probe/mass-assignment-probe.ts +0 -27
- package/src/core/probe/mass-assignment-template.ts +0 -240
- package/src/core/probe/method-probe.ts +0 -164
- package/src/core/probe/method-shared.ts +0 -69
- package/src/core/probe/negative-probe.ts +0 -691
- package/src/core/probe/orphan-tracker.ts +0 -188
- package/src/core/probe/path-discovery.ts +0 -439
- package/src/core/probe/probe-harness.ts +0 -119
- package/src/core/probe/registry.ts +0 -89
- package/src/core/probe/runner.ts +0 -136
- package/src/core/probe/security/baseline.ts +0 -174
- package/src/core/probe/security/classify.ts +0 -341
- package/src/core/probe/security/cleanup.ts +0 -125
- package/src/core/probe/security/detectors.ts +0 -71
- package/src/core/probe/security/digest.ts +0 -104
- package/src/core/probe/security/orchestrator.ts +0 -398
- package/src/core/probe/security/regression.ts +0 -103
- package/src/core/probe/security/types.ts +0 -151
- package/src/core/probe/security-probe-class.ts +0 -207
- package/src/core/probe/security-probe.ts +0 -32
- package/src/core/probe/shared.ts +0 -531
- package/src/core/probe/static-probe-class.ts +0 -125
- package/src/core/probe/types.ts +0 -165
- package/src/core/probe/verdict-aggregator.ts +0 -33
- package/src/core/probe/webhooks-probe.ts +0 -282
- package/src/core/reporter/console.ts +0 -240
- package/src/core/reporter/index.ts +0 -22
- package/src/core/reporter/json.ts +0 -22
- package/src/core/reporter/junit.ts +0 -93
- package/src/core/reporter/ndjson.ts +0 -37
- package/src/core/reporter/types.ts +0 -15
- package/src/core/runner/assertions.ts +0 -383
- package/src/core/runner/async-pool.ts +0 -108
- package/src/core/runner/auth-path.ts +0 -8
- package/src/core/runner/ci-context.ts +0 -72
- package/src/core/runner/executor.ts +0 -652
- package/src/core/runner/expr-eval.ts +0 -41
- package/src/core/runner/form-encode.ts +0 -41
- package/src/core/runner/http-client.ts +0 -224
- package/src/core/runner/learn-drift.ts +0 -293
- package/src/core/runner/preflight-vars.ts +0 -153
- package/src/core/runner/progress-tracker.ts +0 -73
- package/src/core/runner/rate-limiter.ts +0 -185
- package/src/core/runner/run-kind.ts +0 -45
- package/src/core/runner/schema-validator.ts +0 -308
- package/src/core/runner/send-request.ts +0 -232
- package/src/core/runner/transforms.ts +0 -65
- package/src/core/runner/types.ts +0 -94
- package/src/core/secrets/registry.ts +0 -164
- package/src/core/secrets/secrets-file.ts +0 -115
- package/src/core/selectors/operation-filter.ts +0 -144
- package/src/core/setup-api.ts +0 -590
- package/src/core/severity/category.ts +0 -94
- package/src/core/severity/index.ts +0 -58
- package/src/core/spec/infer-schema.ts +0 -102
- package/src/core/spec/layers.ts +0 -154
- package/src/core/spec/merge-specs.ts +0 -156
- package/src/core/spec/schema-from-runs.ts +0 -117
- package/src/core/spec/schema-overlay.ts +0 -130
- package/src/core/util/ajv.ts +0 -13
- package/src/core/util/format-eta.ts +0 -21
- package/src/core/util/headers.ts +0 -9
- package/src/core/util/url.ts +0 -24
- package/src/core/utils.ts +0 -13
- package/src/core/workspace/config.ts +0 -129
- package/src/core/workspace/fixture-gap-report.ts +0 -84
- package/src/core/workspace/fixture-gaps.ts +0 -71
- package/src/core/workspace/manifest.ts +0 -283
- package/src/core/workspace/output-rotation.ts +0 -62
- package/src/core/workspace/root.ts +0 -96
- package/src/core/workspace/triage-path.ts +0 -87
- package/src/db/lint-runs.ts +0 -47
- package/src/db/migrate.ts +0 -128
- package/src/db/migrations/0001_run_kind.sql +0 -25
- package/src/db/migrations/0002_run_kind_request.sql +0 -59
- package/src/db/migrations/sql.d.ts +0 -4
- package/src/db/queries/collections.ts +0 -133
- package/src/db/queries/coverage.ts +0 -9
- package/src/db/queries/dashboard.ts +0 -59
- package/src/db/queries/results.ts +0 -216
- package/src/db/queries/runs.ts +0 -289
- package/src/db/queries/sessions.ts +0 -42
- package/src/db/queries/settings.ts +0 -28
- package/src/db/queries/types.ts +0 -172
- package/src/db/queries.ts +0 -75
- package/src/db/schema.ts +0 -298
|
@@ -1,861 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: zond
|
|
3
|
-
description: |
|
|
4
|
-
Primary skill for any zond work. Loads on workspace touch
|
|
5
|
-
(`apis/<name>/`, `.env.yaml`, `.api-fixtures.yaml`) AND on intent —
|
|
6
|
-
full API audit, "find bugs", "raise coverage", "test the whole API",
|
|
7
|
-
contract-drift check, schema-drift detection, post-deploy regression,
|
|
8
|
-
case study, single-flow scenario authoring, fixture pack. Hand off to
|
|
9
|
-
`zond-checks` for depth-check reference / SARIF / stateful invariants,
|
|
10
|
-
to `zond-triage` for "what broke in the last run".
|
|
11
|
-
allowed-tools: [Read, Write, Edit, Bash(zond *), Bash(bunx zond *), Bash(sqlite3 *)]
|
|
12
|
-
---
|
|
13
|
-
|
|
14
|
-
# zond — Primary skill
|
|
15
|
-
|
|
16
|
-
CLI-only. Run `zond --version` first; if missing:
|
|
17
|
-
`curl -fsSL https://raw.githubusercontent.com/kirrosh/zond/master/install.sh | sh`.
|
|
18
|
-
|
|
19
|
-
For depth-check reference (the catalog of conformance / security /
|
|
20
|
-
stateful checks, SARIF output, m-20 invariants) — see `zond-checks`.
|
|
21
|
-
For triage of a failing run — see `zond-triage`.
|
|
22
|
-
|
|
23
|
-
## Iron rules (read once, apply always)
|
|
24
|
-
|
|
25
|
-
- **NEVER read raw OpenAPI** with Read/cat/grep. The workspace has
|
|
26
|
-
pre-built artifacts (catalog/resources/fixtures) — use those. Drop
|
|
27
|
-
into `apis/<name>/spec.json` only when probe-class authoring needs
|
|
28
|
-
full schemas.
|
|
29
|
-
- **NEVER `curl` or `wget`** — use `zond request <method> <url> --api <name>`
|
|
30
|
-
for ad-hoc HTTP so it lands in the run DB and respects auth. Never
|
|
31
|
-
shell-substitute the token by hand (`$(yq …)` is blocked by the sandbox).
|
|
32
|
-
`--body '{…}'` always sends `application/json`. For form-encoded APIs
|
|
33
|
-
(Stripe v1, classic webforms) pass `--form` — zond URL-encodes the
|
|
34
|
-
JSON body. Auto-detect from spec: if `requestBody.content` in
|
|
35
|
-
`.api-catalog.yaml` declares only `application/x-www-form-urlencoded`,
|
|
36
|
-
use `--form`; mixed content-types → pick by `Content-Type` header you
|
|
37
|
-
want to test.
|
|
38
|
-
- **NEVER hardcode tokens** — put them in `apis/<name>/.secrets.yaml`
|
|
39
|
-
(auto-gitignored), reference from `.env.yaml` as `@secret:auth_token`.
|
|
40
|
-
Plain `${SHELL_VAR}` references also work. Tests read the resolved
|
|
41
|
-
value as `{{auth_token}}`.
|
|
42
|
-
- **NEVER read `.secrets.yaml` directly.** Use `zond doctor --api <name> --json`
|
|
43
|
-
— reports `set | unset` and value length only. To WRITE / rotate a secret
|
|
44
|
-
use `zond secrets set <key> <value> --api <name>` (writes a `.bak`, never
|
|
45
|
-
echoes the value, auto-strips a pasted `Bearer ` prefix — pass `--literal`
|
|
46
|
-
to keep it verbatim).
|
|
47
|
-
- **NEVER edit `.api-*.yaml`** (catalog / resources / fixtures) by hand —
|
|
48
|
-
regenerated by `zond refresh-api`. `.api-resources.local.yaml` IS the
|
|
49
|
-
hand-editable / annotate-target overlay; safe to edit.
|
|
50
|
-
`.api-schema.local.yaml` is the response-schema overlay (ARV-176) —
|
|
51
|
-
written by `refresh-api --merge-schema`, also survives refresh; edit by
|
|
52
|
-
re-merging a `patch.schema.json`, not by hand. `.env.yaml` values are
|
|
53
|
-
editable (see ARV-114).
|
|
54
|
-
- **NEVER invent endpoints.** Only use entries from `.api-catalog.yaml`.
|
|
55
|
-
- **NEVER run destructive ops on a shared / production org without
|
|
56
|
-
`--dry-run` first.** Probes, `prepare-fixtures --apply`, `cleanup` hit
|
|
57
|
-
live APIs and can delete user data. `--dry-run` once → inspect → drop
|
|
58
|
-
the flag.
|
|
59
|
-
- **NEVER share triage artefacts** outbound without `--redact-identity`.
|
|
60
|
-
Secret-redaction is on by default; identity values (org/member slugs,
|
|
61
|
-
real ids) need the extra flag.
|
|
62
|
-
- **NEVER report cleanup failure as an API bug.** POST 200 → DELETE 5xx
|
|
63
|
-
is *probably* a fixture-isolation issue (orphan accumulation, race).
|
|
64
|
-
Re-run with `--no-cleanup` or isolated namespace before filing.
|
|
65
|
-
- **`recommended_action: report_backend_bug` / 5xx → STOP** in interactive
|
|
66
|
-
mode (surface the request/response, get a decision; never `expect:`-mask).
|
|
67
|
-
In autonomous / loop / audit-sweep mode (no user-in-the-loop), log to
|
|
68
|
-
`api-bugs-<NN>.md`, continue the sweep — bailing on bug #1 forfeits #2..N.
|
|
69
|
-
- **CRUD-run ≥80% 401/403 / `permission_denied` → `env_issue`, not bug.**
|
|
70
|
-
Token scope issue. Confirm via `zond db diagnose <run-id> --json`
|
|
71
|
-
(401/403 dominate `by_recommended_action`); do not generate
|
|
72
|
-
case-studies, do not `expect:`-mask.
|
|
73
|
-
- **MUST run `zond doctor --api <name> --missing-only` before generating
|
|
74
|
-
fixtures or touching `.env.yaml`** — identifies unfilled keys early.
|
|
75
|
-
- **`prepare-fixtures` is single-pass, never POST-creates, and never
|
|
76
|
-
harvests values** (ARV-362) — it verifies existing fixtures and reports
|
|
77
|
-
unfilled vars as gaps. Which record/field fills a slot is your call:
|
|
78
|
-
fill fixtures by hand (`fixtures add` / editing `.env.yaml`).
|
|
79
|
-
|
|
80
|
-
## Workspace artifact model
|
|
81
|
-
|
|
82
|
-
After `zond add api <name> --spec <path|url>`, `apis/<name>/` contains:
|
|
83
|
-
|
|
84
|
-
```
|
|
85
|
-
spec.json ← dereferenced OpenAPI (machine source)
|
|
86
|
-
.api-catalog.yaml ← endpoint index (method, path, params)
|
|
87
|
-
.api-resources.yaml ← CRUD chains + FK deps (auto)
|
|
88
|
-
.api-resources.local.yaml ← overlay: extensions + annotate output (hand-edit OK; appears after first `zond api annotate apply` — or create by hand to use as overlay)
|
|
89
|
-
.api-fixtures.yaml ← MANIFEST: required vars + descriptions (read-only)
|
|
90
|
-
.env.yaml ← VALUES: variable values (user / prepare-fixtures writes)
|
|
91
|
-
.secrets.yaml ← raw secret values (auto-gitignored)
|
|
92
|
-
.identity.yaml ← non-secret identifying values (gitignored)
|
|
93
|
-
tests/ ← generated/handwritten suites
|
|
94
|
-
scenarios/ ← handwritten flow suites
|
|
95
|
-
probes/ ← probe-emitted suites
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
### Manifest vs values (the #1 source of confusion)
|
|
99
|
-
|
|
100
|
-
**`.api-fixtures.yaml` = the list of vars this API needs (manifest).
|
|
101
|
-
`.env.yaml` = their values.**
|
|
102
|
-
|
|
103
|
-
| Op | Touches manifest? | Touches env? |
|
|
104
|
-
|---|---|---|
|
|
105
|
-
| `zond add api` / `refresh-api` | rebuilds | seeds skeleton |
|
|
106
|
-
| `zond generate` | extends (adds new `{{vars}}` discovered) | does **not** modify |
|
|
107
|
-
| `zond prepare-fixtures` | reads | writes values |
|
|
108
|
-
| `zond fixtures add` / `import` | reads | writes values (manual) |
|
|
109
|
-
| user editing | never | always |
|
|
110
|
-
|
|
111
|
-
- `{{var}}` in a generated test but NOT in `.api-fixtures.yaml` is a
|
|
112
|
-
bug in the manifest builder / generator. Don't "fix" it by adding to
|
|
113
|
-
`.env.yaml`.
|
|
114
|
-
- A key in `.env.yaml` but NOT in `.api-fixtures.yaml` is shadow —
|
|
115
|
-
`prepare-fixtures` warns `not in manifest, ignored`.
|
|
116
|
-
- "Generate should sync `.env.yaml`" is a rejected design (decision-7,
|
|
117
|
-
m-17).
|
|
118
|
-
|
|
119
|
-
### Manual fixture-bootstrap (you fill every value)
|
|
120
|
-
|
|
121
|
-
ARV-362: `prepare-fixtures` **reports** FK gaps — it never harvests a
|
|
122
|
-
value. Every fixture is filled by hand: read the gap (empty list → create
|
|
123
|
-
a record first; non-empty → pick a value; 404/403 → auth/drift), then
|
|
124
|
-
supply it. Vendor-dashboard ids (Stripe `cus_*`, GitHub PR numbers,
|
|
125
|
-
Sentry issue ids) have no list endpoint at all — same hand-off:
|
|
126
|
-
|
|
127
|
-
| Input you have | Command |
|
|
128
|
-
|---|---|
|
|
129
|
-
| Concrete id values | `zond fixtures add <var>=<id> [--validate] --apply` |
|
|
130
|
-
| Curl from devtools / dashboard | `pbpaste \| zond fixtures import --from-curl --apply` (URL is matched against spec paths to extract `{var}` bindings) |
|
|
131
|
-
|
|
132
|
-
`--validate` GETs the spec's read-by-id endpoint and classifies the
|
|
133
|
-
fixture as `live` / `stale` / `unknown` so dead ids don't silently
|
|
134
|
-
break later runs (ARV-32). Both commands write to `apis/<name>/.env.yaml`
|
|
135
|
-
with a `.env.yaml.bak` backup.
|
|
136
|
-
|
|
137
|
-
### When to read which file
|
|
138
|
-
|
|
139
|
-
| To know… | Read |
|
|
140
|
-
|---|---|
|
|
141
|
-
| What endpoints exist | `.api-catalog.yaml` |
|
|
142
|
-
| How resources chain (CRUD, FK, ETag) | `.api-resources.yaml` |
|
|
143
|
-
| What vars are needed and why | `.api-fixtures.yaml` |
|
|
144
|
-
| What var values are set | `.env.yaml` |
|
|
145
|
-
| Annotation overlay (seed_body, lifecycle, pagination, readback_diff) | `.api-resources.local.yaml` |
|
|
146
|
-
| Full request/response schema (only when needed) | `spec.json` |
|
|
147
|
-
|
|
148
|
-
## --json envelope (uniform across commands)
|
|
149
|
-
|
|
150
|
-
```jsonc
|
|
151
|
-
{
|
|
152
|
-
"ok": true|false,
|
|
153
|
-
"command": "<name>",
|
|
154
|
-
"data": { /* command-specific */ },
|
|
155
|
-
"warnings": ["..."],
|
|
156
|
-
"errors": [{ "code": "ZondErrorCode", "message": "...", "details": {} }],
|
|
157
|
-
"exit_code": 0|1|2
|
|
158
|
-
}
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
Route on `errors[].code`, not message. Stdout discipline: `--json`
|
|
162
|
-
emits only the envelope on stdout; progress/hints to stderr.
|
|
163
|
-
|
|
164
|
-
`--json` ≠ `--report json`. `--json` wraps the **command result**;
|
|
165
|
-
`--report json` wraps the **test-run artefact**. `zond run` doesn't
|
|
166
|
-
accept `--json` — use `--report json`.
|
|
167
|
-
|
|
168
|
-
## Entry points — pick by intent
|
|
169
|
-
|
|
170
|
-
| User asked… | Start at |
|
|
171
|
-
|---|---|
|
|
172
|
-
| "smoke this API", "quick first pass", "demo / CI gate" | `zond audit --api <name>` (safe-mode breadth pass — no live mutations) |
|
|
173
|
-
| "audit this API", "test the whole API", "raise coverage", "deep pass" | walk Phase 0–9 — audit covers smoke; depth (stateful, security, learn-apply) needs phase-level decisions |
|
|
174
|
-
| "full live audit against sandbox", "include mass-assignment/security probes" | `zond audit --api <name> --live --with-mass-assignment --with-security` (REQUIRES throwaway/sandbox account) |
|
|
175
|
-
| "find bugs", "test for 5xx", "probe sweep" | Phase 0 → Phase 7 (Probes) |
|
|
176
|
-
| "security only / SSRF / CRLF" | Phase 7.2 directly with `--dry-run` first |
|
|
177
|
-
| "deep depth-checks", "SARIF", "stateful invariants" | Hand off → `zond-checks` |
|
|
178
|
-
| "what broke in last run", "diagnose run X" | Hand off → `zond-triage` |
|
|
179
|
-
| "write a test for X flow", "smoke this checkout" | Phase S (Scenarios) |
|
|
180
|
-
| "what vars does this API need" | `zond doctor --api <name> --json` |
|
|
181
|
-
| "share results", "case study" | Phase 9 (Share) |
|
|
182
|
-
| "workspace looks messy" | `zond clean --api <name>` (dry-run → `--force`) |
|
|
183
|
-
|
|
184
|
-
## Phase 0 — Orient
|
|
185
|
-
|
|
186
|
-
```bash
|
|
187
|
-
zond doctor --api <name> --json # gaps + freshness
|
|
188
|
-
```
|
|
189
|
-
|
|
190
|
-
Then read the three artifacts (NOT the raw spec):
|
|
191
|
-
|
|
192
|
-
```bash
|
|
193
|
-
cat apis/<name>/.api-catalog.yaml | head -80
|
|
194
|
-
cat apis/<name>/.api-resources.yaml
|
|
195
|
-
cat apis/<name>/.api-fixtures.yaml
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
If doctor reports stale → `zond refresh-api <name>`.
|
|
199
|
-
|
|
200
|
-
## Phase 1 — Fixture pack
|
|
201
|
-
|
|
202
|
-
`zond prepare-fixtures` is **single-pass and deterministic**: it walks
|
|
203
|
-
`.api-resources.yaml`, hits each owner list/read endpoint, and **reports**
|
|
204
|
-
which FK vars are unfilled (empty list → create one first; non-empty →
|
|
205
|
-
pick a value; 404/403 → auth/drift). It does **not** POST-create resources
|
|
206
|
-
and (ARV-362) **never harvests a value** — which record/field fills a slot
|
|
207
|
-
is your judgment. Fill every gap by hand (`fixtures add` / `.env.yaml`).
|
|
208
|
-
|
|
209
|
-
```bash
|
|
210
|
-
zond prepare-fixtures --api <name> # report gaps (never writes values)
|
|
211
|
-
zond prepare-fixtures --api <name> --apply # only unsets stale ids under --refresh; never harvests
|
|
212
|
-
zond prepare-fixtures --api <name> --refresh # = --verify --apply: drop (unset) stale 404 ids → you refill
|
|
213
|
-
```
|
|
214
|
-
|
|
215
|
-
It also scans the generated suites and reports two named gap buckets
|
|
216
|
-
(warnings in text, `summary.fixtureGaps` in `--json`) — report only, no
|
|
217
|
-
values invented, no auto-seed (ARV-349/350):
|
|
218
|
-
- **`unseededRoots`** — required manifest vars that are empty, referenced by
|
|
219
|
-
a suite, and seeded by no step (e.g. `{{account}}` created in `crud-accounts`
|
|
220
|
-
but referenced uncaptured in `persons-crud`). These are chain-heads: one id
|
|
221
|
-
un-skips a whole dependent CRUD suite. Fix these first.
|
|
222
|
-
- **`undefinedVars`** — suite `{{vars}}` no manifest entry, capture, or param
|
|
223
|
-
produces (`{{bank_code}}`, `{{tax_id}}`). Supply by hand.
|
|
224
|
-
|
|
225
|
-
To fill `unseededRoots` autonomously — author create-bodies, POST them, and
|
|
226
|
-
capture the returned ids — hand off to **`zond-seed`** (agent reasons, zond
|
|
227
|
-
executes via `request … --capture`). It seeds only what the API self-serves
|
|
228
|
-
and reports external-input gaps honestly.
|
|
229
|
-
|
|
230
|
-
When a var stays UNSET after `--apply`, read the per-target reason in the
|
|
231
|
-
output, then fill the gap directly:
|
|
232
|
-
|
|
233
|
-
| Input you have | Command |
|
|
234
|
-
|---|---|
|
|
235
|
-
| Concrete id values | `zond fixtures add <var>=<id> [--validate] --apply` |
|
|
236
|
-
| Curl from devtools / dashboard | `pbpaste \| zond fixtures import --from-curl --apply` |
|
|
237
|
-
| Nothing yet (empty list) | Create the resource in the product UI / via API, then harvest its id with `fixtures add` |
|
|
238
|
-
|
|
239
|
-
Common reasons a var stays UNSET (all fixed outside prepare-fixtures):
|
|
240
|
-
- empty list on the target account — no record exists to harvest;
|
|
241
|
-
- the id lives only in a vendor dashboard (Stripe `cus_*`, GitHub PR
|
|
242
|
-
numbers, Sentry issue ids) with no discoverable list endpoint;
|
|
243
|
-
- the value is gated outside the API path (email verify / paid plan /
|
|
244
|
-
SCIM / TOS limits). Document the gap; don't loop.
|
|
245
|
-
|
|
246
|
-
Editing `.env.yaml` by hand is the sanctioned fallback when the CLI
|
|
247
|
-
cannot harvest a value (write-only ingest endpoints, SDK-only ids,
|
|
248
|
-
slugs that live in user's project config). Touch values only — never
|
|
249
|
-
add keys not in `.api-fixtures.yaml`.
|
|
250
|
-
|
|
251
|
-
### Dynamic value functions in `.env.yaml` (ARV-190, m-21)
|
|
252
|
-
|
|
253
|
-
Stale hardcoded UUIDs / dates / idempotency keys pin existing rows or
|
|
254
|
-
expire mid-run. Use `#(funcName)` / `#(funcName(args))` references —
|
|
255
|
-
evaluated at load time, stable within one run:
|
|
256
|
-
|
|
257
|
-
| Token | Resolves to | Use for |
|
|
258
|
-
|---|---|---|
|
|
259
|
-
| `#(uuid)` | fresh UUID v4 (same value across all references in this run) | Idempotency-Key, request id |
|
|
260
|
-
| `#(uuidStable(<seed>))` | deterministic UUID from seed (sha-256 shaped as v4) | reproducible tests |
|
|
261
|
-
| `#(today)` | `YYYY-MM-DD` (UTC) | expiry-from dates |
|
|
262
|
-
| `#(todayPlus(N))` | today + N days (N may be negative) | expires_at, valid_until |
|
|
263
|
-
| `#(now)` | ISO 8601 timestamp | created_at hints |
|
|
264
|
-
| `#(unix)` | seconds since epoch | epoch timestamps |
|
|
265
|
-
| `#(alphanumeric(N))` | N random `[a-z0-9]` chars (default 8) | one-shot tokens |
|
|
266
|
-
| `#(env:VAR)` | `process.env.VAR` (alias for `${VAR}`) | secrets passed through |
|
|
267
|
-
|
|
268
|
-
```yaml
|
|
269
|
-
# apis/foo/.env.yaml
|
|
270
|
-
idempotency_key: "#(uuid)" # same uuid across every step in this run
|
|
271
|
-
trial_expires: "#(todayPlus(30))" # 30 days from now
|
|
272
|
-
request_id: "req-#(uuid)" # embedded; shares uuid with above
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
Resolution order: `${ENV}` → `#(...)` → `@secret:` → `@identity:`. Dynamic
|
|
276
|
-
functions run BEFORE secret/identity refs so a secret value that happens to
|
|
277
|
-
contain `#(...)` stays opaque (no double-expansion). Per-run cache means
|
|
278
|
-
`#(uuid)` referenced twice in the same `.env.yaml` returns the same value
|
|
279
|
-
— critical for idempotency-replay flows.
|
|
280
|
-
|
|
281
|
-
## Phase 2 — Annotate (NEW, m-20 / ARV-187)
|
|
282
|
-
|
|
283
|
-
**Critical**: zond itself does NOT call any LLM. The flow is
|
|
284
|
-
**dump → agent fills YAML → apply**. The agent (you) generates the
|
|
285
|
-
overlay; zond validates and writes it into `.api-resources.local.yaml`.
|
|
286
|
-
|
|
287
|
-
`zond api annotate` has 6 dump-subcommands (one per aspect). Pick what
|
|
288
|
-
your downstream check needs:
|
|
289
|
-
|
|
290
|
-
| Aspect | Dump produces | Feeds into |
|
|
291
|
-
|---|---|---|
|
|
292
|
-
| `--seed-bodies` | request-body schema per resource | all stateful checks (create-body overlay) |
|
|
293
|
-
| `--readback` | create+read pair (write-shape vs read-shape) | `cross_call_references` ignore_fields, write_to_read_map |
|
|
294
|
-
| `--idempotency` | header candidates | `idempotency_replay` (header, scope, body_hash_fields) |
|
|
295
|
-
| `--pagination` | list-endpoint cursor/page detection | `pagination_invariants` (type, cursor_param) |
|
|
296
|
-
| `--lifecycle` | action-endpoint candidates per resource | `lifecycle_transitions` (states, transitions) |
|
|
297
|
-
| `--resources` | orphan endpoints not in resource graph | `.api-resources.local.yaml` extensions |
|
|
298
|
-
|
|
299
|
-
**Workflow** (for ANY aspect):
|
|
300
|
-
|
|
301
|
-
```bash
|
|
302
|
-
# 1. Dump — emits JSON slices on stdout. Restrict scope with --only.
|
|
303
|
-
zond api annotate dump --api <name> --seed-bodies --only r1,r2 > /tmp/dump.json
|
|
304
|
-
|
|
305
|
-
# 2. Agent reads the dump and writes a YAML response file (see format below).
|
|
306
|
-
|
|
307
|
-
# 3. Apply — dry-run first (diff), then --yes to write.
|
|
308
|
-
zond api annotate apply --api <name> --seed-bodies --input /tmp/out.yaml # dry-run + diff
|
|
309
|
-
zond api annotate apply --api <name> --seed-bodies --input /tmp/out.yaml --yes # write
|
|
310
|
-
```
|
|
311
|
-
|
|
312
|
-
YAML response format — a top-level list of entries, each one resource:
|
|
313
|
-
|
|
314
|
-
```yaml
|
|
315
|
-
- resource: <name>
|
|
316
|
-
seed_body: # (--seed-bodies)
|
|
317
|
-
content_type: application/x-www-form-urlencoded
|
|
318
|
-
body:
|
|
319
|
-
amount: 1000
|
|
320
|
-
currency: usd
|
|
321
|
-
rationale: 'why this shape' # optional, helps future review
|
|
322
|
-
confidence: high # high|medium|low — optional
|
|
323
|
-
```
|
|
324
|
-
|
|
325
|
-
Same shape for `readback_diff`, `idempotency`, `pagination`,
|
|
326
|
-
`lifecycle` blocks (see `zond-checks` for the per-aspect schemas).
|
|
327
|
-
|
|
328
|
-
**When to annotate**: run dump+apply BEFORE Phase 6 (stateful checks)
|
|
329
|
-
for any production API. Without annotation, stateful checks fall back
|
|
330
|
-
to defaults and miss API-specific quirks (custom pagination params,
|
|
331
|
-
non-standard lifecycle field names, write-only fields in create body).
|
|
332
|
-
|
|
333
|
-
**For Stripe-style form-encoded APIs**: `--seed-bodies` is the single
|
|
334
|
-
biggest win — the create-body overlay it produces lets stateful checks
|
|
335
|
-
POST valid resources instead of 400-ing on schema-derived random bodies.
|
|
336
|
-
|
|
337
|
-
### Agent-in-the-loop annotate (dump → write → apply, ARV-187 / 277 / 278-282)
|
|
338
|
-
|
|
339
|
-
zond does NOT infer annotations. YOU (the agent) read the spec slice and
|
|
340
|
-
write the overlay; zond only dumps raw material and validates/merges what
|
|
341
|
-
you produce.
|
|
342
|
-
|
|
343
|
-
```bash
|
|
344
|
-
# 1. Dump the raw material for a resource — spec slice + expected YAML
|
|
345
|
-
# shape + (for seed-bodies) the last N seed-POST attempts so you see
|
|
346
|
-
# the error progression, not just the current 400 (ARV-277/278).
|
|
347
|
-
zond api annotate dump --api <name> --seed-bodies --only <res> \
|
|
348
|
-
--with-last-attempt --history 5
|
|
349
|
-
|
|
350
|
-
# 2. YOU write the overlay: read the dumped slice + attempt history,
|
|
351
|
-
# reason about the required create body / pagination / lifecycle /
|
|
352
|
-
# idempotency, and produce one YAML doc per resource. Edit
|
|
353
|
-
# .api-resources.local.yaml directly OR emit a file for apply.
|
|
354
|
-
|
|
355
|
-
# 3. Apply — zond validates your YAML (zod), renders a diff, and (with
|
|
356
|
-
# --yes) merges. With --gap-fill-only (ARV-281) your response can ONLY
|
|
357
|
-
# add missing fields — already-set blocks are protected from overwrite.
|
|
358
|
-
zond api annotate apply --api <name> --seed-bodies --input agent-out.yaml --gap-fill-only --yes
|
|
359
|
-
|
|
360
|
-
# 4. Refresh fixtures — drop (unset) stale (404) ids so they resurface as
|
|
361
|
-
# gaps; refill them by hand / `fixtures add` (discover never harvests).
|
|
362
|
-
zond prepare-fixtures --api <name> --refresh
|
|
363
|
-
```
|
|
364
|
-
|
|
365
|
-
## Phase 3 — Generate tests
|
|
366
|
-
|
|
367
|
-
```bash
|
|
368
|
-
zond generate apis/<name>/spec.json --output apis/<name>/tests [--tag <spec-tag>] [--uncovered-only]
|
|
369
|
-
zond check tests apis/<name>/tests
|
|
370
|
-
```
|
|
371
|
-
|
|
372
|
-
`generate` fills bodies with typed generators (`{{$randomString}}` etc.).
|
|
373
|
-
Format-strict APIs reject many — that's a **test-fix**, not a backend bug.
|
|
374
|
-
Pair with Phase 1 fixtures + Phase 2 annotation.
|
|
375
|
-
|
|
376
|
-
Required FK / reference body fields (`*_id`, unresolved `*Code` like
|
|
377
|
-
`sequenceTypeCode`) are wired to `{{fixture}}` refs (ARV-45) — canonicalised
|
|
378
|
-
(`sequence_type_code`) and listed in `.api-fixtures.yaml` as `source: body-fk`.
|
|
379
|
-
Fill their real values once in `.env.yaml`; no more hand-editing generated
|
|
380
|
-
tests. Blank fixture → the var is unbound and dependent tests skip.
|
|
381
|
-
|
|
382
|
-
If a CRUD chain you expected is missing, run
|
|
383
|
-
`zond generate <spec> --explain` — diagnostic table shows every POST
|
|
384
|
-
endpoint with verdict (no GET-by-id, non-`{id}` item param,
|
|
385
|
-
trailing-slash mismatch, …).
|
|
386
|
-
|
|
387
|
-
## Phase 4 — Spec-lint (hygiene, separate workflow)
|
|
388
|
-
|
|
389
|
-
```bash
|
|
390
|
-
zond lint --api <name> # 0 network requests
|
|
391
|
-
zond lint --api <name> --json | jq '.data.stats'
|
|
392
|
-
# Equivalent: `zond check spec --api <name>`
|
|
393
|
-
```
|
|
394
|
-
|
|
395
|
-
ARV-255 (m-21 pivot): **spec-lint is hygiene, not security or
|
|
396
|
-
contract.** It runs separately from `zond audit` / `zond probe` /
|
|
397
|
-
`zond checks run` — those produce the security/reliability/contract
|
|
398
|
-
report, lint stays out of that pile.
|
|
399
|
-
|
|
400
|
-
Severity capped at **LOW / INFO** by design:
|
|
401
|
-
- LOW: real spec violations (format mismatch in example, missing
|
|
402
|
-
path-param format, response without schema).
|
|
403
|
-
- INFO: style and documentation gaps (additionalProperties missing,
|
|
404
|
-
naming, missing examples).
|
|
405
|
-
|
|
406
|
-
No HIGH/MEDIUM ever — static analysis has no runtime evidence, so the
|
|
407
|
-
severity matrix forbids escalation. `--strict` opts back into a
|
|
408
|
-
non-zero exit when any LOW lands.
|
|
409
|
-
|
|
410
|
-
## Phase 5 — Run (sanity → smoke → CRUD)
|
|
411
|
-
|
|
412
|
-
Wrap multi-run sweeps in a session so `/runs` shows one row:
|
|
413
|
-
|
|
414
|
-
```bash
|
|
415
|
-
zond session start --label "smoke + probes"
|
|
416
|
-
zond run apis/<name>/tests --tag sanity --report json # 5.1 sanity
|
|
417
|
-
zond run apis/<name>/tests --safe --report json # 5.2 smoke (GET-only)
|
|
418
|
-
zond run apis/<name>/tests --tag crud,setup --validate-schema --report json # 5.3 full CRUD
|
|
419
|
-
zond run apis/<name>/tests --tag positive --include 'path:^/emails' # 5.4 narrow
|
|
420
|
-
zond session end
|
|
421
|
-
```
|
|
422
|
-
|
|
423
|
-
**Always pass `--validate-schema` for CRUD** — contract drift is
|
|
424
|
-
invisible without it. `schema_violation` failures are real backend bugs;
|
|
425
|
-
treat them like 5xx.
|
|
426
|
-
|
|
427
|
-
**After any `zond run` with failures → delegate to `zond-triage`.** Do
|
|
428
|
-
NOT parse `runs/run-NN.json` with `jq` by hand. The triage skill reads
|
|
429
|
-
`zond db diagnose --json` (which already buckets by `recommended_action`
|
|
430
|
-
in `data.by_recommended_action`, ARV-228) and routes by closed enum —
|
|
431
|
-
re-implementing that logic with prose heuristics drifts. Only fall back
|
|
432
|
-
to manual CLI queries when the failure is outside the example slice.
|
|
433
|
-
|
|
434
|
-
Rate limit: `zond run` defaults to an adaptive limiter (no-op until
|
|
435
|
-
`RateLimit-*` headers appear). Pass `--rate-limit <N>` for a hard cap;
|
|
436
|
-
`--sequential` for old binaries.
|
|
437
|
-
|
|
438
|
-
**Long runs**: `zond run` shows a periodic progress line on stderr when
|
|
439
|
-
stdout/stderr is a TTY (`zond: [42s] 234/10927 steps (2%), 230 req, ~30
|
|
440
|
-
req/s, ETA ~5m`) — overwrites itself in place. Suppressed by `--quiet`
|
|
441
|
-
and on non-interactive output. For huge probe-suites (e.g. GitHub spec
|
|
442
|
-
→ ~10k probes under `--rate-limit 30` = 6+ min), this signals "alive,
|
|
443
|
-
not hung". `zond probe static` itself prints a scale-warning block with
|
|
444
|
-
ETA-at-rate-limit estimates and a `--max-per-endpoint K` sampling hint
|
|
445
|
-
when `totalProbes ≥ 2000`.
|
|
446
|
-
|
|
447
|
-
**Hard cap on requests**: pass `--max-requests N` to cap outgoing HTTP
|
|
448
|
-
calls across the whole run. Remaining steps short-circuit to `skip` with
|
|
449
|
-
`error: "max-requests-cap-reached"`. Each `retry_until` attempt counts as
|
|
450
|
-
one request. Useful for sampling huge probe runs (`--max-requests 500`)
|
|
451
|
-
and for CI time-boxing.
|
|
452
|
-
|
|
453
|
-
`--all`: fold every `apis/<name>/tests/` dir into one `runs.id` per
|
|
454
|
-
invocation (CI shape). CI context (commit_sha, branch, trigger=ci)
|
|
455
|
-
auto-stamped from env vars.
|
|
456
|
-
|
|
457
|
-
### Spec-drift learning tail (`--learn`)
|
|
458
|
-
|
|
459
|
-
After the initial CRUD run, **always** close with `--learn` /
|
|
460
|
-
`--learn-apply`. R09 measured a 28% → 58% pass-coverage jump from this
|
|
461
|
-
phase alone.
|
|
462
|
-
|
|
463
|
-
```bash
|
|
464
|
-
zond run apis/<name>/tests --learn # detect (read-only)
|
|
465
|
-
zond run apis/<name>/tests --learn-apply --learn-target test # rewrite expect.status
|
|
466
|
-
zond run apis/<name>/tests --learn-apply --learn-target drifts # allowlist in tolerated-drifts.yaml
|
|
467
|
-
```
|
|
468
|
-
|
|
469
|
-
**Caveat — `--learn-apply test` weakens assertions, not the code.**
|
|
470
|
-
Diff `apis/<name>/tests/*.yaml` + `tolerated-drifts.yaml` after every
|
|
471
|
-
apply; treat unexpected widenings as a review-blocker.
|
|
472
|
-
|
|
473
|
-
## Phase 6 — Stateful checks (m-20)
|
|
474
|
-
|
|
475
|
-
After Phase 2 annotation is applied, run the cross-call invariants.
|
|
476
|
-
See `zond-checks` for per-check semantics.
|
|
477
|
-
|
|
478
|
-
```bash
|
|
479
|
-
zond checks run --api <name> --check stateful --report ndjson --live # CRUD create-chains need --live
|
|
480
|
-
```
|
|
481
|
-
|
|
482
|
-
**Safe by default (ARV-299)**: without `--live`, `checks run` skips the
|
|
483
|
-
stateful CRUD create-chains (`ensure_resource_availability`,
|
|
484
|
-
`use_after_free`) — they'd POST real resources. Read-only stateful checks
|
|
485
|
-
(pagination, observation-mode lifecycle, cross-call GET diff) still run.
|
|
486
|
-
Add `--live` **only against a throwaway/sandbox account** to exercise the
|
|
487
|
-
create-chains.
|
|
488
|
-
|
|
489
|
-
`--check stateful` expands to the state-machine set (ARV-325):
|
|
490
|
-
- `cross_call_references` — POST→GET drift (uses readback_diff overlay)
|
|
491
|
-
- `idempotency_replay` — bit-identical replay on `Idempotency-Key`
|
|
492
|
-
- `pagination_invariants` — `?limit=N` + `?after=last_id` non-overlap
|
|
493
|
-
- `lifecycle_transitions` — declared state-machine validity
|
|
494
|
-
- `use_after_free`, `ensure_resource_availability` — CRUD lifecycle
|
|
495
|
-
- `cursor_boundary_fuzzing` — list-cursor edge values
|
|
496
|
-
- `webhooks` (recipe-based, see `docs/recipes/webhook-receiver.md`)
|
|
497
|
-
|
|
498
|
-
`ignored_auth` / `open_cors_on_sensitive` are **not** part of the alias —
|
|
499
|
-
they're auth/security checks that roughly triple run time on wide APIs.
|
|
500
|
-
Run them by explicit id: `--check ignored_auth,open_cors_on_sensitive`.
|
|
501
|
-
|
|
502
|
-
**Iron rule**: don't run `--check stateful` without prior `api annotate`
|
|
503
|
-
review. Defaults catch the obvious; quirks need declared config.
|
|
504
|
-
|
|
505
|
-
## Phase 7 — Proactive bug hunting (probes)
|
|
506
|
-
|
|
507
|
-
```bash
|
|
508
|
-
zond probe static --api <name> # validation + methods (defaults; generator, always safe)
|
|
509
|
-
zond probe mass-assignment --api <name> --live --emit-tests apis/<name>/probes/mass-assignment
|
|
510
|
-
zond probe security ssrf,crlf,open-redirect --api <name> --live \
|
|
511
|
-
--emit-tests apis/<name>/probes/security
|
|
512
|
-
zond run apis/<name>/probes --report json
|
|
513
|
-
```
|
|
514
|
-
|
|
515
|
-
**Safe by default (ARV-299)**: `probe mass-assignment` / `probe security`
|
|
516
|
-
without `--live` plan only (no live attack payloads) — same as `--dry-run`.
|
|
517
|
-
Add `--live` **only against a throwaway/sandbox account** to send probes.
|
|
518
|
-
`probe static` just generates suites, so it's always safe.
|
|
519
|
-
|
|
520
|
-
Flag: 5xx on null/empty/wrong-type body → missing validation. 2xx on
|
|
521
|
-
undeclared method → contract drift. `is_admin: true` echoed in response
|
|
522
|
-
→ HIGH from mass-assignment.
|
|
523
|
-
|
|
524
|
-
**Body-FK auto-discovery** — `mass-assignment` resolves required
|
|
525
|
-
`*_id`/`*_slug`/`*_uuid`/`*_key` fields by hitting sibling list endpoints.
|
|
526
|
-
If digest still says "unresolved body FKs:" — add to `.env.yaml` manually.
|
|
527
|
-
|
|
528
|
-
**Auto-path-FK** — same logic for path params (`{domain_id}`/`{webhook_id}`).
|
|
529
|
-
Cached per run. Pass `--no-discover` to opt out.
|
|
530
|
-
|
|
531
|
-
### 7.1 — Manual mass-assignment catch-up
|
|
532
|
-
|
|
533
|
-
`mass-assignment` digest splits findings: HIGH / MED / LOW /
|
|
534
|
-
INCONCLUSIVE (no valid body — fixture problem) / INCONCLUSIVE-5XX
|
|
535
|
-
(baseline crashed — already in validation-probe). Sweep INCONCLUSIVE
|
|
536
|
-
with `--emit-template`:
|
|
537
|
-
|
|
538
|
-
```bash
|
|
539
|
-
zond probe mass-assignment --api <name> --emit-template "POST:/<resource>" \
|
|
540
|
-
--output apis/<name>/probes/mass-assignment/<resource>.yaml
|
|
541
|
-
```
|
|
542
|
-
|
|
543
|
-
Emits `create → verify → cleanup` chain pre-filled with classic vectors
|
|
544
|
-
(`is_admin`, `role`, `owner_id`). If a privileged field survives the
|
|
545
|
-
round-trip GET → **HIGH**, file via `report bundle --include case-study`.
|
|
546
|
-
|
|
547
|
-
### 7.2 — Security probes
|
|
548
|
-
|
|
549
|
-
```bash
|
|
550
|
-
zond probe security ssrf,crlf --api <name> --dry-run # first run: which (endpoint, field) — same as default --safe
|
|
551
|
-
zond probe security ssrf,crlf,open-redirect --api <name> --live # send, sandbox only
|
|
552
|
-
```
|
|
553
|
-
|
|
554
|
-
**Targeting** (two filters; an endpoint must pass BOTH to be planned):
|
|
555
|
-
|
|
556
|
-
1. **Has a JSON request body** — GET-only routes are skipped with
|
|
557
|
-
`no-body`. The probes work by mutating a body field, so read-only
|
|
558
|
-
endpoints have nothing to attack.
|
|
559
|
-
2. **Has ≥1 field name matching the requested class detectors** — if
|
|
560
|
-
no field matches, the endpoint is skipped with `no-matched-field`.
|
|
561
|
-
|
|
562
|
-
Field autodetection: SSRF (`*_url`/`webhook`/`callback`/`format: uri`),
|
|
563
|
-
CRLF (`subject`/`*_prefix`/`name`), open-redirect (`redirect`/`next`).
|
|
564
|
-
|
|
565
|
-
If `--dry-run` reports `0 planned / 0 skipped / 0 total` with a narrow
|
|
566
|
-
`--include`, the scope likely captured only GET routes — broaden the
|
|
567
|
-
filter or drop `--include` to see the per-endpoint skip reasons. With
|
|
568
|
-
no `--include`, the dry-run shows `no-body` / `no-matched-field` next
|
|
569
|
-
to each skipped endpoint so the gap is obvious.
|
|
570
|
-
|
|
571
|
-
Baseline-OK request first; if baseline ≠ 2xx → `INCONCLUSIVE-BASELINE`,
|
|
572
|
-
attacks skipped (kills 5×404 noise on scope-locked endpoints).
|
|
573
|
-
|
|
574
|
-
Verdict: HIGH (5xx OR payload echoed in 2xx — stored injection),
|
|
575
|
-
LOW (2xx, no echo — verify side-effects manually), OK (4xx).
|
|
576
|
-
|
|
577
|
-
**Cleanup is state-aware**: PUT/PATCH does `GET` → snapshot → attack →
|
|
578
|
-
restore. POST falls back to `DELETE` with eventual-consistency retry.
|
|
579
|
-
Restore failures → `## ⚠️ Cleanup failures` at top of digest.
|
|
580
|
-
|
|
581
|
-
CI exit: non-zero on HIGH > 0 OR `cleanup.error > 0`.
|
|
582
|
-
|
|
583
|
-
### 7.3 — Post-probe hygiene
|
|
584
|
-
|
|
585
|
-
```bash
|
|
586
|
-
zond prepare-fixtures --api <name> --refresh # drop stale FK ids
|
|
587
|
-
zond cleanup --orphans # retry DELETE for ~/.zond/orphans/
|
|
588
|
-
```
|
|
589
|
-
|
|
590
|
-
Skip only with `probe security --isolated` (isolated mode never attacks
|
|
591
|
-
seeded-fixture endpoints).
|
|
592
|
-
|
|
593
|
-
## Phase 8 — Coverage + spec drift
|
|
594
|
-
|
|
595
|
-
```bash
|
|
596
|
-
zond coverage --api <name> --union session # default: dual-metric (test + audit)
|
|
597
|
-
zond coverage --api <name> --union since:1h # last hour
|
|
598
|
-
zond coverage --api <name> --union tag:smoke # tag-filtered
|
|
599
|
-
zond coverage --api <name> --scope audit --union session # only "did zond touch the API"
|
|
600
|
-
zond coverage --api <name> --scope test --union session # legacy single block
|
|
601
|
-
zond coverage --api <name> --fail-on-coverage 50 # CI gate (gates pass-coverage)
|
|
602
|
-
```
|
|
603
|
-
|
|
604
|
-
**Dual-metric output (ARV-265).** `zond coverage` prints two blocks side-by-side by default:
|
|
605
|
-
|
|
606
|
-
- **test-coverage** — runs produced by `zond run` only (`run_kind` ∈ {regular, probe}). Has two sub-metrics:
|
|
607
|
-
- `pass-coverage`: endpoint had a passing 2xx (strict; CI gate via `--fail-on-coverage`)
|
|
608
|
-
- `hit-coverage`: endpoint received any response (loose; breadth proxy)
|
|
609
|
-
Three-bucket JSON: `covered2xx`, `coveredButNon2xx`, `unhit`.
|
|
610
|
-
- **audit-coverage** — any HTTP touch from any producer (`zond run` + `checks run` + `probe` + `request` + `prepare-fixtures`). Use this to answer "did the scan reach the API at all?" — typical safe-mode runs without `zond run` still report >50% on a 1k-endpoint spec from `checks run` alone.
|
|
611
|
-
|
|
612
|
-
Source breakdown (`--scope audit` text or `data.audit_coverage.by_source` JSON):
|
|
613
|
-
|
|
614
|
-
```
|
|
615
|
-
audit-coverage: 619/1184 endpoints (52%, 1500 HTTP touches)
|
|
616
|
-
by source:
|
|
617
|
-
check 619 endpoints, 1500 events
|
|
618
|
-
regular 0 endpoints, 0 events
|
|
619
|
-
probe 0 endpoints, 0 events
|
|
620
|
-
request 3 endpoints, 3 events
|
|
621
|
-
```
|
|
622
|
-
|
|
623
|
-
Producers opt out via `ZOND_CHECKS_PERSIST=0`.
|
|
624
|
-
|
|
625
|
-
**Quality signals to gate CI on** (not raw pass-coverage):
|
|
626
|
-
|
|
627
|
-
| Signal | Command |
|
|
628
|
-
|---|---|
|
|
629
|
-
| Contract drift | `checks run --phase coverage` HIGH count == 0 |
|
|
630
|
-
| Stateful invariants | `checks run --check stateful` HIGH count == 0 |
|
|
631
|
-
| Auth / Authz / injection | `probe security` HIGH count == 0 |
|
|
632
|
-
| Mass-assignment | `probe mass-assignment` HIGH count == 0 |
|
|
633
|
-
| Response conformance | `run --validate-schema` `schema_violation` == 0 |
|
|
634
|
-
| Drift allowlist review | `git diff apis/<name>/tolerated-drifts.yaml` |
|
|
635
|
-
|
|
636
|
-
## Phase 9 — Share findings
|
|
637
|
-
|
|
638
|
-
```bash
|
|
639
|
-
zond report export <run-id> # default: triage/<api>/run-<id>/
|
|
640
|
-
zond report bundle 135..142 -o triage/sweep/ # all artefacts (default): case-study + export + diagnose + index.md
|
|
641
|
-
zond report bundle <run-id> --include case-study,diagnose # subset only (drop export)
|
|
642
|
-
zond report bundle --session <id> -o triage/session/ # group by session
|
|
643
|
-
```
|
|
644
|
-
|
|
645
|
-
Bodies > 8 KB truncated by default; `--no-body-cap` to keep full.
|
|
646
|
-
Existing files at `--output` auto-rotate to `<stem>-vN.<ext>`;
|
|
647
|
-
`--overwrite` to silence. **Always pass `--redact-identity` for
|
|
648
|
-
outbound sharing** (secret-redaction is on by default; identity slugs
|
|
649
|
-
need the extra flag).
|
|
650
|
-
|
|
651
|
-
Offer this proactively after a run surfaces `definitely_bug`
|
|
652
|
-
(5xx / schema violation / mass-assignment 2xx). Skip for `env_issue` /
|
|
653
|
-
`quirk`.
|
|
654
|
-
|
|
655
|
-
## One-shot smoke audit — `zond audit`
|
|
656
|
-
|
|
657
|
-
**`zond audit` is the breadth pass, not "full audit".** It covers the
|
|
658
|
-
*smoke + coverage* slice — prepare-fixtures → generate → probe static
|
|
659
|
-
→ session-wrapped run on tests+probes → coverage → `audit-report.html`.
|
|
660
|
-
Depth (stateful checks, security probes against R18 evidence-gates,
|
|
661
|
-
m-20 annotate, `--learn-apply` iteration) is the **skill's job, not
|
|
662
|
-
audit's**: those stages need decisions audit can't make
|
|
663
|
-
(annotate-before-stateful, scoping after auth-cluster, evidence
|
|
664
|
-
preconditions). Use audit for: CI smoke, demo, "first 5 minutes on a
|
|
665
|
-
new API". For a real depth campaign, walk Phase 0–9 from this skill.
|
|
666
|
-
|
|
667
|
-
```bash
|
|
668
|
-
zond audit --api <name> --dry-run # print stage plan
|
|
669
|
-
zond audit --api <name> # safe mode (default) — no live probes
|
|
670
|
-
zond audit --api <name> --live --with-mass-assignment --with-security # full live pipeline (throwaway/sandbox only)
|
|
671
|
-
zond audit --api <name> --out reports/audit-<name>.html
|
|
672
|
-
```
|
|
673
|
-
|
|
674
|
-
**ARV-264 safe vs live decision matrix.** Default is `--safe` (implicit;
|
|
675
|
-
no flag needed). `--live` is the opt-in for traffic that mutates the
|
|
676
|
-
target API.
|
|
677
|
-
|
|
678
|
-
| Concern | `--safe` (default) | `--live` |
|
|
679
|
-
|---|---|---|
|
|
680
|
-
| `prepare-fixtures` mode | single-pass path-FK discovery (GET) | same — single-pass, never POST-creates |
|
|
681
|
-
| `probe mass-assignment` | skipped even if `--with-mass-assignment` set, with a warning | runs when `--with-mass-assignment` set |
|
|
682
|
-
| `probe security` (SSRF/CRLF/redirect) | skipped even if `--with-security` set, with a warning | runs when `--with-security` set |
|
|
683
|
-
| `probe static` (validation+methods) | always runs — pure spec walk, no live traffic | same |
|
|
684
|
-
| `run tests / probes` | runs against the live API but only generated GET-shape suites by default | same |
|
|
685
|
-
|
|
686
|
-
**Pre-flight warnings (printed before any subprocess fires):** missing
|
|
687
|
-
`auth_token` when the spec declares securitySchemes (probes will skip
|
|
688
|
-
with 401 and the report will read misleadingly green); no
|
|
689
|
-
securitySchemes declared in the spec (open API or incomplete spec —
|
|
690
|
-
auth-related checks will skip).
|
|
691
|
-
|
|
692
|
-
**Rule of thumb.** Unknown-scope PAT against a production API → `--safe`
|
|
693
|
-
(default). Throwaway account / sandbox tenant → `--live` is fine. Never
|
|
694
|
-
run `--live` against shared/production data on first contact.
|
|
695
|
-
|
|
696
|
-
Each stage prints `==> Stage N/M: <name>` and a completion line
|
|
697
|
-
`└─ OK · Ns` / `FAIL (exit K) · Ns` / `SKIPPED (reason)`. Exit 1 if
|
|
698
|
-
any non-coverage stage failed.
|
|
699
|
-
|
|
700
|
-
`generate` skipped via mtime when `tests/` is fresher than `spec.json`
|
|
701
|
-
(pass `--force` to override).
|
|
702
|
-
|
|
703
|
-
**Session reuse (ARV-65)**: if `.zond/current-session` is already set
|
|
704
|
-
when audit starts (i.e. user ran `zond session start` first), audit
|
|
705
|
-
reuses that session — `session-start` and `session-end` stages skip
|
|
706
|
-
with reason `reusing active session <id>`. The user's session stays
|
|
707
|
-
active after audit returns; runs inside audit are stamped with the
|
|
708
|
-
outer `session_id` and roll up under the user's `coverage --union
|
|
709
|
-
session` later.
|
|
710
|
-
|
|
711
|
-
ARV-158: when any run inside the audit session has failures, the
|
|
712
|
-
generated `audit-report.html` embeds a "Failures by run" section with
|
|
713
|
-
collapsible `<details>` per run — `by_recommended_action` buckets +
|
|
714
|
-
first example (method, path, status, reason) + concrete drill-down
|
|
715
|
-
commands (`zond db diagnose --run-id N --json`, `zond report export
|
|
716
|
-
N`). Triage starts from the rendered HTML; you only fall back to CLI
|
|
717
|
-
queries when you need fields outside the example slice.
|
|
718
|
-
|
|
719
|
-
When NOT to use audit: narrow asks ("fix run X", "why this endpoint
|
|
720
|
-
500s") — walk phases.
|
|
721
|
-
|
|
722
|
-
## Phase S — Single-flow scenarios
|
|
723
|
-
|
|
724
|
-
When the user wants to verify one specific flow (not breadth):
|
|
725
|
-
|
|
726
|
-
```bash
|
|
727
|
-
zond doctor --api <name> # confirm fixtures
|
|
728
|
-
# write apis/<name>/scenarios/<flow-slug>.yaml directly (no scaffold cmd)
|
|
729
|
-
zond session start --label "<short reason>"
|
|
730
|
-
zond run apis/<name>/scenarios/<flow>.yaml --report json
|
|
731
|
-
zond session end
|
|
732
|
-
```
|
|
733
|
-
|
|
734
|
-
YAML grammar (zond runner format — NOT Postman / OpenAPI shape):
|
|
735
|
-
|
|
736
|
-
```yaml
|
|
737
|
-
name: cancel-pending-order
|
|
738
|
-
tags: [scenario, orders]
|
|
739
|
-
base_url: "{{base_url}}"
|
|
740
|
-
headers: { Authorization: "Bearer {{auth_token}}" }
|
|
741
|
-
tests:
|
|
742
|
-
- name: create order
|
|
743
|
-
POST: /workspaces/{{ws_id}}/orders
|
|
744
|
-
json: { amount: 100, currency: USD }
|
|
745
|
-
expect:
|
|
746
|
-
status: 201
|
|
747
|
-
body:
|
|
748
|
-
id: { capture: order_id }
|
|
749
|
-
status: { equals: pending }
|
|
750
|
-
- name: cancel
|
|
751
|
-
POST: /orders/{{order_id}}/cancel
|
|
752
|
-
expect: { status: 200 }
|
|
753
|
-
- name: verify cancelled
|
|
754
|
-
GET: /orders/{{order_id}}
|
|
755
|
-
expect:
|
|
756
|
-
status: 200
|
|
757
|
-
body: { status: { equals: cancelled } }
|
|
758
|
-
- name: cleanup
|
|
759
|
-
DELETE: /orders/{{order_id}}
|
|
760
|
-
always: true # runs even on prior failure
|
|
761
|
-
expect: { status: [200, 204, 404] }
|
|
762
|
-
```
|
|
763
|
-
|
|
764
|
-
Grammar:
|
|
765
|
-
- HTTP verb is a top-level key on the step (`GET:`, `POST:`, …).
|
|
766
|
-
- Body: `json: {...}` / `form: {...}` / `multipart:` / `text:`.
|
|
767
|
-
- Captures: `expect.body.<field>: { capture: <var> }`.
|
|
768
|
-
- Assertions: `equals`, `type`, `exists`, `matches`, `gt`/`lt`,
|
|
769
|
-
`length*`, `each`, `contains_item`, `set_equals`, `oneOf`.
|
|
770
|
-
- `expect.status`: number or array. `oneOf` / `anyOf` string forms NOT
|
|
771
|
-
supported — pass an array directly.
|
|
772
|
-
- `always: true` — runs on prior failure (cleanup).
|
|
773
|
-
- `skip_if: "{{var}} =="` — skip when fixture var unset.
|
|
774
|
-
- Generators: `{{$randomEmail}}`, `{{$uuid}}`, `{{$randomString}}`,
|
|
775
|
-
`{{$randomInt}}`, `{{$randomIsoDate}}`. Full list:
|
|
776
|
-
`zond reference random-helpers`.
|
|
777
|
-
|
|
778
|
-
Hand-crafted scenarios are for **bug repro / post-deploy smoke /
|
|
779
|
-
verifying a fix** — small focused work. For breadth, use Phase 3
|
|
780
|
-
(generate) and the full audit chain.
|
|
781
|
-
|
|
782
|
-
## Diagnose failures
|
|
783
|
-
|
|
784
|
-
```bash
|
|
785
|
-
zond db runs --limit 5 --json
|
|
786
|
-
zond db diagnose <run-id> --json # grouped by root_cause
|
|
787
|
-
zond db run <id> --status 500 --json
|
|
788
|
-
zond db compare <idA> <idB> --json # regression diff + field-level body diff (.data.body_changes[], ARV-339)
|
|
789
|
-
zond db run <id> --report yaml # run snapshot as YAML (ARV-338) — keep/diff as text
|
|
790
|
-
```
|
|
791
|
-
|
|
792
|
-
> **db --json envelope shape** (consistent across siblings): the array
|
|
793
|
-
> always lives under `data.<plural>`, not directly under `data`.
|
|
794
|
-
> `db runs` → `.data.runs[]`, `db collections` → `.data.collections[]`,
|
|
795
|
-
> `db run <id> --status …` → `.data.results[]`. Top-level totals on runs
|
|
796
|
-
> are at `runs[].total/passed/failed`, not under a `.summary` wrapper.
|
|
797
|
-
|
|
798
|
-
`recommended_action` enum (closed):
|
|
799
|
-
- `report_backend_bug` — STOP, surface to user
|
|
800
|
-
- `fix_test_logic` — edit the YAML
|
|
801
|
-
- `fix_auth_config` / `fix_network_config` / `fix_env` — config issues
|
|
802
|
-
- `fix_spec` — edit OpenAPI (from `check spec`)
|
|
803
|
-
- `fix_fixture` — fill `.env.yaml` (from `prepare-fixtures` miss-* or
|
|
804
|
-
inconclusive mass-assignment baseline)
|
|
805
|
-
- `update_spec` — status-drift from `--learn`
|
|
806
|
-
|
|
807
|
-
`cascade` failures collapse under root cause — fix upstream, not
|
|
808
|
-
individually.
|
|
809
|
-
|
|
810
|
-
### Fixing 4xx from stub generators (`fix_test_logic`)
|
|
811
|
-
|
|
812
|
-
When body rejected on format (400/422 + "expected ..."):
|
|
813
|
-
|
|
814
|
-
1. Read failure: `zond db run <id> --status 422 --json`.
|
|
815
|
-
2. **Fixture pack first** — if it's a FK id / verified resource /
|
|
816
|
-
constrained enum, add to `.env.yaml` (Phase 1).
|
|
817
|
-
3. **Typed generator** — swap `{{$randomString}}` for matching format
|
|
818
|
-
helper.
|
|
819
|
-
4. **Hardcoded literal** — when typed generators fail (regex too strict).
|
|
820
|
-
5. **Runtime captures** — for resources the test creates (capture from
|
|
821
|
-
prior `create_*` step or `setup: true` suite). For pre-existing FKs,
|
|
822
|
-
prefer step 2.
|
|
823
|
-
|
|
824
|
-
For deep triage of a specific failing run → hand off to `zond-triage`.
|
|
825
|
-
|
|
826
|
-
## Auth / environments
|
|
827
|
-
|
|
828
|
-
- `apis/<name>/.env.yaml` is both auth and fixture pack — any key
|
|
829
|
-
interpolatable as `{{key}}`. Auto-gitignored.
|
|
830
|
-
- Login-flow tokens: `setup: true` suite captures vars that propagate
|
|
831
|
-
to later suites in the same run.
|
|
832
|
-
- `zond run --env <name>` loads `.env.<name>.yaml`. Discovery walks up
|
|
833
|
-
to workspace root; deeper files override shallower.
|
|
834
|
-
|
|
835
|
-
## File lifecycle
|
|
836
|
-
|
|
837
|
-
Everything zond writes is tracked in `.zond/manifest.json` with
|
|
838
|
-
sha256-at-write. Files whose hash no longer matches are **skipped**
|
|
839
|
-
on clean (user edits stay safe). Re-running a generator overwrites
|
|
840
|
-
the entry — manual edits to generator-owned files are lost on
|
|
841
|
-
regenerate; promote them to a separate suite first.
|
|
842
|
-
|
|
843
|
-
```bash
|
|
844
|
-
zond clean --api <name> # dry-run: lists what would be removed
|
|
845
|
-
zond clean --api <name> --force # delete (preserves probes/)
|
|
846
|
-
zond clean --api <name> --probes --force # also probe suites
|
|
847
|
-
zond clean --all --force # nuclear
|
|
848
|
-
```
|
|
849
|
-
|
|
850
|
-
Triage outputs default to
|
|
851
|
-
`triage/<api>/<run-id>/<command>-<ts>.<ext>`. Don't pass `--output`
|
|
852
|
-
unless the user has a specific destination.
|
|
853
|
-
|
|
854
|
-
## When to hand off
|
|
855
|
-
|
|
856
|
-
- "deep depth-checks", "SARIF", "boundary coverage", "stateful
|
|
857
|
-
invariants details", "annotate aspect XYZ" → `zond-checks`
|
|
858
|
-
- "what failed in last run", "why is it red", "case study from run X"
|
|
859
|
-
→ `zond-triage`
|
|
860
|
-
|
|
861
|
-
If the user is mid-workflow in one of those, don't restart from here.
|