@kirrosh/zond 0.23.0 → 0.26.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +176 -1
- package/README.md +8 -7
- package/package.json +2 -3
- package/src/CLAUDE.md +112 -0
- package/src/cli/commands/add-api.ts +19 -7
- package/src/cli/commands/api/annotate/index.ts +359 -4
- package/src/cli/commands/api/annotate/lifecycle.ts +1 -1
- package/src/cli/commands/api/annotate/overlay.ts +1 -1
- package/src/cli/commands/api/annotate/pagination.ts +10 -6
- package/src/cli/commands/api/annotate/prompts.ts +39 -2
- package/src/cli/commands/audit.ts +360 -54
- package/src/cli/commands/check.ts +15 -2
- package/src/cli/commands/checks.ts +352 -36
- package/src/cli/commands/cleanup.ts +4 -30
- package/src/cli/commands/coverage.ts +275 -57
- package/src/cli/commands/db.ts +311 -8
- package/src/cli/commands/discover.ts +281 -161
- package/src/cli/commands/doctor.ts +57 -3
- package/src/cli/commands/fixtures.ts +1 -1
- package/src/cli/commands/generate.ts +24 -7
- package/src/cli/commands/init/bootstrap.ts +4 -1
- package/src/cli/commands/init/index.ts +2 -2
- package/src/cli/commands/init/skills.ts +43 -0
- package/src/cli/commands/init/templates/agents.md +12 -3
- package/src/cli/commands/init/templates/skills/warm-up-target.md +122 -0
- package/src/cli/commands/init/templates/skills/zond-checks.md +268 -44
- package/src/cli/commands/init/templates/skills/zond-seed.md +114 -0
- package/src/cli/commands/init/templates/skills/zond-triage.md +88 -26
- package/src/cli/commands/init/templates/skills/zond.md +274 -64
- package/src/cli/commands/init/templates/zond-config.yml +1 -1
- package/src/cli/commands/prepare-fixtures.ts +14 -52
- package/src/cli/commands/probe/_seed-bodies.ts +52 -0
- package/src/cli/commands/probe/mass-assignment.ts +101 -10
- package/src/cli/commands/probe/security.ts +95 -12
- package/src/cli/commands/probe/webhooks.ts +2 -0
- package/src/cli/commands/probe.ts +87 -11
- package/src/cli/commands/refresh-api.ts +59 -1
- package/src/cli/commands/report-bundle.ts +3 -11
- package/src/cli/commands/request.ts +116 -0
- package/src/cli/commands/run.ts +53 -5
- package/src/cli/commands/schema-from-runs.ts +128 -0
- package/src/cli/commands/secrets.ts +133 -0
- package/src/cli/json-envelope.ts +0 -20
- package/src/cli/json-schemas.ts +51 -0
- package/src/cli/output.ts +17 -1
- package/src/cli/program.ts +5 -4
- package/src/cli/safe-live.ts +24 -0
- package/src/cli/status-filter.ts +0 -10
- package/src/core/audit/persist.ts +183 -0
- package/src/core/checks/budget.ts +59 -0
- package/src/core/checks/checks/cross_call_references.ts +17 -4
- package/src/core/checks/checks/cursor_boundary_fuzzing.ts +219 -0
- package/src/core/checks/checks/idempotency_replay.ts +1 -5
- package/src/core/checks/checks/ignored_auth.ts +44 -1
- package/src/core/checks/checks/index.ts +3 -0
- package/src/core/checks/checks/lifecycle_transitions.ts +169 -26
- package/src/core/checks/checks/negative_data_rejection.ts +119 -16
- package/src/core/checks/checks/not_a_server_error.ts +8 -0
- package/src/core/checks/checks/open_cors_on_sensitive.ts +47 -18
- package/src/core/checks/checks/pagination_invariants.ts +298 -117
- package/src/core/checks/checks/positive_data_acceptance.ts +1 -4
- package/src/core/checks/checks/status_code_conformance.ts +78 -7
- package/src/core/checks/mode.ts +3 -0
- package/src/core/checks/recommended-action.ts +5 -1
- package/src/core/checks/runner.ts +614 -27
- package/src/core/checks/spec-findings.ts +308 -0
- package/src/core/checks/types.ts +117 -1
- package/src/core/checks/zond-extensions.ts +73 -0
- package/src/core/classifier/recommended-action.ts +35 -6
- package/src/core/coverage/loader.ts +31 -0
- package/src/core/diagnostics/db-analysis.ts +200 -106
- package/src/core/diagnostics/failure-class.ts +21 -1
- package/src/core/diagnostics/failure-hints.ts +4 -208
- package/src/core/diagnostics/suggested-fixes.ts +2 -3
- package/src/core/generator/chunker.ts +1 -8
- package/src/core/generator/data-factory.ts +199 -61
- package/src/core/generator/fixtures-builder.ts +38 -31
- package/src/core/generator/index.ts +0 -2
- package/src/core/generator/openapi-reader.ts +98 -4
- package/src/core/generator/path-param-disambig.ts +30 -4
- package/src/core/generator/resources-builder.ts +276 -26
- package/src/core/generator/schema-utils.ts +22 -0
- package/src/core/generator/suite-generator.ts +168 -15
- package/src/core/generator/types.ts +6 -0
- package/src/core/identity/identity-file.ts +0 -0
- package/src/core/output/README.md +11 -29
- package/src/core/output/index.ts +1 -1
- package/src/core/output/run.ts +0 -35
- package/src/core/output/types.ts +0 -7
- package/src/core/parser/dynamic-values.ts +160 -0
- package/src/core/parser/variables.ts +0 -0
- package/src/core/probe/dry-run-envelope.ts +4 -0
- package/src/core/probe/mass-assignment/classify.ts +175 -0
- package/src/core/probe/mass-assignment/cleanup.ts +52 -0
- package/src/core/probe/mass-assignment/digest.ts +114 -0
- package/src/core/probe/mass-assignment/orchestrator.ts +459 -0
- package/src/core/probe/mass-assignment/regression.ts +141 -0
- package/src/core/probe/mass-assignment/suspects.ts +92 -0
- package/src/core/probe/mass-assignment/types.ts +135 -0
- package/src/core/probe/mass-assignment-probe.ts +23 -1118
- package/src/core/probe/mass-assignment-template.ts +32 -4
- package/src/core/probe/path-discovery.ts +3 -4
- package/src/core/probe/probe-harness.ts +21 -22
- package/src/core/probe/security/baseline.ts +174 -0
- package/src/core/probe/security/classify.ts +341 -0
- package/src/core/probe/security/cleanup.ts +125 -0
- package/src/core/probe/security/detectors.ts +71 -0
- package/src/core/probe/security/digest.ts +104 -0
- package/src/core/probe/security/orchestrator.ts +398 -0
- package/src/core/probe/security/regression.ts +103 -0
- package/src/core/probe/security/types.ts +151 -0
- package/src/core/probe/security-probe-class.ts +8 -2
- package/src/core/probe/security-probe.ts +28 -1449
- package/src/core/probe/shared.ts +26 -0
- package/src/core/probe/webhooks-probe.ts +5 -7
- package/src/core/runner/assertions.ts +1 -1
- package/src/core/runner/executor.ts +3 -18
- package/src/core/runner/form-encode.ts +8 -18
- package/src/core/runner/http-client.ts +38 -1
- package/src/core/runner/preflight-vars.ts +19 -15
- package/src/core/runner/rate-limiter.ts +11 -29
- package/src/core/runner/run-kind.ts +7 -1
- package/src/core/runner/schema-validator.ts +2 -6
- package/src/core/runner/send-request.ts +11 -6
- package/src/core/runner/types.ts +6 -0
- package/src/core/setup-api.ts +53 -15
- package/src/core/severity/index.ts +0 -63
- package/src/core/spec/infer-schema.ts +102 -0
- package/src/core/spec/merge-specs.ts +156 -0
- package/src/core/spec/schema-from-runs.ts +117 -0
- package/src/core/spec/schema-overlay.ts +130 -0
- package/src/core/util/ajv.ts +13 -0
- package/src/core/util/headers.ts +9 -0
- package/src/core/util/url.ts +24 -0
- package/src/core/workspace/fixture-gap-report.ts +84 -0
- package/src/core/workspace/fixture-gaps.ts +71 -0
- package/src/core/workspace/root.ts +13 -11
- package/src/db/migrate.ts +2 -0
- package/src/db/migrations/0002_run_kind_request.sql +59 -0
- package/src/db/queries/collections.ts +2 -2
- package/src/db/queries/results.ts +88 -0
- package/src/db/queries/runs.ts +56 -2
- package/src/db/queries.ts +3 -0
- package/src/db/schema.ts +7 -7
- package/src/cli/commands/bootstrap.ts +0 -710
- package/src/core/anti-fp/bootstrap.ts +0 -34
- package/src/core/anti-fp/index.ts +0 -33
- package/src/core/anti-fp/registry.ts +0 -44
- package/src/core/anti-fp/rules/baseline-echo.ts +0 -74
- package/src/core/anti-fp/rules/schemathesis/body_negation_becomes_valid.ts +0 -52
- package/src/core/anti-fp/rules/schemathesis/coverage_phase_boundary_positive.ts +0 -38
- package/src/core/anti-fp/rules/schemathesis/has_unverifiable_mutations.ts +0 -35
- package/src/core/anti-fp/rules/schemathesis/index.ts +0 -24
- package/src/core/anti-fp/rules/schemathesis/string_type_mutation_becomes_valid.ts +0 -53
- package/src/core/anti-fp/rules/subscription-gated/index.ts +0 -11
- package/src/core/anti-fp/rules/subscription-gated/paid-plan-403.ts +0 -75
- package/src/core/anti-fp/types.ts +0 -68
- package/src/core/generator/create-body.ts +0 -89
|
@@ -34,6 +34,7 @@ import { hashSpec } from "../../core/meta/meta-store.ts";
|
|
|
34
34
|
import { jsonOk, jsonError, printJson } from "../json-envelope.ts";
|
|
35
35
|
import { printError } from "../output.ts";
|
|
36
36
|
import { getApi } from "../util/api-context.ts";
|
|
37
|
+
import { detectSkillDrift } from "./init/skills.ts";
|
|
37
38
|
|
|
38
39
|
export interface DoctorOptions {
|
|
39
40
|
api?: string;
|
|
@@ -120,6 +121,10 @@ interface DoctorReport {
|
|
|
120
121
|
exists: boolean;
|
|
121
122
|
sha: string | null;
|
|
122
123
|
};
|
|
124
|
+
endpoints: {
|
|
125
|
+
total: number;
|
|
126
|
+
by_method: Record<string, number>;
|
|
127
|
+
} | null;
|
|
123
128
|
fixtures: {
|
|
124
129
|
required: FixtureMetaRow[];
|
|
125
130
|
optional: FixtureMetaRow[];
|
|
@@ -341,6 +346,21 @@ export async function doctorCommand(opts: DoctorOptions): Promise<number> {
|
|
|
341
346
|
const warnings: string[] = [];
|
|
342
347
|
if (!specExists) warnings.push(`spec.json not found at ${specAbsPath}`);
|
|
343
348
|
if (!manifest) warnings.push(`.api-fixtures.yaml missing — run \`zond refresh-api ${apiName}\``);
|
|
349
|
+
|
|
350
|
+
// ARV-237: warn when workspace .claude/skills/ copy drifts from the
|
|
351
|
+
// in-binary template (binary upgraded but `zond init` not re-run).
|
|
352
|
+
// `missing` is silent — user may have opted out via `--no-skills`.
|
|
353
|
+
try {
|
|
354
|
+
const wsRoot = findWorkspaceRoot().root;
|
|
355
|
+
const outdated = detectSkillDrift(wsRoot).filter(d => d.status === "outdated");
|
|
356
|
+
if (outdated.length > 0) {
|
|
357
|
+
warnings.push(
|
|
358
|
+
`workspace skills outdated (${outdated.map(d => d.name).join(", ")}) — run \`zond init\` to refresh .claude/skills/`,
|
|
359
|
+
);
|
|
360
|
+
}
|
|
361
|
+
} catch {
|
|
362
|
+
// not in a workspace — skip
|
|
363
|
+
}
|
|
344
364
|
const placeholderRows = [...requiredOut, ...optionalOut].filter(r => r.placeholder);
|
|
345
365
|
if (placeholderRows.length > 0) {
|
|
346
366
|
warnings.push(
|
|
@@ -348,6 +368,32 @@ export async function doctorCommand(opts: DoctorOptions): Promise<number> {
|
|
|
348
368
|
);
|
|
349
369
|
}
|
|
350
370
|
|
|
371
|
+
// Endpoint counts from .api-catalog.yaml — read what `zond add api` already
|
|
372
|
+
// computed, so `--json` consumers (e.g. /zond-scan PF2 budget estimate)
|
|
373
|
+
// don't have to walk raw spec.json themselves (which would violate the
|
|
374
|
+
// "never read raw OpenAPI" iron rule in the zond skill).
|
|
375
|
+
let endpoints: DoctorReport["endpoints"] = null;
|
|
376
|
+
const catalogPath = join(baseDir, ".api-catalog.yaml");
|
|
377
|
+
if (existsSync(catalogPath)) {
|
|
378
|
+
const catalog = readYamlIfExists<{
|
|
379
|
+
endpointCount?: number;
|
|
380
|
+
endpoints?: Array<{ method?: string }>;
|
|
381
|
+
}>(catalogPath);
|
|
382
|
+
if (catalog) {
|
|
383
|
+
const list = Array.isArray(catalog.endpoints) ? catalog.endpoints : [];
|
|
384
|
+
const byMethod: Record<string, number> = {};
|
|
385
|
+
for (const ep of list) {
|
|
386
|
+
const m = (ep.method ?? "").toUpperCase();
|
|
387
|
+
if (!m) continue;
|
|
388
|
+
byMethod[m] = (byMethod[m] ?? 0) + 1;
|
|
389
|
+
}
|
|
390
|
+
endpoints = {
|
|
391
|
+
total: typeof catalog.endpointCount === "number" ? catalog.endpointCount : list.length,
|
|
392
|
+
by_method: byMethod,
|
|
393
|
+
};
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
|
|
351
397
|
const report: DoctorReport = {
|
|
352
398
|
api: apiName,
|
|
353
399
|
mode: "spec",
|
|
@@ -357,6 +403,7 @@ export async function doctorCommand(opts: DoctorOptions): Promise<number> {
|
|
|
357
403
|
exists: specExists,
|
|
358
404
|
sha: specSha,
|
|
359
405
|
},
|
|
406
|
+
endpoints,
|
|
360
407
|
fixtures: {
|
|
361
408
|
required: requiredOut,
|
|
362
409
|
optional: optionalOut,
|
|
@@ -378,12 +425,18 @@ export async function doctorCommand(opts: DoctorOptions): Promise<number> {
|
|
|
378
425
|
if (opts.query) {
|
|
379
426
|
const resolved = resolveDotPath(presented, opts.query);
|
|
380
427
|
if (resolved === undefined) {
|
|
381
|
-
const message = `--query path '${opts.query}' did not resolve in the doctor report (canonical paths: api, spec, fixtures.required, fixtures.optional, fixtures.extraInEnv, staleArtifacts, warnings)`;
|
|
428
|
+
const message = `--query path '${opts.query}' did not resolve in the doctor report (canonical paths: api, spec, endpoints, fixtures.required, fixtures.optional, fixtures.extraInEnv, staleArtifacts, warnings)`;
|
|
382
429
|
if (opts.json) printJson(jsonError("doctor", [message]));
|
|
383
430
|
else printError(message);
|
|
384
431
|
return 2;
|
|
385
432
|
}
|
|
386
433
|
process.stdout.write(JSON.stringify(resolved, null, 2) + "\n");
|
|
434
|
+
// ARV-274: --json (incl. --query) returns 0 on successful emit.
|
|
435
|
+
// Fixture/staleness state is *data*, not command failure — non-zero is
|
|
436
|
+
// reserved for command failure (missing api, parse error). Text mode
|
|
437
|
+
// still surfaces state via exit code so `if zond doctor; then ...`
|
|
438
|
+
// remains a useful gate.
|
|
439
|
+
if (opts.json) return 0;
|
|
387
440
|
if (blockedRequired > 0) return 1;
|
|
388
441
|
if (staleArtifacts.some(s => !s.fresh) || !specExists || !manifest) return 2;
|
|
389
442
|
return 0;
|
|
@@ -392,10 +445,11 @@ export async function doctorCommand(opts: DoctorOptions): Promise<number> {
|
|
|
392
445
|
// ── Output ──
|
|
393
446
|
if (opts.json) {
|
|
394
447
|
printJson(jsonOk("doctor", presented));
|
|
395
|
-
|
|
396
|
-
printHuman(presented, envVars, { missingOnly: opts.missingOnly === true });
|
|
448
|
+
return 0; // ARV-274: JSON envelope emit success → exit 0
|
|
397
449
|
}
|
|
398
450
|
|
|
451
|
+
printHuman(presented, envVars, { missingOnly: opts.missingOnly === true });
|
|
452
|
+
|
|
399
453
|
if (blockedRequired > 0) return 1;
|
|
400
454
|
if (staleArtifacts.some(s => !s.fresh) || !specExists || !manifest) return 2;
|
|
401
455
|
return 0;
|
|
@@ -100,7 +100,7 @@ async function readEnv(envPath: string): Promise<Record<string, string>> {
|
|
|
100
100
|
return (await loadEnvFile(envPath)) ?? {};
|
|
101
101
|
}
|
|
102
102
|
|
|
103
|
-
async function applyEnvWrites(
|
|
103
|
+
export async function applyEnvWrites(
|
|
104
104
|
envPath: string,
|
|
105
105
|
writes: Record<string, string>,
|
|
106
106
|
): Promise<{ backup: string | null }> {
|
|
@@ -68,10 +68,9 @@ export interface GenerateOptions {
|
|
|
68
68
|
* exits — no files written. Use to debug "why didn't generate emit a
|
|
69
69
|
* CRUD chain for resource X?" on real specs. */
|
|
70
70
|
explain?: boolean;
|
|
71
|
-
/** TASK-219:
|
|
72
|
-
*
|
|
73
|
-
*
|
|
74
|
-
* future fix will gate sha-mismatched user edits behind this flag. */
|
|
71
|
+
/** TASK-219/ARV-361: overwrite hand-edited suite files (those whose
|
|
72
|
+
* auto-gen header was removed). Default false = preserve them; files that
|
|
73
|
+
* still carry the header are regenerated regardless. */
|
|
75
74
|
force?: boolean;
|
|
76
75
|
json?: boolean;
|
|
77
76
|
/** ARV-9 unified filter: path:<regex> / method:<csv> / tag:<csv> /
|
|
@@ -371,6 +370,22 @@ export async function generateCommand(options: GenerateOptions): Promise<number>
|
|
|
371
370
|
const yaml = serializeSuite(suite);
|
|
372
371
|
const fileName = `${suite.fileStem ?? suite.name}.yaml`;
|
|
373
372
|
const filePath = join(options.output, fileName);
|
|
373
|
+
// m-24 (ARV-361): don't silently clobber agent-edited suites. The
|
|
374
|
+
// auto-gen header already promises "drop the header (or rename) to
|
|
375
|
+
// keep changes" — honour it deterministically: a file whose first line
|
|
376
|
+
// is no longer the auto-gen marker is agent-owned, so preserve it
|
|
377
|
+
// unless --force. Files that still carry the header get overwritten,
|
|
378
|
+
// exactly as the header warns. Makes the previously no-op --force real.
|
|
379
|
+
if (!options.force && existsSync(filePath)) {
|
|
380
|
+
// Small yaml — whole-file read is fine; we only need the first line.
|
|
381
|
+
const firstLine = (await Bun.file(filePath).text()).split("\n", 1)[0] ?? "";
|
|
382
|
+
if (!firstLine.startsWith("# Auto-generated by")) {
|
|
383
|
+
warnings.push(
|
|
384
|
+
`Preserved ${fileName} — no auto-gen header, treated as hand-edited. Pass --force to overwrite.`,
|
|
385
|
+
);
|
|
386
|
+
continue;
|
|
387
|
+
}
|
|
388
|
+
}
|
|
374
389
|
const header = autoGenHeader("zond generate", `zond generate --api <name> --output ${options.output}`);
|
|
375
390
|
await Bun.write(filePath, header + yaml);
|
|
376
391
|
createdFiles.push({
|
|
@@ -505,7 +520,9 @@ export async function generateCommand(options: GenerateOptions): Promise<number>
|
|
|
505
520
|
if (options.json) {
|
|
506
521
|
printJson(jsonError("generate", [message]));
|
|
507
522
|
} else {
|
|
508
|
-
|
|
523
|
+
// ARV-203: pass err so ZOND_DEBUG=1 dumps the stack and the user
|
|
524
|
+
// gets a hint about the env var when running without it.
|
|
525
|
+
printError(message, err);
|
|
509
526
|
}
|
|
510
527
|
return 2;
|
|
511
528
|
}
|
|
@@ -518,7 +535,7 @@ import { getApi } from "../util/api-context.ts";
|
|
|
518
535
|
export function registerGenerate(program: Command): void {
|
|
519
536
|
program
|
|
520
537
|
.command("generate [spec]")
|
|
521
|
-
.description("Generate test suites from OpenAPI spec
|
|
538
|
+
.description("Generate test suites from OpenAPI spec. Re-run regenerates files that still carry the auto-gen header; hand-edited suites (header removed) are preserved unless --force (ARV-361). Body fields are filled with `{{$random*}}` helpers (slug/email/url/uuid/…) — see `zond reference random-helpers` or docs/random-helpers.md for the full list (TASK-267).")
|
|
522
539
|
.option("--api <name>", "Use the registered API's spec (apis/<name>/spec.json)")
|
|
523
540
|
.option("--db <path>", "Path to SQLite database file")
|
|
524
541
|
.option("--output <dir>", "Output directory for generated test files (required unless --explain)")
|
|
@@ -534,7 +551,7 @@ export function registerGenerate(program: Command): void {
|
|
|
534
551
|
.option("--uncovered-only", "Skip endpoints already covered by existing tests")
|
|
535
552
|
.option("--include-deprecated", "Generate suites for deprecated endpoints too (filtered out by default)")
|
|
536
553
|
.option("--explain", "Print the CRUD detection table (which resources became chain candidates and why) without writing files (TASK-139)")
|
|
537
|
-
.option("--force, --overwrite", "
|
|
554
|
+
.option("--force, --overwrite", "ARV-361: overwrite even hand-edited suite files (those whose auto-gen header was removed). Default: preserve them and warn — files that still carry the header are always regenerated.")
|
|
538
555
|
.action(async (specPos: string | undefined, opts, cmd: Command) => {
|
|
539
556
|
const resolved = resolveSpecArg(specPos, opts.api, opts.db);
|
|
540
557
|
if ("error" in resolved) { printError(resolved.error); process.exitCode = 2; return; }
|
|
@@ -44,7 +44,10 @@ export interface BootstrapResult {
|
|
|
44
44
|
* (unless `writeAgents` is false) `AGENTS.md`.
|
|
45
45
|
*/
|
|
46
46
|
export function bootstrapWorkspace(opts: BootstrapOptions = {}): BootstrapResult {
|
|
47
|
-
|
|
47
|
+
// Honor ZOND_WORKSPACE so `zond init` writes the workspace at the same root
|
|
48
|
+
// that findWorkspaceRoot() reads from (headless/CI runs anchor both here).
|
|
49
|
+
// Explicit opts.cwd (tests) still wins.
|
|
50
|
+
const cwd = resolve(opts.cwd ?? process.env.ZOND_WORKSPACE?.trim() ?? process.cwd());
|
|
48
51
|
const warnings: string[] = [];
|
|
49
52
|
const writeAgents = opts.writeAgents ?? true;
|
|
50
53
|
const writeSkills = opts.writeSkills ?? true;
|
|
@@ -163,7 +163,7 @@ function printBootstrapResult(b: BootstrapResult, writeAgents: boolean): void {
|
|
|
163
163
|
`\nNext steps:\n` +
|
|
164
164
|
` 1. zond add api <name> --spec <path|url> # register API → builds .api-fixtures.yaml (manifest)\n` +
|
|
165
165
|
` 2. zond doctor --api <name> # gap report: which vars are UNSET in .env.yaml\n` +
|
|
166
|
-
` 3. zond prepare-fixtures --api <name> --apply
|
|
166
|
+
` 3. zond prepare-fixtures --api <name> --apply # fill .env.yaml values (single-pass); fill any gaps by hand\n` +
|
|
167
167
|
`\nNote: zond init only refreshes workspace files (skills, AGENTS.md, zond.config.yml).\n` +
|
|
168
168
|
` It does NOT touch fixtures or .env.yaml — that's the doctor/prepare-fixtures loop above.\n`
|
|
169
169
|
);
|
|
@@ -173,7 +173,7 @@ function printBootstrapResult(b: BootstrapResult, writeAgents: boolean): void {
|
|
|
173
173
|
`\nFixtures untouched. zond init only refreshes skills/AGENTS.md/zond.config.yml.\n` +
|
|
174
174
|
`Verify env state with:\n` +
|
|
175
175
|
` zond doctor --api ${sample} --missing-only # show UNSET vars + blocked endpoints\n` +
|
|
176
|
-
` zond prepare-fixtures --api ${sample} --apply
|
|
176
|
+
` zond prepare-fixtures --api ${sample} --apply # discover values (single-pass); fill any gaps by hand\n`
|
|
177
177
|
);
|
|
178
178
|
}
|
|
179
179
|
}
|
|
@@ -4,6 +4,8 @@ import { dirname, join } from "node:path";
|
|
|
4
4
|
import zondSkill from "./templates/skills/zond.md" with { type: "text" };
|
|
5
5
|
import checksSkill from "./templates/skills/zond-checks.md" with { type: "text" };
|
|
6
6
|
import triageSkill from "./templates/skills/zond-triage.md" with { type: "text" };
|
|
7
|
+
import seedSkill from "./templates/skills/zond-seed.md" with { type: "text" };
|
|
8
|
+
import warmUpSkill from "./templates/skills/warm-up-target.md" with { type: "text" };
|
|
7
9
|
|
|
8
10
|
export interface SkillResult {
|
|
9
11
|
name: string;
|
|
@@ -27,6 +29,13 @@ const SKILLS: SkillTemplate[] = [
|
|
|
27
29
|
{ name: "zond-checks", body: checksSkill },
|
|
28
30
|
// Read-only triage of a finished run / probe artifact.
|
|
29
31
|
{ name: "zond-triage", body: triageSkill },
|
|
32
|
+
// ARV-355: agent-orchestrated auto-seed loop (read gaps → order by
|
|
33
|
+
// fkDependencies → author body → request POST --capture → fix 4xx + retry).
|
|
34
|
+
{ name: "zond-seed", body: seedSkill },
|
|
35
|
+
// ARV-366: warm up external-input fixtures (issue_id/file_id/integration_id)
|
|
36
|
+
// via the target's own SDK/CLI/replay — the honest-2xx ceiling zond-seed
|
|
37
|
+
// can't reach with a plain POST. Agent warms, zond stores + measures.
|
|
38
|
+
{ name: "warm-up-target", body: warmUpSkill },
|
|
30
39
|
];
|
|
31
40
|
|
|
32
41
|
/**
|
|
@@ -47,6 +56,40 @@ export interface StaleSkill {
|
|
|
47
56
|
path: string;
|
|
48
57
|
}
|
|
49
58
|
|
|
59
|
+
/**
|
|
60
|
+
* Per-skill drift status vs the in-binary template. `missing` is
|
|
61
|
+
* non-actionable on its own (user may have intentionally passed
|
|
62
|
+
* `--no-skills`); `outdated` is the actionable one — workspace skill
|
|
63
|
+
* predates a binary upgrade and `zond init` will refresh it.
|
|
64
|
+
*/
|
|
65
|
+
export type SkillDriftStatus = "fresh" | "outdated" | "missing";
|
|
66
|
+
|
|
67
|
+
export interface SkillDrift {
|
|
68
|
+
name: string;
|
|
69
|
+
path: string;
|
|
70
|
+
status: SkillDriftStatus;
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
/**
|
|
74
|
+
* Compare each in-binary skill template against the workspace copy at
|
|
75
|
+
* `<cwd>/.claude/skills/<name>/SKILL.md`. Returns one entry per skill —
|
|
76
|
+
* callers filter by status (`outdated` → warn the user to re-run
|
|
77
|
+
* `zond init`).
|
|
78
|
+
*/
|
|
79
|
+
export function detectSkillDrift(cwd: string): SkillDrift[] {
|
|
80
|
+
return SKILLS.map(({ name, body }) => {
|
|
81
|
+
const path = join(cwd, ".claude", "skills", name, "SKILL.md");
|
|
82
|
+
const desired = body.endsWith("\n") ? body : body + "\n";
|
|
83
|
+
if (!existsSync(path)) return { name, path, status: "missing" as const };
|
|
84
|
+
const current = readFileSync(path, "utf-8");
|
|
85
|
+
return {
|
|
86
|
+
name,
|
|
87
|
+
path,
|
|
88
|
+
status: (current === desired ? "fresh" : "outdated") as SkillDriftStatus,
|
|
89
|
+
};
|
|
90
|
+
});
|
|
91
|
+
}
|
|
92
|
+
|
|
50
93
|
/**
|
|
51
94
|
* Returns directories under `<cwd>/.claude/skills/` whose name is in
|
|
52
95
|
* `LEGACY_SKILL_NAMES`. User-authored skill directories (any other
|
|
@@ -16,8 +16,14 @@ only, no MCP server in this workspace.
|
|
|
16
16
|
lifecycle_transitions) and the `zond api annotate dump+apply` flow.
|
|
17
17
|
- **`.claude/skills/zond-triage/SKILL.md`** — read-only triage of a
|
|
18
18
|
finished run / probe artifact. Routes by `recommended_action` enum.
|
|
19
|
+
- **`.claude/skills/zond-seed/SKILL.md`** — agent-orchestrated seed loop:
|
|
20
|
+
read fixture gaps → order by FK deps → author body → `request POST
|
|
21
|
+
--capture` → fix 4xx + retry. Fills what `prepare-fixtures` reports missing.
|
|
22
|
+
- **`.claude/skills/warm-up-target/SKILL.md`** — warm external-input
|
|
23
|
+
fixtures (issue_id/file_id/integration_id) via the target's own
|
|
24
|
+
SDK/CLI/replay — the honest-2xx ceiling a plain POST can't reach.
|
|
19
25
|
|
|
20
|
-
|
|
26
|
+
These skills work off the per-API artifacts written by `zond add api`:
|
|
21
27
|
|
|
22
28
|
```
|
|
23
29
|
apis/<name>/
|
|
@@ -41,7 +47,8 @@ an env fix.
|
|
|
41
47
|
zond init # bootstrap workspace (no fixture changes)
|
|
42
48
|
zond add api <name> --spec <path-or-url> # register API + emit manifest + seed empty .env.yaml
|
|
43
49
|
zond doctor --api <name> --missing-only # gap report: which vars are UNSET
|
|
44
|
-
zond prepare-fixtures --api <name>
|
|
50
|
+
zond prepare-fixtures --api <name> # gap report: verify + which FK vars need a value
|
|
51
|
+
# → fill each gap yourself: `zond fixtures add` / edit .env.yaml (you pick the value)
|
|
45
52
|
zond doctor --api <name> # re-check (exit 0 = ready)
|
|
46
53
|
```
|
|
47
54
|
|
|
@@ -52,7 +59,9 @@ What each step does to `.env.yaml`:
|
|
|
52
59
|
| `zond init` | no — only writes workspace/skills files |
|
|
53
60
|
| `zond add api` | seeds skeleton with empty placeholders for every required var |
|
|
54
61
|
| `zond doctor` | no — read-only diagnostic |
|
|
55
|
-
| `zond prepare-fixtures
|
|
62
|
+
| `zond prepare-fixtures` | no — reports gaps only; **never harvests a value** (which record/field fills a slot is your call) |
|
|
63
|
+
| `zond prepare-fixtures --refresh` | unsets stale (404) ids so they resurface as gaps (`.bak` backup); does not re-resolve |
|
|
64
|
+
| `zond fixtures add` / `import` | writes the value you supply (`.bak` backup) |
|
|
56
65
|
| `zond refresh-api` | no — only re-snapshots `spec.json` and rebuilds the manifest |
|
|
57
66
|
|
|
58
67
|
`zond refresh-api <name> [--spec <new-source>]` re-snapshots when the upstream
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: warm-up-target
|
|
3
|
+
description: |
|
|
4
|
+
Warm up the TARGET API's environment so honest-2xx coverage stops being
|
|
5
|
+
capped by empty state. Picks up where `zond-seed` stops: the fixtures the
|
|
6
|
+
API cannot self-create via a plain POST — a real error `issue_id` (needs an
|
|
7
|
+
actual event), a `file_id` (needs a sourcemap/artifact upload), an
|
|
8
|
+
`integration_id` (needs an OAuth/app install), a `webhook_event_id` (needs a
|
|
9
|
+
replay). YOU decide HOW to warm each one using the target's OWN tooling (its
|
|
10
|
+
SDK, CLI, dashboard, replay/trigger endpoints); then harvest the resulting
|
|
11
|
+
LIVE id into `.env.yaml`. Use when honest-2xx is stuck low (~30%), when
|
|
12
|
+
`zond-seed` / `prepare-fixtures` report roots as un-seedable / external-input,
|
|
13
|
+
or the user asks to "warm up the target", "raise honest-2xx", "seed via the
|
|
14
|
+
SDK", "generate real test data". zond only STORES the id and MEASURES the
|
|
15
|
+
lift — the warm-up judgment is yours. Hand back to `zond` once warm.
|
|
16
|
+
allowed-tools: [Read, Write, Edit, Bash]
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# warm-up-target — get the API into a testable state
|
|
20
|
+
|
|
21
|
+
`prepare-fixtures` reports fixture gaps; `zond-seed` fills the ones the API
|
|
22
|
+
self-serves (plain `POST` + capture). Whatever is left — resources that exist
|
|
23
|
+
only after a **real-world event** — is the honest-2xx ceiling. This skill
|
|
24
|
+
breaks that ceiling by warming the target with **its own means**, then storing
|
|
25
|
+
the live ids zond will reuse.
|
|
26
|
+
|
|
27
|
+
This is **not** core zond and **not** a heuristic: zond never learns how to
|
|
28
|
+
warm an API (that judgment has an infinite tail — every provider is different).
|
|
29
|
+
YOU read the provider's docs/SDK and drive it; zond only records the id
|
|
30
|
+
(`fixtures add`) and measures the coverage delta (`coverage`). Keep it that way
|
|
31
|
+
— no warm-up logic goes into zond core (litmus test).
|
|
32
|
+
|
|
33
|
+
## Iron rules
|
|
34
|
+
|
|
35
|
+
- **You warm, zond stores + measures.** You run the SDK/CLI/dashboard action
|
|
36
|
+
that creates the real resource; you read the returned id; `zond fixtures add`
|
|
37
|
+
writes it. Never ask zond to "figure out" how to warm a resource.
|
|
38
|
+
- **Live + throwaway/sandbox + cleanup.** Warm-ups create real state (events,
|
|
39
|
+
files, installs). Confirm `base_url` is a sandbox/test account first. Track
|
|
40
|
+
what you create; tear it down after (or use a disposable org). If you can't
|
|
41
|
+
guarantee cleanup, ask before creating.
|
|
42
|
+
- **Hand off to `zond-seed` first.** Only warm what `zond-seed` marked
|
|
43
|
+
un-seedable. Don't reinvent plain-POST creation here — that's zond-seed's job.
|
|
44
|
+
- **Ask early, don't reverse-engineer.** Give yourself a hard budget of ~1-2
|
|
45
|
+
attempts to derive a create-recipe from the spec/API. If a blind create 4xx/5xx
|
|
46
|
+
(missing an integer id the API doesn't expose, an opaque body shape), STOP —
|
|
47
|
+
ask the user for an example request (a working `curl`/HAR/body from their UI).
|
|
48
|
+
Extract the ids from it deterministically and replay. Do NOT spelunk 10 endpoints
|
|
49
|
+
guessing how creation works — that's the slow path the user hates.
|
|
50
|
+
- **Delete only by self-captured id.** Tear-down must target the id YOUR create
|
|
51
|
+
returned, never a harvested read-fixture value. Deleting by `{{key}}` when that
|
|
52
|
+
key came from a live resource wipes pre-existing data (see ARV-368). Track
|
|
53
|
+
created ids explicitly; DELETE those and only those.
|
|
54
|
+
- **Report honestly what you can't warm.** Some fixtures need a human (a paid
|
|
55
|
+
plan, a KYC step, a physical device, a manual dashboard toggle). Surface those
|
|
56
|
+
as a short "needs you" list with the exact action — never fake a value (it
|
|
57
|
+
just 422s and lies about coverage).
|
|
58
|
+
- **Measure the lift.** Snapshot honest-2xx before and after so the warm-up's
|
|
59
|
+
value is visible (`zond coverage --api <name> --union session`).
|
|
60
|
+
|
|
61
|
+
## Inputs — what needs warming
|
|
62
|
+
|
|
63
|
+
| Signal | Command | What it tells you |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| Un-seedable roots | `zond-seed` handoff / `zond prepare-fixtures --api <name> --json` → `summary.fixtureGaps.unseededRoots[]` | fixtures no plain-POST can create |
|
|
66
|
+
| Empty-list gaps | `prepare-fixtures` items with `status: miss-empty` | resource exists in spec, zero records in the target |
|
|
67
|
+
| What each id is for | `apis/<name>/.api-resources.yaml` (the owning resource + its create/read endpoints) | which real-world action mints this id |
|
|
68
|
+
| Current honest-2xx | `zond coverage --api <name> --union session --json` | the ceiling you're trying to raise |
|
|
69
|
+
|
|
70
|
+
## Warm-up patterns (pick per resource)
|
|
71
|
+
|
|
72
|
+
The resource's *nature* tells you the warm-up path. Common families:
|
|
73
|
+
|
|
74
|
+
| Fixture shape | Warm-up path (via the target's own tooling) |
|
|
75
|
+
|---|---|
|
|
76
|
+
| Error / issue id (`issue_id`, `event_id`) | Trigger a real event: `sentry-cli send-event`, the app's error-report endpoint, or an SDK `captureException`. Poll the list endpoint until the id appears, capture it. |
|
|
77
|
+
| Uploaded-artifact id (`file_id`, `release`, `sourcemap`) | Upload via the provider CLI/SDK (`sentry-cli releases files upload`, a `POST .../files` multipart). Capture the returned id. |
|
|
78
|
+
| Integration / connection id (`integration_id`, `installation_id`) | Install the app/integration into a **sandbox** org (OAuth flow / provider dashboard "add to test workspace"). Capture the id from the callback or the list endpoint. |
|
|
79
|
+
| Delivered-webhook id (`delivery_id`, `webhook_event_id`) | Register a webhook, trigger the source event, replay/list deliveries, capture the id. |
|
|
80
|
+
| Provisioned-async id (needs a background job to finish) | Kick off the job, poll the status endpoint until ready, capture. |
|
|
81
|
+
|
|
82
|
+
These are *examples*, not a lookup table — read the actual provider's docs for
|
|
83
|
+
the target you're on.
|
|
84
|
+
|
|
85
|
+
## The loop
|
|
86
|
+
|
|
87
|
+
For each un-seedable root:
|
|
88
|
+
|
|
89
|
+
1. **Identify the warm-up path.** From `.api-resources.yaml` + the provider
|
|
90
|
+
docs, decide which real-world action mints this id. If it needs a human,
|
|
91
|
+
add it to the "needs you" list and move on.
|
|
92
|
+
|
|
93
|
+
2. **Warm it via the target's own tooling.** Run the SDK/CLI/curl that creates
|
|
94
|
+
the real resource in the sandbox. Read the id from the response (or poll the
|
|
95
|
+
list endpoint until it shows up — async resources aren't instant).
|
|
96
|
+
|
|
97
|
+
3. **Store the live id.**
|
|
98
|
+
```bash
|
|
99
|
+
zond fixtures add --api <name> issue_id=<real-id-you-got> --validate --apply
|
|
100
|
+
```
|
|
101
|
+
`--validate` GETs the read-by-id endpoint and confirms the id is `live`
|
|
102
|
+
before writing (so a warm-up that half-failed doesn't poison later runs).
|
|
103
|
+
|
|
104
|
+
4. **Verify the gap closed.** Re-run `zond prepare-fixtures --api <name> --json`
|
|
105
|
+
— the warmed root should drop out of `unseededRoots` / flip off `miss-empty`.
|
|
106
|
+
|
|
107
|
+
5. **Measure + hand back.** When the worklist is drained (or only "needs you"
|
|
108
|
+
items remain), snapshot honest-2xx again and report the delta, then hand
|
|
109
|
+
back to `zond` for the audit:
|
|
110
|
+
```bash
|
|
111
|
+
zond coverage --api <name> --union session # after — compare to the before snapshot
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Output to the user
|
|
115
|
+
|
|
116
|
+
- **Warmed:** table of `fixture → how it was warmed → live id captured`.
|
|
117
|
+
- **Needs you:** fixtures that require a human action, each with the exact step
|
|
118
|
+
(e.g. "enable the Slack integration in the sandbox workspace, then re-run").
|
|
119
|
+
- **Coverage:** honest-2xx before → after, so the lift is measurable.
|
|
120
|
+
|
|
121
|
+
Cleanup: DELETE the resources you created (or note they live in a disposable
|
|
122
|
+
sandbox). Never leave test events/files/installs on a shared account.
|