@reqvire-org/ads 0.3.12 → 0.3.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -1
- package/dist/capture/worker.mjs +38 -0
- package/dist/capture-store.mjs +123 -0
- package/dist/capture.mjs +117 -0
- package/dist/config.mjs +6 -1
- package/dist/mcp/context.mjs +1 -1
- package/dist/mcp/mutations.mjs +25 -5
- package/dist/mcp/server.mjs +62 -15
- package/dist/scenarios.mjs +9 -3
- package/dist/showcase/client.bundle.js +18 -18
- package/dist/showcase/style.css +2 -0
- package/dist/skills/ads-break-ui/SKILL.md +6 -1
- package/dist/skills/ads-break-ui/references/captures.md +147 -0
- package/dist/skills/ads-break-ui/references/guarded-editing.md +58 -0
- package/dist/skills/ads-break-ui/references/scenarios.md +5 -0
- package/dist/skills/ads-designer/SKILL.md +6 -1
- package/dist/skills/ads-designer/references/captures.md +147 -0
- package/dist/skills/ads-designer/references/guarded-editing.md +58 -0
- package/dist/skills/ads-designer/references/scenarios.md +5 -0
- package/dist/skills/ads-import/SKILL.md +6 -1
- package/dist/skills/ads-import/references/captures.md +147 -0
- package/dist/skills/ads-import/references/guarded-editing.md +58 -0
- package/dist/skills/ads-import/references/scenarios.md +5 -0
- package/dist/skills/ads-migrate/SKILL.md +6 -1
- package/dist/skills/ads-migrate/references/captures.md +147 -0
- package/dist/skills/ads-migrate/references/guarded-editing.md +58 -0
- package/dist/skills/ads-migrate/references/scenarios.md +5 -0
- package/dist/skills/ads-motion/SKILL.md +6 -1
- package/dist/skills/ads-motion/references/captures.md +147 -0
- package/dist/skills/ads-motion/references/guarded-editing.md +58 -0
- package/dist/skills/ads-motion/references/scenarios.md +5 -0
- package/dist/verification/browser.mjs +8 -3
- package/dist/verification/playwright-reporter.cjs +23 -8
- package/dist/verification.mjs +137 -33
- package/docs/ADS.md +5 -0
- package/docs/ARCHITECTURE.md +39 -2
- package/docs/BEHAVIORS.md +57 -3
- package/docs/CHANGELOG.md +17 -0
- package/docs/GUIDE.md +48 -1
- package/docs/MCP.md +49 -1
- package/package.json +1 -1
|
@@ -6,6 +6,64 @@ tools, request `action:"diff"`, then apply the returned patchFile. Discard the
|
|
|
6
6
|
draft afterward. Remote clients can keep sending inline Git diffs. Neither draft
|
|
7
7
|
preparation nor diff generation validates or changes live source.
|
|
8
8
|
|
|
9
|
+
To verify a draft before applying it, diff it and dry-run the returned patch:
|
|
10
|
+
|
|
11
|
+
```js
|
|
12
|
+
ads_prepare_patch({"unit":"components/Example","action":"diff","draft":"<returned draft ID>"})
|
|
13
|
+
ads_apply_patch({"unit":"components/Example","patchFile":"<returned patchFile>","dryRun":true})
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
The dry run validates the full candidate, including affected registered behavior
|
|
17
|
+
checks in an isolated project copy. Configured browser checks use that candidate's
|
|
18
|
+
managed preview, not the live showcase. Inspect the returned `behavior` failures
|
|
19
|
+
or the registered behavior-check results in `checks`, fix the draft and diff again.
|
|
20
|
+
Keep behavior.json/specs with their owner and use registered checks; a scratchpad
|
|
21
|
+
run against live source cannot verify unapplied changes. After validation, apply
|
|
22
|
+
with `dryRun:false`; ADS revalidates before writing. Dry-run evidence is not saved
|
|
23
|
+
as passing evidence for the live project.
|
|
24
|
+
|
|
25
|
+
`ads_verify({unit,checks?})` selects checks for current project source; it has no
|
|
26
|
+
draft argument. There is no dedicated selected-check runner for drafts yet.
|
|
27
|
+
Use a dry-run transaction for genuinely coupled draft patches.
|
|
28
|
+
|
|
29
|
+
Behavior-run performance belongs to both orchestration and project checks. ADS
|
|
30
|
+
selects affected owners, filters registered Playwright case tags and shares one
|
|
31
|
+
candidate host per run. The project owns assertions, browser setup, worker count,
|
|
32
|
+
per-test timeouts and its profile's whole-run `timeoutMs`. The latter covers the
|
|
33
|
+
entire runner process, not each test. Do not rewrite APIs, drop guarantees or
|
|
34
|
+
disable checks to fit the budget. Inspect measured durations before tuning it.
|
|
35
|
+
|
|
36
|
+
During verification, inspect `ads_status.behaviors.progress`; during candidate
|
|
37
|
+
validation, inspect `ads_status.mutations.progress`. These show completed/active/
|
|
38
|
+
pending checks and timings. MCP clients requesting progress also receive updates.
|
|
39
|
+
A timeout retains reported case outcomes and identifies active/unstarted checks;
|
|
40
|
+
it still blocks the mutation. `reportedStatus:"passed"` within an incomplete run
|
|
41
|
+
is diagnostic information, not a passing guarantee. Reports/artifacts under the
|
|
42
|
+
live project's `.ds/verification/` survive candidate cleanup. Uninstrumented custom
|
|
43
|
+
runners may report an unknown execution state instead of claiming a case started.
|
|
44
|
+
|
|
45
|
+
For independent browser specs, consider a small parallel worker count and measure
|
|
46
|
+
on the target machine. Use separate contexts/pages, independent state and the
|
|
47
|
+
candidate URLs provided by `openBehaviorScenario`; avoid fixed live-server URLs,
|
|
48
|
+
shared mutable fixtures and test-order assumptions. Files can run in parallel;
|
|
49
|
+
tests in one file remain sequential by default. Prefer readiness assertions over
|
|
50
|
+
blanket sleeps. Fake clocks suit timer-only guarantees; keep real time for actual
|
|
51
|
+
gesture/motion/layout guarantees. Project settings belong in guarded configuration
|
|
52
|
+
changes; ADS does not silently raise budgets or override the worker count.
|
|
53
|
+
|
|
54
|
+
Named barrel imports can narrow affected checks when the project accurately
|
|
55
|
+
declares package `sideEffects`; retain CSS/global effects in that policy. Missing
|
|
56
|
+
or unsupported policies and dynamic/wildcard dependencies stay conservative.
|
|
57
|
+
Do not change the policy merely to suppress verification of real side effects.
|
|
58
|
+
|
|
59
|
+
For registered measured-geometry style writes, use direct `element.style.transform =
|
|
60
|
+
value` or `element.style.setProperty("--ds-measured-value", value)` with the exact
|
|
61
|
+
file/property boundary. The current guard rejects a style-object alias such as
|
|
62
|
+
`const style = element.style` as an escape; it does not trace the alias to later
|
|
63
|
+
writes. Keep writes explicit when adapting local controller code. This syntax
|
|
64
|
+
constraint is separate from deciding which measured properties belong to the
|
|
65
|
+
controller; it does not authorize unregistered visual writes.
|
|
66
|
+
|
|
9
67
|
Shared provenance uses the existing `documentation` unit. Read its `filePolicy`
|
|
10
68
|
from `ads_units`: one patch can create `CREDITS.md`, shared `licenses/*.txt` and
|
|
11
69
|
README links together. Conventional root credits/license/notice names support
|
|
@@ -18,6 +18,11 @@ recovers missed atomic replacements. Do not touch source to force freshness;
|
|
|
18
18
|
inspect build diagnostics. Upgrade the project's compiler dependency together
|
|
19
19
|
with MCP when an older reporter lacks source evidence.
|
|
20
20
|
|
|
21
|
+
Optional static `captureTargets` names CSS selectors within the scenario's App for
|
|
22
|
+
on-demand element screenshots. It grants no render/behavior coverage and no extra
|
|
23
|
+
styling permission. Use [captures.md](captures.md) for current/candidate review,
|
|
24
|
+
target selection, resource reads and storage lifecycle.
|
|
25
|
+
|
|
21
26
|
## Owner-local token previews load their own definitions
|
|
22
27
|
|
|
23
28
|
Keep the registered token sheet imported by its runtime owner for product use.
|
|
@@ -11,6 +11,11 @@ export async function openBehaviorScenario(page, checkId, { timeout = 30000 } =
|
|
|
11
11
|
const check = request.checks.find(check => check.id === checkId);
|
|
12
12
|
const url = check && request.scenarios[check.scenario];
|
|
13
13
|
if (!url) throw new Error("This check has no managed browser scenario. StrictMode profiles must supply a project development harness; production preview bundles do not replay React development effects.");
|
|
14
|
+
return openPreviewScenario(page, url, check.scenario, { timeout });
|
|
15
|
+
}
|
|
16
|
+
|
|
17
|
+
/** Shared readiness for registered verification and on-demand captures. */
|
|
18
|
+
export async function openPreviewScenario(page, url, scenario, { timeout = 30000 } = {}) {
|
|
14
19
|
const failures = [];
|
|
15
20
|
const failed = request => {
|
|
16
21
|
if (failures.length < 5) failures.push(`${request.method()} ${request.url().slice(0, 300)}: ${request.failure()?.errorText ?? "request failed"}`);
|
|
@@ -22,8 +27,8 @@ export async function openBehaviorScenario(page, checkId, { timeout = 30000 } =
|
|
|
22
27
|
while (Date.now() < end) {
|
|
23
28
|
for (const frame of page.frames().filter(frame => frame !== page.mainFrame())) {
|
|
24
29
|
const state = await frame.evaluate(() => ({ state: document.documentElement.dataset.adsPreviewState, id: document.documentElement.dataset.adsScenario, message: document.getElementById("ads-preview-status")?.textContent })).catch(() => ({}));
|
|
25
|
-
if (state.id ===
|
|
26
|
-
if (state.id ===
|
|
30
|
+
if (state.id === scenario && state.state === "ready") return frame;
|
|
31
|
+
if (state.id === scenario && state.state === "error") throw new Error(state.message || "The MCP App failed before verification.");
|
|
27
32
|
}
|
|
28
33
|
const error = await page.evaluate(() => document.querySelector('#status[data-error="true"]')?.textContent).catch(() => null);
|
|
29
34
|
if (error) throw new Error(error);
|
|
@@ -31,6 +36,6 @@ export async function openBehaviorScenario(page, checkId, { timeout = 30000 } =
|
|
|
31
36
|
}
|
|
32
37
|
throw new Error("Timed out waiting for the selected MCP App scenario to become current and ready. Inspect ads_status and ads://project/build.");
|
|
33
38
|
} catch (error) {
|
|
34
|
-
throw new Error(`Scenario ${
|
|
39
|
+
throw new Error(`Scenario ${scenario}: ${error.message}${failures.length ? `\nFailed browser requests:\n${failures.join("\n")}` : ""}`, { cause: error });
|
|
35
40
|
} finally { page.off("requestfailed", failed); }
|
|
36
41
|
}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
const { readFileSync, writeFileSync } = require("node:fs");
|
|
1
|
+
const { readFileSync, writeFileSync, renameSync, rmSync } = require("node:fs");
|
|
2
2
|
const path = require("node:path");
|
|
3
3
|
|
|
4
4
|
/** Case identity is an explicit project test tag: @ads:<case-id>. */
|
|
@@ -7,18 +7,33 @@ module.exports = class AdsBehaviorReporter {
|
|
|
7
7
|
this.request = JSON.parse(readFileSync(process.env.ADS_VERIFY_REQUEST, "utf8"));
|
|
8
8
|
this.cases = [];
|
|
9
9
|
this.errors = [];
|
|
10
|
+
this.active = new Map();
|
|
11
|
+
this.startedAt = Date.now();
|
|
12
|
+
this.totalTests = 0;
|
|
10
13
|
}
|
|
11
|
-
|
|
14
|
+
onBegin(config, suite) { this.workers = config.workers; this.totalTests = suite.allTests().length; this.publish(); }
|
|
15
|
+
entries(test) {
|
|
12
16
|
const file = path.relative(this.request.projectRoot, test.location.file).replaceAll("\\", "/");
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
}
|
|
17
|
+
return (test.tags ?? []).filter(tag => tag.startsWith("@ads:")).map(tag => ({ id: tag.slice(5), file }));
|
|
18
|
+
}
|
|
19
|
+
onTestBegin(test, result) {
|
|
20
|
+
this.active.set(test.id, this.entries(test).map(entry => ({ ...entry, startedAt: (result.startTime ?? new Date()).toISOString(), workerIndex: result.workerIndex, retry: result.retry })));
|
|
21
|
+
this.publish();
|
|
22
|
+
}
|
|
23
|
+
onTestEnd(test, result) {
|
|
24
|
+
this.active.delete(test.id);
|
|
25
|
+
for (const entry of this.entries(test)) this.cases.push({ ...entry, status: result.status === "passed" && test.expectedStatus === "passed" ? "passed" : ["skipped", "interrupted"].includes(result.status) ? "skipped" : "failed", durationMs: result.duration, workerIndex: result.workerIndex, retry: result.retry, completedAt: new Date().toISOString(), message: result.errors?.map(error => error.message || error.stack || error.value || "The browser runner reported an error without details.").join("\n").slice(0, 1000) });
|
|
26
|
+
this.publish();
|
|
17
27
|
}
|
|
18
28
|
onError(error) {
|
|
19
29
|
if (this.errors.length < 5) this.errors.push(String(error.message || error.stack || error.value || "The browser runner reported a suite error without details.").slice(0, 4000));
|
|
30
|
+
this.publish();
|
|
20
31
|
}
|
|
21
|
-
|
|
22
|
-
|
|
32
|
+
publish(complete = false) {
|
|
33
|
+
const file = process.env.ADS_VERIFY_REPORT, temporary = `${file}.${process.pid}.tmp`;
|
|
34
|
+
const report = { schemaVersion: 1, runId: this.request.runId, complete, startedAt: new Date(this.startedAt).toISOString(), elapsedMs: Date.now() - this.startedAt, workers: this.workers, totalTests: this.totalTests, cases: this.cases.slice(0, 500), active: [...this.active.values()].flat().slice(0, 100), errors: this.errors };
|
|
35
|
+
try { writeFileSync(temporary, JSON.stringify(report)); renameSync(temporary, file); }
|
|
36
|
+
finally { rmSync(temporary, { force: true }); }
|
|
23
37
|
}
|
|
38
|
+
onEnd() { this.publish(true); }
|
|
24
39
|
};
|
package/dist/verification.mjs
CHANGED
|
@@ -19,13 +19,55 @@ const slash = file => file.replaceAll("\\", "/");
|
|
|
19
19
|
const resultFile = root => path.join(root, ".ds/behavior-results.json");
|
|
20
20
|
|
|
21
21
|
/** Follow real project imports; package/configuration changes also invalidate evidence. */
|
|
22
|
-
function inputContext(options) {
|
|
22
|
+
function inputContext(options) {
|
|
23
|
+
const sources = new Map(), digests = new Map(), packages = new Map(), resolve = importResolver(options), resolutions = new Map();
|
|
24
|
+
return { inventory: inspectUnits(options), scenarios: inspectScenarios(options).scenarios,
|
|
25
|
+
seeds: [...configurationFiles(options)].map(file => path.join(options.root, file)).concat(sourceFiles(path.join(options.dsRoot, options.config.tokenDir)), path.join(options.dsRoot, options.config.stylesEntry)),
|
|
26
|
+
digest(file) { if (!digests.has(file)) digests.set(file, hash(readFileSync(file))); return digests.get(file); },
|
|
27
|
+
ast(file) { if (!sources.has(file)) sources.set(file, ts.createSourceFile(file, readFileSync(file, "utf8"), ts.ScriptTarget.Latest, true)); return sources.get(file); },
|
|
28
|
+
resolve(specifier, file) { const key = `${file}\0${specifier}`; if (!resolutions.has(key)) resolutions.set(key, resolve(specifier, file)); return resolutions.get(key); },
|
|
29
|
+
hasSideEffects(file, record) {
|
|
30
|
+
let directory = path.dirname(file);
|
|
31
|
+
while (isInside(options.root, directory)) {
|
|
32
|
+
const manifest = path.join(directory, "package.json");
|
|
33
|
+
if (existsSync(manifest)) {
|
|
34
|
+
record(manifest);
|
|
35
|
+
if (!packages.has(manifest)) { try { packages.set(manifest, JSON.parse(readFileSync(manifest, "utf8")).sideEffects); } catch { packages.set(manifest, true); } }
|
|
36
|
+
const policy = packages.get(manifest), relative = slash(path.relative(directory, file));
|
|
37
|
+
if (policy === false) return false;
|
|
38
|
+
// Only the simple package sideEffects glob subset is narrowed. Unknown
|
|
39
|
+
// syntax and undeclared policies preserve the entire dependency graph.
|
|
40
|
+
if (!Array.isArray(policy) || policy.some(pattern => typeof pattern !== "string" || !/^[\w./*?-]+$/.test(pattern))) return true;
|
|
41
|
+
return policy.some(pattern => {
|
|
42
|
+
pattern = pattern.replace(/^\.\//, ""); if (!pattern.includes("/")) pattern = `**/${pattern}`;
|
|
43
|
+
const expression = pattern.match(/\*\*\/|\*\*|\*|\?|[^*?]+/g)?.map(part => part === "**/" ? "(?:.*/)?" : part === "**" ? ".*" : part === "*" ? "[^/]*" : part === "?" ? "[^/]" : part.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")).join("");
|
|
44
|
+
return new RegExp(`^${expression}$`).test(relative);
|
|
45
|
+
});
|
|
46
|
+
}
|
|
47
|
+
directory = path.dirname(directory);
|
|
48
|
+
}
|
|
49
|
+
return true;
|
|
50
|
+
},
|
|
51
|
+
};
|
|
52
|
+
}
|
|
23
53
|
|
|
24
54
|
export function behaviorInputs(options, contract, check, context = inputContext(options)) {
|
|
25
|
-
const files = new Set(),
|
|
26
|
-
|
|
55
|
+
const files = new Set(), demands = new Map(), effectsVisited = new Set();
|
|
56
|
+
const resolve = context.resolve;
|
|
57
|
+
function effects(file) {
|
|
58
|
+
if (effectsVisited.has(file) || !isInside(options.root, file) || file.includes(`${path.sep}node_modules${path.sep}`) || !existsSync(file)) return;
|
|
59
|
+
effectsVisited.add(file);
|
|
60
|
+
if (context.hasSideEffects(file, visit)) { visit(file); return; }
|
|
61
|
+
if (!/\.[cm]?[jt]sx?$/.test(file) || !lstatSync(file).isFile()) return;
|
|
62
|
+
if (lstatSync(file).isSymbolicLink() || realpathSync(file) !== file) throw new Error(`Behavior inputs cannot resolve through symlinks: ${file}`);
|
|
63
|
+
for (const statement of context.ast(file).statements) if ((ts.isImportDeclaration(statement) || ts.isExportDeclaration(statement)) && statement.moduleSpecifier && ts.isStringLiteralLike(statement.moduleSpecifier)) for (const target of resolve(statement.moduleSpecifier.text, file)) effects(target);
|
|
64
|
+
}
|
|
65
|
+
function visit(file, names = null) {
|
|
27
66
|
file = path.resolve(file);
|
|
28
|
-
if (
|
|
67
|
+
if (!isInside(options.root, file) || file.includes(`${path.sep}node_modules${path.sep}`) || !existsSync(file)) return;
|
|
68
|
+
const prior = demands.get(file);
|
|
69
|
+
if (demands.has(file) && (prior === null || names && names.every(name => prior.has(name)))) return;
|
|
70
|
+
demands.set(file, names === null ? null : new Set([...(prior ?? []), ...names]));
|
|
29
71
|
if (lstatSync(file).isSymbolicLink() || realpathSync(file) !== file) throw new Error(`Behavior inputs cannot resolve through symlinks: ${file}`);
|
|
30
72
|
if (!lstatSync(file).isFile()) return;
|
|
31
73
|
files.add(file);
|
|
@@ -39,12 +81,31 @@ export function behaviorInputs(options, contract, check, context = inputContext(
|
|
|
39
81
|
css.walk(node => { if (node.type === "decl" || node.type === "atrule") valueParser(node.type === "decl" ? node.value : node.params).walk(value => { if (value.type === "function" && value.value.toLowerCase() === "url") localAsset(value.nodes.filter(part => part.type !== "space").map(part => part.value).join("")); }); });
|
|
40
82
|
}
|
|
41
83
|
if (!/\.[cm]?[jt]sx?$/.test(file)) return;
|
|
42
|
-
const ast =
|
|
84
|
+
const ast = context.ast(file);
|
|
85
|
+
const barrel = names && ast.statements.every(statement => ts.isEmptyStatement(statement) || ts.isImportDeclaration(statement) && !statement.importClause || ts.isExportDeclaration(statement) && (!statement.exportClause || ts.isNamedExports(statement.exportClause)) && (!statement.exportClause || !statement.exportClause.elements.length || statement.moduleSpecifier));
|
|
86
|
+
if (barrel) {
|
|
87
|
+
for (const statement of ast.statements) {
|
|
88
|
+
if (!statement.moduleSpecifier || !ts.isStringLiteralLike(statement.moduleSpecifier)) continue;
|
|
89
|
+
for (const target of resolve(statement.moduleSpecifier.text, file)) {
|
|
90
|
+
if (ts.isImportDeclaration(statement) || !statement.exportClause) visit(target);
|
|
91
|
+
else {
|
|
92
|
+
const imported = statement.exportClause.elements.filter(entry => names.includes(entry.name.text)).map(entry => (entry.propertyName ?? entry.name).text);
|
|
93
|
+
if (imported.length) visit(target, imported);
|
|
94
|
+
effects(target);
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
return;
|
|
99
|
+
}
|
|
43
100
|
function walk(node) {
|
|
44
101
|
let specifier;
|
|
45
102
|
if ((ts.isImportDeclaration(node) || ts.isExportDeclaration(node)) && node.moduleSpecifier && ts.isStringLiteralLike(node.moduleSpecifier)) specifier = node.moduleSpecifier.text;
|
|
46
103
|
if (ts.isCallExpression(node) && (node.expression.kind === ts.SyntaxKind.ImportKeyword || ts.isIdentifier(node.expression) && node.expression.text === "require") && node.arguments[0] && ts.isStringLiteralLike(node.arguments[0])) specifier = node.arguments[0].text;
|
|
47
|
-
if (specifier)
|
|
104
|
+
if (specifier) {
|
|
105
|
+
const clause = ts.isImportDeclaration(node) && node.importClause;
|
|
106
|
+
const names = clause && !clause.namedBindings ? clause.name ? ["default"] : null : clause?.namedBindings && ts.isNamedImports(clause.namedBindings) ? [...(clause.name ? ["default"] : []), ...clause.namedBindings.elements.map(entry => (entry.propertyName ?? entry.name).text)] : null;
|
|
107
|
+
for (const target of resolve(specifier, file)) visit(target, names);
|
|
108
|
+
}
|
|
48
109
|
if (ts.isNewExpression(node) && ts.isIdentifier(node.expression) && node.expression.text === "URL" && node.arguments?.[0] && ts.isStringLiteralLike(node.arguments[0])) localAsset(node.arguments[0].text);
|
|
49
110
|
if (ts.isCallExpression(node) && node.expression.kind === ts.SyntaxKind.ImportKeyword && (!node.arguments[0] || !ts.isStringLiteralLike(node.arguments[0]))) {
|
|
50
111
|
// Computed module selection cannot provide a precise dependency closure.
|
|
@@ -56,13 +117,11 @@ export function behaviorInputs(options, contract, check, context = inputContext(
|
|
|
56
117
|
walk(ast);
|
|
57
118
|
}
|
|
58
119
|
for (const file of context.inventory.units.find(unit => unit.id === contract.unit)?.files ?? []) visit(path.join(options.root, file));
|
|
59
|
-
for (const file of
|
|
60
|
-
for (const file of sourceFiles(path.join(options.dsRoot, options.config.tokenDir))) visit(file);
|
|
61
|
-
visit(path.join(options.dsRoot, options.config.stylesEntry));
|
|
120
|
+
for (const file of context.seeds) visit(file);
|
|
62
121
|
const scenario = check.scenario && context.scenarios.find(item => item.id === check.scenario);
|
|
63
122
|
if (scenario) visit(path.join(options.root, scenario.file));
|
|
64
|
-
const rows = [...files].sort().map(file => [slash(path.relative(options.root, file)),
|
|
65
|
-
for (const file of ["package-lock.json", "bun.lock", "bun.lockb", "pnpm-lock.yaml", "yarn.lock", "node_modules/.package-lock.json"]) rows.push([file, existsSync(path.join(options.root, file)) ?
|
|
123
|
+
const rows = [...files].sort().map(file => [slash(path.relative(options.root, file)), context.digest(file)]);
|
|
124
|
+
for (const file of ["package-lock.json", "bun.lock", "bun.lockb", "pnpm-lock.yaml", "yarn.lock", "node_modules/.package-lock.json"]) rows.push([file, existsSync(path.join(options.root, file)) ? context.digest(path.join(options.root, file)) : null]);
|
|
66
125
|
rows.push(["profile", JSON.stringify(options.config.mcp.behaviorProfiles[check.profile])], ["node", process.version]);
|
|
67
126
|
for (const file of ["./verification.mjs", "./verification/browser.mjs", "./verification/playwright-reporter.cjs"]) rows.push([`tool:${file}`, hash(readFileSync(new URL(file, import.meta.url)))]);
|
|
68
127
|
return { revision: hash(JSON.stringify(rows)), files: rows.map(([file]) => file) };
|
|
@@ -104,9 +163,11 @@ export function affectedBehaviorUnits(options, changedFiles) {
|
|
|
104
163
|
return contracts.filter(contract => Object.values(contract.checks).some(check => behaviorInputs(options, contract, check, context).files.some(file => changed.has(file)))).map(contract => contract.unit);
|
|
105
164
|
}
|
|
106
165
|
|
|
107
|
-
function
|
|
166
|
+
export function executeDevelopmentCommand(profile, args, root, env, signal) {
|
|
108
167
|
return new Promise((resolve, reject) => {
|
|
109
|
-
|
|
168
|
+
const label = profile.label ?? "Behavior verification";
|
|
169
|
+
if (signal.aborted) { reject(new Error(`${label} cancelled.`)); return; }
|
|
170
|
+
const startedAt = Date.now(), timeoutMs = profile.timeoutMs ?? 60000;
|
|
110
171
|
let output = "", failure, child, guard, done = false;
|
|
111
172
|
const kill = () => {
|
|
112
173
|
if (!child?.pid) return;
|
|
@@ -121,28 +182,47 @@ function execute(profile, args, root, env, signal) {
|
|
|
121
182
|
guard.once("error", error => { failure = error; kill(); });
|
|
122
183
|
guard.once("exit", () => { if (!done) { failure ??= new Error("Verification ownership guard exited unexpectedly."); kill(); } });
|
|
123
184
|
}
|
|
124
|
-
const abort = () => { failure = new Error(
|
|
185
|
+
const abort = () => { failure = Object.assign(new Error(`${label} cancelled.`), { reason: "cancelled" }); kill(); };
|
|
125
186
|
signal.addEventListener("abort", abort, { once: true });
|
|
126
|
-
const timeout = setTimeout(() => { failure = new Error(
|
|
187
|
+
const timeout = setTimeout(() => { failure = Object.assign(new Error(`${label} timed out after ${timeoutMs} ms for the complete profile run, not each check.`), { reason: "timeout", timeoutMs }); kill(); }, timeoutMs);
|
|
127
188
|
for (const stream of [child.stdout, child.stderr]) stream.on("data", chunk => { output = (output + chunk.toString()).slice(-12000); });
|
|
128
189
|
child.once("error", error => { failure = error; });
|
|
129
190
|
child.once("exit", kill);
|
|
130
191
|
child.once("close", code => {
|
|
131
192
|
done = true; kill(); guard?.stdin.end("disarm\n"); clearTimeout(timeout); signal.removeEventListener("abort", abort);
|
|
132
|
-
|
|
133
|
-
|
|
193
|
+
const elapsedMs = Date.now() - startedAt;
|
|
194
|
+
if (failure) reject(Object.assign(failure, { output, elapsedMs }));
|
|
195
|
+
else resolve({ code, output, elapsedMs });
|
|
134
196
|
});
|
|
135
197
|
});
|
|
136
198
|
}
|
|
137
199
|
|
|
200
|
+
function readReport(file, runId) {
|
|
201
|
+
if (!existsSync(file) || realpathSync(file) !== file || !lstatSync(file).isFile() || lstatSync(file).size > 1024 * 1024) throw new Error("Runner produced no valid bounded verification report. A successful exit alone is not passing evidence.");
|
|
202
|
+
const value = JSON.parse(readFileSync(file, "utf8"));
|
|
203
|
+
if (value.schemaVersion !== 1 || value.runId !== runId || !Array.isArray(value.cases) || value.cases.length > 500 || value.active !== undefined && (!Array.isArray(value.active) || value.active.length > 100)) throw new Error("Runner report has the wrong run ID/schema or too many cases.");
|
|
204
|
+
return value;
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
function runProgress(group, report, startedAt, profile, timeoutMs) {
|
|
208
|
+
const selected = group.map(check => ({ unit: check.unit, check: check.id, id: check.case, file: slash(path.posix.join(path.posix.dirname(slash(check.contract.file)), check.file)) }));
|
|
209
|
+
const cases = selected.map(check => {
|
|
210
|
+
const matches = report?.cases.filter(item => item?.id === check.id && item.file === check.file) ?? [];
|
|
211
|
+
const running = report?.active?.find(item => item?.id === check.id && item.file === check.file);
|
|
212
|
+
return { unit: check.unit, check: check.check, state: running ? "running" : matches.length ? "completed" : Array.isArray(report?.active) ? "not_started" : "not_reported", ...(running ? { elapsedMs: Math.max(0, Date.now() - Date.parse(running.startedAt)) || 0, workerIndex: running.workerIndex } : {}), ...(matches.length ? { reportedStatus: matches.at(-1).status, durationMs: matches.reduce((sum, item) => sum + (Number.isFinite(item.durationMs) ? Math.max(0, item.durationMs) : 0), 0) } : {}) };
|
|
213
|
+
});
|
|
214
|
+
return { profile, elapsedMs: Date.now() - startedAt, timeoutMs, total: cases.length, completed: cases.filter(item => item.state === "completed").length, active: cases.filter(item => item.state === "running"), pending: cases.filter(item => ["not_started", "not_reported"].includes(item.state)).length, cases, ...(Number.isSafeInteger(report?.workers) ? { workers: report.workers } : {}) };
|
|
215
|
+
}
|
|
216
|
+
|
|
138
217
|
/** Registered development commands are trusted project code, never model-supplied commands. */
|
|
139
|
-
export function createVerificationController(root) {
|
|
218
|
+
export function createVerificationController(root, { reportRoot = root, onProgress } = {}) {
|
|
140
219
|
root = path.resolve(root);
|
|
141
|
-
|
|
220
|
+
reportRoot = path.resolve(reportRoot);
|
|
221
|
+
let active, progress;
|
|
142
222
|
return {
|
|
143
|
-
status: () => ({ running: !!active }),
|
|
223
|
+
status: () => ({ running: !!active, ...(active && progress ? { progress: { ...progress, elapsedMs: Date.now() - progress.startedAt } } : {}) }),
|
|
144
224
|
async close() { active?.abort(); await active?.completion; },
|
|
145
|
-
async run({ unit, units, checks, persist = true } = {}, signal) {
|
|
225
|
+
async run({ unit, units, checks, persist = true } = {}, signal, notifyProgress) {
|
|
146
226
|
if (active) return { ok: false, diagnostics: [{ rule: "behavior-busy", message: "A behavior verification is already running." }], results: [] };
|
|
147
227
|
active = new AbortController();
|
|
148
228
|
const controller = active;
|
|
@@ -162,7 +242,7 @@ export function createVerificationController(root) {
|
|
|
162
242
|
for (const profileName of new Set(selection.map(check => check.profile))) {
|
|
163
243
|
const profile = options.config.mcp.behaviorProfiles[profileName];
|
|
164
244
|
const group = selection.filter(check => check.profile === profileName);
|
|
165
|
-
const directory = path.join(
|
|
245
|
+
const directory = path.join(reportRoot, ".ds/verification", randomUUID());
|
|
166
246
|
mkdirSync(directory, { recursive: true });
|
|
167
247
|
if (realpathSync(directory) !== directory) throw new Error("Verification output cannot resolve through a symlink.");
|
|
168
248
|
const runId = randomUUID(), report = path.join(directory, "report.json"), request = path.join(directory, "request.json");
|
|
@@ -179,31 +259,55 @@ export function createVerificationController(root) {
|
|
|
179
259
|
const requestContext = { schemaVersion: 1, runId, projectRoot: root, framework: options.config.framework, profile: profileName, mode: profile.mode ?? "unit", checks: group.map(({ contract: _contract, ...check }) => check), scenarios };
|
|
180
260
|
writeFileSync(request, JSON.stringify(requestContext));
|
|
181
261
|
const args = [...profile.args];
|
|
182
|
-
if (profile.runner === "playwright")
|
|
262
|
+
if (profile.runner === "playwright") {
|
|
263
|
+
const tags = [...new Set(group.map(check => check.case))].map(id => id.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"));
|
|
264
|
+
args.push(...new Set(group.map(check => slash(path.relative(root, path.resolve(root, path.dirname(check.contract.file), check.file))))), `--grep=(?:^|\\s)@ads:(?:${tags.join("|")})(?=\\s|$)`, `--reporter=${fileURLToPath(new URL("./verification/playwright-reporter.cjs", import.meta.url))}`, `--output=${path.join(directory, "artifacts")}`);
|
|
265
|
+
}
|
|
183
266
|
let run, reportValue, runnerError;
|
|
267
|
+
const startedAt = Date.now();
|
|
268
|
+
const update = value => {
|
|
269
|
+
progress = { ...runProgress(group, value, startedAt, profileName, profile.timeoutMs ?? 60000), startedAt };
|
|
270
|
+
onProgress?.(progress); notifyProgress?.(progress);
|
|
271
|
+
};
|
|
272
|
+
update();
|
|
273
|
+
let lastSnapshot = "", lastUpdate = Date.now();
|
|
274
|
+
const monitor = setInterval(() => {
|
|
275
|
+
let value;
|
|
276
|
+
try { value = readReport(report, runId); } catch { /* An absent/invalid intermediate snapshot cannot provide evidence. */ }
|
|
277
|
+
const snapshot = JSON.stringify(value);
|
|
278
|
+
if (snapshot !== lastSnapshot || Date.now() - lastUpdate >= 5000) { lastSnapshot = snapshot; lastUpdate = Date.now(); update(value); }
|
|
279
|
+
}, 250);
|
|
184
280
|
try {
|
|
185
|
-
run = await
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
281
|
+
run = await executeDevelopmentCommand(profile, args, root, { ADS_VERIFY_REQUEST: request, ADS_VERIFY_REPORT: report, ADS_VERIFY_MODE: requestContext.mode }, signal);
|
|
282
|
+
} catch (error) { runnerError = error; }
|
|
283
|
+
finally { clearInterval(monitor); }
|
|
284
|
+
try { reportValue = readReport(report, runId); }
|
|
285
|
+
catch (error) { runnerError ??= error; }
|
|
286
|
+
update(reportValue);
|
|
287
|
+
if (runnerError) {
|
|
288
|
+
const detail = `${progress.completed}/${progress.total} checks completed; ${progress.pending} without a completed/active result. Active: ${progress.active.map(item => `${item.unit}#${item.check} (${Math.round(item.elapsedMs)} ms)`).join(", ") || "none reported"}.`;
|
|
289
|
+
diagnostics.push({ rule: "behavior-runner", message: `${runnerError.message} ${detail}`, reason: runnerError.reason, elapsedMs: runnerError.elapsedMs ?? Date.now() - startedAt, progress, output: runnerError.output?.slice(-4000), profile: profileName });
|
|
290
|
+
}
|
|
190
291
|
const suiteErrors = Array.isArray(reportValue?.errors) ? reportValue.errors.slice(0, 5).map(error => String(error).slice(0, 1000)) : [];
|
|
191
|
-
const runnerFailure = run && (run.code !== 0 || suiteErrors.length) ? `Registered runner ${profileName} ${run.code !== 0 ? `exited with code ${run.code ?? "unknown"}` : "reported a suite error"}. ${suiteErrors.join("; ") || reportValue?.cases.filter(item => item?.status === "failed").slice(0, 3).map(item => `${item.id}: ${item.message || "failed without a case diagnostic"}`).join("; ") || "Inspect the runner output and verification report."}`.slice(0, 3000) : undefined;
|
|
292
|
+
const runnerFailure = run && (run.code !== 0 || suiteErrors.length || reportValue?.complete === false) ? `Registered runner ${profileName} ${run.code !== 0 ? `exited with code ${run.code ?? "unknown"}` : reportValue?.complete === false ? "left an incomplete report" : "reported a suite error"}. ${suiteErrors.join("; ") || reportValue?.cases.filter(item => item?.status === "failed").slice(0, 3).map(item => `${item.id}: ${item.message || "failed without a case diagnostic"}`).join("; ") || "Inspect the runner output and verification report."}`.slice(0, 3000) : undefined;
|
|
192
293
|
if (runnerFailure) diagnostics.push({ rule: "behavior-runner", message: runnerFailure, output: run.output?.slice(-4000), profile: profileName });
|
|
193
294
|
const afterOptions = loadConfig(root), afterContext = inputContext(afterOptions);
|
|
194
295
|
for (const check of group) {
|
|
195
296
|
const file = slash(path.relative(root, path.resolve(root, path.dirname(check.contract.file), check.file)));
|
|
196
297
|
const matches = reportValue?.cases.filter(item => item && item.id === check.case && item.file === file) ?? [];
|
|
197
298
|
const reportedStatus = matches.length && matches.every(item => item.status === "passed") ? "passed" : matches.length && matches.every(item => item.status === "skipped") ? "skipped" : "failed";
|
|
198
|
-
|
|
299
|
+
const interrupted = !!runnerError && !!reportValue;
|
|
300
|
+
let status = interrupted ? matches.some(item => item.status === "failed") ? "failed" : "incomplete" : !run || runnerError || runnerFailure ? "failed" : reportedStatus;
|
|
199
301
|
if (runnerError?.code === "ENOENT") status = "unavailable";
|
|
200
302
|
const current = behaviorInputs(afterOptions, check.contract, check, afterContext).revision === check.inputRevision;
|
|
201
303
|
if (!current) status = "stale";
|
|
202
304
|
if (signal.aborted) status = "cancelled";
|
|
203
305
|
results.push({ unit: check.unit, check: check.id, file: check.file, case: check.case, profile: profileName, status, current, revision: check.inputRevision, runId, completedAt: new Date().toISOString(),
|
|
204
306
|
previewRevision: profile.mode === "browser" ? host?.previewRevision() ?? readPreviewManifest(options)?.revision ?? null : null,
|
|
205
|
-
...(
|
|
206
|
-
|
|
307
|
+
...(matches.length ? { reportedStatus, durationMs: matches.reduce((sum, item) => sum + (Number.isFinite(item.durationMs) ? Math.max(0, item.durationMs) : 0), 0) } : {}),
|
|
308
|
+
...(runnerFailure ? { runnerExitCode: run.code } : {}),
|
|
309
|
+
execution: progress.cases.find(item => item.unit === check.unit && item.check === check.id)?.state,
|
|
310
|
+
message: [matches.filter(item => item.status !== "passed").map(item => String(item.message || `Case reported ${item.status} without a diagnostic.`).slice(0, 2000)).join("\n"), runnerError ? `${runnerError.message} This check ${matches.length ? `reported ${reportedStatus}` : `was ${progress.cases.find(item => item.unit === check.unit && item.check === check.id)?.state ?? "not reported"}`}; the complete run did not pass.` : !matches.length ? "The selected case did not execute/report. Missing and skipped cases do not pass." : runnerFailure ? `This case reported ${reportedStatus}, but the complete run did not pass. ${runnerFailure}` : ""].filter(Boolean).join("\n"), report: slash(path.relative(reportRoot, report)) });
|
|
207
311
|
}
|
|
208
312
|
if (signal.aborted) break;
|
|
209
313
|
}
|
|
@@ -217,7 +321,7 @@ export function createVerificationController(root) {
|
|
|
217
321
|
}
|
|
218
322
|
return { ok: diagnostics.length === 0 && results.length > 0 && results.every(result => result.status === "passed" && result.current), diagnostics, results };
|
|
219
323
|
} catch (error) { return { ok: false, diagnostics: [{ rule: "behavior-verification", message: error.message }], results }; }
|
|
220
|
-
finally { try { await host?.close(); } finally { if (active === controller) active = undefined; finish(); } }
|
|
324
|
+
finally { try { await host?.close(); } finally { if (active === controller) { active = undefined; progress = undefined; } finish(); } }
|
|
221
325
|
},
|
|
222
326
|
};
|
|
223
327
|
}
|
package/docs/ADS.md
CHANGED
|
@@ -107,6 +107,11 @@ framework/dependency gates and MCP execution contract.
|
|
|
107
107
|
Flat owners may use `Button.scenario.tsx` and `Button.fixture.ts` instead.
|
|
108
108
|
`Scenario.tsx` exports a default independently mountable view and literal metadata
|
|
109
109
|
`export const scenario = { title: "Button states", covers: ["Button"] };`.
|
|
110
|
+
Optional static `captureTargets` maps up to 20 names to CSS selectors scoped to
|
|
111
|
+
that scenario's App. These support explicitly requested element screenshots; they
|
|
112
|
+
do not grant coverage or styling exceptions. PNGs and current/candidate metadata
|
|
113
|
+
are generated development evidence, outside source ownership and product bundles.
|
|
114
|
+
See the portable [capture workflow](../skills/ads-designer/references/captures.md).
|
|
110
115
|
Token entries under the configured token directory declare
|
|
111
116
|
`export const scenario = { title: fixture.title, covers: ["TokenSamples"], samples: fixture.samples };`.
|
|
112
117
|
`TokenSamples` is any project-owned public review pattern, not a required name.
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -336,6 +336,39 @@ The MCP server serves compiled bytes without evaluating project TSX. Compiler
|
|
|
336
336
|
adapters normalize each entry to mount/unmount, retaining native React or Preact
|
|
337
337
|
semantics; choosing a framework does not install one into the tooling host.
|
|
338
338
|
|
|
339
|
+
## On-demand visual evidence
|
|
340
|
+
|
|
341
|
+
The capture unit is a registered, independently mountable scenario or one of its
|
|
342
|
+
project-declared `captureTargets`. Components, patterns, compositions and token
|
|
343
|
+
previews use the same real MCP App, fixtures and adapter as interactive review.
|
|
344
|
+
Shared controllers/hooks need a visible demonstration. Capture introduces no
|
|
345
|
+
tool-owned component templates, styling or fixture inference.
|
|
346
|
+
|
|
347
|
+
ADS owns isolated preview hosting, App readiness and an owned Playwright process;
|
|
348
|
+
the receiving project supplies Playwright and its installed browser. The capture
|
|
349
|
+
host is the existing minimal MCP client/bridge with navigation hidden, giving the
|
|
350
|
+
sandboxed App the requested viewport. Project runtime, fonts and styles remain
|
|
351
|
+
inside the App resource graph. The image excludes showcase chrome.
|
|
352
|
+
|
|
353
|
+
Captures exist only when requested. `ads_capture` replaces current and removes any
|
|
354
|
+
candidate for that scenario/target. Guarded patch/transaction dry runs can opt into
|
|
355
|
+
`screenshot:true` with explicit selected captures: after the normal gates, ADS
|
|
356
|
+
renders current and candidate under matching settings without changing live source.
|
|
357
|
+
A new scenario has no fabricated baseline. Final freshness/conflict checks precede
|
|
358
|
+
image publication. Matching accepted source promotes candidate to current.
|
|
359
|
+
|
|
360
|
+
There are at most two slots per target: current and candidate. Revisions/settings
|
|
361
|
+
are metadata, not new resource identities. There is no previous slot, accumulating
|
|
362
|
+
history or TTL. Stale source is identified explicitly; deleted targets are collected.
|
|
363
|
+
PNG/pixel/store limits and explicit clearing bound generated storage. These images
|
|
364
|
+
are visual evidence for one state, not passing behavioral guarantees.
|
|
365
|
+
|
|
366
|
+
One MCP capture index and a PNG resource template expose stable image slots.
|
|
367
|
+
Subscriptions receive resource updates when images change or disappear. The pair
|
|
368
|
+
is published as one atomic generated record under `.ds/captures/` and stays outside
|
|
369
|
+
source ownership and product bundles. Detailed usage lives in each portable skill's
|
|
370
|
+
[capture reference](../skills/ads-designer/references/captures.md).
|
|
371
|
+
|
|
339
372
|
## Build checks keep the architecture in place
|
|
340
373
|
|
|
341
374
|
The project build runs ADS checks before producing application or showcase
|
|
@@ -369,8 +402,12 @@ context, or the shared token foundation. ADS resolves file ownership, reconstruc
|
|
|
369
402
|
candidate files locally and validates a temporary project copy with architecture
|
|
370
403
|
checks, registered lint rules and project validation scripts. Affected registered
|
|
371
404
|
behavior checks, including consumers of a changed shared controller, also run in
|
|
372
|
-
the isolated candidate project.
|
|
373
|
-
|
|
405
|
+
the isolated candidate project. The project owns its test code and runner policy;
|
|
406
|
+
ADS owns affected-check selection, candidate hosting, progress, evidence freshness
|
|
407
|
+
and process cleanup. One profile timeout bounds its entire runner process. Parallel
|
|
408
|
+
workers can share that candidate host with independent App state. Partial outcomes
|
|
409
|
+
remain diagnostic when execution is interrupted; they cannot approve a mutation.
|
|
410
|
+
Passing checks describe their registered assertions, not every natural-language claim. This keeps edits
|
|
374
411
|
focused without requiring entire file bodies in model context. Peer units and
|
|
375
412
|
unrelated owners remain outside the selected unit's edit boundary. Shared foundation,
|
|
376
413
|
branding and guides have their own units. Shared credits and text license notices
|
package/docs/BEHAVIORS.md
CHANGED
|
@@ -139,8 +139,25 @@ configuration must discover colocated spec files. Configure browser installation
|
|
|
139
139
|
and any required test setup in that project. The ADS tooling host gains no
|
|
140
140
|
framework or browser dependency from a profile.
|
|
141
141
|
|
|
142
|
-
The Playwright adapter appends selected file paths
|
|
143
|
-
artifact directory.
|
|
142
|
+
The Playwright adapter appends selected file paths and case-tag filtering, its
|
|
143
|
+
reporter and a generated artifact directory. Unselected cases in the same spec
|
|
144
|
+
do not run. It preserves the project's worker configuration.
|
|
145
|
+
|
|
146
|
+
`timeoutMs` bounds the complete runner process for one profile, including runner
|
|
147
|
+
startup, hooks, retries and all selected cases. It is not a per-test timeout.
|
|
148
|
+
Playwright's own test/hook timeouts remain separate project settings. Set the
|
|
149
|
+
profile budget from measured suite durations with headroom; increasing it does
|
|
150
|
+
not repair a stalled assertion or cleanup.
|
|
151
|
+
|
|
152
|
+
Independent specs can use multiple Playwright workers against the same managed
|
|
153
|
+
candidate host. Each check needs its own page/context and state; avoid fixed ports,
|
|
154
|
+
shared mutable fixtures and test-order dependencies. Start with a small worker
|
|
155
|
+
count and measure on the target machine. Files normally run in parallel; tests
|
|
156
|
+
within one file normally remain sequential. ADS does not force parallelism or
|
|
157
|
+
change verification policy. See [Playwright parallelism](https://playwright.dev/docs/test-parallel).
|
|
158
|
+
Prefer readiness/assertion polling over arbitrary sleeps. Timer-only guarantees
|
|
159
|
+
may use [Playwright's clock](https://playwright.dev/docs/clock); retain real time
|
|
160
|
+
when browser motion, pointer gestures or layout timing are what the check verifies.
|
|
144
161
|
|
|
145
162
|
Runner failures remain blocking even when some selected cases reported passed.
|
|
146
163
|
Those results retain `reportedStatus` and `runnerExitCode`, and explain the suite
|
|
@@ -150,6 +167,24 @@ scenario ID and failed browser requests in startup failures; App resource errors
|
|
|
150
167
|
identify the resource URI. Failed assertions and startup errors are not retried
|
|
151
168
|
automatically. Fix or investigate the failure before explicitly verifying again.
|
|
152
169
|
|
|
170
|
+
The reporter atomically publishes partial snapshots at test start/end. While a
|
|
171
|
+
run is active, `ads_status.behaviors.progress` exposes the profile, runner budget,
|
|
172
|
+
elapsed time, completed/active/pending counts, worker count when available and
|
|
173
|
+
case identities/timings. Candidate mutation verification exposes the same data at
|
|
174
|
+
`ads_status.mutations.progress`. Clients requesting MCP progress receive updates
|
|
175
|
+
and a heartbeat during the runner phase; client-side request timeouts still apply.
|
|
176
|
+
`durationMs` records completed case execution, including attempts, separately from
|
|
177
|
+
the suite's wall time. Parallel case durations can sum to more than wall time.
|
|
178
|
+
|
|
179
|
+
After a deadline, valid completed outcomes remain visible through `reportedStatus`
|
|
180
|
+
and durations, alongside active and unstarted case identities. A passed case from
|
|
181
|
+
an interrupted run has blocking status `incomplete`; real assertion failures retain
|
|
182
|
+
their own messages. No incomplete run can accept a mutation or certify a guarantee.
|
|
183
|
+
Reports and artifacts stay under the live project's `.ds/verification/` even after
|
|
184
|
+
the candidate copy is removed. This is diagnostic output, not passing live-source
|
|
185
|
+
evidence. Old/custom runners without incremental progress use `not_reported` for
|
|
186
|
+
cases whose execution state cannot be established.
|
|
187
|
+
|
|
153
188
|
Use native test tags for stable case identity:
|
|
154
189
|
|
|
155
190
|
```ts
|
|
@@ -210,7 +245,13 @@ selected `checks` and optional `scenarios` URLs. Write a bounded JSON report:
|
|
|
210
245
|
}
|
|
211
246
|
```
|
|
212
247
|
|
|
213
|
-
Case statuses are `passed`, `failed` or `skipped`, with optional `message
|
|
248
|
+
Case statuses are `passed`, `failed` or `skipped`, with optional `message` and
|
|
249
|
+
`durationMs`. Protocol runners may atomically publish intermediate reports with
|
|
250
|
+
`complete:false` and an `active` list of `{id,file,startedAt,workerIndex?}`, then
|
|
251
|
+
publish `complete:true` at completion. Older final reports without `complete`
|
|
252
|
+
remain supported after normal process completion. An explicitly incomplete report
|
|
253
|
+
cannot pass even with exit code zero. Partial reports retain the same schema/run
|
|
254
|
+
identity and size limits as final reports.
|
|
214
255
|
`file` is project-relative. Nonzero process exit, missing cases, skipped cases,
|
|
215
256
|
wrong run IDs and unavailable reports do not pass. Test commands are trusted
|
|
216
257
|
project code; an adapter cannot prove the quality or honesty of their assertions.
|
|
@@ -245,10 +286,23 @@ Computed module selection conservatively includes all ADS owners. Environment,
|
|
|
245
286
|
network and other unregistered external inputs remain the runner's responsibility;
|
|
246
287
|
an evidence record is not a proof about arbitrary external state.
|
|
247
288
|
|
|
289
|
+
For named imports through simple re-export barrels, ADS follows selected bindings
|
|
290
|
+
and retains dependencies with declared side effects. This narrowing requires a
|
|
291
|
+
package `sideEffects:false` or a supported simple `sideEffects` glob array; preserve
|
|
292
|
+
CSS/global effects in that policy. Unknown policies/patterns, wildcard exports and
|
|
293
|
+
computed imports remain conservative. Barrel source changes still invalidate
|
|
294
|
+
dependents. ADS caches source parsing, resolution and digests within an inspection
|
|
295
|
+
only; later runs read current source again.
|
|
296
|
+
|
|
248
297
|
Guarded mutations execute affected registered behavior contracts after core,
|
|
249
298
|
Oxlint and configured project checks, before writing source. Imported shared
|
|
250
299
|
controller changes also select dependent contracts conservatively. Rejected and
|
|
251
300
|
dry-run patches publish no source changes. Verification output stays under `.ds/`.
|
|
301
|
+
To check a prepared draft, request its diff and pass the returned `patchFile` to
|
|
302
|
+
`ads_apply_patch` with `dryRun:true`. This checks the isolated candidate, including
|
|
303
|
+
affected behavior contracts and its configured browser preview. Candidate evidence
|
|
304
|
+
is not persisted as passing evidence for live source. `ads_verify` selects checks
|
|
305
|
+
for current source and has no draft argument.
|
|
252
306
|
|
|
253
307
|
## Packaging and performance
|
|
254
308
|
|
package/docs/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,22 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.13
|
|
4
|
+
|
|
5
|
+
- Add on-demand scenario/named-element PNG captures and screenshot-enabled guarded
|
|
6
|
+
dry runs. Expose stable current/candidate resources, freshness and notifications;
|
|
7
|
+
standalone captures clear earlier candidates, matching applies promote them, and
|
|
8
|
+
bounded storage keeps no previous/history/TTL. Document architecture and portable
|
|
9
|
+
skill usage without introducing project-specific renderers or fixtures.
|
|
10
|
+
- Report behavior progress, worker activity and per-case durations, preserving
|
|
11
|
+
partial outcomes and candidate artifacts on timeout while blocking incomplete
|
|
12
|
+
runs. Send requested MCP progress updates and expose candidate progress in status.
|
|
13
|
+
- Filter selected Playwright case tags, preserve project worker configuration and
|
|
14
|
+
verify concurrent Apps against one candidate host. Narrow named barrel dependencies
|
|
15
|
+
under declared side-effect policies and reuse parsing/digests within an inspection.
|
|
16
|
+
- Explain draft behavior verification through diff preparation and candidate
|
|
17
|
+
dry runs in all five portable skills. Distinguish candidate checks from
|
|
18
|
+
selected live-source verification through `ads_verify`.
|
|
19
|
+
|
|
3
20
|
## 0.3.12
|
|
4
21
|
|
|
5
22
|
- Let the documentation unit create shared credits, conventional root license
|