ccqa 1.51.0 → 1.51.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin/ccqa.mjs +13 -0
- package/dist/package.json +1 -1
- package/package.json +1 -1
package/dist/bin/ccqa.mjs
CHANGED
|
@@ -11126,6 +11126,7 @@ ${stepsText}
|
|
|
11126
11126
|
- Judge ONLY this step's \`Expected\` condition. Do not infer pass/fail from steps that have not run yet.
|
|
11127
11127
|
- If the page shows an error banner, a 404, a login wall, or any blocker that prevents the expected outcome — fail.
|
|
11128
11128
|
- If the expected outcome is partially satisfied (e.g. the page loaded but the asserted element is missing) — fail, and say which part is missing.
|
|
11129
|
+
- **An \`Expected\` may label its parts.** A \`must:\` line is the assertion — the rule above applies to it. A \`when present:\` line is observed, not required: judge it when it is there, and when it is absent say so in your reason and pass on the \`must:\` lines alone. Every line of a labelled \`Expected\` carries a label; text with no labels at all is all \`must:\`.
|
|
11129
11130
|
- Pass only when you have *positive* evidence (a successful snapshot, a verified URL, a wait that resolved). "No error shown" is not enough on its own.
|
|
11130
11131
|
- Do not invent success when blocked: fail honestly with a short reason.
|
|
11131
11132
|
- **Evidence discipline**: when the assertion target is a specific row / message / banner / URL, scroll it into view (or focus the relevant pane) before letting the step end. The "after" screenshot is captured for you automatically — your job is to make sure that screenshot shows the thing your STEP_RESULT line is talking about.
|
|
@@ -12634,12 +12635,21 @@ function isStateSelector(selector) {
|
|
|
12634
12635
|
function plainSelectorOf(action) {
|
|
12635
12636
|
return action.locator && action.index === void 0 ? locatorToSelector(action.locator) : void 0;
|
|
12636
12637
|
}
|
|
12638
|
+
/**
|
|
12639
|
+
* A `wait` the post-trace validator actually ran and saw fail. Cascade-skipped
|
|
12640
|
+
* actions carry the flag without ever having been attempted, so they are not
|
|
12641
|
+
* evidence of anything and stay.
|
|
12642
|
+
*/
|
|
12643
|
+
function isProvenFailedWait(action) {
|
|
12644
|
+
return action.replayUnstable === true && !(action.replayReason ?? "").includes("skipped after a preceding action failed");
|
|
12645
|
+
}
|
|
12637
12646
|
function actionToLine$1(action) {
|
|
12638
12647
|
if (isRefSelector(plainSelectorOf(action))) return null;
|
|
12639
12648
|
if (action.action === "assert" && action.replayUnstable && typeof action.replayReason === "string" && action.replayReason.includes("selector not present")) {
|
|
12640
12649
|
const sel = plainSelectorOf(action) ?? action.observation ?? "(unknown)";
|
|
12641
12650
|
return `// [warn] replay-unstable: dropped over-assertion (${action.assert ?? "assert"} ${sel}) — selector not present on replay`;
|
|
12642
12651
|
}
|
|
12652
|
+
if (action.action === "wait" && isProvenFailedWait(action)) return `// [warn] replay-unstable: dropped wait (${plainSelectorOf(action) ?? "(unknown)"}) — did not resolve on replay`;
|
|
12643
12653
|
switch (action.action) {
|
|
12644
12654
|
case "snapshot": return action.observation ? `// ${action.observation}` : null;
|
|
12645
12655
|
case "wait": {
|
|
@@ -13725,6 +13735,7 @@ function actionToLine(action) {
|
|
|
13725
13735
|
const sel = action.locator?.value ?? action.observation ?? "(unknown)";
|
|
13726
13736
|
return `// [warn] replay-unstable: dropped over-assertion (${action.assert ?? "assert"} ${sel}) — selector not present on replay`;
|
|
13727
13737
|
}
|
|
13738
|
+
if (action.action === "wait" && action.replayUnstable === true && !(action.replayReason ?? "").includes("skipped after a preceding action failed")) return `// [warn] replay-unstable: dropped wait (${action.locator?.value ?? "(unknown)"}) — did not resolve on replay`;
|
|
13728
13739
|
const locator = action.locator ? locatorToPlaywright(action.locator, action.index) : null;
|
|
13729
13740
|
const subject = locator !== null && action.index === void 0 ? `${locator}.first()` : locator;
|
|
13730
13741
|
switch (action.action) {
|
|
@@ -16829,6 +16840,8 @@ CCQA_STEP=<step-id> CCQA_ASSERT=url_contains:/dashboard agent-browser --session
|
|
|
16829
16840
|
|
|
16830
16841
|
- **Assert what the step asks about, nothing else.** The \`expected\` is the contract; anything else you happened to see on the way is not. A nav item, a heading or a greeting that the step never mentions adds no coverage, differs between recordings of the same spec, and is the first thing to break on replay — so the next recording quietly drops it and the test gets weaker without anyone deciding that.
|
|
16831
16842
|
|
|
16843
|
+
- **An \`expected\` that labels its parts is recorded down to the \`must:\` lines.** A \`when present:\` line names something the product does not produce on every run, so **nothing that fails on its absence may be recorded for it** — no assert, and no \`wait --text\` either: a recorded wait throws on the next run that legitimately lacks it, which is the same failure an assert would have caused. Look at it with \`snapshot\` or \`get count\` if you want to see it, and record nothing. When the step needs to wait for something asynchronous to arrive, anchor that wait on a \`must:\` signal — waiting on the optional one makes the whole step hinge on it. Every line of a labelled \`expected\` carries a label; text with no labels at all is all \`must:\`.
|
|
16844
|
+
|
|
16832
16845
|
- **\`url_contains\` is opt-in, not a habit.** The same rule, in the form that gets broken most. Emit it ONLY when a step's own \`expected\` explicitly asks about the URL or path. Do NOT add a \`url_contains\` to "prove" a login succeeded, a page loaded, or a navigation happened — confirm those with \`text_visible\` / \`element_visible\` on something the destination page renders. An unrequested URL assertion adds no coverage the visible-content assert doesn't already give, and is the single most common way an environment gets baked into a test.
|
|
16833
16846
|
- **When you do assert a URL, the substring may come from ONE place only:** a \`\${VAR}\` URL that *this step's own instruction* opened, written as that \`\${VAR}\` followed by the literal tail after it. If the step opens \`\${APP_URL}/policies\`, assert \`\${APP_URL}/policies\` (or the tail \`/policies\`); the recorder resolves \`\${APP_URL}\` per environment. Never assert on a URL you merely *observed* — login redirects, identity-provider pages, and OAuth callbacks all live on a **different, environment-named origin** than the app, so any substring of them (host, origin, OR path) names the environment.
|
|
16834
16847
|
- **A leading slash does NOT make a substring safe.** \`/auth-staging\`, \`/env-qa\`, \`/tenant-acme\` look like paths but are environment labels — the first segment of an identity-provider or tenant URL, not an application route. If a substring contains an environment name, a stage token (\`dev\`, \`stg\`, \`prod\`), a tenant/org name, or any fragment of a hostname, it is forbidden even with a leading slash. The only safe path substrings are stable *application* routes off the app's own origin (\`/dashboard\`, \`/policies/new\`), taken from a \`\${VAR}\` you opened — not from a redirect you watched.
|
package/dist/package.json
CHANGED