@andwhy/e2e-pipeline 0.11.1 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@andwhy/e2e-pipeline",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.12.0",
|
|
4
4
|
"description": "Scaffolds the full e2e testing pipeline: agent-assisted Playwright tests, CI runs against Vercel previews, and Checkly monitors that follow production deploys",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -88,9 +88,10 @@ The scaffold writes templates, not a working setup. Walk these with the user:
|
|
|
88
88
|
- `VERCEL_AUTOMATION_BYPASS_SECRET` — Vercel → project → Settings →
|
|
89
89
|
Deployment Protection → Protection Bypass for Automation. Per project —
|
|
90
90
|
never copied from another repo.
|
|
91
|
-
- `SLACK_WEBHOOK_URL` —
|
|
92
|
-
|
|
93
|
-
|
|
91
|
+
- `SLACK_WEBHOOK_URL` — the **org-shared** e2e-channel webhook: paste the
|
|
92
|
+
value from 1Password ("e2e Slack webhook"). Same value in every repo;
|
|
93
|
+
do not walk the user through creating a Slack app or webhook — routing
|
|
94
|
+
to the project channel happens via a Slack rule in Phase 5.
|
|
94
95
|
- `ANTHROPIC_API_KEY` needs **no action**: it is an organization-level
|
|
95
96
|
secret inherited by every repo. Do not tell the user to add it.
|
|
96
97
|
5. **Existing tests under `tests/`** (unit, a11y — the scaffold lists them as
|
|
@@ -242,10 +243,21 @@ Set up once per project:
|
|
|
242
243
|
5. **Do not also subscribe a Slack alert channel to these suites.** The
|
|
243
244
|
pipeline is the single alert path; a parallel raw Checkly alert double-pings
|
|
244
245
|
the channel and trains people to ignore one of them.
|
|
245
|
-
6.
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
246
|
+
6. **One Slack Workflow Builder rule** routes this project's messages from
|
|
247
|
+
the shared e2e channel to the project channel. All notifications post to
|
|
248
|
+
the e2e channel tagged `[e2e:<repo-name>]`; the rule matches the **full
|
|
249
|
+
bracketed tag** — never a short name like `gbk`, which collides with
|
|
250
|
+
other text — and sends the message to the project channel. The sent
|
|
251
|
+
message's template is just the message-text variable (plus the
|
|
252
|
+
link-to-original variable when available, so threads happen on the
|
|
253
|
+
original) — no static prefix text. Have the user add a **second
|
|
254
|
+
manager** to the rule and record it in the rule registry in the e2e
|
|
255
|
+
channel's canvas.
|
|
256
|
+
7. Dry run: GitHub → Actions → "Checkly auto-heal" → Run workflow, with the
|
|
257
|
+
suite's check name. Confirm the run completes and the message shows up in
|
|
258
|
+
the e2e channel and, via the rule, in the project channel (a passing
|
|
259
|
+
check should end as "not reproducible" — summary only, no message — so a
|
|
260
|
+
real dry run is easiest right after any genuine failure).
|
|
249
261
|
|
|
250
262
|
What the user should expect afterwards: a failing monitor produces either a
|
|
251
263
|
`checkly-heal` PR plus a Slack note (stale or flaky test — **review
|
|
@@ -10,9 +10,11 @@ name: Checkly auto-heal
|
|
|
10
10
|
# (.github, .claude, configs, lockfiles). The agent never pushes: branch,
|
|
11
11
|
# commit, PR, and Slack are workflow steps.
|
|
12
12
|
# Needs repo secrets CHECKLY_API_KEY, CHECKLY_ACCOUNT_ID, SLACK_WEBHOOK_URL
|
|
13
|
-
# (values in 1Password); ANTHROPIC_API_KEY
|
|
14
|
-
#
|
|
15
|
-
#
|
|
13
|
+
# (the org-shared e2e-channel webhook; values in 1Password); ANTHROPIC_API_KEY
|
|
14
|
+
# is inherited from the organization. Slack messages carry an [e2e:<repo>]
|
|
15
|
+
# tag so Slack Workflow Builder rules route them from the shared channel to
|
|
16
|
+
# project channels. Setup docs: the playwright-test-pipeline skill walks
|
|
17
|
+
# through the Checkly webhook channel and the Slack rule.
|
|
16
18
|
on:
|
|
17
19
|
repository_dispatch:
|
|
18
20
|
types: [checkly-alert]
|
|
@@ -295,6 +297,10 @@ jobs:
|
|
|
295
297
|
*)
|
|
296
298
|
text=":rotating_light: *Checkly: \`$CHECK_NAME\` failed and the auto-heal run did not finish cleanly.* ${SUMMARY:-Check the run logs.} This needs a developer." ;;
|
|
297
299
|
esac
|
|
300
|
+
# Routing tag: Slack Workflow Builder rules match the full bracketed
|
|
301
|
+
# tag ([e2e:<repo>]) to fan messages out from the shared e2e channel
|
|
302
|
+
# to project channels without substring collisions.
|
|
303
|
+
text="[e2e:${GITHUB_REPOSITORY##*/}] $text"
|
|
298
304
|
[ "$APP_CHANGE" = "true" ] && text="$text\nIncludes app-side test hooks — the PR's preview run validates them."
|
|
299
305
|
[ -n "$RESULT_LINK" ] && text="$text\nCheckly result: $RESULT_LINK"
|
|
300
306
|
text="$text\nRun: $RUN_URL"
|