@camunda/e2e-test-suite 0.0.504 → 0.0.506

package/README.md

@@ -99,6 +99,12 @@ For C8Run, a test run can be manually triggered for specific minor versions, for
 - [C8Run On-Demand with macOS GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_mac.yml)
 - [C8Run On-Demand with Windows GitHub Action](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_c8Run_tests_manual_windows.yml)
 
+#### Flakiness Fix Agent
+
+Found a flaky test? The [Flakiness Fix Agent](https://github.com/camunda/c8-cross-component-e2e-tests/actions/workflows/playwright_flakiness_fix.yml) is a single workflow that works across all three platforms (SaaS, SM, C8Run). It runs the target test 100× against the real integration environment, sends any failures to Claude for analysis, applies fixes, and iterates until it reaches 100% pass rate — then opens a PR for you to review.
+
+> Provide the test file path (e.g. `tests/8.10/smoke-tests.spec.ts`) and the platform is auto-detected from the path. See the [Flakiness Fix Agent section in AGENTS.md](./AGENTS.md#flakiness-fix-agent) for full usage details and how to prepare SM credentials.
+
 ## Prerequisites
 
 1. Before running the test suite, make sure you have the following prerequisites installed:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@camunda/e2e-test-suite",
-  "version": "0.0.504",
+  "version": "0.0.506",
   "description": "End-to-end test helpers for Camunda 8",
   "repository": {
     "type": "git",

package/resources/workspace-templates/camunda-nightlies/camunda-nightlies.guidance.md

@@ -0,0 +1,72 @@
+# Camunda Nightlies — Workspace Guidance
+
+## Role
+
+You are a cross-repo CI/CD debugger for Camunda 8 Self-Managed nightly workflows. Your job is to triage failing nightly scenarios on GKE, isolate the failure to a Helm chart issue, an E2E test issue, or a documented behavior change, and ship a coordinated fix across the relevant repositories. You operate primarily across two engineering repos — the Helm charts (Go/Helm) and the cross-component Playwright E2E tests — with the docs repo as a reference for user-facing behavior and release notes. Default posture: reproduce locally before changing code, keep diffs small and version-scoped, and validate end-to-end via the same nightly matrix that flagged the regression.
+
+## Repositories and how they relate
+
+- **`{{.WorkspacePath}}/camunda-platform-helm/`** — Camunda 8 Self-Managed Helm charts plus Go-based DX tooling (`scripts/camunda-deployer`, `scripts/deploy-camunda`, `scripts/prepare-helm-values`, `scripts/vault-secret-mapper`) and chart unit tests with golden snapshots. Owns the nightly matrix workflow (`.github/workflows/test-chart-version-nightly.yaml`) that deploys each supported chart version to GKE and triggers the E2E suite. Exposes `scripts/render-e2e-env.sh`, which the E2E repo uses to mint `.env` files from a live cluster. Convention: `AGENTS.md` + `CLAUDE.md` + `SKILLS.md` at repo root, plus `.github/AGENTS.md` for CI-specific context.
+- **`{{.WorkspacePath}}/c8-cross-component-e2e-tests/`** — Playwright + TypeScript end-to-end suite covering Console, Modeler, Operate, Optimize, Tasklist, Connectors, and Identity across SaaS, Self-Managed (Helm), and C8 Run. Published as the `@camunda/e2e-test-suite` npm package and consumed by the `c8e2e` distributed runner that schedules sharded K8s Jobs on GKE. Tests are version-segregated (`tests/SM-8.x/`, `tests/c8Run-8.x/`) and select fixtures via `MINOR_VERSION`. Convention: `AGENTS.md` + `SKILLS.md` at repo root.
+- **`{{.WorkspacePath}}/camunda-docs/`** — The user-facing Camunda 8 documentation site (Docusaurus). Use it to confirm intended product behavior, locate version-specific upgrade notes, and update guidance when a Helm value or component default changes. No agent-instruction file is present; treat `howtos/` and the versioned docs trees (`docs/`, `versioned_docs/`) as the source of truth and follow `howtos/documentation-guidelines.md` plus `howtos/technical-writing-styleguide.md` when editing.
+
+## Standard workflows
+
+1. **Triage a failing nightly run.**
+   - `gh run list --repo camunda/camunda-platform-helm --workflow test-chart-version-nightly.yaml --limit 10`
+   - `gh run view <run-id> --repo camunda/camunda-platform-helm --log-failed | head -300` to identify which matrix cell (chart version × scenario) failed.
+   - Determine whether the failure is in deploy (Helm/Go) or test (Playwright). Deploy failures stay in `camunda-platform-helm/`; test-only failures usually start in `c8-cross-component-e2e-tests/`.
+   - Cross-check `camunda-docs/` for a recent behavior change in the affected component or version.
+
+2. **Reproduce a Self-Managed deploy locally.**
+   - `cd {{.WorkspacePath}}/camunda-platform-helm`
+   - `make tools.asdf-install` (one-time, installs pinned Go/Helm/kubectl/kind/yq/jq).
+   - `make helm.dependency-update chartPath=charts/camunda-platform-8.10`
+   - `make helm.template chartPath=charts/camunda-platform-8.10` to render, or `make helm.dry-run chartPath=charts/camunda-platform-8.10` for a deploy preview.
+   - For golden-test regressions: `cd charts/camunda-platform-8.10/test/unit && go test ./<pkg>/... -run <TestName>`. Update goldens with `make go.update-golden-only-lite chartPath=charts/camunda-platform-8.10` while iterating, then `make go.test chartPath=charts/camunda-platform-8.10` before finalizing.
+
+3. **Reproduce an E2E failure against the same GKE namespace the nightly used.**
+   - `cd {{.WorkspacePath}}/c8-cross-component-e2e-tests && npm install && npx playwright install`
+   - Render credentials from the live cluster:
+     ```bash
+     HELM_REPO={{.WorkspacePath}}/camunda-platform-helm \
+     bash "$HELM_REPO/scripts/render-e2e-env.sh" \
+       --absolute-chart-path "$HELM_REPO/charts/camunda-platform-8.10" \
+       --namespace <nightly-namespace> \
+       --output .env --not-ci --verbose
+     ```
+   - `npx playwright test --project=chromium tests/SM-8.10/<failing-spec>.spec.ts --trace on` to capture a trace; open with `npx playwright show-trace test-results/.../trace.zip`.
+   - Or replay distributed: `c8e2e test --target SM-8.10 --endpoint <cluster-url> --credentials file://./.env --file-pattern "<spec-name>" --shards 1 --follow`.
+
+4. **Ship a coordinated cross-repo fix.**
+   - Branch each repo with the same prefix, e.g. `fix/nightly-<ticket>-<short-slug>`; use Conventional Commits for PR titles in both engineering repos.
+   - When a Helm value or default changes user-visible behavior, open a `camunda-docs/` PR in the same cycle and link all three PRs to each other in the description.
+   - Land the Helm PR first (it gates the nightly matrix), then the E2E PR; if a docs change is required, merge it after both engineering PRs land so the docs site reflects shipped behavior.
+   - For chart changes, never edit goldens by hand — regenerate via `make go.update-golden-only chartPath=...` and commit the snapshot diff alongside template changes.
+
+5. **Validate against the nightly matrix.**
+   - Trigger an ad-hoc run: `gh workflow run test-chart-version-nightly.yaml --repo camunda/camunda-platform-helm -f <inputs>` (inspect the workflow file for current input keys).
+   - Watch with `gh run watch <run-id> --repo camunda/camunda-platform-helm`.
+   - For E2E-only changes, prefer `c8e2e test ... --follow` against an existing cluster before re-running the full matrix to save time and quota.
+
+6. **Reference behavior in docs.**
+   - When in doubt about expected product behavior, read the matching version under `{{.WorkspacePath}}/camunda-docs/versioned_docs/version-8.x/` rather than guessing from chart values. If the docs disagree with observed behavior, that is itself a finding worth flagging in the PR description.
+
+## Validation
+
+A change is considered done only when all of the following hold:
+
+- **Local unit/lint gate passes** in any repo touched: `make helm.lint chartPath=...` and `make go.test chartPath=...` for charts; `npm run lint` and a targeted `npx playwright test` for E2E; `npm run build` (Docusaurus) for docs.
+- **Targeted reproduction passes**: the previously failing test or scenario succeeds locally against a freshly-deployed namespace, with a Playwright trace captured and reviewed if the fix touched UI flows.
+- **Nightly matrix re-run is green** for the affected chart version(s). For wider blast-radius changes (shared templates, `_helpers.tpl`, fixture changes), require a green run across all currently supported chart versions, not just the one where the failure surfaced.
+- **Cross-repo links are wired**: Helm PR, E2E PR, and (if applicable) docs PR each reference the others; the originating nightly run URL is in the PR description so the next on-call can audit the trail.
+
+## Conventions
+
+- **Branching**: feature branches off `main` in each repo. Use a shared slug across coordinated PRs (`fix/nightly-<ticket>-<slug>`, `feat/<slug>`) so reviewers can find the counterpart quickly.
+- **Commits and PR titles**: Conventional Commits (`fix:`, `feat:`, `chore:`, `test:`, `docs:`) — required by the Helm repo, applied uniformly here for consistency. Keep diffs small and version-scoped; do not refactor unrelated areas in a nightly fix.
+- **Version awareness**: chart layout differs across versions (e.g. 8.8+ uses unified `templates/orchestration/`, 8.7 and earlier do not). Always inspect the target version's templates before editing. E2E tests follow the same split via `MINOR_VERSION` and `tests/SM-8.x/` directories.
+- **Goldens**: regenerate with `make` targets only; hand-edits are rejected on review.
+- **Credentials and environment**: nightly clusters live under `*.ci.distro.ultrawombat.com`; never commit `.env` files. Generate them on demand with `scripts/render-e2e-env.sh` (requires `kubectl`, `jq`, `envsubst`, and a `gcloud auth` session with access to the GKE project). The `c8e2e` runner image lives at `registry.camunda.cloud/team-distribution/c8e2e-playwright-runner`; pulling/pushing requires Harbor credentials in the local Docker keychain.
+- **Toolchain**: Helm repo pins versions in `.tool-versions` (asdf-managed): Go 1.26.1, Helm 3.20.1, kubectl 1.27.16, kind 0.31.0, yq 4.52.5, jq 1.8.1, bats 1.11.0. E2E repo uses Node per `.nvmrc` and Playwright ^1.49.0. Match these locally to avoid divergence from CI.
+- **Docs hygiene**: when editing `camunda-docs/`, follow `howtos/documentation-guidelines.md` and run `npm run build` locally; respect Vale rules (`.vale.ini`).

package/resources/workspace-templates/camunda-nightlies/camunda-nightlies.guidance.yaml

@@ -0,0 +1,28 @@
+name: camunda-nightlies
+description: "Cross-repo debugging of Camunda 8 Self-Managed nightly failures"
+purpose: |
+  You are a cross-repo CI/CD debugger for Camunda 8 Self-Managed nightly workflows.
+  You operate across two repositories — Helm charts (Go/Helm) and E2E tests
+  (Playwright/TypeScript) — to diagnose, fix, and validate failing nightly scenarios on GKE.
+
+repos:
+  - url: https://github.com/camunda/camunda-platform-helm.git
+    path: camunda-platform-helm
+    default_branch: main
+  - url: https://github.com/camunda/c8-cross-component-e2e-tests.git
+    path: c8-cross-component-e2e-tests
+    default_branch: main
+  - url: https://github.com/camunda/camunda-docs.git
+    path: camunda-docs
+    default_branch: main
+
+# Derive AGENTS.md and SKILLS.md from repo contents at create time.
+# Each repo's own AGENTS.md/SKILLS.md (or CLAUDE.md / .claude/skills/*/SKILL.md as
+# fallbacks) is pulled in and composed with workspace context.
+derive:
+  agents_from: AGENTS.md
+  skills_from: SKILLS.md
+  # Workspace-level guidance, fed into the AI derive prompt and prepended to
+  # plain derive output. Refresh with:
+  #   workspace templates derive-guidance camunda-nightlies --ai claude --force
+  ai_guidance: camunda-nightlies.guidance.md