@camunda/e2e-test-suite 0.0.1037 → 0.0.1038
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -415,6 +415,26 @@ See [docs/ci-failure-categories.md](docs/ci-failure-categories.md) for the full
|
|
|
415
415
|
- **C8 SM with Helm**: Tests are run against the alpha branch of the [camunda-platform-helm repository](https://github.com/camunda/camunda-platform-helm), using SNAPSHOT images for each component. This is peformed with both Elasticsearch and OpenSearch.
|
|
416
416
|
- **C8 Run**: Tests are run against the main branch of the [camunda/camunda repository](https://github.com/camunda/camunda).
|
|
417
417
|
|
|
418
|
+
## PR-Level vs Nightly Coverage (AlwaysGreen)
|
|
419
|
+
|
|
420
|
+
When a PR opens in [camunda/camunda](https://github.com/camunda/camunda) or [camunda/connectors](https://github.com/camunda/connectors), AlwaysGreen dispatches a subset of this suite against that PR's build. This is separate from this repo's own [Automatic test runs on PR](#automatic-test-runs-on-pr) above, which cover changes to the test suite itself rather than to the product.
|
|
421
|
+
|
|
422
|
+
| leg | scope | blocking | runtime |
|
|
423
|
+
| --- | --- | --- | --- |
|
|
424
|
+
| SaaS smoke | `test-setup`, `navigation`, `smoke-tests` | yes | 9–12 min |
|
|
425
|
+
| SaaS full | remaining specs (minus exclusions below) | no, until Nov 2026 | ~40–48 min |
|
|
426
|
+
| SM smoke | smoke suite | yes | — |
|
|
427
|
+
| SM full | full suite | no | — |
|
|
428
|
+
|
|
429
|
+
**Excluded from PR level entirely — nightly only.** Third-party model-endpoint specs bill an external provider on every run they're included in, so they stay out of the PR-blocking path:
|
|
430
|
+
|
|
431
|
+
- By spec file: `agentic-ai-user-flows.spec.ts`, `idp-user-flows.spec.ts`, `aws-connectors-user-flows.spec.ts` (AWS Bedrock, AgentCore, Vertex, Azure OpenAI)
|
|
432
|
+
- By tag `@ai-external`: the AI-generated-form tests inside `web-modeler-user-flows.spec.ts` (`Form.js Integration with User Task and AI Generated Form`, `Form.js Integration with AI generated Start form`)
|
|
433
|
+
|
|
434
|
+
**Kept in at PR level despite touching external systems:** Connector Secrets tests (reads from our own AWS/GCP secret managers — no model endpoint, no meaningful per-run cost) and MCP tests (run against our own cluster — no cost).
|
|
435
|
+
|
|
436
|
+
See [AlwaysGreen Iteration 2 - Alpha 5: Coverage & Optimisation](https://github.com/camunda/team-test-automation/issues/78) for the full rationale and current SaaS/SM stability data behind these boundaries.
|
|
437
|
+
|
|
418
438
|
## Test Suite Structure
|
|
419
439
|
|
|
420
440
|
The test suite is organized using the Page Object Model design pattern, where each test file contains test scenarios related to a specific component or feature of the Camunda C8 product stack. Additionally, it leverages separate page objects for each web page or component being tested.
|