@jterrats/open-orchestra 1.0.7 → 1.0.9

package/docs/site-manifest.json

@@ -112,6 +112,7 @@
     "links": [
       { "title": "Adoption guide", "source": "docs/adoption-guide.md", "heading": "Open Orchestra 1.0.0 Adoption Guide" },
       { "title": "Core command surface", "source": "docs/core-command-surface.md", "heading": "Core Command Surface" },
+      { "title": "E2E test batteries", "source": "docs/e2e-test-batteries.md", "heading": "End-to-End Test Batteries" },
       { "title": "Duplicate-code enforcement", "source": "docs/duplicate-code-enforcement.md", "heading": "Duplicate-Code Enforcement" },
       { "title": "Sonar quality gates", "source": "docs/sonar-quality-gates.md", "heading": "Sonar Quality Gates" },
       { "title": "Sonar architecture model", "source": "docs/sonar-architecture-model.md", "heading": "Sonar Architecture Model" },
@@ -122,6 +123,7 @@
   "releaseDocs": {
     "links": [
       { "title": "Release test matrix", "source": "docs/release-test-matrix.md", "heading": "1.0.0 Release Test Matrix" },
+      { "title": "E2E test batteries", "source": "docs/e2e-test-batteries.md", "heading": "End-to-End Test Batteries" },
       { "title": "Sonar quality gates", "source": "docs/sonar-quality-gates.md", "heading": "Sonar Quality Gates" },
       { "title": "Sonar architecture model", "source": "docs/sonar-architecture-model.md", "heading": "Sonar Architecture Model" },
       { "title": "QA evidence", "source": "docs/site-content-workflow.md", "heading": "QA Evidence" },

package/docs/sonar-quality-gates.md

@@ -22,10 +22,13 @@ Required GitHub secret when the GitHub Actions workflow is enabled:
 
 - `SONAR_TOKEN`: token for SonarQube Cloud or SonarQube Server.
 
-Optional GitHub secret:
+Optional GitHub secrets:
 
 - `SONAR_HOST_URL`: required for self-hosted SonarQube Server. Leave unset for
   SonarQube Cloud, or set `http://localhost:9000` only for local commands.
+- `CF_ACCESS_CLIENT_ID` and `CF_ACCESS_CLIENT_SECRET`: Cloudflare Access service
+  token credentials for GitHub-hosted runners that must reach a private
+  self-hosted SonarQube URL protected by Zero Trust.
 
 Optional GitHub variables:
 
@@ -37,6 +40,12 @@ Optional GitHub variables:
   `workflow_dispatch`.
 - `SONAR_QUALITY_GATE_WAIT`: set to `true` to fail the workflow when the remote
   quality gate fails.
+- `SONAR_RUNNER`: set to `self-hosted` to run the Sonar workflow on a local
+  runner that can reach the shared SonarQube runtime directly. When this is set,
+  the workflow uses `http://localhost:9001` by default and skips Cloudflare
+  Access service-token checks.
+- `SONAR_LOCAL_HOST_URL`: optional override for self-hosted runner mode when the
+  runner reaches SonarQube through a different local-only URL.
 
 The workflow skips analysis when `SONAR_TOKEN` is not configured. This keeps
 forks and offline development usable. For private repositories, keep
@@ -55,29 +64,142 @@ gate status. If the scanner can upload analysis but the wait step fails with
 
 ## Local SonarQube
 
-Open Orchestra includes `docker-compose.sonar.yml` for local SonarQube
-dogfooding:
+Open Orchestra does not own the long-lived local SonarQube containers. The
+shared laptop/VPS runtime lives in `~/dev/sonarqube_jeterrats_dev` so multiple
+projects can use the same SonarQube server without tying its lifecycle to this
+repository.
 
 ```bash
-docker compose -f docker-compose.sonar.yml up -d
+cd ~/dev/sonarqube_jeterrats_dev
+docker compose up -d
 ```
 
-Open `http://localhost:9000`, complete the SonarQube first-run setup, create a
-project key, and generate a project token. Then run scanner/import commands
-against the local host. Example import after analysis is available:
+The shared runtime binds SonarQube to `127.0.0.1:${SONAR_PORT:-9001}` by
+default, persists data in Docker volumes, and routes
+`sonarqube.jterrats.dev` through the Cloudflare Tunnel named
+`open-orchestra-sonar-local`.
+The local database password is a rotated strong value stored only in the shared
+infra `.env` file with owner-only file permissions; do not reset it to the
+default `sonar` password.
+
+```bash
+cd ~/dev/sonarqube_jeterrats_dev
+docker compose ps
+docker compose logs -f sonarqube
+docker compose logs -f cloudflared
+```
+
+This repository keeps only project-specific assets: `sonar-project.properties`,
+scanner scripts, import commands, and release evidence. That separation avoids
+one project accidentally stopping, deleting, changing ports, or rotating
+credentials for every other project using the same SonarQube server.
+
+Open `http://localhost:9001`, complete the SonarQube first-run setup if needed,
+create the `jterrats_open-orchestra` project key, and generate a project token.
+The scanner and `orchestra sonar import` both authenticate with the token as
+SonarQube Basic auth (`<token>:`), so the token must be valid for analysis and
+API reads on the target project. Then run scanner/import commands against the
+local or tunnel host. Example local scan:
+
+```bash
+SONAR_HOST_URL=http://localhost:9001 SONAR_TOKEN=<local-token> npm run sonar:scan:local
+```
+
+Example import after analysis is available:
 
 ```bash
 SONAR_TOKEN=<local-token> node bin/orchestra.js sonar import \
   --provider sonarqube-local \
-  --host-url http://localhost:9000 \
-  --project-key open-orchestra \
+  --host-url http://localhost:9001 \
+  --project-key jterrats_open-orchestra \
   --branch main \
   --task GH-368-LOCAL-SONARQUBE-PROVIDER \
   --json
 ```
 
-HTTP is accepted only for `sonarqube-local` on localhost. Self-hosted and cloud
-hosts must use HTTPS.
+HTTP is accepted only for `sonarqube-local` on localhost. Shared tunnel and
+cloud hosts must use HTTPS.
+
+### Private Cloudflare Access
+
+Do not expose SonarQube as a public DNS-only origin. If remote access is needed,
+use Cloudflare Tunnel with Cloudflare Access so `sonarqube.jterrats.dev` is an
+authenticated private entry point, not an open public service.
+
+Minimum Cloudflare setup:
+
+- Create a tunnel for this laptop or temporary VPS. The current tunnel name is
+  `open-orchestra-sonar-local`.
+- Route `sonarqube.jterrats.dev` to the Sonar service behind the tunnel. The DNS
+  record is a proxied CNAME to
+  `6fb60222-1427-4ca1-bf11-9e19375d39ff.cfargotunnel.com`.
+- Protect the hostname with a Cloudflare Access self-hosted application.
+- Restrict Access to named users, groups, or a short-lived maintainer policy.
+- Require MFA at the identity provider when possible.
+- Keep SonarQube itself authenticated; Cloudflare Access is an outer gate, not a
+  replacement for Sonar users and tokens.
+
+When the tunnel hostname is active, CI can use
+`SONAR_HOST_URL=https://sonarqube.jterrats.dev` only if the runner has an
+approved Access path. For GitHub-hosted runners, create a Cloudflare Access
+service token, add a Service Auth policy scoped to the SonarQube application,
+and configure `CF_ACCESS_CLIENT_ID` plus `CF_ACCESS_CLIENT_SECRET` as GitHub
+secrets. The workflow starts an ephemeral localhost proxy that injects those
+headers for SonarScanner and Orchestra import calls; browser login remains
+required for human access. The proxy readiness check validates SonarQube through
+the configured `SONAR_TOKEN` so private SonarQube instances that require
+authentication do not fail on anonymous health endpoints.
+
+If the Access service token secrets are not configured, the workflow keeps the
+normal direct Sonar URL behavior. Use local analysis evidence or a self-hosted
+runner when GitHub-hosted runners cannot access the private endpoint.
+
+### Self-Hosted Runner Mode
+
+For private local SonarQube on a laptop or low-cost VPS, prefer a self-hosted
+GitHub Actions runner over exposing the analyzer path through Cloudflare Access.
+Configure the repository or organization variable:
+
+```bash
+gh variable set SONAR_RUNNER --repo jterratsdev/open-orchestra --body self-hosted
+```
+
+Register the runner with dedicated labels so only the Sonar job can claim it.
+Do not include OS-specific labels in the workflow unless the Sonar runtime truly
+depends on that operating system; this keeps the same CI definition usable from
+a macOS laptop today and a Linux host later.
+
+```text
+self-hosted
+sonar
+local-sonar
+```
+
+For the current laptop setup, use the macOS ARM64 runner package and configure
+the runner with `--labels sonar,local-sonar`. GitHub automatically adds the
+platform labels such as `self-hosted`, `macOS`, and `ARM64`.
+
+Keep the shared SonarQube stack running locally:
+
+```bash
+cd ~/dev/sonarqube_jterrats_dev
+docker compose up -d
+```
+
+When `SONAR_RUNNER=self-hosted`, the workflow resolves SonarQube to
+`http://localhost:9001` unless `SONAR_LOCAL_HOST_URL` is set. This intentionally
+ignores `SONAR_HOST_URL`, so organization-level Cloudflare tunnel secrets do not
+pull local machine analysis back through Zero Trust. Cloudflare Access remains
+available for human remote browser usage and for GitHub-hosted runner access to
+private SonarQube only. The CI scan uses
+`continue-on-error` on the scanner step so Orchestra can still import and upload
+Sonar evidence when the quality gate fails; a final workflow step re-fails the
+job after evidence is captured.
+
+If the runner itself runs inside a container, `localhost` points at the runner
+container. In that case either run the runner process on the host, attach the
+runner container to the SonarQube Docker network, or set `SONAR_LOCAL_HOST_URL`
+to the host/network address that reaches SonarQube without Cloudflare.
 
 Sonar reads TypeScript through `tsconfig.sonar.json`, a standalone analyzer
 config that mirrors the build compiler options but lowers only the analyzer

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jterrats/open-orchestra",
-  "version": "1.0.7",
+  "version": "1.0.9",
   "type": "module",
   "workspaces": [
     "extensions/vscode-open-orchestra",
@@ -17,16 +17,20 @@
     "test:coverage": "npm run build && c8 --reporter=lcov --reports-dir coverage --exclude \"test/**\" --exclude \"e2e/**\" --exclude \"extensions/**/test/**\" --exclude \"dist/assets/**\" --exclude \"dist/web-console/**\" node --test test/**/*.js extensions/**/*.test.cjs",
     "test:e2e": "npm run build && npm run site:build && playwright test",
     "test:e2e:init": "node --test e2e/init-onboarding.test.js",
+    "test:e2e:runtime": "node --test e2e/runtime-manual-queue.test.js",
+    "test:e2e:runtime:ollama": "npm run build && node --test e2e/runtime-ollama-provider.test.js",
     "lint": "eslint . && prettier --check \"{bin,e2e,scripts,test,src}/**/*.js\" \"{site,web-console}/src/**/*.{css,js,jsx}\" \"{site,web-console}/*.{html,js,json}\" \"extensions/**/*.{cjs,json,md}\" \"src/**/*.ts\" \"*.{js,json}\"",
     "format": "prettier --write \"{bin,e2e,scripts,test,src}/**/*.js\" \"{site,web-console}/src/**/*.{css,js,jsx}\" \"{site,web-console}/*.{html,js,json}\" \"extensions/**/*.{cjs,json,md}\" \"src/**/*.ts\" \"*.{js,json}\"",
     "secret-scan": "node scripts/secret-scan.js",
     "security:audit": "node scripts/security-audit.js",
+    "architecture:inventory": "npm run build && node scripts/architecture-debt-inventory.js",
     "duplicates": "jscpd --config .jscpd.json",
     "validate:workflow": "node scripts/validate-workflow.js",
     "release:matrix": "node scripts/release-test-matrix.js",
     "performance:bench": "npm run build && node scripts/performance-benchmark.js",
     "precommit": "npm run lint && npm run typecheck && npm run secret-scan && npm run security:audit && npm test && npm run validate:workflow",
     "prepack": "npm run build",
+    "sonar:scan:local": "sonar-scanner -Dsonar.host.url=${SONAR_HOST_URL:-http://localhost:9001}",
     "hooks:install": "git config core.hooksPath .githooks",
     "build:web": "npm run build:web:legacy && npm run build:web:react",
     "build:web:legacy": "esbuild src/web-console-client.js --bundle --format=esm --platform=browser --target=es2022 --outfile=dist/assets/web-console.js",

package/rules/delivery-quality-gates.mdc

@@ -19,6 +19,10 @@ Development work is not complete when code compiles. Every implementation must m
 
 - QA receives the Developer handoff before release approval.
 - QA must produce a test plan covering acceptance criteria, regression areas, edge cases, data setup, and environment assumptions.
+- QA must block test planning when acceptance criteria are fragmented, non-verifiable, or only role/phase headings. Return those findings to PO/BA before release evidence is generated.
+- QA plans must include an AC-to-evidence matrix with expected observable result, actual result, artifact/command, and pass/fail/deferred status for each acceptance criterion.
+- QA must validate that the planned tests exercise the actual risk, not a weaker surrogate. For scope/split, handoff, workflow, runtime, queueing, failback, or release-gate bugs, the test data must create the condition that should trigger the guardrail.
+- QA must block when the plan substitutes a weaker surface for the requested behavior, such as browser smoke for workflow/CLI behavior, command execution without stdout/files/events checks, or API response checks without receiver-side effects for integrations.
 - QA must execute or explicitly defer each test case with a reason.
 - QA findings must include severity, reproduction steps, expected result, actual result, and evidence.
 - QA execution must be reviewable through a sprint-review-style evidence demo before release approval. Analyst/BA compares the executed evidence against the GitHub issue, user story, acceptance criteria, and Orchestra task; Architect reviews whether the tests cover architecture contracts, boundaries, integrations, data flow, and risk areas.
@@ -29,6 +33,8 @@ Development work is not complete when code compiles. Every implementation must m
 
 - QA and Developer must identify which manual checks should become automated tests.
 - Prefer Playwright for browser-based E2E, smoke, and regression flows.
+- Automation for every product surface must use isolated deterministic fixtures: web, mobile, desktop, CLI, API, integrations, workflow/runtime, installer, data, and generated-artifact flows. Use the configured E2E fixture path, sandbox org, emulator, container, device farm, or test environment when the user/project provides one; otherwise default local disposable fixtures to `/tmp`.
+- Automation must assert state transitions and final artifacts. For workflow-style systems, assert the automaton path: valid transition, blocked transition, loop/return transition, resume, and final release transition when applicable. For UI/mobile/API/integration systems, assert the user-visible state, device/responsive behavior, API contract, receiver-side side effect, persisted data, async job/event, or external sandbox state that proves the acceptance criterion.
 - Use Page Object pattern for Playwright suites. Selectors belong in page objects or stable test helpers, not scattered through test bodies.
 - Automated tests must be deterministic and avoid real network, clock, or randomness unless controlled by fixtures, mocks, or seeded data.
 

package/rules/devops-tooling.mdc

@@ -30,6 +30,7 @@ DevOps decisions must cover deployability, scalability, downtime strategy, obser
 - Prefer managed services when they reduce operational risk without creating unacceptable lock-in or cost exposure.
 - Record tool choices and major operational trade-offs in an ADR when they affect long-term operations.
 - CI/CD, IaC, runbooks, and operational scripts that repeat command matrices, provider lists, environment maps, or resource collections must load the `collection-standards` skill.
+- Local Docker stacks must publish ports on `127.0.0.1` unless the task explicitly requires LAN/public access and Security has accepted the risk. Databases, caches, queues, admin UIs, metrics backends, and Docker socket access are private by default.
 
 ## Scalability
 - Define expected traffic, data volume, concurrency, growth assumptions, and bottlenecks.

package/rules/security-guardrails.mdc

@@ -35,3 +35,6 @@ These are non-negotiable. Violations must be fixed before code review.
 
 - For databases, encryption, IaC, environment segregation, secrets management, scalability, and vulnerability management, see **infra-data-encryption.mdc**.
 - Production networked services must consider TLS 1.2+, certificate management, HSTS where applicable, secure cookies, least privilege, and secret rotation.
+- Local Docker Compose, dev servers, databases, caches, observability tools, and admin UIs must bind published ports to `127.0.0.1` by default.
+- Binding to `0.0.0.0`, `[::]`, or a LAN interface is a security exception. It needs a linked task, explicit rationale, no default credentials, and a time-bounded review.
+- Never expose Redis, Postgres, admin consoles, Docker socket, or internal APIs on public/LAN interfaces unless Security approves the exact use case and compensating controls.

package/rules/testing-discipline.mdc

@@ -40,6 +40,11 @@ alwaysApply: true
 - Use the Page Object pattern for UI tests. Selectors live in page objects, not test bodies.
 - Tag tests by speed/scope (`@smoke`, `@regression`) so CI can run fast feedback loops.
 - Capture evidence for E2E failures with traces, screenshots, or videos when supported by the framework.
+- E2E tests for any product surface must run against isolated disposable fixtures: web apps, mobile apps, desktop apps, CLI, APIs, integrations, workflows, runtimes, installers, file-system flows, data pipelines, and generated artifacts. Use the user/project-configured E2E fixture path, device farm, sandbox org, emulator, container, or test environment when one is explicitly provided; otherwise default local disposable fixtures to `/tmp`. Each test creates its own users/data/project/tasks/roles/acceptance criteria/expected artifacts as applicable; never rely on the developer's current repo state as the tested product state.
+- QA must choose fixture data that reproduces the risk being validated. If the acceptance criterion is about oversized stories, split decisions, phase returns, queueing, or specialist roles, the E2E must create an oversized/cross-cutting task with enough paths, roles, acceptance criteria, and risk signals to force that behavior.
+- Every E2E must assert the resulting state, not only that the action executed. Validate UI-visible state, navigation, accessibility, device/responsive behavior, API contracts, receiver-side integration state, database/mock records, files, events, handoff contents, stdout/stderr, exit code, push notifications, background jobs, generated artifacts, or release-readiness output as applicable.
+- For workflow automata, E2E must validate state transitions and loops: allowed transition, blocked transition, return-to-dev/architect/BA when findings fail, and resume behavior after the corrective state is satisfied.
+- Include negative and edge scenarios when they are the reason for the change. A happy-path smoke is insufficient for bugs involving guardrails, split detection, security boundaries, QA failback, or release blocking.
 - QA, SDET, Developer, BA, Architect, and Release work that produces or reviews evidence must load the `qa-evidence-pack` skill when it involves acceptance criteria coverage, Playwright/browser artifacts, CLI stdout/stderr, API contracts, integration side effects, screenshots, visual diffs, or annotated defect evidence.
 - Keep large screenshots, videos, traces, logs, API payloads, and visual diffs as files. Summarize them in a compact evidence report so agents do not consume context with raw artifacts.
 
@@ -47,6 +52,10 @@ alwaysApply: true
 
 - Developer must provide QA with test commands run, pass/fail results, covered scenarios, and known gaps.
 - QA must produce a test plan before release approval and map every acceptance criterion to automated, manual, contract/mock, or deferred evidence.
+- QA must reject fragmented acceptance criteria before planning tests. Role names, phase names, headings, and partial clauses are not executable criteria and must return to PO/BA for rewrite.
+- QA plans must include an AC-to-evidence matrix with: acceptance criterion, test type, fixture/setup, command or artifact, expected observable result, actual result, and pass/fail/deferred status.
+- QA must challenge weak tests before approving: if the test fixture is too small to exercise the bug, uses only the happy path, does not create the expected failure/return condition, or does not inspect the artifact/state that proves the acceptance criterion, QA must block or request changes.
+- QA must reject weaker surrogate tests. Validate the affected product surface directly: workflow/CLI/API/integration/generated artifact/mobile/desktop/data behavior cannot be approved only by a generic browser smoke or command-executed check.
 - QA evidence must validate observable outcomes, not only execution. CLI checks assert exit code, stdout/stderr, files, events, or final state; browser checks assert visible user-facing state; API checks assert response contract and side effects; integration checks assert sandbox/mock/contract/webhook/event/log outcomes or defer with owner and rationale.
 - Evidence summaries or metadata must name the covered acceptance criterion or explicitly state that all acceptance criteria are covered. Smoke and regression checks are useful but do not count as acceptance coverage unless they map to an acceptance criterion.
 - Visual/UI/diagram defect evidence must include source or expected image when available, actual screenshot/render, diff image when practical, and an annotated screenshot for ambiguous failures. Use red boxes for broken bounds/overlap, orange arrows for wrong connectors or flow, yellow translucent areas for excess spacing, blue guide lines for alignment, and short defect labels.