froid-plane-test-architecture-enterprise 1.23.4

package/SECURITY.md

@@ -0,0 +1,85 @@
+# Security Policy
+
+## Supported Versions
+
+We release security patches for the following versions:
+
+| Version  | Supported          |
+| -------- | ------------------ |
+| Latest   | :white_check_mark: |
+| < Latest | :x:                |
+
+We recommend always using the latest version of Froid Plane to ensure you have the most recent security updates.
+
+## Reporting a Vulnerability
+
+We take security vulnerabilities seriously. If you discover a security issue, please report it responsibly.
+
+### How to Report
+
+**Do NOT report security vulnerabilities through public GitHub issues.**
+
+Instead, please report them via one of these methods:
+
+1. **GitHub Security Advisories** (Preferred): Use [GitHub's private vulnerability reporting](https://github.com/cordya-ai/froid-plane/security/advisories/new) to submit a confidential report.
+
+2. **Discord**: Contact a maintainer directly via DM on our [Discord server](https://discord.gg/gk8jAdXWmj).
+
+### What to Include
+
+Please include as much of the following information as possible:
+
+- Type of vulnerability (e.g., prompt injection, path traversal, etc.)
+- Full paths of source file(s) related to the vulnerability
+- Step-by-step instructions to reproduce the issue
+- Proof-of-concept or exploit code (if available)
+- Impact assessment of the vulnerability
+
+### Response Timeline
+
+- **Initial Response**: Within 48 hours of receiving your report
+- **Status Update**: Within 7 days with our assessment
+- **Resolution Target**: Critical issues within 30 days; other issues within 90 days
+
+### What to Expect
+
+1. We will acknowledge receipt of your report
+2. We will investigate and validate the vulnerability
+3. We will work on a fix and coordinate disclosure timing with you
+4. We will credit you in the security advisory (unless you prefer to remain anonymous)
+
+## Security Scope
+
+### In Scope
+
+- Vulnerabilities in Froid Plane core framework code
+- Security issues in agent definitions or workflows that could lead to unintended behavior
+- Path traversal or file system access issues
+- Prompt injection vulnerabilities that bypass intended agent behavior
+- Supply chain vulnerabilities in dependencies
+
+### Out of Scope
+
+- Security issues in user-created custom agents or modules
+- Vulnerabilities in third-party AI providers (Claude, GPT, etc.)
+- Issues that require physical access to a user's machine
+- Social engineering attacks
+- Denial of service attacks that don't exploit a specific vulnerability
+
+## Security Best Practices for Users
+
+When using Froid Plane:
+
+1. **Review Agent Outputs**: Always review AI-generated code before executing it
+2. **Limit File Access**: Configure your AI IDE to limit file system access where possible
+3. **Keep Updated**: Regularly update to the latest version
+4. **Validate Dependencies**: Review any dependencies added by generated code
+5. **Environment Isolation**: Consider running AI-assisted development in isolated environments
+
+## Acknowledgments
+
+We appreciate the security research community's efforts in helping keep Froid Plane secure. Contributors who report valid security issues will be acknowledged in our security advisories.
+
+---
+
+Thank you for helping keep Froid Plane and our community safe.

package/cli/examples/README.md

@@ -0,0 +1,53 @@
+# CLI examples
+
+`pr-test-review.yml` is the full annotated template, start there. These two cover real adaptations.
+
+## A central reusable-workflows repo
+
+Some orgs centralize CI logic: one repo owns `workflow_call` workflows, consuming repos call them. If that repo already has a comment-triggered `@claude` reviewer (opt-in, advisory, fired from `issue_comment`), don't graft TEA onto it. TEA needs to run on every PR automatically and gate merges, a different job than an on-demand advisory review. Add it as its own pair, same shape as whatever pattern you already use:
+
+- **Reusable workflow** (central repo, e.g. `rwf-tea-test-review.yml`): copy the `review` and `comment` jobs from `pr-test-review.yml` into a `workflow_call` workflow. Promote `--min-score`, `--max-critical`, `--min-files`, and the pinned `TEA_VERSION` to `inputs:`, and the Anthropic key to a required secret.
+- **Caller** (each consuming repo, or the central repo itself for dogfooding): a thin `pull_request`-triggered workflow that does `uses: <org>/<central-repo>/.github/workflows/rwf-tea-test-review.yml@<ref>`.
+
+Keep the trigger on `pull_request`. That's what makes it a required check: it runs automatically, no one has to remember to summon it.
+
+```yaml
+# rwf-tea-test-review.yml (central repo): only what differs from pr-test-review.yml
+on:
+  workflow_call:
+    inputs:
+      min_score: { type: number, required: false, default: 80 }
+    secrets:
+      anthropic_api_key:
+        required: true
+jobs:
+  review:
+    # ...same steps as pr-test-review.yml's `review` job...
+    run: tea-test-review --base "$BASE_REF" --min-score ${{ inputs.min_score }} --agent claude --skill-root "$GITHUB_WORKSPACE/_froid/tea/workflows/testarch/froid-testarch-test-review" --output test-review.md --json test-review.json
+```
+
+```yaml
+# .github/workflows/tea-test-review.yml (caller, per consuming repo)
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+jobs:
+  tea-test-review:
+    uses: <org>/<central-repo>/.github/workflows/rwf-tea-test-review.yml@v1
+    with:
+      min_score: 80
+    secrets:
+      anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+```
+
+## A repo already using a third-party review bot (CodeRabbit, etc.)
+
+Third-party review bots are configured entirely through their own SaaS-side file, there's no hook in there for invoking an external CLI. Leave that file alone. Add a new, independent `pull_request`-triggered workflow (same shape as `pr-test-review.yml`) next to it.
+
+The two don't compete. Check whether the bot's config sets a required commit status or a request-changes gate. If it doesn't (most default/free configs are advisory-only, commenting on the diff without blocking merges), TEA test-review can be the actual required check that config deliberately leaves open, scoped specifically to test quality rather than the whole diff.
+
+Adjust flags to your layout, for example a monorepo with tests outside the default directory:
+
+```yaml
+run: tea-test-review --base "$BASE_REF" --test-dir playwright --min-score 80 --agent claude --skill-root "$GITHUB_WORKSPACE/_froid/tea/workflows/testarch/froid-testarch-test-review" --output test-review.md --json test-review.json
+```

package/cli/examples/pr-test-review.yml

@@ -0,0 +1,315 @@
+# tea-test-review — a starting template for a required test-review gate.
+#
+# Two jobs: `review` runs the headless TEA test review against the PR's
+# changed test files and uploads the report artifacts; `comment` publishes the
+# outcome as a single upserted PR comment. Make the `review` job a required
+# status check to gate merges on the review verdict.
+#
+# Prerequisites:
+#   - An ANTHROPIC_API_KEY repository secret (this template's review step uses
+#     --agent claude, executed by the claude CLI installed below). --agent
+#     codex is also supported (OPENAI_API_KEY instead) — see the --agent flag
+#     in docs/reference/tea-test-review-cli.md and swap the installed package
+#     and secret accordingly.
+#   - A vetted version of the froid-plane-test-architecture-enterprise npm
+#     package, pinned exactly (see TEA_VERSION below).
+#
+# Honest limitations — read before requiring this check:
+#   - This workflow cannot be the sole required check for repositories that
+#     accept fork pull requests: forks receive no secrets by design, so the
+#     fork guard below skips the review for them and the gate never runs.
+#     Fork coverage needs a separate privileged design (for example a
+#     pull_request_target workflow with strict controls), which is out of
+#     scope for this template.
+#   - The review skill is installed from the pinned npm package below, not
+#     from the PR checkout, and the run step pins it explicitly with
+#     --skill-root — the CLI never probes the PR checkout for the reviewer,
+#     closing the vendored-checkout trust gap for this shipped path. The
+#     residual trust assumption: you are trusting the pinned version as
+#     published on the npm registry — vet that version once, pin it exactly,
+#     and bump the pin deliberately.
+#
+# Gate-policy flags (all optional; tune to your gate policy):
+#   --min-score <n>            fail below a quality-score floor (0-100); tune to your gate policy
+#   --max-critical <n>         fail above a Critical-violation cap (default: no cap); tune to your gate policy
+#   --min-files <n>            fail when fewer than n files were reviewed (default 1); tune to your gate policy
+#   --waive/--waive-until      record a time-boxed waiver: any verdict failure exits 0 with a WAIVED
+#                              banner and waived fields in the JSON (exit 2/3 are never waivable);
+#                              tune to your gate policy
+name: TEA Test Review
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+
+# Default deny; each job grants exactly the permissions it needs.
+permissions: {}
+
+jobs:
+  review:
+    name: Headless test review
+    runs-on: ubuntu-latest
+    # Forks receive no secrets, so the review cannot run for them; skip
+    # honestly instead of failing (see the header comment).
+    if: github.event.pull_request.head.repo.full_name == github.repository
+    permissions:
+      contents: read
+    outputs:
+      verdict: ${{ steps.verdict.outputs.verdict }}
+    env:
+      # Single source of truth for both install steps below, so bumping the
+      # pin can't update one and silently leave the other on the old version.
+      # 1.20.0 is the first release that ships the tea-test-review bin; every
+      # earlier version publishes the skill with an empty `bin`. An exact
+      # version is required either way, because the tar step below reconstructs
+      # the tarball filename from it.
+      TEA_VERSION: 1.20.0
+    steps:
+      - name: Checkout (full history for the PR diff)
+        uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+          # The review only reads the tree; never hand the PR checkout a
+          # credentials-bearing git config.
+          persist-credentials: false
+
+      - name: Setup Node
+        uses: actions/setup-node@v6
+        with:
+          node-version: 22
+
+      - name: Install the review skill from the pinned package
+        # Deterministic, non-interactive install: the skill content the CLI
+        # discovers under _froid/ comes from the vetted package tarball, never
+        # from the PR checkout.
+        run: |
+          npm pack "froid-plane-test-architecture-enterprise@${TEA_VERSION}"
+          tar -xzf "froid-plane-test-architecture-enterprise-${TEA_VERSION}.tgz"
+          mkdir -p _froid/tea/workflows/testarch
+          cp -R package/src/workflows/testarch/froid-testarch-test-review _froid/tea/workflows/testarch/froid-testarch-test-review
+
+      - name: Install the pinned CLI and agent
+        env:
+          # Both pinned: the reviewer's control plane must be the exact code
+          # you vetted, not whatever `latest` resolves to when a PR lands.
+          CLAUDE_CODE_VERSION: 2.1.220
+        run: npm install --global "froid-plane-test-architecture-enterprise@${TEA_VERSION}" "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}"
+
+      - name: Run headless test review
+        id: run-review
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          BASE_REF: origin/${{ github.base_ref }}
+        # No ${{ }} expansion inside run text: the base ref arrives via the
+        # environment so a crafted branch name cannot inject shell.
+        # --skill-root pins the reviewer to the pinned npm copy installed
+        # above: the CLI skips probing the PR checkout for the skill, so a PR
+        # that edits its own vendored _froid/ copy cannot rewrite the reviewer.
+        #
+        # TEA config: this job installs the skill from a tarball rather than
+        # running the interactive installer, so _froid/tea/config.yaml does not
+        # exist and the module defaults apply (Playwright Utils on, Pact off,
+        # no Pact MCP). Those defaults are stated in the prompt, so the run is
+        # deterministic either way. If this repository uses contract testing,
+        # say so or the review loads the generic contract-testing fragment
+        # instead of the pactjs-utils set:
+        #   --use-pactjs-utils [--pact-mcp mcp]
+        # Committing _froid/tea/config.yaml works too; the flags win over it.
+        run: tea-test-review --base "$BASE_REF" --agent claude --skill-root "$GITHUB_WORKSPACE/_froid/tea/workflows/testarch/froid-testarch-test-review" --output test-review.md --json test-review.json
+
+      - name: Upload review artifacts
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: tea-test-review
+          path: |
+            test-review.md
+            test-review.json
+
+      - name: Export the verdict as a job output
+        id: verdict
+        if: always()
+        env:
+          # The CLI exit code is the verdict (it already folds in --fail-on,
+          # --min-score and deletions-only handling), so the step outcome is
+          # the faithful pass/fail signal; the JSON carries the detail.
+          REVIEW_OUTCOME: ${{ steps.run-review.outcome }}
+        run: |
+          verdict="failed"
+          if [ "$REVIEW_OUTCOME" = "success" ]; then
+            verdict="passed"
+          fi
+          echo "verdict=$verdict" >> "$GITHUB_OUTPUT"
+
+  comment:
+    name: Comment the outcome
+    needs: review
+    # Always run after the review (success or failure) so a failed review is
+    # still reported; never run for forks (nothing was reviewed there).
+    if: always() && github.event.pull_request.head.repo.full_name == github.repository
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: write
+    steps:
+      # No PR checkout and no ANTHROPIC_API_KEY in this job: it only reads
+      # the uploaded artifacts and writes a PR comment.
+      - name: Download review artifacts
+        id: download
+        uses: actions/download-artifact@v4
+        # A missing artifact is reported below as an infrastructure failure,
+        # not raised as a step failure.
+        continue-on-error: true
+        with:
+          name: tea-test-review
+
+      - name: Find-and-update the review comment
+        uses: actions/github-script@v7
+        env:
+          DOWNLOAD_OUTCOME: ${{ steps.download.outcome }}
+          REVIEW_RESULT: ${{ needs.review.result }}
+          REVIEW_VERDICT: ${{ needs.review.outputs.verdict }}
+        with:
+          # Kept in sync by hand with .github/workflows/tea-test-review.yaml's
+          # "Find-and-update the review comment" step: same comment-building
+          # logic, duplicated because this file is a standalone copy-paste
+          # template for other repos and cannot depend on that one.
+          script: |
+            const fs = require("fs");
+            const marker = "<!-- tea-test-review -->";
+            const artifactsUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}#artifacts`;
+            // GitHub caps a comment body at 65536 chars; this leaves headroom
+            // for the digest and wrapper around the inlined report.
+            const MAX_INLINE_REPORT_CHARS = 40000;
+
+            let body = null;
+            const artifactAvailable = process.env.DOWNLOAD_OUTCOME === "success" && fs.existsSync("test-review.json");
+            if (artifactAvailable) {
+              try {
+                const verdict = JSON.parse(fs.readFileSync("test-review.json", "utf8"));
+                if (verdict.skipped) {
+                  body = [
+                    marker,
+                    "## TEA Test Review: skipped",
+                    "",
+                    `${verdict.reason ?? "No changed test files in this PR"}.`,
+                    "",
+                    `[Review job artifacts](${artifactsUrl})`,
+                  ].join("\n");
+                } else {
+                  const counts = verdict.violations ?? {};
+                  const violations = `${counts.critical ?? 0} Critical / ${counts.high ?? 0} High / ${counts.medium ?? 0} Medium / ${counts.low ?? 0} Low`;
+                  const weaknesses = (verdict.keyWeaknesses ?? []).slice(0, 3);
+                  // Named, not just counted: a finding that cites "line 200" is
+                  // unattributable on a PR touching more than one test file.
+                  const reviewed = Array.isArray(verdict.reviewedFiles) ? verdict.reviewedFiles : [];
+                  const markdownCodeSpan = (value) => {
+                    const text = String(value);
+                    const longestBacktickRun = Math.max(0, ...(text.match(/`+/g) ?? []).map((run) => run.length));
+                    const fence = "`".repeat(longestBacktickRun + 1);
+                    const padding = text.startsWith("`") || text.endsWith("`") ? " " : "";
+                    return `${fence}${padding}${text}${padding}${fence}`;
+                  };
+
+                  const lines = [
+                    marker,
+                    `## TEA Test Review: ${verdict.recommendation}`,
+                    "",
+                    `- **Quality score**: ${verdict.qualityScore ?? "n/a"}/100`,
+                    `- **Recommendation**: ${verdict.recommendation}`,
+                    `- **Violations**: ${violations}`,
+                    `- **Reviewed files**: ${reviewed.length}`,
+                    ...reviewed.slice(0, 10).map((f) => `  - ${markdownCodeSpan(f)}`),
+                    ...(reviewed.length > 10 ? [`  - … and ${reviewed.length - 10} more`] : []),
+                  ];
+                  // A score with no attribution cannot be compared to another
+                  // score. The verdict JSON already carries the resolved agent
+                  // and model for that reason; without this line the human
+                  // surface loses it, and 82 from one vendor reads as
+                  // interchangeable with 82 from another.
+                  const reviewer = [verdict.agent, verdict.model].filter((v) => typeof v === "string" && v.length > 0);
+                  if (reviewer.length > 0) {
+                    lines.push(`- **Reviewer**: ${reviewer.join(" / ")}`);
+                  }
+                  if (weaknesses.length > 0) {
+                    lines.push("", "**Key weaknesses**:", ...weaknesses.map((w) => `- ${w}`));
+                  }
+                  lines.push("");
+
+                  // Inline the full report (not just the digest above) so a
+                  // reviewer can paste it straight into an AI coding agent to
+                  // apply the fixes, no artifact download required. Falls back
+                  // to the artifact link alone if the report is too large or
+                  // missing from the download.
+                  const reportText = fs.existsSync("test-review.md") ? fs.readFileSync("test-review.md", "utf8") : null;
+                  if (reportText && reportText.length <= MAX_INLINE_REPORT_CHARS) {
+                    // The report opens with `---\nkey: value\n---`, and markdown
+                    // reads that closing `---` as a setext underline, so the
+                    // resume-state block renders as an <h2> above the report's
+                    // own title. Fencing keeps it verbatim for the paste path at
+                    // code-block size. Only a closing block at the very start is
+                    // touched, and never one whose values contain a fence.
+                    const fenceFrontmatter = (text) => {
+                      const match = /^---\r?\n([\s\S]*?)\r?\n---(\r?\n|$)/.exec(text);
+                      if (!match || match[1].includes("```")) return text;
+                      return `\`\`\`yaml\n${match[1]}\n\`\`\`${match[2]}${text.slice(match[0].length)}`;
+                    };
+                    // A reviewed file's own content can contain a literal
+                    // </details>; inserting a zero-width space breaks that as
+                    // an HTML closing tag while leaving the visible text
+                    // effectively unchanged, so it cannot end this block early
+                    // and spill raw markdown into the rest of the comment.
+                    const safeReportText = fenceFrontmatter(reportText).replace(/<\/details>/gi, "<\u200B/details>");
+                    lines.push(
+                      "<details>",
+                      "<summary>Full report (paste into your AI coding agent to apply the fixes)</summary>",
+                      "",
+                      safeReportText,
+                      "",
+                      "</details>",
+                      "",
+                      `[Verdict JSON](${artifactsUrl})`,
+                    );
+                  } else {
+                    lines.push(`[Full report and verdict JSON](${artifactsUrl})`);
+                  }
+                  body = lines.join("\n");
+                }
+              } catch {
+                body = null;
+              }
+            }
+            if (body === null) {
+              body = [
+                marker,
+                "## TEA Test Review: infrastructure failure",
+                "",
+                "The review job did not produce a readable verdict artifact.",
+                "This is **not** a review verdict — treat the gate as broken, not as approved tests.",
+                "",
+                `Review job result: ${process.env.REVIEW_RESULT} (verdict output: ${process.env.REVIEW_VERDICT || "n/a"}).`,
+                `[Review job artifacts](${artifactsUrl})`,
+              ].join("\n");
+            }
+
+            const { data: comments } = await github.rest.issues.listComments({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number,
+              per_page: 100,
+            });
+            const existing = comments.find((comment) => comment.body && comment.body.includes(marker));
+            if (existing) {
+              await github.rest.issues.updateComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                comment_id: existing.id,
+                body,
+              });
+            } else {
+              await github.rest.issues.createComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                issue_number: context.issue.number,
+                body,
+              });
+            }