forgedock 1.0.20 → 1.0.21

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -55,7 +55,7 @@ ForgeDock is **not another AI coding agent.** It's a set of prompt-engineered co
55
55
  | **Full-lifecycle automation** | `/work-on #42` — investigates the issue, architects a fix, builds it, runs quality gates, opens a PR, and reviews it. You click merge. |
56
56
  | **Persistent agent memory** | Structured `FORGE:` annotations on GitHub issues/PRs survive context resets and session boundaries. Agents never start blind. |
57
57
  | **9 specialist review agents** | Security, billing, database, concurrency, auth, frontend, API, performance, infrastructure — every PR gets domain-expert review. |
58
- | **Cross-issue knowledge graph** | Agent fixing issue #43 reads the investigation from #42 and applies the known pattern — no re-investigation. |
58
+ | **Institutional memory that compounds** | Agent fixing issue #43 reads the investigation from #42 and applies the known pattern — no re-investigation. |
59
59
  | **Self-improving pipeline** | Review agents learn from past findings — recurring patterns automatically become new quality gate checks. |
60
60
  | **Parallel orchestration** | `/orchestrate` decomposes milestones into waves and runs `/work-on` on each in parallel. |
61
61
 
@@ -223,11 +223,26 @@ Removes all ForgeDock command symlinks from `~/.claude/commands/`. Your `forge.y
223
223
 
224
224
  ---
225
225
 
226
+ ## FAQ — How is ForgeDock different from X?
227
+
228
+ **vs. DeepWiki (Devin)** — DeepWiki generates a read-only wiki from your codebase once. ForgeDock's FORGE annotations are written by pipeline agents during every issue run — capturing root causes, architectural decisions, and review findings as they happen, not as a snapshot of static code.
229
+
230
+ **vs. AGENTS.md / CLAUDE.md** — These files are written and maintained by you. They capture conventions and rules. ForgeDock annotations are written automatically by pipeline agents and capture execution history — what bugs were confirmed, what fixes were shipped, what review findings recurred. Use both: CLAUDE.md for conventions, ForgeDock for institutional memory.
231
+
232
+ **vs. Cursor Memories** — Cursor Memories are scoped to the Cursor IDE's backend and don't survive a Claude Code context reset. ForgeDock annotations live on GitHub — any agent, any tool, any human can read them. When Claude's context window compacts mid-session, a ForgeDock agent queries GitHub and resumes exactly where it left off.
233
+
234
+ **vs. Devin Knowledge Base** — Devin's KB is proprietary, opaque, and vendor-locked. ForgeDock context is plain GitHub issue comments — readable, editable, and queryable with `gh api`. No vendor lock-in, no black box.
235
+
236
+ [Full comparison →](docs/comparison.md)
237
+
238
+ ---
239
+
226
240
  ## Documentation
227
241
 
228
242
  - [Getting Started in 5 Minutes](docs/site/getting-started.md) — install, configure, first pipeline run
229
243
  - [How the Knowledge Graph Works](docs/site/how-it-works.md) — FORGE annotations, context relay, compaction resilience
230
244
  - [ForgeDock vs. Manual Workflows](docs/site/vs-manual-workflows.md) — structured pipelines vs. ad-hoc prompting
245
+ - [ForgeDock vs. DeepWiki, AGENTS.md, and Cursor Memories](docs/comparison.md) — structural differences between context tools
231
246
  - [FORGE Annotation Protocol](docs/site/forge-annotation-protocol.md) — open standard spec for AI context passing
232
247
  - [Command Reference](docs/site/command-reference.md) — all 25+ commands with usage and examples
233
248
 
@@ -0,0 +1,469 @@
1
+ ---
2
+ description: Stack-aware CI gap detection — audits a project's GitHub Actions workflows against its declared tech stack and files issues for missing config validation checks
3
+ argument-hint: [--repo <prefix>] [--dry-run]
4
+ ---
5
+ <!-- SPDX-FileCopyrightText: Copyright (c) RapierCraft Studios -->
6
+ <!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
7
+
8
+ # /ci-audit — Stack-Aware CI Gap Detection
9
+
10
+ **Input**: $ARGUMENTS
11
+
12
+ You are the CI hygiene auditor. Given a project's declared tech stack (from `forge.yaml`), determine what config validation tools exist for each stack component and audit whether the project's GitHub Actions workflows include those validation steps. File GitHub issues for any gaps found.
13
+
14
+ This command enforces ForgeDock's separation-of-concerns principle:
15
+
16
+ | Layer | What runs | Tokens | Example |
17
+ |-------|-----------|--------|---------|
18
+ | **CI (automated)** | Deterministic tool validators | Zero | `traefik validate`, `nginx -t`, `kubectl --dry-run` |
19
+ | **ForgeDock (autonomous)** | Reasoning about architecture, logic, security | Yes | "Is this timeout chain coherent?", "Does this wiring match the deploy script?" |
20
+ | **ForgeDock CI hygiene** | Audit that CI has the right checks for the stack | Once | "Project uses Traefik v3 but CI has no `traefik validate` step → file issue" |
21
+
22
+ Config validation runs in CI (zero tokens). ForgeDock's role is to ensure the right CI checks exist — not to run them itself. <!-- Added: forge#1104 -->
23
+
24
+ This command is designed to be run:
25
+ - **Once after project setup** to establish CI hygiene baseline
26
+ - **After stack changes** (adding Traefik, upgrading K8s, adopting Terraform)
27
+ - **As part of `/autopilot`** periodic review cycle
28
+ - **After a production incident** caused by a structurally invalid config
29
+
30
+ This is NOT a PR review — it does not approve or block. It creates issues for each CI gap found.
31
+
32
+ **Agent model policy**: Default `model: "sonnet"`. If Sonnet is rate-limited, fall back to `model: "opus"`.
33
+ **NEVER use plan mode (EnterPlanMode).**
34
+
35
+ ---
36
+
37
+ ## Config Preamble
38
+
39
+ Before executing any phase, read `forge.yaml` to resolve project references:
40
+
41
+ ```bash
42
+ GH_REPO=$(yq '.project.owner + "/" + .project.repo' forge.yaml 2>/dev/null)
43
+ GH_FLAG="-R ${GH_REPO}"
44
+ REPO_PATH=$(yq '.paths.root' forge.yaml 2>/dev/null || echo ".")
45
+ TECH_STACK=$(yq '.review.tech_stack // ""' forge.yaml 2>/dev/null || echo "")
46
+ TOOL_VERSIONS=$(yq '.review.tool_versions // {}' forge.yaml 2>/dev/null || echo '{}')
47
+ ```
48
+
49
+ **Optional `forge.yaml` fields used by this command**:
50
+
51
+ ```yaml
52
+ review:
53
+ tech_stack: "Traefik v3, nginx 1.25, Terraform 1.6, Kubernetes 1.28" # Free-form stack description
54
+ tool_versions: # Structured version declarations
55
+ traefik: "3"
56
+ nginx: "1.25"
57
+ terraform: "1.6"
58
+ kubernetes: "1.28"
59
+ docker_compose: "2"
60
+
61
+ # Optional: override which CI workflows to audit (default: all files in .github/workflows/)
62
+ ci_audit:
63
+ workflow_paths:
64
+ - ".github/workflows/ci.yml"
65
+ - ".github/workflows/deploy-production.yml"
66
+ dry_run: false # When true: print findings without creating issues
67
+ ```
68
+
69
+ If `forge.yaml` is missing: stop and tell the user to run `npx forgedock init` to generate it.
70
+
71
+ ---
72
+
73
+ ## Phase 1: Discover Stack Components
74
+
75
+ ### 1A: Parse declared stack
76
+
77
+ Extract stack components from `forge.yaml → review.tech_stack` (free-form) and `forge.yaml → review.tool_versions` (structured).
78
+
79
+ ```bash
80
+ echo "=== Stack from forge.yaml ==="
81
+ echo "TECH_STACK: $TECH_STACK"
82
+ echo "TOOL_VERSIONS: $TOOL_VERSIONS"
83
+ ```
84
+
85
+ **Parse `TECH_STACK` (free-form text) for known tool names**:
86
+
87
+ Scan the `TECH_STACK` string for these tool keywords (case-insensitive):
88
+
89
+ | Keyword pattern | Tool | Validator(s) |
90
+ |-----------------|------|-------------|
91
+ | `traefik` | Traefik | `traefik validate`, `traefik healthcheck` |
92
+ | `nginx` | nginx | `nginx -t`, `nginx -T` |
93
+ | `kubernetes\|k8s\|kubectl` | Kubernetes | `kubectl apply --dry-run=client`, `kubeval`, `kustomize build \| kubectl apply --dry-run=client` |
94
+ | `terraform` | Terraform | `terraform validate`, `terraform plan`, `tflint` |
95
+ | `helm` | Helm | `helm lint`, `helm template \| kubectl apply --dry-run=client` |
96
+ | `docker.compose\|docker-compose` | Docker Compose | `docker compose config` |
97
+ | `pydantic\|fastapi` | Python (Pydantic) | `python -c "from app.main import app"` (startup validation) |
98
+ | `ansible` | Ansible | `ansible-playbook --syntax-check`, `ansible-lint` |
99
+ | `pulumi` | Pulumi | `pulumi preview --non-interactive` |
100
+
101
+ **Parse `TOOL_VERSIONS` (structured)** — for each key present, add to the detected components map with its version.
102
+
103
+ **Version-aware validator selection**:
104
+
105
+ The correct validator depends on the tool version. Version mismatches (e.g., using Traefik v2 flags on a v3 install) produce misleading results.
106
+
107
+ | Tool | Version | Preferred validator | Notes |
108
+ |------|---------|---------------------|-------|
109
+ | Traefik | v2.x | `traefik validate --configFile=traefik.yml` | v2 flag syntax |
110
+ | Traefik | v3.x | `traefik validate` (no `--configFile` flag needed in v3) | v3 changed CLI |
111
+ | nginx | any | `nginx -t -c /etc/nginx/nginx.conf` | Standard across versions |
112
+ | Terraform | ≥0.13 | `terraform validate` (in-directory) | Requires `terraform init` first |
113
+ | Terraform | all | `tflint` (linter, catches schema errors `terraform validate` misses) | Complementary |
114
+ | Kubernetes | any | `kubectl apply --dry-run=client -f k8s/` | Client-side, no cluster needed |
115
+ | Helm | ≥3.x | `helm lint ./chart` | v3 only (v2 deprecated) |
116
+ | Docker Compose | v2 | `docker compose config` | v2 CLI (`docker compose`, not `docker-compose`) |
117
+ | Docker Compose | v1 | `docker-compose config` | v1 CLI (legacy) |
118
+
119
+ If tool version is unknown: use the most permissive/universal validator form and note version ambiguity in the filed issue.
120
+
121
+ ### 1B: Discover config file locations
122
+
123
+ For each detected tool, check whether config files exist in the repo:
124
+
125
+ ```bash
126
+ REPO_PATH="${REPO_PATH:-.}"
127
+
128
+ # Traefik
129
+ TRAEFIK_CONFIGS=$(find "$REPO_PATH/traefik" -name "*.yml" -o -name "*.yaml" -o -name "*.toml" 2>/dev/null | head -10)
130
+ [ -n "$TRAEFIK_CONFIGS" ] && echo "Traefik configs found: $TRAEFIK_CONFIGS"
131
+
132
+ # nginx
133
+ NGINX_CONFIGS=$(find "$REPO_PATH/infra/nginx" "$REPO_PATH/nginx" -name "*.conf" 2>/dev/null | head -10)
134
+ [ -n "$NGINX_CONFIGS" ] && echo "nginx configs found: $NGINX_CONFIGS"
135
+
136
+ # Kubernetes
137
+ K8S_CONFIGS=$(find "$REPO_PATH/k8s" "$REPO_PATH/kubernetes" "$REPO_PATH/manifests" -name "*.yml" -o -name "*.yaml" 2>/dev/null | head -10)
138
+ [ -n "$K8S_CONFIGS" ] && echo "Kubernetes manifests found: $K8S_CONFIGS"
139
+
140
+ # Terraform
141
+ TF_CONFIGS=$(find "$REPO_PATH/terraform" "$REPO_PATH/infra/terraform" -name "*.tf" 2>/dev/null | head -10)
142
+ [ -n "$TF_CONFIGS" ] && echo "Terraform configs found: $TF_CONFIGS"
143
+
144
+ # Helm charts
145
+ HELM_CHARTS=$(find "$REPO_PATH" -name "Chart.yaml" 2>/dev/null | head -10)
146
+ [ -n "$HELM_CHARTS" ] && echo "Helm charts found: $HELM_CHARTS"
147
+
148
+ # Docker Compose
149
+ COMPOSE_FILES=$(find "$REPO_PATH" -maxdepth 2 -name "docker-compose*.yml" 2>/dev/null | head -10)
150
+ [ -n "$COMPOSE_FILES" ] && echo "Docker Compose files found: $COMPOSE_FILES"
151
+ ```
152
+
153
+ **Skip any tool whose config files are NOT found in the repo** — a declared stack component with no config files has no CI validation gap to audit.
154
+
155
+ ---
156
+
157
+ ## Phase 2: Audit GitHub Actions Workflows
158
+
159
+ ### 2A: Read all workflow files
160
+
161
+ ```bash
162
+ WORKFLOW_DIR="$REPO_PATH/.github/workflows"
163
+ WORKFLOW_FILES=$(ls "$WORKFLOW_DIR"/*.yml "$WORKFLOW_DIR"/*.yaml 2>/dev/null | head -30)
164
+
165
+ if [ -z "$WORKFLOW_FILES" ]; then
166
+ echo "AUDIT GAP: No GitHub Actions workflows found in $WORKFLOW_DIR"
167
+ echo "A project without CI has no automated validation at all — see Phase 3 for issue creation."
168
+ fi
169
+
170
+ echo "=== Workflows to audit ==="
171
+ echo "$WORKFLOW_FILES"
172
+ ```
173
+
174
+ **Override with `forge.yaml → ci_audit.workflow_paths`** if that section is present:
175
+ ```bash
176
+ CUSTOM_PATHS=$(yq '.ci_audit.workflow_paths // []' forge.yaml 2>/dev/null | yq '.[]' 2>/dev/null || echo "")
177
+ [ -n "$CUSTOM_PATHS" ] && WORKFLOW_FILES="$CUSTOM_PATHS"
178
+ ```
179
+
180
+ ### 2B: Check for each tool's validator
181
+
182
+ For each tool detected in Phase 1 with config files present, search all workflow files for a validation step:
183
+
184
+ ```bash
185
+ # Build a combined search corpus from all workflow files
186
+ WORKFLOW_CORPUS=$(cat $WORKFLOW_FILES 2>/dev/null)
187
+
188
+ declare -A TOOL_VALIDATOR_FOUND
189
+ declare -A TOOL_VALIDATOR_PATTERN
190
+
191
+ # Traefik
192
+ TOOL_VALIDATOR_PATTERN[traefik]="traefik.*validate|traefik.*healthcheck|traefik.*check"
193
+
194
+ # nginx
195
+ TOOL_VALIDATOR_PATTERN[nginx]="nginx.*-t|nginx.*-T|nginx.*configtest|nginx.*test"
196
+
197
+ # Kubernetes
198
+ TOOL_VALIDATOR_PATTERN[kubernetes]="kubectl.*--dry-run|kubeval|kustomize.*\|.*kubectl|kubectl.*validate"
199
+
200
+ # Terraform
201
+ TOOL_VALIDATOR_PATTERN[terraform]="terraform.*validate|terraform.*plan|tflint"
202
+
203
+ # Helm
204
+ TOOL_VALIDATOR_PATTERN[helm]="helm.*lint|helm.*template"
205
+
206
+ # Docker Compose
207
+ TOOL_VALIDATOR_PATTERN[docker_compose]="docker.*compose.*config|docker-compose.*config"
208
+
209
+ for tool in "${!TOOL_VALIDATOR_PATTERN[@]}"; do
210
+ PATTERN="${TOOL_VALIDATOR_PATTERN[$tool]}"
211
+ if echo "$WORKFLOW_CORPUS" | grep -qiE "$PATTERN"; then
212
+ TOOL_VALIDATOR_FOUND[$tool]="YES"
213
+ FOUND_IN=$(grep -liE "$PATTERN" $WORKFLOW_FILES 2>/dev/null | head -3 | tr '\n' ', ')
214
+ echo "OK: $tool validator found in: $FOUND_IN"
215
+ else
216
+ TOOL_VALIDATOR_FOUND[$tool]="NO"
217
+ echo "GAP: No $tool validation step found in any CI workflow"
218
+ fi
219
+ done
220
+ ```
221
+
222
+ ### 2C: Check validator placement
223
+
224
+ A validation step that runs only in a deployment workflow (not in the CI workflow that runs on every PR) provides partial coverage — config errors reach `main` undetected if the deploy-only validator only runs at deploy time.
225
+
226
+ ```bash
227
+ CI_WORKFLOW=$(ls "$WORKFLOW_DIR/ci.yml" "$WORKFLOW_DIR/test.yml" "$WORKFLOW_DIR/build.yml" 2>/dev/null | head -1)
228
+
229
+ if [ -n "$CI_WORKFLOW" ]; then
230
+ echo "=== Checking validator placement (CI workflow: $CI_WORKFLOW) ==="
231
+ for tool in "${!TOOL_VALIDATOR_FOUND[@]}"; do
232
+ if [ "${TOOL_VALIDATOR_FOUND[$tool]}" = "YES" ]; then
233
+ PATTERN="${TOOL_VALIDATOR_PATTERN[$tool]}"
234
+ if grep -qiE "$PATTERN" "$CI_WORKFLOW" 2>/dev/null; then
235
+ echo "OK: $tool validator is in the CI workflow (runs on every PR)"
236
+ else
237
+ echo "PLACEMENT GAP: $tool validator exists but is NOT in $CI_WORKFLOW — it may only run at deploy time, missing PR-time validation"
238
+ fi
239
+ fi
240
+ done
241
+ fi
242
+ ```
243
+
244
+ ---
245
+
246
+ ## Phase 3: File Issues for Gaps
247
+
248
+ For each gap detected (tool present in repo but no CI validation step found), file a GitHub issue.
249
+
250
+ **Skip issue creation if `--dry-run` flag is passed** (or `forge.yaml → ci_audit.dry_run: true`):
251
+ ```bash
252
+ DRY_RUN=$(yq '.ci_audit.dry_run // "false"' forge.yaml 2>/dev/null || echo "false")
253
+ [ "$1" = "--dry-run" ] || [ "$2" = "--dry-run" ] && DRY_RUN="true"
254
+ ```
255
+
256
+ ### 3A: Determine validator recommendation
257
+
258
+ For each gap, compose a concrete, version-aware recommendation:
259
+
260
+ ```bash
261
+ get_validator_recommendation() {
262
+ local tool="$1"
263
+ local version="$2" # may be empty if unknown
264
+
265
+ case "$tool" in
266
+ traefik)
267
+ if [ "${version%%.*}" = "3" ] || echo "$version" | grep -q "^v3"; then
268
+ echo "Add a CI step that runs: \`docker run --rm -v \$(pwd)/traefik:/etc/traefik traefik:v${version:-3} validate\`"
269
+ echo "Traefik v3 schema differs from v2 — the validator must use the same version as production."
270
+ else
271
+ echo "Add a CI step that runs: \`docker run --rm -v \$(pwd)/traefik:/etc/traefik traefik:v${version:-2} validate --configFile=/etc/traefik/traefik.yml\`"
272
+ fi
273
+ ;;
274
+ nginx)
275
+ echo "Add a CI step that runs: \`docker run --rm -v \$(pwd)/infra/nginx:/etc/nginx nginx:${version:-alpine} nginx -t\`"
276
+ ;;
277
+ kubernetes)
278
+ echo "Add a CI step that runs: \`kubectl apply --dry-run=client -f k8s/\` (requires \`kubectl\` in CI environment, no cluster needed for client-side validation)"
279
+ ;;
280
+ terraform)
281
+ echo "Add CI steps that run: (1) \`terraform init -backend=false\` then (2) \`terraform validate\`. Optionally add \`tflint\` for schema linting beyond what terraform validate catches."
282
+ ;;
283
+ helm)
284
+ echo "Add a CI step that runs: \`helm lint ./chart\` (or the path to your chart directory)"
285
+ ;;
286
+ docker_compose)
287
+ if [ "${version%%.*}" = "2" ]; then
288
+ echo "Add a CI step that runs: \`docker compose config\` (validates compose file structure and interpolation)"
289
+ else
290
+ echo "Add a CI step that runs: \`docker-compose config\` (validates compose file structure and interpolation)"
291
+ fi
292
+ ;;
293
+ esac
294
+ }
295
+ ```
296
+
297
+ ### 3B: Create issues for gaps
298
+
299
+ For each tool with `TOOL_VALIDATOR_FOUND[$tool] = "NO"`:
300
+
301
+ ```bash
302
+ for tool in "${!TOOL_VALIDATOR_FOUND[@]}"; do
303
+ if [ "${TOOL_VALIDATOR_FOUND[$tool]}" = "NO" ]; then
304
+ TOOL_VERSION=$(yq ".review.tool_versions.${tool} // \"\"" forge.yaml 2>/dev/null || echo "")
305
+ RECOMMENDATION=$(get_validator_recommendation "$tool" "$TOOL_VERSION")
306
+ VERSION_NOTE=""
307
+ [ -n "$TOOL_VERSION" ] && VERSION_NOTE="**Declared version**: \`$tool $TOOL_VERSION\`" || VERSION_NOTE="**Version**: Unknown — declare in \`forge.yaml → review.tool_versions.$tool\` for version-aware validation"
308
+
309
+ ISSUE_BODY="## Problem
310
+
311
+ The project uses \`$tool\` (config files present in repo) but the CI pipeline has no structural validation step for \`$tool\` config files. Structural config errors (wrong nesting, missing required keys, version-incompatible syntax) are therefore not caught before merge or deploy.
312
+
313
+ **Upstream incident**: This CI gap was identified via \`/ci-audit\` following incident analysis showing that review agents verify logical correctness (coherent values, naming consistency, cross-file wiring) but cannot substitute for deterministic structural validators. <!-- Added: forge#1104 -->
314
+
315
+ ## Root Cause
316
+
317
+ **Failure pattern**: The \`$tool\` validator has not been added to CI. Config changes are reviewed for logical correctness by the INFRA review agent but structural correctness (schema conformance) is not deterministically checked before merge.
318
+
319
+ **Separation of concerns**:
320
+ - ForgeDock review agents: verify logical correctness (architecture, wiring, security)
321
+ - CI validators: verify structural correctness (schema conformance, tool version compatibility)
322
+
323
+ Both layers are required. This issue tracks the CI layer gap.
324
+
325
+ ## Affected Files
326
+
327
+ Files to be identified during investigation:
328
+ 1. \`.github/workflows/ci.yml\` (or equivalent) — add validation step
329
+ 2. \`$tool/\` config directory — files to be validated
330
+
331
+ ## Expected Behavior
332
+
333
+ Every PR that changes \`$tool\` config files triggers a CI step that runs the \`$tool\` validator. Structural errors (wrong YAML nesting, missing required keys, version-incompatible syntax) are caught before merge — not after deploy.
334
+
335
+ ## Acceptance Criteria
336
+
337
+ - [ ] A CI step runs \`$tool\` structural validation on every PR that changes \`$tool\` config files
338
+ - [ ] The CI step uses the same \`$tool\` version as production (version-pinned)
339
+ - [ ] The CI step runs in the PR-time workflow (not only at deploy time)
340
+ - [ ] Validation failure blocks merge
341
+ - [ ] No regression in existing CI workflows
342
+
343
+ ## Recommendation
344
+
345
+ ${RECOMMENDATION}
346
+
347
+ ${VERSION_NOTE}
348
+
349
+ **Where to add**: Add the validation step to the CI workflow that runs on every PR (\`ci.yml\` or equivalent), not only the deploy workflow. A validator that only runs at deploy time allows structural errors to merge to \`staging\` undetected.
350
+
351
+ **Implementation**: File this issue → \`/work-on\` pipeline → builder adds the CI step → review verifies version pinning and trigger conditions → merge.
352
+
353
+ ## Severity
354
+
355
+ **P2** — Structural config errors are silently ignored by the tool at runtime. A project without a \`$tool\` validator in CI will ship broken configs whenever a config refactor changes YAML nesting. The failure mode is silent and only surfaces at deploy time (or not at all, if the tool silently ignores the misconfiguration)."
356
+
357
+ if [ "$DRY_RUN" = "true" ]; then
358
+ echo "=== DRY RUN: Would create issue for $tool CI gap ==="
359
+ echo "Title: ci($tool): add structural config validation step to CI pipeline"
360
+ echo "Body:"
361
+ echo "$ISSUE_BODY"
362
+ echo "Labels: bug,P2,audit-finding"
363
+ else
364
+ CREATED_ISSUE=$(gh issue create $GH_FLAG \
365
+ --title "ci($tool): add structural config validation step to CI pipeline" \
366
+ --body "$ISSUE_BODY" \
367
+ --label "bug,P2,audit-finding" 2>&1)
368
+ echo "Created issue for $tool gap: $CREATED_ISSUE"
369
+ fi
370
+ fi
371
+ done
372
+ ```
373
+
374
+ ### 3C: Handle placement gaps
375
+
376
+ For each tool whose validator exists but is only in the deploy workflow (not in the CI workflow):
377
+
378
+ ```bash
379
+ # (Populated in Phase 2C)
380
+ for tool in "${!PLACEMENT_GAPS[@]}"; do
381
+ TOOL_VERSION=$(yq ".review.tool_versions.${tool} // \"\"" forge.yaml 2>/dev/null || echo "")
382
+
383
+ ISSUE_BODY="## Problem
384
+
385
+ The project has a \`$tool\` validation step in CI, but it only runs in the deployment workflow — not in the PR-time CI workflow. This means structural config errors in \`$tool\` configs can merge to \`staging\` undetected and only surface during deployment.
386
+
387
+ ## Root Cause
388
+
389
+ **Failure pattern**: Validator placement is deploy-time only. PRs that introduce structural config errors pass all checks and merge normally. The error is only caught when the deploy pipeline runs.
390
+
391
+ ## Acceptance Criteria
392
+
393
+ - [ ] The \`$tool\` validation step runs in the PR-time CI workflow (\`ci.yml\` or equivalent)
394
+ - [ ] Validation failure blocks merge (not just alerts)
395
+ - [ ] No regression in the existing deploy-time validation
396
+
397
+ ## Severity
398
+
399
+ **P3** — Partial coverage. Structural errors are caught at deploy time but not at PR time, creating a window where broken configs exist on \`staging\` until a deploy is attempted."
400
+
401
+ if [ "$DRY_RUN" = "true" ]; then
402
+ echo "=== DRY RUN: Would create placement issue for $tool ==="
403
+ else
404
+ gh issue create $GH_FLAG \
405
+ --title "ci($tool): move config validation to PR-time CI workflow" \
406
+ --body "$ISSUE_BODY" \
407
+ --label "bug,P3,audit-finding" 2>&1
408
+ fi
409
+ done
410
+ ```
411
+
412
+ ---
413
+
414
+ ## Phase 4: Post Audit Summary
415
+
416
+ ```bash
417
+ gh issue comment $(gh issue list $GH_FLAG --search "ci-audit" --limit 1 --json number --jq '.[0].number // ""' 2>/dev/null || echo "") $GH_FLAG --body "## CI Audit Summary
418
+
419
+ **Audit date**: $(date -u +"%Y-%m-%dT%H:%M:%SZ")
420
+ **Stack audited**: ${TECH_STACK:-"(no tech_stack declared in forge.yaml)"}
421
+
422
+ ### Results
423
+
424
+ | Tool | Config Files Found | CI Validator Found | Placement | Issues Filed |
425
+ |------|--------------------|-------------------|-----------|-------------|
426
+ $(for tool in "${!TOOL_VALIDATOR_FOUND[@]}"; do
427
+ CONFIG_FOUND=$([ -n "${TOOL_CONFIG_FOUND[$tool]}" ] && echo "Yes" || echo "No")
428
+ VALIDATOR="${TOOL_VALIDATOR_FOUND[$tool]:-NO}"
429
+ PLACEMENT="${TOOL_PLACEMENT[$tool]:-"N/A"}"
430
+ echo "| $tool | $CONFIG_FOUND | $VALIDATOR | $PLACEMENT | (see above) |"
431
+ done)
432
+
433
+ ### Actions Required
434
+
435
+ $([ -n "$(for t in "${!TOOL_VALIDATOR_FOUND[@]}"; do [ "${TOOL_VALIDATOR_FOUND[$t]}" = "NO" ] && echo "$t"; done)" ] && \
436
+ echo "Issues filed for: $(for t in "${!TOOL_VALIDATOR_FOUND[@]}"; do [ "${TOOL_VALIDATOR_FOUND[$t]}" = "NO" ] && echo "$t"; done | tr '\n' ' ')" || \
437
+ echo "No CI gaps found — all detected stack components have CI validators.")
438
+
439
+ ### Next Steps
440
+
441
+ 1. Run \`/work-on\` on each filed issue to implement the CI validation step
442
+ 2. Re-run \`/ci-audit\` after implementation to verify gaps are closed
443
+ 3. Add \`forge.yaml → review.tool_versions\` entries for any tools whose version was marked Unknown
444
+ " 2>/dev/null || echo "Audit complete. Summary printed above (no issue to comment on in dry-run mode)."
445
+ ```
446
+
447
+ Print the summary to stdout regardless:
448
+
449
+ ```bash
450
+ echo ""
451
+ echo "=== /ci-audit Complete ==="
452
+ echo "Stack: ${TECH_STACK:-"(no tech_stack declared)"}"
453
+ echo "Workflows audited: $(echo $WORKFLOW_FILES | tr ' ' '\n' | wc -l | tr -d ' ')"
454
+ echo "Tools with config files: $(echo "${!TOOL_CONFIG_FOUND[@]}" | wc -w | tr -d ' ')"
455
+ echo "CI gaps found: $(for t in "${!TOOL_VALIDATOR_FOUND[@]}"; do [ "${TOOL_VALIDATOR_FOUND[$t]}" = "NO" ] && echo "$t"; done | wc -w | tr -d ' ')"
456
+ echo ""
457
+ echo "Issues filed:"
458
+ echo "${ISSUES_FILED:-" (none — dry-run or no gaps)"}"
459
+ ```
460
+
461
+ ---
462
+
463
+ ## Error Handling
464
+
465
+ - **`forge.yaml` missing**: Stop and tell the user to run `npx forgedock init`
466
+ - **No stack declared** (`TECH_STACK` empty and `TOOL_VERSIONS` empty): Print advisory — "No tech stack declared in forge.yaml. Add `review.tech_stack` or `review.tool_versions` to enable stack-aware CI auditing." Do not file issues.
467
+ - **No config files found for a declared tool**: Skip that tool's CI audit — presence in `TECH_STACK` without actual config files may mean the project is in the process of adopting the tool, not that CI is missing a validator.
468
+ - **No GitHub Actions workflows found**: File a single high-severity issue: "Project has no CI — all stack components lack automated validation."
469
+ - **`gh` auth failure**: Print `gh auth status` output and stop.