agent-orchestrator-kit 0.1.5 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +100 -5
- package/bin/agent-orchestrator.js +262 -9
- package/package.json +1 -1
- package/templates/.agents/skills/agent-orchestration/SKILL.md +5 -2
- package/templates/.github/workflows/agent-verify.yml +5 -0
- package/templates/.github/workflows/spec-verify.yml +82 -0
- package/templates/.gitlab/agent-verify.yml +3 -0
- package/templates/.gitlab/spec-verify.yml +47 -0
- package/templates/.gitlab-ci.starter.yml.example +2 -0
- package/templates/AGENTS.md +6 -2
- package/templates/scripts/post-mr-verdict.sh +116 -0
- package/templates/scripts/post-pr-verdict-github.sh +129 -0
- package/templates/scripts/verify-specs.sh +196 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,32 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.1.7] - 2026-07-07
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
- **`agent-orchestrator status`** — dashboard for active OpenSpec changes: task progress (`N/M`), review verdict, `ready to archive` flag
|
|
9
|
+
- **`agent-orchestrator gate-check`** — deterministic review-gate check; exits non-zero when `src/` changed without an approved `review.md` (`require_spec_review: true`); warns (non-blocking) when active changes exceed `max_active_changes`; graceful no-op without `.agents/orchestrator.yaml`, when review isn't required, or when the diff can't be computed
|
|
10
|
+
- **GitHub AI Spec Verifier** — `init --ci github --spec-verify` installs `.github/workflows/spec-verify.yml` + `scripts/post-pr-verdict-github.sh` (via `gh pr comment`), reusing the existing stack-agnostic `scripts/verify-specs.sh`
|
|
11
|
+
- **OpenSpec specs** — `orchestrator-cli-controls`, `github-spec-verify`
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
- **`sync`** (CLI) now removes stale skills/rules no longer present in `.agents/skills|rules` — matches `sync-local-agent-skills.sh` (`rsync --delete`); does not touch `memory.json`, `.mcp.json`, `CLAUDE.md`, `settings.json`
|
|
15
|
+
- **`agent-verify.yml`** (both GitHub and GitLab fragments) now run `gate-check` as part of verify
|
|
16
|
+
- **`--spec-verify`** — no longer GitLab-only; valid with `--ci gitlab` or `--ci github`
|
|
17
|
+
- **README / AGENTS.md / agent-orchestration skill** — document `status`, `gate-check`, and GitHub Spec Verifier parity
|
|
18
|
+
|
|
19
|
+
## [0.1.6] - 2026-07-02
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
- **`init --spec-verify`** — opt-in AI Spec Verifier for GitLab consumers: on MRs changing `src/`, an Amp agent verifies code against `openspec/specs/`, posts PASS/BLOCKED to the MR, and fails the pipeline on BLOCKED
|
|
23
|
+
- **Templates** — `.gitlab/spec-verify.yml` (blocking job, commented Phase 1 `allow_failure` fallback), `scripts/verify-specs.sh` (stack-agnostic prompt with project context from `openspec/config.yaml`, graceful skips, secret-safe), `scripts/post-mr-verdict.sh` (GitLab MR comment)
|
|
24
|
+
- **Orchestrator gate** — `spec-verify-blocking` auto-added to `roles.verifier.gates` (idempotent)
|
|
25
|
+
- **OpenSpec spec** — `spec-verify-consumer`
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
- **`update`** refreshes spec-verify files via `KIT_OPTIN_PATHS` — only in projects that already installed them
|
|
29
|
+
- **README / AGENTS.md template** — AI Spec Verifier documented (install, CI variables, verdict schema, Phase 1 rollout)
|
|
30
|
+
|
|
5
31
|
## [0.1.5] - 2026-06-27
|
|
6
32
|
|
|
7
33
|
### Added
|
|
@@ -63,6 +89,7 @@ All notable changes to this project will be documented in this file.
|
|
|
63
89
|
### Added
|
|
64
90
|
- Initial release: 5-role orchestration pipeline, `/opsx:*` commands, IDE sync
|
|
65
91
|
|
|
92
|
+
[0.1.7]: https://github.com/makshc2/agent-orchestrator-kit/compare/v0.1.6...v0.1.7
|
|
66
93
|
[0.1.5]: https://github.com/makshc2/agent-orchestrator-kit/compare/v0.1.4...v0.1.5
|
|
67
94
|
[0.1.4]: https://github.com/makshc2/agent-orchestrator-kit/compare/v0.1.3...v0.1.4
|
|
68
95
|
[0.1.3]: https://github.com/makshc2/agent-orchestrator-kit/compare/v0.1.2...v0.1.3
|
package/README.md
CHANGED
|
@@ -99,7 +99,9 @@ your-project/
|
|
|
99
99
|
├── AGENTS.md
|
|
100
100
|
├── CLAUDE.md
|
|
101
101
|
├── .github/workflows/agent-verify.yml # CI (default --ci github)
|
|
102
|
+
├── .github/workflows/spec-verify.yml # AI Spec Verifier (--ci github --spec-verify, opt-in)
|
|
102
103
|
├── .gitlab/agent-verify.yml # CI fragment (--ci gitlab)
|
|
104
|
+
├── .gitlab/spec-verify.yml # AI Spec Verifier (--ci gitlab --spec-verify, opt-in)
|
|
103
105
|
├── .agents/
|
|
104
106
|
│ ├── orchestrator.yaml
|
|
105
107
|
│ ├── mcp.json.example # Cursor MCP template
|
|
@@ -115,7 +117,8 @@ your-project/
|
|
|
115
117
|
│ ├── openspec-archive-change/
|
|
116
118
|
│ ├── openspec-sync-specs/
|
|
117
119
|
│ └── spec-workflow-openspec/
|
|
118
|
-
|
|
120
|
+
├── scripts/sync-local-agent-skills.sh
|
|
121
|
+
└── scripts/verify-specs.sh + post-mr-verdict.sh / post-pr-verdict-github.sh # (--spec-verify, opt-in)
|
|
119
122
|
```
|
|
120
123
|
|
|
121
124
|
### Included in kit
|
|
@@ -124,8 +127,10 @@ your-project/
|
|
|
124
127
|
|----------|----------|
|
|
125
128
|
| Orchestration | 5-role pipeline, `AGENTS.md`, `orchestrator.yaml`, review command |
|
|
126
129
|
| OpenSpec skills | All 7 skills for `/opsx:*` workflow |
|
|
127
|
-
| IDE sync | Cursor + Claude Code sync script |
|
|
128
|
-
|
|
|
130
|
+
| IDE sync | Cursor + Claude Code sync script (`--delete` semantics — removes stale skills) |
|
|
131
|
+
| CLI gates | `agent-orchestrator status` / `gate-check` — deterministic review-gate checks |
|
|
132
|
+
| CI | `agent-verify.yml` — GitHub (default) or GitLab fragment + `prebuild` hook, both run `gate-check` |
|
|
133
|
+
| AI Spec Verifier | `spec-verify.yml` + verifier scripts — GitLab or GitHub, opt-in (`--spec-verify`) |
|
|
129
134
|
| MCP templates | Memory MCP for Cursor and Amp |
|
|
130
135
|
|
|
131
136
|
### Not included (install separately)
|
|
@@ -326,6 +331,72 @@ Optional: include `.gitlab/agent-verify.yml` in `.gitlab-ci.yml` for full lint/b
|
|
|
326
331
|
|
|
327
332
|
Blocks merge if any gate fails.
|
|
328
333
|
|
|
334
|
+
Every CI fragment (`agent-verify.yml`, GitHub and GitLab) also runs `npx agent-orchestrator-kit gate-check` — see [Deterministic gates](#deterministic-gates-status--gate-check) below. It never fails the pipeline for projects without `.agents/orchestrator.yaml`.
|
|
335
|
+
|
|
336
|
+
#### AI Spec Verifier (GitLab or GitHub, opt-in)
|
|
337
|
+
|
|
338
|
+
```bash
|
|
339
|
+
npx agent-orchestrator-kit init --ci gitlab --spec-verify
|
|
340
|
+
npx agent-orchestrator-kit init --ci github --spec-verify
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
Installs an AI verification layer on top of the deterministic gates: on every merge/pull request that changes `src/`, an Amp agent reads `openspec/specs/`, checks the changed code against every relevant requirement, posts a **PASS / BLOCKED** comment to the MR/PR, and **fails the pipeline on BLOCKED** — specs become an enforceable merge contract, not just documentation.
|
|
344
|
+
|
|
345
|
+
Installed files (GitLab):
|
|
346
|
+
|
|
347
|
+
| File | Purpose |
|
|
348
|
+
|------|---------|
|
|
349
|
+
| `.gitlab/spec-verify.yml` | CI fragment — hidden `.spec-verify-base` + blocking `spec-verify` job (MR + `src/**/*` only) |
|
|
350
|
+
| `scripts/verify-specs.sh` | Collects changed files + specs, builds prompt (project context from `openspec/config.yaml`), calls `amp -x`, writes `artifacts/verdict.json` |
|
|
351
|
+
| `scripts/post-mr-verdict.sh` | Posts the verdict as an MR comment via GitLab API |
|
|
352
|
+
|
|
353
|
+
Installed files (GitHub):
|
|
354
|
+
|
|
355
|
+
| File | Purpose |
|
|
356
|
+
|------|---------|
|
|
357
|
+
| `.github/workflows/spec-verify.yml` | Workflow triggered on `pull_request` for `src/**` — same verdict evaluation, `permissions: pull-requests: write` |
|
|
358
|
+
| `scripts/verify-specs.sh` | Same script as GitLab — stack-agnostic, reused as-is |
|
|
359
|
+
| `scripts/post-pr-verdict-github.sh` | Posts the verdict as a PR comment via `gh pr comment` |
|
|
360
|
+
|
|
361
|
+
The flag also adds `spec-verify-blocking` to `roles.verifier.gates` in `.agents/orchestrator.yaml`.
|
|
362
|
+
|
|
363
|
+
Setup after install (GitLab):
|
|
364
|
+
|
|
365
|
+
1. Include the fragment from `.gitlab-ci.yml`:
|
|
366
|
+
|
|
367
|
+
```yaml
|
|
368
|
+
include:
|
|
369
|
+
- local: '.gitlab/spec-verify.yml'
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
2. Add CI/CD variables (Settings → CI/CD → Variables, masked): `AMP_API_KEY`, `GITLAB_VERIFIER_TOKEN` (project access token with `api` scope).
|
|
373
|
+
|
|
374
|
+
Setup after install (GitHub): the workflow runs automatically on `pull_request` — just add the repo secret `AMP_API_KEY` (Settings → Secrets and variables → Actions). `GITHUB_TOKEN` is provided by Actions automatically.
|
|
375
|
+
|
|
376
|
+
Verdict schema (`artifacts/verdict.json`): `pass`, `score` (0–100), `summary`, `findings[]` with `severity` (`error` fails the job), `spec`, `requirement`, `message`, `file`. The script degrades gracefully — no `src/` changes, no specs, missing `amp` CLI, or missing `AMP_API_KEY` produce a skipped passing verdict and never block the pipeline. Secrets are never logged; `.env`/key/token files are excluded from prompts.
|
|
377
|
+
|
|
378
|
+
**Warning-only rollout (Phase 1):** uncomment `allow_failure: true` (GitLab) or `continue-on-error: true` (GitHub) to keep the pipeline green while the team builds trust in verdicts, then remove it to enforce blocking (Phase 2).
|
|
379
|
+
|
|
380
|
+
`update` refreshes the spec-verify files only in projects that already installed them — the feature stays opt-in.
|
|
381
|
+
|
|
382
|
+
---
|
|
383
|
+
|
|
384
|
+
### Deterministic gates: `status` / `gate-check`
|
|
385
|
+
|
|
386
|
+
Orchestration hard rules (review approval, one active change) used to rely entirely on the agent remembering to check them in chat. Two CLI commands make them checkable and CI-enforceable:
|
|
387
|
+
|
|
388
|
+
```bash
|
|
389
|
+
npx agent-orchestrator-kit status
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
Prints every active OpenSpec change with task progress (`N/M tasks`), review verdict (`APPROVE` / `REQUEST CHANGES` / `none`), and a `ready to archive` flag once all tasks are `[x]` — no more running `openspec status` per change by hand.
|
|
393
|
+
|
|
394
|
+
```bash
|
|
395
|
+
npx agent-orchestrator-kit gate-check [change-name] [--src-glob src/] [--base HEAD~1]
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
Fails (non-zero exit) when `pipeline.require_spec_review: true`, the diff against `--base` touches `--src-glob`, and the active change has no `review.md` with `Verdict: APPROVE`. It degrades gracefully to exit 0 (with a message, not silently) when: `.agents/orchestrator.yaml` is missing, review isn't required, the diff can't be computed (e.g. shallow clone), or nothing under `--src-glob` changed. It also warns (never fails) when active changes exceed `pipeline.max_active_changes`. Both `agent-verify.yml` fragments (GitHub and GitLab) call `gate-check` automatically.
|
|
399
|
+
|
|
329
400
|
---
|
|
330
401
|
|
|
331
402
|
### Archive — `/opsx:archive`
|
|
@@ -460,14 +531,26 @@ npx agent-orchestrator-kit init [options]
|
|
|
460
531
|
--lang <code> Agent language: en | uk | ...
|
|
461
532
|
--name <name> Project name (default: directory name)
|
|
462
533
|
--ci <provider> CI provider: gitlab | github | none (default: github)
|
|
534
|
+
--spec-verify Install AI Spec Verifier blocking gate (GitLab or GitHub)
|
|
463
535
|
--force Overwrite existing files
|
|
464
536
|
|
|
465
537
|
npx agent-orchestrator-kit update
|
|
466
538
|
Updates kit-managed files, preserves project overlay
|
|
467
539
|
|
|
468
540
|
npx agent-orchestrator-kit sync [options]
|
|
469
|
-
--target <ide> cursor | claude | all (default: all)
|
|
470
|
-
Copies .agents/ to local IDE directories
|
|
541
|
+
--target <ide> cursor | claude | amp | all (default: all)
|
|
542
|
+
Copies .agents/ to local IDE directories, removing skills/rules no longer
|
|
543
|
+
present in .agents/ (does not touch memory.json, .mcp.json, CLAUDE.md, etc.)
|
|
544
|
+
|
|
545
|
+
npx agent-orchestrator-kit status
|
|
546
|
+
Show progress, review verdict, and archive-readiness for active changes
|
|
547
|
+
|
|
548
|
+
npx agent-orchestrator-kit gate-check [change-name] [options]
|
|
549
|
+
--src-glob <glob> Source path filter used to detect code changes (default: src/)
|
|
550
|
+
--base <ref> Git ref to diff against (default: HEAD~1)
|
|
551
|
+
Exit non-zero when require_spec_review is true, src/ changed, and the
|
|
552
|
+
active change has no review.md with Verdict: APPROVE. Graceful no-op
|
|
553
|
+
otherwise (missing config, review not required, no relevant diff).
|
|
471
554
|
```
|
|
472
555
|
|
|
473
556
|
## Directory Reference
|
|
@@ -501,6 +584,18 @@ openspec/ # Committed — spec-driven workflow
|
|
|
501
584
|
|
|
502
585
|
## Changelog
|
|
503
586
|
|
|
587
|
+
### 0.1.7
|
|
588
|
+
- `agent-orchestrator status` — dashboard for active OpenSpec changes: task progress, review verdict, archive readiness
|
|
589
|
+
- `agent-orchestrator gate-check` — deterministic review-gate check (exit non-zero without an approved `review.md`); wired into both `agent-verify.yml` CI fragments
|
|
590
|
+
- `init --ci github --spec-verify` — GitHub parity for the AI Spec Verifier (`.github/workflows/spec-verify.yml`, `scripts/post-pr-verdict-github.sh`), reusing the existing stack-agnostic `verify-specs.sh`
|
|
591
|
+
- `sync` (CLI) now removes skills/rules that no longer exist in `.agents/` — matches `sync-local-agent-skills.sh` (`rsync --delete`) behavior; leaves `memory.json`, `.mcp.json`, `CLAUDE.md`, `settings.json` untouched
|
|
592
|
+
|
|
593
|
+
### 0.1.6
|
|
594
|
+
- `init --ci gitlab --spec-verify` — opt-in AI Spec Verifier: blocking MR gate via Amp CLI
|
|
595
|
+
- Templates: `.gitlab/spec-verify.yml`, `scripts/verify-specs.sh`, `scripts/post-mr-verdict.sh`
|
|
596
|
+
- `spec-verify-blocking` gate auto-added to `roles.verifier.gates`
|
|
597
|
+
- `update` refreshes spec-verify files only where already installed
|
|
598
|
+
|
|
504
599
|
### 0.1.5
|
|
505
600
|
- `init --ci gitlab|github|none` — GitLab verify via prebuild hook + CI fragment
|
|
506
601
|
- PM-aware `verify:openspec` / `prebuild` injection for GitLab projects
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
import { program } from 'commander';
|
|
3
3
|
import pc from 'picocolors';
|
|
4
|
-
import { readFileSync, existsSync, mkdirSync, copyFileSync, readdirSync, statSync, writeFileSync } from 'fs';
|
|
4
|
+
import { readFileSync, existsSync, mkdirSync, copyFileSync, readdirSync, statSync, writeFileSync, rmSync } from 'fs';
|
|
5
5
|
import { join, dirname, basename } from 'path';
|
|
6
6
|
import { fileURLToPath } from 'url';
|
|
7
7
|
import { execSync } from 'child_process';
|
|
@@ -32,6 +32,26 @@ const KIT_MANAGED_PATHS = [
|
|
|
32
32
|
'scripts/sync-local-agent-skills.sh',
|
|
33
33
|
];
|
|
34
34
|
|
|
35
|
+
// Opt-in AI Spec Verifier files, per CI provider. `scripts/verify-specs.sh` is
|
|
36
|
+
// shared — it is stack- and CI-agnostic already.
|
|
37
|
+
const GITLAB_SPEC_VERIFY_PATHS = [
|
|
38
|
+
'.gitlab/spec-verify.yml',
|
|
39
|
+
'scripts/verify-specs.sh',
|
|
40
|
+
'scripts/post-mr-verdict.sh',
|
|
41
|
+
];
|
|
42
|
+
const GITHUB_SPEC_VERIFY_PATHS = [
|
|
43
|
+
'.github/workflows/spec-verify.yml',
|
|
44
|
+
'scripts/verify-specs.sh',
|
|
45
|
+
'scripts/post-pr-verdict-github.sh',
|
|
46
|
+
];
|
|
47
|
+
|
|
48
|
+
// Opt-in files: refreshed by `update` only when already present in the project
|
|
49
|
+
const KIT_OPTIN_PATHS = [...new Set([...GITLAB_SPEC_VERIFY_PATHS, ...GITHUB_SPEC_VERIFY_PATHS])];
|
|
50
|
+
|
|
51
|
+
function specVerifyPathsFor(ci) {
|
|
52
|
+
return ci === 'github' ? GITHUB_SPEC_VERIFY_PATHS : GITLAB_SPEC_VERIFY_PATHS;
|
|
53
|
+
}
|
|
54
|
+
|
|
35
55
|
const VALID_CI_PROVIDERS = ['gitlab', 'github', 'none'];
|
|
36
56
|
const VERIFY_OPENSPEC_SCRIPT = 'npx openspec validate --all --strict';
|
|
37
57
|
|
|
@@ -46,10 +66,11 @@ const log = {
|
|
|
46
66
|
};
|
|
47
67
|
|
|
48
68
|
function copyDir(src, dest, opts = {}) {
|
|
49
|
-
const { overwrite = true, skip = [] } = opts;
|
|
69
|
+
const { overwrite = true, skip = [], delete: deleteStale = false } = opts;
|
|
50
70
|
if (!existsSync(src)) return;
|
|
51
71
|
mkdirSync(dest, { recursive: true });
|
|
52
|
-
|
|
72
|
+
const srcEntries = readdirSync(src);
|
|
73
|
+
for (const entry of srcEntries) {
|
|
53
74
|
if (skip.includes(entry)) continue;
|
|
54
75
|
const srcPath = join(src, entry);
|
|
55
76
|
const destPath = join(dest, entry);
|
|
@@ -64,6 +85,17 @@ function copyDir(src, dest, opts = {}) {
|
|
|
64
85
|
log.ok(destPath.replace(process.cwd() + '/', ''));
|
|
65
86
|
}
|
|
66
87
|
}
|
|
88
|
+
|
|
89
|
+
// Remove entries that exist in dest but no longer exist in src (e.g. skills
|
|
90
|
+
// removed by a kit `update`), keeping .cursor/.claude in sync with .agents/.
|
|
91
|
+
if (deleteStale && existsSync(dest)) {
|
|
92
|
+
for (const entry of readdirSync(dest)) {
|
|
93
|
+
if (skip.includes(entry) || srcEntries.includes(entry)) continue;
|
|
94
|
+
const destPath = join(dest, entry);
|
|
95
|
+
rmSync(destPath, { recursive: true, force: true });
|
|
96
|
+
log.warn(`removed stale: ${destPath.replace(process.cwd() + '/', '')}`);
|
|
97
|
+
}
|
|
98
|
+
}
|
|
67
99
|
}
|
|
68
100
|
|
|
69
101
|
function gitignoreLines(content) {
|
|
@@ -121,6 +153,59 @@ function hasOpenSpec(projectDir) {
|
|
|
121
153
|
return existsSync(join(projectDir, 'openspec', 'config.yaml'));
|
|
122
154
|
}
|
|
123
155
|
|
|
156
|
+
function listActiveChanges(projectDir) {
|
|
157
|
+
const changesDir = join(projectDir, 'openspec', 'changes');
|
|
158
|
+
if (!existsSync(changesDir)) return [];
|
|
159
|
+
return readdirSync(changesDir)
|
|
160
|
+
.filter((name) => name !== 'archive' && statSync(join(changesDir, name)).isDirectory())
|
|
161
|
+
.sort();
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
function parseTasksProgress(changeDir) {
|
|
165
|
+
const tasksPath = join(changeDir, 'tasks.md');
|
|
166
|
+
if (!existsSync(tasksPath)) return null;
|
|
167
|
+
const content = readFileSync(tasksPath, 'utf-8');
|
|
168
|
+
const total = (content.match(/^\s*- \[[ xX]\]/gm) || []).length;
|
|
169
|
+
const done = (content.match(/^\s*- \[[xX]\]/gm) || []).length;
|
|
170
|
+
return { total, done };
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
function parseReviewVerdict(changeDir) {
|
|
174
|
+
const reviewPath = join(changeDir, 'review.md');
|
|
175
|
+
if (!existsSync(reviewPath)) return null;
|
|
176
|
+
const content = readFileSync(reviewPath, 'utf-8');
|
|
177
|
+
const match = content.match(/\*\*Verdict:\*\*\s*(.+)/);
|
|
178
|
+
return match ? match[1].trim() : 'unknown';
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
function readPipelineConfig(projectDir) {
|
|
182
|
+
const orchPath = join(projectDir, '.agents', 'orchestrator.yaml');
|
|
183
|
+
if (!existsSync(orchPath)) return null;
|
|
184
|
+
const content = readFileSync(orchPath, 'utf-8');
|
|
185
|
+
const requireReviewMatch = content.match(/require_spec_review:\s*(true|false)/);
|
|
186
|
+
const maxActiveMatch = content.match(/max_active_changes:\s*(\d+)/);
|
|
187
|
+
return {
|
|
188
|
+
requireSpecReview: requireReviewMatch ? requireReviewMatch[1] === 'true' : true,
|
|
189
|
+
maxActiveChanges: maxActiveMatch ? parseInt(maxActiveMatch[1], 10) : null,
|
|
190
|
+
};
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
// Returns true/false when the diff is known, or null when it could not be
|
|
194
|
+
// determined (no git repo, invalid base ref, shallow clone, etc.) — callers
|
|
195
|
+
// must treat null as "skip gracefully", never as "block".
|
|
196
|
+
function gitDiffTouchesGlob(projectDir, base, srcGlob) {
|
|
197
|
+
try {
|
|
198
|
+
const out = execSync(`git diff --name-only ${base}...HEAD -- "${srcGlob}"`, {
|
|
199
|
+
cwd: projectDir,
|
|
200
|
+
stdio: ['ignore', 'pipe', 'ignore'],
|
|
201
|
+
encoding: 'utf-8',
|
|
202
|
+
});
|
|
203
|
+
return out.trim().length > 0;
|
|
204
|
+
} catch {
|
|
205
|
+
return null;
|
|
206
|
+
}
|
|
207
|
+
}
|
|
208
|
+
|
|
124
209
|
function installOpenspecConfigExample(projectDir, profile, vars, force) {
|
|
125
210
|
const src = resolveTemplate('openspec-config.yaml.example', profile);
|
|
126
211
|
if (!existsSync(src)) return;
|
|
@@ -223,6 +308,43 @@ function installCi(projectDir, templateDir, ci, force) {
|
|
|
223
308
|
}
|
|
224
309
|
}
|
|
225
310
|
|
|
311
|
+
function installSpecVerify(projectDir, templateDir, force, ci) {
|
|
312
|
+
const paths = specVerifyPathsFor(ci);
|
|
313
|
+
for (const rel of paths) {
|
|
314
|
+
const src = join(templateDir, rel);
|
|
315
|
+
const dest = join(projectDir, rel);
|
|
316
|
+
if (!existsSync(src)) continue;
|
|
317
|
+
if (!force && existsSync(dest)) {
|
|
318
|
+
log.warn(`skip (exists): ${rel}`);
|
|
319
|
+
continue;
|
|
320
|
+
}
|
|
321
|
+
mkdirSync(dirname(dest), { recursive: true });
|
|
322
|
+
copyFileSync(src, dest);
|
|
323
|
+
log.ok(rel);
|
|
324
|
+
}
|
|
325
|
+
const scripts = paths.filter((rel) => rel.startsWith('scripts/')).map((rel) => join(projectDir, rel));
|
|
326
|
+
try {
|
|
327
|
+
execSync(`chmod +x ${scripts.join(' ')}`);
|
|
328
|
+
} catch {}
|
|
329
|
+
}
|
|
330
|
+
|
|
331
|
+
function patchOrchestratorSpecVerify(projectDir) {
|
|
332
|
+
const orchPath = join(projectDir, '.agents', 'orchestrator.yaml');
|
|
333
|
+
if (!existsSync(orchPath)) return;
|
|
334
|
+
|
|
335
|
+
let content = readFileSync(orchPath, 'utf-8');
|
|
336
|
+
if (content.includes('spec-verify-blocking')) return;
|
|
337
|
+
|
|
338
|
+
const anchor = /^(\s*)- openspec-validate-strict\s*$/m;
|
|
339
|
+
if (!anchor.test(content)) {
|
|
340
|
+
log.warn('could not add spec-verify-blocking gate: openspec-validate-strict anchor not found in orchestrator.yaml');
|
|
341
|
+
return;
|
|
342
|
+
}
|
|
343
|
+
content = content.replace(anchor, '$1- openspec-validate-strict\n$1- spec-verify-blocking');
|
|
344
|
+
writeFileSync(orchPath, content);
|
|
345
|
+
log.ok('spec-verify-blocking gate added to orchestrator.yaml');
|
|
346
|
+
}
|
|
347
|
+
|
|
226
348
|
function patchOrchestratorVerifier(projectDir, pm) {
|
|
227
349
|
const orchPath = join(projectDir, '.agents', 'orchestrator.yaml');
|
|
228
350
|
if (!existsSync(orchPath)) return;
|
|
@@ -245,7 +367,7 @@ function patchOrchestratorVerifier(projectDir, pm) {
|
|
|
245
367
|
writeFileSync(orchPath, content);
|
|
246
368
|
}
|
|
247
369
|
|
|
248
|
-
function printNextSteps(profile, projectDir, ci = 'github') {
|
|
370
|
+
function printNextSteps(profile, projectDir, ci = 'github', specVerify = false) {
|
|
249
371
|
const pm = detectPackageManager(projectDir);
|
|
250
372
|
const openspecReady = hasOpenSpec(projectDir);
|
|
251
373
|
const lines = [`${pc.bold('Next steps:')}`];
|
|
@@ -288,6 +410,19 @@ function printNextSteps(profile, projectDir, ci = 'github') {
|
|
|
288
410
|
lines.push(` ${pc.dim('Optional dev CI: include local .gitlab/agent-verify.yml (see kit templates/.gitlab-ci.starter.yml.example)')}`);
|
|
289
411
|
}
|
|
290
412
|
|
|
413
|
+
if (specVerify) {
|
|
414
|
+
lines.push(` ${pc.bold('AI Spec Verifier:')}`);
|
|
415
|
+
if (ci === 'gitlab') {
|
|
416
|
+
lines.push(` - include ${pc.cyan(".gitlab/spec-verify.yml")} from your .gitlab-ci.yml`);
|
|
417
|
+
lines.push(` - add CI/CD variables: ${pc.cyan('AMP_API_KEY')}, ${pc.cyan('GITLAB_VERIFIER_TOKEN')} (masked)`);
|
|
418
|
+
lines.push(` - BLOCKED verdict fails the MR pipeline (uncomment allow_failure for warning-only rollout)`);
|
|
419
|
+
} else if (ci === 'github') {
|
|
420
|
+
lines.push(` - workflow ${pc.cyan('.github/workflows/spec-verify.yml')} runs automatically on pull_request`);
|
|
421
|
+
lines.push(` - add repo secret: ${pc.cyan('AMP_API_KEY')} (Settings → Secrets and variables → Actions)`);
|
|
422
|
+
lines.push(` - BLOCKED verdict fails the PR pipeline (remove continue-on-error for warning-only rollout)`);
|
|
423
|
+
}
|
|
424
|
+
}
|
|
425
|
+
|
|
291
426
|
console.log('\n' + lines.join('\n') + '\n');
|
|
292
427
|
}
|
|
293
428
|
|
|
@@ -319,6 +454,7 @@ program
|
|
|
319
454
|
.option('--name <name>', 'Project name (defaults to directory name)')
|
|
320
455
|
.option('--force', 'Overwrite existing files', false)
|
|
321
456
|
.option('--ci <provider>', 'CI provider: gitlab | github | none', 'github')
|
|
457
|
+
.option('--spec-verify', 'Install AI Spec Verifier blocking gate (GitLab or GitHub)', false)
|
|
322
458
|
.action((opts) => {
|
|
323
459
|
const projectDir = process.cwd();
|
|
324
460
|
const projectName = opts.name || basename(projectDir);
|
|
@@ -344,7 +480,10 @@ program
|
|
|
344
480
|
}
|
|
345
481
|
|
|
346
482
|
log.title('Installing scripts/');
|
|
347
|
-
copyDir(join(templateDir, 'scripts'), join(projectDir, 'scripts'), {
|
|
483
|
+
copyDir(join(templateDir, 'scripts'), join(projectDir, 'scripts'), {
|
|
484
|
+
overwrite: opts.force,
|
|
485
|
+
skip: ['verify-specs.sh', 'post-mr-verdict.sh'],
|
|
486
|
+
});
|
|
348
487
|
try {
|
|
349
488
|
execSync(`chmod +x ${join(projectDir, 'scripts', 'sync-local-agent-skills.sh')}`);
|
|
350
489
|
} catch {}
|
|
@@ -355,6 +494,15 @@ program
|
|
|
355
494
|
injectVerifyScripts(projectDir, { pm });
|
|
356
495
|
}
|
|
357
496
|
|
|
497
|
+
const specVerify = Boolean(opts.specVerify) && (ci === 'gitlab' || ci === 'github');
|
|
498
|
+
if (opts.specVerify && !specVerify) {
|
|
499
|
+
log.warn('--spec-verify requires --ci gitlab or --ci github — skipping AI Spec Verifier install');
|
|
500
|
+
}
|
|
501
|
+
if (specVerify) {
|
|
502
|
+
log.title('Installing AI Spec Verifier (opt-in)');
|
|
503
|
+
installSpecVerify(projectDir, templateDir, opts.force, ci);
|
|
504
|
+
}
|
|
505
|
+
|
|
358
506
|
log.title('Installing root files');
|
|
359
507
|
for (const f of ['AGENTS.md', 'CLAUDE.md']) {
|
|
360
508
|
const src = resolveTemplate(f, profile);
|
|
@@ -380,6 +528,9 @@ program
|
|
|
380
528
|
patchOrchestratorVerifier(projectDir, pm);
|
|
381
529
|
log.ok('.agents/orchestrator.yaml');
|
|
382
530
|
}
|
|
531
|
+
if (specVerify) {
|
|
532
|
+
patchOrchestratorSpecVerify(projectDir);
|
|
533
|
+
}
|
|
383
534
|
|
|
384
535
|
log.title('OpenSpec config template');
|
|
385
536
|
installOpenspecConfigExample(projectDir, profile, vars, opts.force);
|
|
@@ -389,7 +540,7 @@ program
|
|
|
389
540
|
|
|
390
541
|
log.title('Done');
|
|
391
542
|
log.ok(`agent-orchestrator-kit v${KIT_VERSION} installed`);
|
|
392
|
-
printNextSteps(profile, projectDir, ci);
|
|
543
|
+
printNextSteps(profile, projectDir, ci, specVerify);
|
|
393
544
|
});
|
|
394
545
|
|
|
395
546
|
program
|
|
@@ -414,6 +565,14 @@ program
|
|
|
414
565
|
}
|
|
415
566
|
}
|
|
416
567
|
|
|
568
|
+
for (const rel of KIT_OPTIN_PATHS) {
|
|
569
|
+
const src = join(templateDir, rel);
|
|
570
|
+
const dest = join(projectDir, rel);
|
|
571
|
+
if (!existsSync(src) || !existsSync(dest)) continue;
|
|
572
|
+
copyFileSync(src, dest);
|
|
573
|
+
log.ok(`${rel} (opt-in)`);
|
|
574
|
+
}
|
|
575
|
+
|
|
417
576
|
log.ok(`Updated to v${KIT_VERSION}`);
|
|
418
577
|
log.info('Run ./scripts/sync-local-agent-skills.sh to sync to local IDE');
|
|
419
578
|
});
|
|
@@ -433,8 +592,8 @@ program
|
|
|
433
592
|
|
|
434
593
|
if (syncCursor) {
|
|
435
594
|
log.info('Syncing .agents/ → .cursor/');
|
|
436
|
-
copyDir(join(projectDir, '.agents', 'skills'), join(projectDir, '.cursor', 'skills'), { overwrite: true });
|
|
437
|
-
copyDir(join(projectDir, '.agents', 'rules'), join(projectDir, '.cursor', 'rules'), { overwrite: true });
|
|
595
|
+
copyDir(join(projectDir, '.agents', 'skills'), join(projectDir, '.cursor', 'skills'), { overwrite: true, delete: true });
|
|
596
|
+
copyDir(join(projectDir, '.agents', 'rules'), join(projectDir, '.cursor', 'rules'), { overwrite: true, delete: true });
|
|
438
597
|
|
|
439
598
|
const mcpExample = join(projectDir, '.agents', 'mcp.json.example');
|
|
440
599
|
const mcpDest = join(projectDir, '.mcp.json');
|
|
@@ -446,7 +605,7 @@ program
|
|
|
446
605
|
|
|
447
606
|
if (syncClaude) {
|
|
448
607
|
log.info('Syncing .agents/ → .claude/');
|
|
449
|
-
copyDir(join(projectDir, '.agents', 'skills'), join(projectDir, '.claude', 'skills'), { overwrite: true });
|
|
608
|
+
copyDir(join(projectDir, '.agents', 'skills'), join(projectDir, '.claude', 'skills'), { overwrite: true, delete: true });
|
|
450
609
|
|
|
451
610
|
const claudeMd = join(projectDir, 'CLAUDE.md');
|
|
452
611
|
const claudeDir = join(projectDir, '.claude');
|
|
@@ -467,4 +626,98 @@ program
|
|
|
467
626
|
log.warn('.cursor/, .claude/, .amp/ are local only — not committed to git');
|
|
468
627
|
});
|
|
469
628
|
|
|
629
|
+
program
|
|
630
|
+
.command('status')
|
|
631
|
+
.description('Show status of active OpenSpec changes (tasks progress, review verdict, archive readiness)')
|
|
632
|
+
.action(() => {
|
|
633
|
+
const projectDir = process.cwd();
|
|
634
|
+
log.title('agent-orchestrator status');
|
|
635
|
+
|
|
636
|
+
const changes = listActiveChanges(projectDir);
|
|
637
|
+
if (changes.length === 0) {
|
|
638
|
+
log.info('No active changes');
|
|
639
|
+
return;
|
|
640
|
+
}
|
|
641
|
+
|
|
642
|
+
for (const name of changes) {
|
|
643
|
+
const changeDir = join(projectDir, 'openspec', 'changes', name);
|
|
644
|
+
const progress = parseTasksProgress(changeDir);
|
|
645
|
+
const verdict = parseReviewVerdict(changeDir);
|
|
646
|
+
const progressStr = progress ? `${progress.done}/${progress.total} tasks` : 'no tasks.md';
|
|
647
|
+
const verdictStr = verdict || 'none';
|
|
648
|
+
const readyToArchive = Boolean(progress && progress.total > 0 && progress.done === progress.total);
|
|
649
|
+
|
|
650
|
+
console.log(`\n${pc.bold(name)}`);
|
|
651
|
+
console.log(` tasks: ${progressStr}`);
|
|
652
|
+
console.log(` review: ${verdictStr}`);
|
|
653
|
+
if (readyToArchive) log.ok('ready to archive');
|
|
654
|
+
}
|
|
655
|
+
console.log('');
|
|
656
|
+
});
|
|
657
|
+
|
|
658
|
+
program
|
|
659
|
+
.command('gate-check [change-name]')
|
|
660
|
+
.description('Deterministically check the review gate before apply/merge (exit non-zero if unmet)')
|
|
661
|
+
.option('--src-glob <glob>', 'source path filter used to detect code changes', 'src/')
|
|
662
|
+
.option('--base <ref>', 'git ref to diff against', 'HEAD~1')
|
|
663
|
+
.action((changeName, opts) => {
|
|
664
|
+
const projectDir = process.cwd();
|
|
665
|
+
log.title('agent-orchestrator gate-check');
|
|
666
|
+
|
|
667
|
+
const config = readPipelineConfig(projectDir);
|
|
668
|
+
if (!config) {
|
|
669
|
+
log.info('.agents/orchestrator.yaml not found — nothing to gate');
|
|
670
|
+
return;
|
|
671
|
+
}
|
|
672
|
+
|
|
673
|
+
if (!config.requireSpecReview) {
|
|
674
|
+
log.ok('review not required (pipeline.require_spec_review: false)');
|
|
675
|
+
return;
|
|
676
|
+
}
|
|
677
|
+
|
|
678
|
+
const touchesSrc = gitDiffTouchesGlob(projectDir, opts.base, opts.srcGlob);
|
|
679
|
+
if (touchesSrc === false) {
|
|
680
|
+
log.ok(`no changes under ${opts.srcGlob} — nothing to gate`);
|
|
681
|
+
return;
|
|
682
|
+
}
|
|
683
|
+
if (touchesSrc === null) {
|
|
684
|
+
log.warn('could not compute git diff — skipping gate-check');
|
|
685
|
+
return;
|
|
686
|
+
}
|
|
687
|
+
|
|
688
|
+
const changes = listActiveChanges(projectDir);
|
|
689
|
+
if (config.maxActiveChanges && changes.length > config.maxActiveChanges) {
|
|
690
|
+
log.warn(`${changes.length} active changes exceed pipeline.max_active_changes (${config.maxActiveChanges})`);
|
|
691
|
+
}
|
|
692
|
+
|
|
693
|
+
let target = changeName;
|
|
694
|
+
if (!target) {
|
|
695
|
+
if (changes.length === 0) {
|
|
696
|
+
log.warn(`${opts.srcGlob} changed but no active OpenSpec change found — cannot verify review gate`);
|
|
697
|
+
return;
|
|
698
|
+
}
|
|
699
|
+
target = changes
|
|
700
|
+
.map((name) => ({ name, mtime: statSync(join(projectDir, 'openspec', 'changes', name)).mtimeMs }))
|
|
701
|
+
.sort((a, b) => b.mtime - a.mtime)[0].name;
|
|
702
|
+
log.info(`auto-selected change: ${target} (override: gate-check <name>)`);
|
|
703
|
+
}
|
|
704
|
+
|
|
705
|
+
const changeDir = join(projectDir, 'openspec', 'changes', target);
|
|
706
|
+
if (!existsSync(changeDir)) {
|
|
707
|
+
log.err(`change not found: ${target}`);
|
|
708
|
+
process.exitCode = 1;
|
|
709
|
+
return;
|
|
710
|
+
}
|
|
711
|
+
|
|
712
|
+
const verdict = parseReviewVerdict(changeDir);
|
|
713
|
+
if (verdict && /^APPROVE/i.test(verdict)) {
|
|
714
|
+
log.ok(`review gate passed — ${target}: APPROVE`);
|
|
715
|
+
return;
|
|
716
|
+
}
|
|
717
|
+
|
|
718
|
+
log.err(`review gate failed — change "${target}" has ${verdict ? `verdict "${verdict}"` : 'no review.md'}`);
|
|
719
|
+
log.err(`Run /opsx:review ${target} and get an explicit APPROVE before apply/merge.`);
|
|
720
|
+
process.exitCode = 1;
|
|
721
|
+
});
|
|
722
|
+
|
|
470
723
|
program.parse();
|
package/package.json
CHANGED
|
@@ -74,6 +74,8 @@ Before apply, check `.agents/orchestrator.yaml`:
|
|
|
74
74
|
|
|
75
75
|
If Request Changes — fix artifacts, re-run `/opsx:review`.
|
|
76
76
|
|
|
77
|
+
This is no longer only a chat convention: `agent-orchestrator-kit gate-check` runs in CI (both `agent-verify.yml` fragments) and fails the pipeline if `src/` changed without an approved `review.md` — a forgotten or skipped review is caught at merge time, not just at apply time.
|
|
78
|
+
|
|
77
79
|
### apply → verify
|
|
78
80
|
Exit Implementer when:
|
|
79
81
|
- All `tasks.md` boxes `[x]`
|
|
@@ -91,7 +93,7 @@ After PR merged + CI green:
|
|
|
91
93
|
|
|
92
94
|
**Start of each session:**
|
|
93
95
|
1. Announce role: "Starting Spec Reviewer session for change: <name>"
|
|
94
|
-
2.
|
|
96
|
+
2. Run `agent-orchestrator-kit status` (or `openspec list`) — confirm active change limit (`max_active_changes` in orchestrator.yaml) and see task/review progress for every active change at a glance
|
|
95
97
|
3. Read `orchestrator.yaml` for project config and review gate
|
|
96
98
|
|
|
97
99
|
**During session:**
|
|
@@ -135,7 +137,8 @@ At start of new session: read relevant entities to restore context without re-ex
|
|
|
135
137
|
- [ ] explicit **Approve** received before apply (when `require_spec_review: true`)
|
|
136
138
|
- [ ] `review.md` with `Verdict: APPROVE` exists (when review required)
|
|
137
139
|
- [ ] all tasks `[x]` + build OK before PR
|
|
138
|
-
- [ ]
|
|
140
|
+
- [ ] `agent-orchestrator-kit gate-check` passes locally before pushing (mirrors the CI gate)
|
|
141
|
+
- [ ] `/opsx:archive` run after merge — `agent-orchestrator-kit status` shows "ready to archive"
|
|
139
142
|
|
|
140
143
|
## Anti-patterns
|
|
141
144
|
|
|
@@ -10,6 +10,8 @@ jobs:
|
|
|
10
10
|
runs-on: ubuntu-latest
|
|
11
11
|
steps:
|
|
12
12
|
- uses: actions/checkout@v4
|
|
13
|
+
with:
|
|
14
|
+
fetch-depth: 0
|
|
13
15
|
|
|
14
16
|
- uses: actions/setup-node@v4
|
|
15
17
|
with:
|
|
@@ -45,6 +47,9 @@ jobs:
|
|
|
45
47
|
- name: OpenSpec validate
|
|
46
48
|
run: npx openspec validate --all --strict
|
|
47
49
|
|
|
50
|
+
- name: Gate check (review gate)
|
|
51
|
+
run: npx agent-orchestrator-kit gate-check --base "${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || 'HEAD~1' }}"
|
|
52
|
+
|
|
48
53
|
- name: Lint (npm)
|
|
49
54
|
if: steps.pm.outputs.manager == 'npm'
|
|
50
55
|
run: npm run lint --if-present
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
name: spec-verify
|
|
2
|
+
|
|
3
|
+
# ──────────────────────────────────────────────────────────────
|
|
4
|
+
# AI Spec Verifier — blocking gate on pull requests.
|
|
5
|
+
# Verifies that changed src/ code complies with openspec/specs/
|
|
6
|
+
# via Amp CLI, posts a PASS / BLOCKED comment to the PR, and
|
|
7
|
+
# fails the pipeline on a BLOCKED verdict.
|
|
8
|
+
#
|
|
9
|
+
# Installed by agent-orchestrator-kit (init --ci github --spec-verify).
|
|
10
|
+
#
|
|
11
|
+
# Required repo secret (Settings → Secrets and variables → Actions):
|
|
12
|
+
# AMP_API_KEY — Amp API key
|
|
13
|
+
# GITHUB_TOKEN is provided automatically by Actions.
|
|
14
|
+
#
|
|
15
|
+
# Phase 1 (warning-only rollout): uncomment `continue-on-error: true` on the
|
|
16
|
+
# "Evaluate verdict" step below, then remove it once the team trusts the
|
|
17
|
+
# verdicts (Phase 2, blocking by default).
|
|
18
|
+
# ──────────────────────────────────────────────────────────────
|
|
19
|
+
|
|
20
|
+
on:
|
|
21
|
+
pull_request:
|
|
22
|
+
paths:
|
|
23
|
+
- 'src/**'
|
|
24
|
+
|
|
25
|
+
permissions:
|
|
26
|
+
pull-requests: write
|
|
27
|
+
contents: read
|
|
28
|
+
|
|
29
|
+
jobs:
|
|
30
|
+
spec-verify:
|
|
31
|
+
runs-on: ubuntu-latest
|
|
32
|
+
steps:
|
|
33
|
+
- uses: actions/checkout@v4
|
|
34
|
+
with:
|
|
35
|
+
fetch-depth: 0
|
|
36
|
+
|
|
37
|
+
- uses: actions/setup-node@v4
|
|
38
|
+
with:
|
|
39
|
+
node-version: 20
|
|
40
|
+
|
|
41
|
+
- name: Install Amp CLI
|
|
42
|
+
run: npm install -g @sourcegraph/amp@latest
|
|
43
|
+
|
|
44
|
+
- name: Make scripts executable
|
|
45
|
+
run: chmod +x scripts/verify-specs.sh scripts/post-pr-verdict-github.sh
|
|
46
|
+
|
|
47
|
+
- name: Run spec verifier
|
|
48
|
+
env:
|
|
49
|
+
AMP_API_KEY: ${{ secrets.AMP_API_KEY }}
|
|
50
|
+
# verify-specs.sh is shared with the GitLab variant and reads this
|
|
51
|
+
# variable name as the diff base — reused as-is, not GitHub-specific.
|
|
52
|
+
CI_MERGE_REQUEST_DIFF_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
|
53
|
+
run: bash scripts/verify-specs.sh
|
|
54
|
+
|
|
55
|
+
- name: Post PR comment
|
|
56
|
+
if: always()
|
|
57
|
+
env:
|
|
58
|
+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
59
|
+
run: bash scripts/post-pr-verdict-github.sh
|
|
60
|
+
|
|
61
|
+
- name: Upload verifier artifacts
|
|
62
|
+
if: always()
|
|
63
|
+
uses: actions/upload-artifact@v4
|
|
64
|
+
with:
|
|
65
|
+
name: spec-verify-artifacts
|
|
66
|
+
path: |
|
|
67
|
+
artifacts/verdict.json
|
|
68
|
+
artifacts/verifier-prompt.md
|
|
69
|
+
retention-days: 7
|
|
70
|
+
if-no-files-found: ignore
|
|
71
|
+
|
|
72
|
+
- name: Evaluate verdict
|
|
73
|
+
# Phase 1 (warning-only rollout): uncomment the next line to let the
|
|
74
|
+
# pipeline stay green while the team builds trust in verdicts.
|
|
75
|
+
# continue-on-error: true
|
|
76
|
+
run: |
|
|
77
|
+
PASS=$(python3 -c "import json; v=json.load(open('artifacts/verdict.json')); print(str(v.get('pass', True)).lower())")
|
|
78
|
+
if [ "$PASS" != "true" ]; then
|
|
79
|
+
echo "Spec verifier found errors. See PR comment for details."
|
|
80
|
+
exit 1
|
|
81
|
+
fi
|
|
82
|
+
echo "Spec verification passed."
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
.agent-verify-base:
|
|
2
2
|
image: node:20
|
|
3
|
+
variables:
|
|
4
|
+
GIT_DEPTH: 0
|
|
3
5
|
before_script:
|
|
4
6
|
- |
|
|
5
7
|
if [ -f pnpm-lock.yaml ]; then
|
|
@@ -15,6 +17,7 @@
|
|
|
15
17
|
fi
|
|
16
18
|
script:
|
|
17
19
|
- npx openspec validate --all --strict
|
|
20
|
+
- npx agent-orchestrator-kit gate-check --base "${CI_MERGE_REQUEST_DIFF_BASE_SHA:-HEAD~1}"
|
|
18
21
|
- |
|
|
19
22
|
run_pm_script() {
|
|
20
23
|
local name=$1
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# ──────────────────────────────────────────────────────────────
|
|
2
|
+
# AI Spec Verifier — blocking gate on merge requests.
|
|
3
|
+
# Verifies that changed src/ code complies with openspec/specs/
|
|
4
|
+
# via Amp CLI, posts a PASS / BLOCKED comment to the MR, and
|
|
5
|
+
# fails the pipeline on a BLOCKED verdict.
|
|
6
|
+
#
|
|
7
|
+
# Installed by agent-orchestrator-kit (init --ci gitlab --spec-verify).
|
|
8
|
+
# Include from .gitlab-ci.yml:
|
|
9
|
+
# include:
|
|
10
|
+
# - local: '.gitlab/spec-verify.yml'
|
|
11
|
+
#
|
|
12
|
+
# Required CI/CD variables (Settings → CI/CD → Variables):
|
|
13
|
+
# AMP_API_KEY — Amp API key (masked, protected)
|
|
14
|
+
# GITLAB_VERIFIER_TOKEN — Project token with api scope (masked)
|
|
15
|
+
# ──────────────────────────────────────────────────────────────
|
|
16
|
+
.spec-verify-base:
|
|
17
|
+
image: node:20
|
|
18
|
+
script:
|
|
19
|
+
- apt-get update -qq && apt-get install -y -qq python3 curl git > /dev/null
|
|
20
|
+
- npm install -g @sourcegraph/amp@latest
|
|
21
|
+
- chmod +x scripts/verify-specs.sh scripts/post-mr-verdict.sh
|
|
22
|
+
- bash scripts/verify-specs.sh
|
|
23
|
+
- bash scripts/post-mr-verdict.sh
|
|
24
|
+
# Evaluate verdict — exit 1 if verifier says fail
|
|
25
|
+
- |
|
|
26
|
+
PASS=$(python3 -c "import json; v=json.load(open('artifacts/verdict.json')); print(str(v.get('pass',True)).lower())")
|
|
27
|
+
if [ "$PASS" != "true" ]; then
|
|
28
|
+
echo "Spec verifier found errors. See MR comment for details."
|
|
29
|
+
exit 1
|
|
30
|
+
fi
|
|
31
|
+
echo "Spec verification passed."
|
|
32
|
+
artifacts:
|
|
33
|
+
paths:
|
|
34
|
+
- artifacts/verdict.json
|
|
35
|
+
- artifacts/verifier-prompt.md
|
|
36
|
+
expire_in: 7 days
|
|
37
|
+
when: always
|
|
38
|
+
|
|
39
|
+
spec-verify:
|
|
40
|
+
extends: .spec-verify-base
|
|
41
|
+
rules:
|
|
42
|
+
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
|
|
43
|
+
changes:
|
|
44
|
+
- "src/**/*"
|
|
45
|
+
# Phase 1 (warning-only rollout): uncomment the next line to let
|
|
46
|
+
# the pipeline stay green while the team builds trust in verdicts.
|
|
47
|
+
# allow_failure: true
|
package/templates/AGENTS.md
CHANGED
|
@@ -26,6 +26,10 @@ Never mix phases in one chat — this is the single most important rule.
|
|
|
26
26
|
|
|
27
27
|
Verifier runs on **GitHub Actions** (default) or **GitLab** via `prebuild` → `verify:openspec` when using `init --ci gitlab`. GitLab projects do not use `.github/workflows/`.
|
|
28
28
|
|
|
29
|
+
With `init --ci gitlab --spec-verify` or `init --ci github --spec-verify`, an **AI Spec Verifier** also runs on MRs/PRs changing `src/`: an Amp agent checks the changed code against `openspec/specs/` and a **BLOCKED verdict fails the pipeline** (gate `spec-verify-blocking` in `.agents/orchestrator.yaml`).
|
|
30
|
+
|
|
31
|
+
Both CI fragments also run `agent-orchestrator-kit gate-check` — a deterministic check that fails the pipeline when `src/` changed but the active change has no `review.md` with `Verdict: APPROVE` (when `require_spec_review: true`). Run `agent-orchestrator-kit status` at the start of any session to see task progress, review verdict, and archive readiness for every active change without querying `openspec` per change.
|
|
32
|
+
|
|
29
33
|
## Hard Rules
|
|
30
34
|
|
|
31
35
|
- **One active change per developer** at a time.
|
|
@@ -40,9 +44,9 @@ Verifier runs on **GitHub Actions** (default) or **GitLab** via `prebuild` → `
|
|
|
40
44
|
|------------|------|
|
|
41
45
|
| explore → propose | Decision brief written; change name chosen |
|
|
42
46
|
| propose → review | `openspec validate --strict` passes ✓ |
|
|
43
|
-
| review → apply | Reviewer writes explicit **Approve** |
|
|
47
|
+
| review → apply | Reviewer writes explicit **Approve** — enforced in CI by `gate-check` |
|
|
44
48
|
| apply → verify | All `tasks.md` checkboxes `[x]`; local build OK |
|
|
45
|
-
| verify → archive | CI green; PR merged |
|
|
49
|
+
| verify → archive | CI green; PR merged — check `agent-orchestrator status` for "ready to archive" |
|
|
46
50
|
|
|
47
51
|
## Context to Pin per Role
|
|
48
52
|
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# ──────────────────────────────────────────────────────────────
|
|
3
|
+
# Posts spec verifier verdict as a GitLab MR comment.
|
|
4
|
+
#
|
|
5
|
+
# Installed by agent-orchestrator-kit (init --ci gitlab --spec-verify).
|
|
6
|
+
#
|
|
7
|
+
# Usage: ./scripts/post-mr-verdict.sh
|
|
8
|
+
# Env: CI_API_V4_URL, CI_PROJECT_ID, CI_MERGE_REQUEST_IID,
|
|
9
|
+
# GITLAB_VERIFIER_TOKEN (CI/CD variable, masked)
|
|
10
|
+
#
|
|
11
|
+
# Security: GITLAB_VERIFIER_TOKEN is a project access token with
|
|
12
|
+
# api scope. It is NEVER logged or echoed.
|
|
13
|
+
# ──────────────────────────────────────────────────────────────
|
|
14
|
+
set -euo pipefail
|
|
15
|
+
|
|
16
|
+
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
17
|
+
VERDICT_FILE="$ROOT/artifacts/verdict.json"
|
|
18
|
+
|
|
19
|
+
if [ ! -f "$VERDICT_FILE" ]; then
|
|
20
|
+
echo "No verdict file found — skipping MR comment."
|
|
21
|
+
exit 0
|
|
22
|
+
fi
|
|
23
|
+
|
|
24
|
+
if [ -z "${CI_API_V4_URL:-}" ] || [ -z "${CI_PROJECT_ID:-}" ] || [ -z "${CI_MERGE_REQUEST_IID:-}" ]; then
|
|
25
|
+
echo "Not running in MR context — skipping MR comment."
|
|
26
|
+
exit 0
|
|
27
|
+
fi
|
|
28
|
+
|
|
29
|
+
if [ -z "${GITLAB_VERIFIER_TOKEN:-}" ]; then
|
|
30
|
+
echo "GITLAB_VERIFIER_TOKEN not set — skipping MR comment."
|
|
31
|
+
exit 0
|
|
32
|
+
fi
|
|
33
|
+
|
|
34
|
+
# Parse verdict
|
|
35
|
+
PASS=$(python3 -c "import json,sys; v=json.load(open('$VERDICT_FILE')); print(str(v.get('pass',True)).lower())")
|
|
36
|
+
SCORE=$(python3 -c "import json,sys; v=json.load(open('$VERDICT_FILE')); print(v.get('score',0))")
|
|
37
|
+
SUMMARY=$(python3 -c "import json,sys; v=json.load(open('$VERDICT_FILE')); print(v.get('summary',''))")
|
|
38
|
+
SKIPPED=$(python3 -c "import json,sys; v=json.load(open('$VERDICT_FILE')); print(str(v.get('skipped',False)).lower())")
|
|
39
|
+
|
|
40
|
+
if [ "$SKIPPED" = "true" ]; then
|
|
41
|
+
ICON="⏭️"
|
|
42
|
+
STATUS="SKIPPED"
|
|
43
|
+
elif [ "$PASS" = "true" ]; then
|
|
44
|
+
ICON="✅"
|
|
45
|
+
STATUS="PASS"
|
|
46
|
+
else
|
|
47
|
+
ICON="❌"
|
|
48
|
+
STATUS="BLOCKED"
|
|
49
|
+
fi
|
|
50
|
+
|
|
51
|
+
# Build findings table
|
|
52
|
+
FINDINGS_TABLE=$(python3 <<'PYEOF'
|
|
53
|
+
import json, sys
|
|
54
|
+
|
|
55
|
+
with open("artifacts/verdict.json") as f:
|
|
56
|
+
v = json.load(f)
|
|
57
|
+
|
|
58
|
+
findings = v.get("findings", [])
|
|
59
|
+
if not findings:
|
|
60
|
+
print("_No findings._")
|
|
61
|
+
sys.exit(0)
|
|
62
|
+
|
|
63
|
+
severity_icons = {"error": "🔴", "warning": "🟡", "info": "🔵"}
|
|
64
|
+
|
|
65
|
+
print("| | Severity | Spec | Requirement | Message | File |")
|
|
66
|
+
print("|---|----------|------|-------------|---------|------|")
|
|
67
|
+
for f in findings:
|
|
68
|
+
icon = severity_icons.get(f.get("severity", "info"), "⚪")
|
|
69
|
+
sev = f.get("severity", "—")
|
|
70
|
+
spec = f.get("spec", "—")
|
|
71
|
+
req = f.get("requirement", "—")
|
|
72
|
+
msg = f.get("message", "—")
|
|
73
|
+
file = f.get("file", "—")
|
|
74
|
+
line = f.get("line")
|
|
75
|
+
if line:
|
|
76
|
+
file = f"{file}:{line}"
|
|
77
|
+
print(f"| {icon} | {sev} | {spec} | {req} | {msg} | {file} |")
|
|
78
|
+
PYEOF
|
|
79
|
+
)
|
|
80
|
+
|
|
81
|
+
# Build comment body
|
|
82
|
+
COMMENT_BODY="## ${ICON} Spec Verifier — ${STATUS}
|
|
83
|
+
|
|
84
|
+
**Score:** ${SCORE}/100
|
|
85
|
+
**Verdict:** ${STATUS}
|
|
86
|
+
|
|
87
|
+
### Summary
|
|
88
|
+
${SUMMARY}
|
|
89
|
+
|
|
90
|
+
### Findings
|
|
91
|
+
${FINDINGS_TABLE}
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
_AI Spec Verifier • agent-orchestrator-kit • Pipeline: ${CI_PIPELINE_ID:-local}_"
|
|
95
|
+
|
|
96
|
+
# Escape for JSON
|
|
97
|
+
COMMENT_JSON=$(python3 -c "
|
|
98
|
+
import json, sys
|
|
99
|
+
body = sys.stdin.read()
|
|
100
|
+
print(json.dumps({'body': body}))
|
|
101
|
+
" <<< "$COMMENT_BODY")
|
|
102
|
+
|
|
103
|
+
# Post to GitLab API
|
|
104
|
+
# Security: token is passed via header, never logged
|
|
105
|
+
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
|
106
|
+
--request POST \
|
|
107
|
+
--header "PRIVATE-TOKEN: ${GITLAB_VERIFIER_TOKEN}" \
|
|
108
|
+
--header "Content-Type: application/json" \
|
|
109
|
+
--data "$COMMENT_JSON" \
|
|
110
|
+
"${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes")
|
|
111
|
+
|
|
112
|
+
if [ "$HTTP_STATUS" -ge 200 ] && [ "$HTTP_STATUS" -lt 300 ]; then
|
|
113
|
+
echo "MR comment posted successfully (HTTP $HTTP_STATUS)"
|
|
114
|
+
else
|
|
115
|
+
echo "Failed to post MR comment (HTTP $HTTP_STATUS)"
|
|
116
|
+
fi
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# ──────────────────────────────────────────────────────────────
|
|
3
|
+
# Posts spec verifier verdict as a GitHub PR comment via gh CLI.
|
|
4
|
+
#
|
|
5
|
+
# Installed by agent-orchestrator-kit (init --ci github --spec-verify).
|
|
6
|
+
#
|
|
7
|
+
# Usage: ./scripts/post-pr-verdict-github.sh
|
|
8
|
+
# Env: GH_TOKEN / GITHUB_TOKEN — provided by GitHub Actions
|
|
9
|
+
# GITHUB_EVENT_PATH, GITHUB_REPOSITORY, GITHUB_RUN_ID — Actions runtime
|
|
10
|
+
# PR_NUMBER — optional override, auto-detected from the pull_request event otherwise
|
|
11
|
+
#
|
|
12
|
+
# Security: GH_TOKEN/GITHUB_TOKEN is never logged or echoed.
|
|
13
|
+
# ──────────────────────────────────────────────────────────────
|
|
14
|
+
set -euo pipefail
|
|
15
|
+
|
|
16
|
+
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
17
|
+
VERDICT_FILE="$ROOT/artifacts/verdict.json"
|
|
18
|
+
|
|
19
|
+
if [ ! -f "$VERDICT_FILE" ]; then
|
|
20
|
+
echo "No verdict file found — skipping PR comment."
|
|
21
|
+
exit 0
|
|
22
|
+
fi
|
|
23
|
+
|
|
24
|
+
TOKEN="${GH_TOKEN:-${GITHUB_TOKEN:-}}"
|
|
25
|
+
if [ -z "$TOKEN" ]; then
|
|
26
|
+
echo "GH_TOKEN/GITHUB_TOKEN not set — skipping PR comment."
|
|
27
|
+
exit 0
|
|
28
|
+
fi
|
|
29
|
+
|
|
30
|
+
if ! command -v gh &>/dev/null; then
|
|
31
|
+
echo "gh CLI not found — skipping PR comment."
|
|
32
|
+
exit 0
|
|
33
|
+
fi
|
|
34
|
+
|
|
35
|
+
PR_NUMBER="${PR_NUMBER:-}"
|
|
36
|
+
if [ -z "$PR_NUMBER" ] && [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
|
|
37
|
+
PR_NUMBER=$(python3 -c "
|
|
38
|
+
import json
|
|
39
|
+
try:
|
|
40
|
+
data = json.load(open('$GITHUB_EVENT_PATH'))
|
|
41
|
+
print(data.get('pull_request', {}).get('number', ''))
|
|
42
|
+
except Exception:
|
|
43
|
+
print('')
|
|
44
|
+
" 2>/dev/null || true)
|
|
45
|
+
fi
|
|
46
|
+
|
|
47
|
+
if [ -z "$PR_NUMBER" ]; then
|
|
48
|
+
echo "Not running in a pull_request context — skipping PR comment."
|
|
49
|
+
exit 0
|
|
50
|
+
fi
|
|
51
|
+
|
|
52
|
+
export GH_TOKEN="$TOKEN"
|
|
53
|
+
|
|
54
|
+
PASS=$(python3 -c "import json; v=json.load(open('$VERDICT_FILE')); print(str(v.get('pass',True)).lower())")
|
|
55
|
+
SCORE=$(python3 -c "import json; v=json.load(open('$VERDICT_FILE')); print(v.get('score',0))")
|
|
56
|
+
SUMMARY=$(python3 -c "import json; v=json.load(open('$VERDICT_FILE')); print(v.get('summary',''))")
|
|
57
|
+
SKIPPED=$(python3 -c "import json; v=json.load(open('$VERDICT_FILE')); print(str(v.get('skipped',False)).lower())")
|
|
58
|
+
|
|
59
|
+
if [ "$SKIPPED" = "true" ]; then
|
|
60
|
+
ICON="⏭️"
|
|
61
|
+
STATUS="SKIPPED"
|
|
62
|
+
elif [ "$PASS" = "true" ]; then
|
|
63
|
+
ICON="✅"
|
|
64
|
+
STATUS="PASS"
|
|
65
|
+
else
|
|
66
|
+
ICON="❌"
|
|
67
|
+
STATUS="BLOCKED"
|
|
68
|
+
fi
|
|
69
|
+
|
|
70
|
+
# Build findings table
|
|
71
|
+
FINDINGS_TABLE=$(python3 <<'PYEOF'
|
|
72
|
+
import json, sys
|
|
73
|
+
|
|
74
|
+
with open("artifacts/verdict.json") as f:
|
|
75
|
+
v = json.load(f)
|
|
76
|
+
|
|
77
|
+
findings = v.get("findings", [])
|
|
78
|
+
if not findings:
|
|
79
|
+
print("_No findings._")
|
|
80
|
+
sys.exit(0)
|
|
81
|
+
|
|
82
|
+
severity_icons = {"error": "🔴", "warning": "🟡", "info": "🔵"}
|
|
83
|
+
|
|
84
|
+
print("| | Severity | Spec | Requirement | Message | File |")
|
|
85
|
+
print("|---|----------|------|-------------|---------|------|")
|
|
86
|
+
for f in findings:
|
|
87
|
+
icon = severity_icons.get(f.get("severity", "info"), "⚪")
|
|
88
|
+
sev = f.get("severity", "—")
|
|
89
|
+
spec = f.get("spec", "—")
|
|
90
|
+
req = f.get("requirement", "—")
|
|
91
|
+
msg = f.get("message", "—")
|
|
92
|
+
file = f.get("file", "—")
|
|
93
|
+
line = f.get("line")
|
|
94
|
+
if line:
|
|
95
|
+
file = f"{file}:{line}"
|
|
96
|
+
print(f"| {icon} | {sev} | {spec} | {req} | {msg} | {file} |")
|
|
97
|
+
PYEOF
|
|
98
|
+
)
|
|
99
|
+
|
|
100
|
+
# Build comment body
|
|
101
|
+
COMMENT_BODY="## ${ICON} Spec Verifier — ${STATUS}
|
|
102
|
+
|
|
103
|
+
**Score:** ${SCORE}/100
|
|
104
|
+
**Verdict:** ${STATUS}
|
|
105
|
+
|
|
106
|
+
### Summary
|
|
107
|
+
${SUMMARY}
|
|
108
|
+
|
|
109
|
+
### Findings
|
|
110
|
+
${FINDINGS_TABLE}
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
_AI Spec Verifier • agent-orchestrator-kit • Run: ${GITHUB_RUN_ID:-local}_"
|
|
114
|
+
|
|
115
|
+
TMP_BODY_FILE=$(mktemp)
|
|
116
|
+
trap 'rm -f "$TMP_BODY_FILE"' EXIT
|
|
117
|
+
echo "$COMMENT_BODY" > "$TMP_BODY_FILE"
|
|
118
|
+
|
|
119
|
+
GH_REPO_FLAG=()
|
|
120
|
+
if [ -n "${GITHUB_REPOSITORY:-}" ]; then
|
|
121
|
+
GH_REPO_FLAG=(--repo "$GITHUB_REPOSITORY")
|
|
122
|
+
fi
|
|
123
|
+
|
|
124
|
+
# Security: token is passed via GH_TOKEN env var, never logged
|
|
125
|
+
if gh pr comment "$PR_NUMBER" "${GH_REPO_FLAG[@]}" --body-file "$TMP_BODY_FILE" >/dev/null 2>&1; then
|
|
126
|
+
echo "PR comment posted successfully"
|
|
127
|
+
else
|
|
128
|
+
echo "Failed to post PR comment"
|
|
129
|
+
fi
|
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# ──────────────────────────────────────────────────────────────
|
|
3
|
+
# AI Spec Verifier — collects changed src/ files, concatenates
|
|
4
|
+
# openspec/specs/, builds a prompt, calls Amp CLI, and produces
|
|
5
|
+
# artifacts/verdict.json.
|
|
6
|
+
#
|
|
7
|
+
# Installed by agent-orchestrator-kit (init --ci gitlab --spec-verify).
|
|
8
|
+
#
|
|
9
|
+
# Usage: ./scripts/verify-specs.sh
|
|
10
|
+
# Env: CI_MERGE_REQUEST_DIFF_BASE_SHA (GitLab CI provides it)
|
|
11
|
+
# AMP_API_KEY — Amp API key (CI/CD variable, masked)
|
|
12
|
+
# SRC_GLOB — source path filter (default: src/)
|
|
13
|
+
#
|
|
14
|
+
# Security: this script NEVER logs tokens, keys, or .env content.
|
|
15
|
+
# ──────────────────────────────────────────────────────────────
|
|
16
|
+
set -euo pipefail
|
|
17
|
+
|
|
18
|
+
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
19
|
+
ARTIFACTS_DIR="$ROOT/artifacts"
|
|
20
|
+
SPECS_DIR="$ROOT/openspec/specs"
|
|
21
|
+
PROJECT_CONFIG="$ROOT/openspec/config.yaml"
|
|
22
|
+
VERDICT_FILE="$ARTIFACTS_DIR/verdict.json"
|
|
23
|
+
PROMPT_FILE="$ARTIFACTS_DIR/verifier-prompt.md"
|
|
24
|
+
SRC_GLOB="${SRC_GLOB:-src/}"
|
|
25
|
+
|
|
26
|
+
mkdir -p "$ARTIFACTS_DIR"
|
|
27
|
+
|
|
28
|
+
write_skipped_verdict() {
|
|
29
|
+
local summary="$1"
|
|
30
|
+
cat > "$VERDICT_FILE" <<EOF
|
|
31
|
+
{
|
|
32
|
+
"pass": true,
|
|
33
|
+
"score": 100,
|
|
34
|
+
"skipped": true,
|
|
35
|
+
"summary": "${summary}",
|
|
36
|
+
"findings": []
|
|
37
|
+
}
|
|
38
|
+
EOF
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
# ── 1. Collect changed source files ──────────────────────────
|
|
42
|
+
BASE_SHA="${CI_MERGE_REQUEST_DIFF_BASE_SHA:-HEAD~1}"
|
|
43
|
+
|
|
44
|
+
CHANGED_FILES=$(git diff --name-only "$BASE_SHA"...HEAD -- "$SRC_GLOB" || true)
|
|
45
|
+
|
|
46
|
+
if [ -z "$CHANGED_FILES" ]; then
|
|
47
|
+
echo "No ${SRC_GLOB} files changed — skipping spec verification."
|
|
48
|
+
write_skipped_verdict "No ${SRC_GLOB} files changed — verification skipped."
|
|
49
|
+
exit 0
|
|
50
|
+
fi
|
|
51
|
+
|
|
52
|
+
echo "Changed files:"
|
|
53
|
+
echo "$CHANGED_FILES"
|
|
54
|
+
|
|
55
|
+
# ── 2. Collect all spec files ────────────────────────────────
|
|
56
|
+
SPEC_FILES=$(find "$SPECS_DIR" -name '*.md' -type f 2>/dev/null || true)
|
|
57
|
+
|
|
58
|
+
if [ -z "$SPEC_FILES" ]; then
|
|
59
|
+
echo "No spec files found in $SPECS_DIR — skipping."
|
|
60
|
+
write_skipped_verdict "No spec files found — verification skipped."
|
|
61
|
+
exit 0
|
|
62
|
+
fi
|
|
63
|
+
|
|
64
|
+
# ── 3. Build spec content block ──────────────────────────────
|
|
65
|
+
SPECS_CONTENT=""
|
|
66
|
+
for spec_file in $SPEC_FILES; do
|
|
67
|
+
rel_path="${spec_file#$ROOT/}"
|
|
68
|
+
SPECS_CONTENT+="
|
|
69
|
+
--- FILE: $rel_path ---
|
|
70
|
+
$(cat "$spec_file")
|
|
71
|
+
"
|
|
72
|
+
done
|
|
73
|
+
|
|
74
|
+
# ── 4. Build changed file content block ──────────────────────
|
|
75
|
+
# Security: skip .env, secrets, tokens, keys from content
|
|
76
|
+
CHANGED_CONTENT=""
|
|
77
|
+
for file in $CHANGED_FILES; do
|
|
78
|
+
full_path="$ROOT/$file"
|
|
79
|
+
if [ -f "$full_path" ]; then
|
|
80
|
+
case "$file" in
|
|
81
|
+
*.env*|*secret*|*token*|*key*|*.pem|*.p12) continue ;;
|
|
82
|
+
esac
|
|
83
|
+
CHANGED_CONTENT+="
|
|
84
|
+
--- FILE: $file ---
|
|
85
|
+
$(cat "$full_path")
|
|
86
|
+
"
|
|
87
|
+
fi
|
|
88
|
+
done
|
|
89
|
+
|
|
90
|
+
# ── 5. Project context from openspec/config.yaml ─────────────
|
|
91
|
+
PROJECT_CONTEXT=""
|
|
92
|
+
if [ -f "$PROJECT_CONFIG" ]; then
|
|
93
|
+
PROJECT_CONTEXT="
|
|
94
|
+
## Project Context (openspec/config.yaml)
|
|
95
|
+
|
|
96
|
+
$(cat "$PROJECT_CONFIG")
|
|
97
|
+
"
|
|
98
|
+
fi
|
|
99
|
+
|
|
100
|
+
# ── 6. Build verifier prompt ─────────────────────────────────
|
|
101
|
+
cat > "$PROMPT_FILE" <<PROMPT
|
|
102
|
+
You are a SPEC VERIFIER.
|
|
103
|
+
Your job: verify that changed source code complies with project specifications.
|
|
104
|
+
${PROJECT_CONTEXT}
|
|
105
|
+
## Instructions
|
|
106
|
+
1. Read ALL specs below carefully.
|
|
107
|
+
2. Read ALL changed source files below.
|
|
108
|
+
3. For each changed file, check if it relates to any spec requirement.
|
|
109
|
+
4. Verify that every relevant requirement/scenario from specs is satisfied.
|
|
110
|
+
5. Respect project conventions from the project context above, if provided.
|
|
111
|
+
|
|
112
|
+
## Output
|
|
113
|
+
Return ONLY valid JSON (no markdown fences, no extra text) with this structure:
|
|
114
|
+
{
|
|
115
|
+
"pass": true|false,
|
|
116
|
+
"score": 0-100,
|
|
117
|
+
"summary": "Brief summary",
|
|
118
|
+
"findings": [
|
|
119
|
+
{
|
|
120
|
+
"severity": "error"|"warning"|"info",
|
|
121
|
+
"spec": "spec file path",
|
|
122
|
+
"requirement": "requirement name from spec",
|
|
123
|
+
"message": "what is wrong or missing",
|
|
124
|
+
"file": "affected source file",
|
|
125
|
+
"line": null
|
|
126
|
+
}
|
|
127
|
+
]
|
|
128
|
+
}
|
|
129
|
+
|
|
130
|
+
Rules for verdict:
|
|
131
|
+
- "pass": false if ANY finding has severity "error"
|
|
132
|
+
- "pass": true if only "warning" or "info" findings, or no findings
|
|
133
|
+
- "score": 100 minus 10 per error, 3 per warning (minimum 0)
|
|
134
|
+
- If a changed file has no related spec, add an "info" finding noting it
|
|
135
|
+
|
|
136
|
+
## SPECIFICATIONS
|
|
137
|
+
|
|
138
|
+
${SPECS_CONTENT}
|
|
139
|
+
|
|
140
|
+
## CHANGED SOURCE FILES
|
|
141
|
+
|
|
142
|
+
${CHANGED_CONTENT}
|
|
143
|
+
PROMPT
|
|
144
|
+
|
|
145
|
+
echo "Verifier prompt built ($(wc -c < "$PROMPT_FILE") bytes)"
|
|
146
|
+
|
|
147
|
+
# ── 7. Call Amp CLI ──────────────────────────────────────────
|
|
148
|
+
if ! command -v amp &>/dev/null; then
|
|
149
|
+
echo "amp CLI not found — writing fallback verdict."
|
|
150
|
+
write_skipped_verdict "amp CLI not installed — verification skipped."
|
|
151
|
+
exit 0
|
|
152
|
+
fi
|
|
153
|
+
|
|
154
|
+
if [ -z "${AMP_API_KEY:-}" ]; then
|
|
155
|
+
echo "AMP_API_KEY not set — skipping Amp call."
|
|
156
|
+
write_skipped_verdict "AMP_API_KEY not set in CI — Amp verification skipped."
|
|
157
|
+
exit 0
|
|
158
|
+
fi
|
|
159
|
+
|
|
160
|
+
echo "Running Amp verifier agent..."
|
|
161
|
+
export AMP_API_KEY
|
|
162
|
+
AMP_RESPONSE=$(amp -x < "$PROMPT_FILE" 2>/dev/null || true)
|
|
163
|
+
|
|
164
|
+
# ── 8. Extract JSON from response ────────────────────────────
|
|
165
|
+
# The response might contain markdown fences — strip them
|
|
166
|
+
CLEAN_JSON=$(echo "$AMP_RESPONSE" | sed -n '/^{/,/^}/p' | head -200)
|
|
167
|
+
|
|
168
|
+
if echo "$CLEAN_JSON" | python3 -m json.tool > /dev/null 2>&1; then
|
|
169
|
+
echo "$CLEAN_JSON" > "$VERDICT_FILE"
|
|
170
|
+
else
|
|
171
|
+
echo "Failed to parse verifier response as JSON."
|
|
172
|
+
echo "Raw response (first 500 chars):"
|
|
173
|
+
echo "$AMP_RESPONSE" | head -c 500
|
|
174
|
+
# Write a cautious pass — don't block on verifier infrastructure failure
|
|
175
|
+
cat > "$VERDICT_FILE" <<'EOF'
|
|
176
|
+
{
|
|
177
|
+
"pass": true,
|
|
178
|
+
"score": 50,
|
|
179
|
+
"skipped": false,
|
|
180
|
+
"summary": "Verifier did not return valid JSON — result inconclusive.",
|
|
181
|
+
"findings": [
|
|
182
|
+
{
|
|
183
|
+
"severity": "warning",
|
|
184
|
+
"spec": "N/A",
|
|
185
|
+
"requirement": "N/A",
|
|
186
|
+
"message": "Verifier agent response was not valid JSON. Manual review recommended.",
|
|
187
|
+
"file": "N/A",
|
|
188
|
+
"line": null
|
|
189
|
+
}
|
|
190
|
+
]
|
|
191
|
+
}
|
|
192
|
+
EOF
|
|
193
|
+
fi
|
|
194
|
+
|
|
195
|
+
echo "Verdict written to $VERDICT_FILE"
|
|
196
|
+
cat "$VERDICT_FILE" | python3 -m json.tool 2>/dev/null || cat "$VERDICT_FILE"
|