opencode-codeops 2.4.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/README.md +49 -2
- package/_shared/discovery.md +2 -1
- package/_shared/spec-first-ordering.md +6 -2
- package/_shared/test-selection.md +130 -0
- package/bin/codeops-test.mjs +855 -0
- package/bin/lib/test-branch-policy.d.mts +51 -0
- package/bin/lib/test-branch-policy.mjs +185 -0
- package/bin/lib/test-map-validate.d.mts +42 -0
- package/bin/lib/test-map-validate.mjs +618 -0
- package/bin/lib/test-map.d.mts +114 -0
- package/bin/lib/test-map.mjs +384 -0
- package/bin/lib/test-provenance.d.mts +73 -0
- package/bin/lib/test-provenance.mjs +188 -0
- package/bin/lib/test-runner.d.mts +72 -0
- package/bin/lib/test-runner.mjs +350 -0
- package/bin/lib/test-selection-changes.d.mts +76 -0
- package/bin/lib/test-selection-changes.mjs +272 -0
- package/bin/lib/test-selection-explain.mjs +75 -0
- package/bin/lib/test-selection-glob.d.mts +32 -0
- package/bin/lib/test-selection-glob.mjs +206 -0
- package/bin/lib/test-selection.d.mts +80 -0
- package/bin/lib/test-selection.mjs +418 -0
- package/package.json +4 -2
- package/schemas/test-map.schema.json +164 -0
- package/scripts/__pycache__/codeops_plan.cpython-314.pyc +0 -0
- package/scripts/__pycache__/install_agents.cpython-314.pyc +0 -0
- package/scripts/discovery-toggle.mjs +236 -0
- package/skills/analyze-project/SKILL.md +9 -2
- package/skills/exec-plan/SKILL.md +16 -8
- package/skills/exec-plan/execution-protocol.md +15 -2
- package/skills/git-commit/SKILL.md +1 -1
- package/skills/make-plan/SKILL.md +10 -3
- package/skills/make-plan/quality-checklist.md +4 -0
- package/skills/make-plan/templates.md +19 -1
- package/skills/organize-tests/SKILL.md +169 -0
- package/skills/organize-tests/references/agents-testing-block.md +80 -0
- package/skills/organize-tests/references/analysis-protocol.md +94 -0
- package/skills/organize-tests/references/ci-github-actions.md +178 -0
- package/skills/organize-tests/references/language-adapters.md +54 -0
- package/skills/organize-tests/references/map-health.md +47 -0
- package/skills/organize-tests/references/project-wrapper.md +72 -0
- package/skills/organize-tests/references/shadow-mode.md +60 -0
- package/skills/setup-codeops/SKILL.md +30 -3
- package/standards/coding-standards-full.md +28 -0
- package/standards/coding-standards.md +29 -0
- package/scripts/__pycache__/codeops_plan.cpython-312.pyc +0 -0
- package/scripts/__pycache__/install_agents.cpython-312.pyc +0 -0
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# AGENTS.md Testing Block
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
The testing block tells an agent how to verify this project without re-reading
|
|
6
|
+
the map. It lives inside the existing `CODEOPS-PROJECT` managed block and is
|
|
7
|
+
owned by one writer.
|
|
8
|
+
|
|
9
|
+
## Ownership
|
|
10
|
+
|
|
11
|
+
`organize-tests` is the **sole writer** of the block. `analyze-project` detects
|
|
12
|
+
the block and preserves it byte-for-byte on every refresh; it never renders or
|
|
13
|
+
overwrites it, and it warns when the block is missing or stale. A project that
|
|
14
|
+
has no map is not given a block by `analyze-project`.
|
|
15
|
+
|
|
16
|
+
The block markers are:
|
|
17
|
+
|
|
18
|
+
```markdown
|
|
19
|
+
<!-- CODEOPS-TESTING:START -->
|
|
20
|
+
<!-- CODEOPS-TESTING:END -->
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## With a map
|
|
24
|
+
|
|
25
|
+
```markdown
|
|
26
|
+
<!-- CODEOPS-TESTING:START -->
|
|
27
|
+
- Test map: `codeops/test-map/` (8 groups, 3 slices)
|
|
28
|
+
- Task verify: `npm run verify:task` (affected task-tier groups + dependents)
|
|
29
|
+
- Phase verify: `npm run verify:phase` (task + phase tiers)
|
|
30
|
+
- Gate verify: `npm run verify` (full suite; release/acceptance/CI)
|
|
31
|
+
- Map check: `npm run test:map:check`
|
|
32
|
+
- Last map check: 2026-11-11 23:36
|
|
33
|
+
<!-- CODEOPS-TESTING:END -->
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The commands are the project's real commands. Never invent a script name; if the
|
|
37
|
+
project does not want a wrapper, point at the CLI command instead.
|
|
38
|
+
|
|
39
|
+
## Optional Test slices table
|
|
40
|
+
|
|
41
|
+
A project that wants a map-less reference may add a human-readable table after
|
|
42
|
+
the commands. It is informational; the map remains the source of truth.
|
|
43
|
+
|
|
44
|
+
| Slice | Tier | Tests | Sources | Gate |
|
|
45
|
+
| --- | --- | --- | --- | --- |
|
|
46
|
+
| `billing.invoice-dialog` | task | `web/billing/**/*.test.tsx` | `web/billing/**` | task |
|
|
47
|
+
| `billing.invoice-dialog` | release | `tests/e2e/billing/**` | `web/billing/**` | release |
|
|
48
|
+
| `shared.auth` | task | `shared/auth/**/*.test.ts` | `shared/auth/**` | always-run |
|
|
49
|
+
|
|
50
|
+
The table columns are slice, tier, tests, sources, and gate. Keep it short; a
|
|
51
|
+
large table belongs in the map, not in `AGENTS.md`.
|
|
52
|
+
|
|
53
|
+
## Without a map
|
|
54
|
+
|
|
55
|
+
```markdown
|
|
56
|
+
<!-- CODEOPS-TESTING:START -->
|
|
57
|
+
- Testing: full suite only (no test map)
|
|
58
|
+
- Gate verify: `npm run verify`
|
|
59
|
+
<!-- CODEOPS-TESTING:END -->
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
The no-map text preserves today's behavior. `analyze-project` leaves an existing
|
|
63
|
+
no-map block untouched.
|
|
64
|
+
|
|
65
|
+
## Refresh rules
|
|
66
|
+
|
|
67
|
+
- Replace only the block contents; preserve surrounding managed and
|
|
68
|
+
hand-authored text byte-for-byte.
|
|
69
|
+
- Keep exactly one block; stop on duplicate or reversed markers and name the
|
|
70
|
+
manual fix.
|
|
71
|
+
- Update `Last map check` only after a successful `check`.
|
|
72
|
+
- Write with the file's dominant line ending; keep exactly one block; fail without writing on duplicate or reversed markers.
|
|
73
|
+
- Never put secrets, absolute machine paths, or test output in the block.
|
|
74
|
+
|
|
75
|
+
## Wrapper guidance
|
|
76
|
+
|
|
77
|
+
A thin project wrapper is optional. If the project wants npm scripts, generate a
|
|
78
|
+
small file that calls the plugin CLI with the project's map path and forwards
|
|
79
|
+
arguments. The wrapper contains no selection logic; it must not grow into a
|
|
80
|
+
second implementation. Record the wrapper path in the block.
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Analysis and Inventory Protocol
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
This reference describes how `organize-tests` turns a repository into a proposed
|
|
6
|
+
test map. It is evidence-first: every group traces to a real source path, a real
|
|
7
|
+
test unit, and a command that was observed to run at least one test.
|
|
8
|
+
|
|
9
|
+
## 1. Project reconnaissance
|
|
10
|
+
|
|
11
|
+
Read, without writing:
|
|
12
|
+
|
|
13
|
+
- the Git root and current branch;
|
|
14
|
+
- `codeops/test-map.json` and `codeops/test-map/` when present;
|
|
15
|
+
- manifests and lockfiles (`package.json`, `Cargo.toml`, `go.mod`,
|
|
16
|
+
`pyproject.toml`, `pom.xml`, `build.gradle`, `*.sln`, `CMakeLists.txt`);
|
|
17
|
+
- runner configuration (`vitest.config.*`, `jest.config.*`, `pytest.ini`,
|
|
18
|
+
`nextest.toml`, CTest files);
|
|
19
|
+
- CI workflows, especially the full verify job;
|
|
20
|
+
- the `CODEOPS-PROJECT` and `CODEOPS-TESTING` blocks in `AGENTS.md`.
|
|
21
|
+
|
|
22
|
+
Record the project's full verify command and the exact file or script that
|
|
23
|
+
defines it. A missing full command stops adoption.
|
|
24
|
+
|
|
25
|
+
## 2. Test inventory
|
|
26
|
+
|
|
27
|
+
For each test unit, record:
|
|
28
|
+
|
|
29
|
+
| Field | Meaning |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| `test` | Repo-relative file path, target name, or test ID |
|
|
32
|
+
| `source` | The production path or target that owns it |
|
|
33
|
+
| `runner` | Detected adapter |
|
|
34
|
+
| `count` | Observed number of tests |
|
|
35
|
+
| `durationSec` | Observed or estimated duration |
|
|
36
|
+
|
|
37
|
+
Use native discovery where possible. Do not guess a test count; run the
|
|
38
|
+
candidate command and read the reported number.
|
|
39
|
+
|
|
40
|
+
## 3. Slice proposal
|
|
41
|
+
|
|
42
|
+
Group units by the feature or pipeline stage a developer changes, not by folder
|
|
43
|
+
alone:
|
|
44
|
+
|
|
45
|
+
- business app: dialog, invoice, inventory, auth;
|
|
46
|
+
- compiler: lexer, parser, codegen, assembler, linker, e2e;
|
|
47
|
+
- library: public API, adapters, compatibility;
|
|
48
|
+
- monorepo: workspace package boundaries.
|
|
49
|
+
|
|
50
|
+
One unit belongs to exactly one group. A slice carries one or more tiers of the
|
|
51
|
+
same feature. Split a slice when its task tier exceeds the budget or when its
|
|
52
|
+
tests cover unrelated behavior.
|
|
53
|
+
|
|
54
|
+
## 4. Dependency graph
|
|
55
|
+
|
|
56
|
+
Collect edges from the native graph first. Fall back to production imports, not
|
|
57
|
+
test imports. `dependsOn` lists upstream groups: a parser group is upstream of a
|
|
58
|
+
codegen group. Selection walks reverse edges, so changing the parser selects
|
|
59
|
+
codegen and its dependents, while changing codegen does not select the parser.
|
|
60
|
+
|
|
61
|
+
Shared test utilities are not a dependency shortcut. If a helper is used by
|
|
62
|
+
unrelated slices, keep it in a narrow shared group or mark the consumers
|
|
63
|
+
`alwaysRun`; never merge unrelated slices to satisfy the helper.
|
|
64
|
+
|
|
65
|
+
## 5. Command synthesis
|
|
66
|
+
|
|
67
|
+
Build an argv command per group. Verify it:
|
|
68
|
+
|
|
69
|
+
1. run it once on the current tree;
|
|
70
|
+
2. confirm it exits zero;
|
|
71
|
+
3. confirm the reported test count is greater than zero;
|
|
72
|
+
4. confirm it runs a subset, not the whole suite.
|
|
73
|
+
|
|
74
|
+
Record the count in the preview. A zero-test command is rejected. A command that
|
|
75
|
+
runs the whole suite defeats the purpose and is moved back to the full gate.
|
|
76
|
+
|
|
77
|
+
## 6. Preview contents
|
|
78
|
+
|
|
79
|
+
The preview must show:
|
|
80
|
+
|
|
81
|
+
- the chosen layout (single file or policy plus fragments);
|
|
82
|
+
- every slice, tier, source glob, test glob, dependency, and duration;
|
|
83
|
+
- the exact command per group and its observed count;
|
|
84
|
+
- files that will be created or changed;
|
|
85
|
+
- the AGENTS block diff and the CI diff;
|
|
86
|
+
- validation findings from the staged map.
|
|
87
|
+
|
|
88
|
+
Nothing is written before the preview is accepted.
|
|
89
|
+
|
|
90
|
+
## 7. Evidence to keep
|
|
91
|
+
|
|
92
|
+
Record the observed counts, the shadow verdict, the `check --strict` result, and
|
|
93
|
+
the date of the last map check. Evidence lives in the plan or the adoption
|
|
94
|
+
report, not in shipped source comments.
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
# GitHub Actions Test-Selection Templates
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
These templates are advisory. `organize-tests` previews every diff and never
|
|
6
|
+
removes an existing job. A project can keep its current full gate and adopt only
|
|
7
|
+
the map, or adopt the affected PR job later.
|
|
8
|
+
|
|
9
|
+
## Shape
|
|
10
|
+
|
|
11
|
+
```text
|
|
12
|
+
PR:
|
|
13
|
+
plan(affected) -> test shards -> verify aggregator (required)
|
|
14
|
+
|
|
15
|
+
release/** / hotfix/** / v* tag:
|
|
16
|
+
full gate -> build artifact -> package -> provenance -> deploy
|
|
17
|
+
|
|
18
|
+
acceptance/**:
|
|
19
|
+
full gate + acceptance groups -> provenance
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
## Affected plan and shards
|
|
23
|
+
|
|
24
|
+
```yaml
|
|
25
|
+
jobs:
|
|
26
|
+
plan:
|
|
27
|
+
runs-on: ubuntu-latest
|
|
28
|
+
outputs:
|
|
29
|
+
matrix: ${{ steps.plan.outputs.matrix }}
|
|
30
|
+
mode: ${{ steps.plan.outputs.mode }}
|
|
31
|
+
steps:
|
|
32
|
+
- uses: actions/checkout@v7
|
|
33
|
+
with:
|
|
34
|
+
fetch-depth: 0
|
|
35
|
+
- run: npm ci --ignore-scripts
|
|
36
|
+
- id: plan
|
|
37
|
+
run: node bin/codeops-test.mjs plan --gate pr --since "origin/${{ github.base_ref }}" --json > plan.json
|
|
38
|
+
- run: echo "matrix=$(jq -c .matrix plan.json)" >> "$GITHUB_OUTPUT"
|
|
39
|
+
- run: echo "mode=$(jq -r .mode plan.json)" >> "$GITHUB_OUTPUT"
|
|
40
|
+
- uses: actions/upload-artifact@v4
|
|
41
|
+
with:
|
|
42
|
+
name: test-plan
|
|
43
|
+
path: plan.json
|
|
44
|
+
|
|
45
|
+
test:
|
|
46
|
+
needs: plan
|
|
47
|
+
if: needs.plan.outputs.mode == 'groups' && needs.plan.outputs.matrix != '{"include":[]}'
|
|
48
|
+
strategy:
|
|
49
|
+
fail-fast: false
|
|
50
|
+
matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
|
|
51
|
+
runs-on: ubuntu-latest
|
|
52
|
+
steps:
|
|
53
|
+
- uses: actions/checkout@v7
|
|
54
|
+
- run: npm ci --ignore-scripts
|
|
55
|
+
- uses: actions/download-artifact@v4
|
|
56
|
+
with:
|
|
57
|
+
name: test-plan
|
|
58
|
+
- run: node bin/codeops-test.mjs run --selection plan.json --group "${{ matrix.group }}"
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The plan has four modes: `groups`, `full`, `none`, and `skip`. Every mode
|
|
62
|
+
returns a matrix object. `none` and `skip` use an empty matrix. A full run is a
|
|
63
|
+
separate job, never a fake group entry, so `run --group full` remains a usage
|
|
64
|
+
error.
|
|
65
|
+
|
|
66
|
+
```yaml
|
|
67
|
+
full:
|
|
68
|
+
needs: plan
|
|
69
|
+
if: needs.plan.outputs.mode == 'full'
|
|
70
|
+
uses: ./.github/workflows/verify-full.yml
|
|
71
|
+
|
|
72
|
+
check:
|
|
73
|
+
needs: plan
|
|
74
|
+
runs-on: ubuntu-latest
|
|
75
|
+
steps:
|
|
76
|
+
- uses: actions/checkout@v7
|
|
77
|
+
- run: npm ci --ignore-scripts
|
|
78
|
+
- run: node bin/codeops-test.mjs check --strict
|
|
79
|
+
|
|
80
|
+
verify:
|
|
81
|
+
needs: [plan, test, full, check]
|
|
82
|
+
if: always()
|
|
83
|
+
runs-on: ubuntu-latest
|
|
84
|
+
steps:
|
|
85
|
+
- run: |
|
|
86
|
+
test "${{ needs.plan.result }}" = "success"
|
|
87
|
+
test "${{ needs.check.result }}" = "success"
|
|
88
|
+
if [ "${{ needs.plan.outputs.mode }}" = "groups" ]; then
|
|
89
|
+
test "${{ needs.test.result }}" = "success"
|
|
90
|
+
fi
|
|
91
|
+
if [ "${{ needs.plan.outputs.mode }}" = "full" ]; then
|
|
92
|
+
test "${{ needs.full.result }}" = "success"
|
|
93
|
+
fi
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The stable `verify` job is the required branch-protection check. Dynamic matrix
|
|
97
|
+
names are never required checks. `check --strict` runs beside it so a malformed
|
|
98
|
+
map stays visible even when selection falls back to full.
|
|
99
|
+
|
|
100
|
+
## Release and acceptance
|
|
101
|
+
|
|
102
|
+
Release and acceptance always run the full gate. Build the artifact only after
|
|
103
|
+
the gate, or build first and let the gate consume the artifact.
|
|
104
|
+
|
|
105
|
+
```yaml
|
|
106
|
+
verify:
|
|
107
|
+
uses: ./.github/workflows/verify-full.yml
|
|
108
|
+
|
|
109
|
+
package:
|
|
110
|
+
needs: verify
|
|
111
|
+
runs-on: ubuntu-latest
|
|
112
|
+
steps:
|
|
113
|
+
- uses: actions/checkout@v7
|
|
114
|
+
- run: npm ci --ignore-scripts
|
|
115
|
+
- run: npm run build
|
|
116
|
+
- run: node bin/codeops-test.mjs provenance write --gate release --artifact dist/app.tar.gz --source-commit "$GITHUB_SHA" --record release.json
|
|
117
|
+
- uses: actions/upload-artifact@v4
|
|
118
|
+
with:
|
|
119
|
+
name: release-evidence
|
|
120
|
+
path: release.json
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`--source-commit` records the commit the full gate tested. When the gate ran on
|
|
124
|
+
source, the artifact must be reproducible; `provenance verify` fails on a
|
|
125
|
+
different digest or source commit. Acceptance uses `--gate acceptance` and
|
|
126
|
+
records the manual or external result.
|
|
127
|
+
|
|
128
|
+
## Nightly
|
|
129
|
+
|
|
130
|
+
```yaml
|
|
131
|
+
name: Nightly Full
|
|
132
|
+
on:
|
|
133
|
+
schedule: [{ cron: "0 3 * * *" }]
|
|
134
|
+
workflow_dispatch:
|
|
135
|
+
concurrency:
|
|
136
|
+
group: nightly-full
|
|
137
|
+
cancel-in-progress: false
|
|
138
|
+
jobs:
|
|
139
|
+
plan:
|
|
140
|
+
runs-on: ubuntu-latest
|
|
141
|
+
outputs:
|
|
142
|
+
run_full: ${{ steps.plan.outputs.run_full }}
|
|
143
|
+
steps:
|
|
144
|
+
- uses: actions/checkout@v7
|
|
145
|
+
with: { fetch-depth: 0 }
|
|
146
|
+
- run: npm ci --ignore-scripts
|
|
147
|
+
- id: plan
|
|
148
|
+
env:
|
|
149
|
+
CODEOPS_LAST_FULL_SHA: ${{ vars.CODEOPS_LAST_FULL_SHA }}
|
|
150
|
+
run: node bin/codeops-test.mjs plan --gate nightly --last-full-sha "$CODEOPS_LAST_FULL_SHA" --json > plan.json
|
|
151
|
+
- run: echo "run_full=$(jq -r '.mode != "skip"' plan.json)" >> "$GITHUB_OUTPUT"
|
|
152
|
+
full:
|
|
153
|
+
needs: plan
|
|
154
|
+
if: needs.plan.outputs.run_full == 'true'
|
|
155
|
+
uses: ./.github/workflows/verify-full.yml
|
|
156
|
+
update-marker:
|
|
157
|
+
needs: full
|
|
158
|
+
if: needs.full.result == 'success'
|
|
159
|
+
runs-on: ubuntu-latest
|
|
160
|
+
permissions:
|
|
161
|
+
actions: write
|
|
162
|
+
steps:
|
|
163
|
+
- run: gh variable set CODEOPS_LAST_FULL_SHA --body "$GITHUB_SHA"
|
|
164
|
+
env:
|
|
165
|
+
GH_TOKEN: ${{ github.token }}
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The marker updates only after a successful full run. Never update it from a
|
|
169
|
+
failed or skipped job.
|
|
170
|
+
|
|
171
|
+
## Governance
|
|
172
|
+
|
|
173
|
+
- Protect `codeops/test-map/**` with a `CODEOWNERS` entry; map changes are
|
|
174
|
+
reviewed like CI changes.
|
|
175
|
+
- Use `pull_request`, never `pull_request_target` with fork code. The map
|
|
176
|
+
contains commands and must be treated as code.
|
|
177
|
+
- Preserve existing caching, toolchain setup, permissions, and branch filters.
|
|
178
|
+
- Do not add a path filter that lets release or acceptance skip the full gate.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Language Adapter Recipes
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
The selector core is language-neutral. Each adapter is only a discovery method
|
|
6
|
+
and an argv command stored in the map. Never add a language branch to the core.
|
|
7
|
+
|
|
8
|
+
| Ecosystem | Natural unit | Discovery | Subset command |
|
|
9
|
+
| --- | --- | --- | --- |
|
|
10
|
+
| Node `node --test` | test file | file globs | `node --test {tests}` |
|
|
11
|
+
| Vitest | test file | config/project | `npx vitest run {tests}` |
|
|
12
|
+
| Jest | test file | config | `npx jest --runTestsByPath {tests}` |
|
|
13
|
+
| Rust | crate or target | `cargo metadata` | `cargo test -p <crate> --lib --tests` |
|
|
14
|
+
| Python | file or node ID | `pytest --collect-only` | `python3 -m pytest {tests}` |
|
|
15
|
+
| Go | package | `go list ./...` | `go test ./pkg/...` |
|
|
16
|
+
| JVM | class or package | Gradle/Maven tasks | `./gradlew test --tests com.example.core.*` |
|
|
17
|
+
| .NET | project or class | `dotnet test --list-tests` | `dotnet test --filter FullyQualifiedName~Core` |
|
|
18
|
+
| C/C++ | binary or target | `ctest -N`, Bazel query | `ctest -R ^core_`, Bazel target |
|
|
19
|
+
| Ruby | file | runner config | `rspec {files}` |
|
|
20
|
+
| PHP | file | runner config | `phpunit {files}` |
|
|
21
|
+
| Custom | project unit | project command | project argv |
|
|
22
|
+
|
|
23
|
+
## Target-based groups
|
|
24
|
+
|
|
25
|
+
Rust unit tests are inline in source, Go tests are package-level, and C/C++
|
|
26
|
+
tests are binary/target-level. These runners discover their own unit, so a
|
|
27
|
+
target-based group may omit `tests`. Ownership is then validated through
|
|
28
|
+
`sources` plus a smoke run of the command; the group is exempt from file-glob
|
|
29
|
+
ownership checks.
|
|
30
|
+
|
|
31
|
+
A target-based command must still prove a nonzero test count. The preview records
|
|
32
|
+
that count.
|
|
33
|
+
|
|
34
|
+
## File-based groups
|
|
35
|
+
|
|
36
|
+
Node, Vitest, Jest, Python, Ruby, and PHP groups use `tests` globs. `{tests}`
|
|
37
|
+
expands to every file the group owns, independent of the change set. A
|
|
38
|
+
source-only change therefore still runs the group's whole test set.
|
|
39
|
+
|
|
40
|
+
## Custom adapter
|
|
41
|
+
|
|
42
|
+
Use `--custom` when no recipe fits. Ask for both commands:
|
|
43
|
+
|
|
44
|
+
- the full verify command;
|
|
45
|
+
- the subset command, as argv tokens, with `{tests}` where file paths belong.
|
|
46
|
+
|
|
47
|
+
Run the subset once and record the count. If the command cannot run a subset,
|
|
48
|
+
keep the project on the full gate; do not invent a selector.
|
|
49
|
+
|
|
50
|
+
## Cross-language projects
|
|
51
|
+
|
|
52
|
+
Use one fragment per ecosystem and a shared `policy.json`. The full command may
|
|
53
|
+
be a single umbrella script or an array of argv commands. Never let one
|
|
54
|
+
ecosystem's group command run another ecosystem's suite.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Map Health and Audit
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
A map can be valid JSON and still be unhealthy. The audit looks for drift and
|
|
6
|
+
ownership defects that make selection quietly less useful.
|
|
7
|
+
|
|
8
|
+
## Checks
|
|
9
|
+
|
|
10
|
+
| Finding | Meaning | Action |
|
|
11
|
+
| --- | --- | --- |
|
|
12
|
+
| `map.orphan-test` | A test under a test root belongs to no group | Add it to an owning group, or record an explicit ignore/acceptance decision; the skill never deletes a test |
|
|
13
|
+
| `map.overlap` | A test belongs to more than one group | Give it exactly one owner |
|
|
14
|
+
| `map.unknown-source` | A source path is owned, full-on, or ignored by nothing | Add ownership, a full-on rule, or a documented ignore |
|
|
15
|
+
| `map.slice-no-task` | A slice has no fast task tier | Add a task group or split the slice |
|
|
16
|
+
| `map.slice-no-release` | A slice has no release/acceptance path | Add a release group or a per-slice acceptance record |
|
|
17
|
+
| `map.group.slow` | A task group estimates above 180 s | Split it or move slow tests to a higher tier |
|
|
18
|
+
| `map.empty-group` | A file-based group matches no tracked test | Fix the glob or remove the group |
|
|
19
|
+
| never selected | A group has never run in recorded evidence | Check its sources and dependency edges |
|
|
20
|
+
| flaky candidate | A group fails intermittently | Stabilize the test; do not hide it in a higher tier |
|
|
21
|
+
|
|
22
|
+
A file is a test candidate when a group's `tests` glob claims it or its name follows a
|
|
23
|
+
test convention (`.test.`, `.spec.`, `_test.`, `test_*`, `*Test`, or a `.rs` file under a
|
|
24
|
+
test root). Fixtures and helpers under a test root are not orphan tests.
|
|
25
|
+
|
|
26
|
+
## Strict mode
|
|
27
|
+
|
|
28
|
+
`check --strict` promotes drift warnings to errors and exits 4. Malformed map
|
|
29
|
+
rules still exit 3. Release and acceptance CI runs strict mode so a stale map
|
|
30
|
+
cannot silently widen selection.
|
|
31
|
+
|
|
32
|
+
## Refresh
|
|
33
|
+
|
|
34
|
+
`--refresh` may update globs, dependency edges, and timings. It preserves manual
|
|
35
|
+
group edits, owner fields, and acceptance records. It never renames an ID; a
|
|
36
|
+
renamed group is a new group with a new owner decision.
|
|
37
|
+
|
|
38
|
+
## Slow groups
|
|
39
|
+
|
|
40
|
+
A slow task tier is a design smell, not a selector failure. Split the slice or
|
|
41
|
+
move the slow test to `phase` or `release`. Never drop the test from the map to
|
|
42
|
+
make the budget green.
|
|
43
|
+
|
|
44
|
+
## Reporting
|
|
45
|
+
|
|
46
|
+
Every audit finding names the path or group ID and the rule code. The report is
|
|
47
|
+
read-only; the user decides whether to refresh, split, or accept the finding.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Project Wrapper Guidance
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
A wrapper is optional. It gives a project short npm scripts without copying
|
|
6
|
+
selector logic. The wrapper is a thin adapter over the plugin CLI.
|
|
7
|
+
|
|
8
|
+
## Rules
|
|
9
|
+
|
|
10
|
+
- No selection logic in the wrapper.
|
|
11
|
+
- No language detection, glob parsing, or command rewriting.
|
|
12
|
+
- Forward every argument unchanged.
|
|
13
|
+
- Resolve the project root and map path once, then call the CLI.
|
|
14
|
+
- Keep the file small enough to review in one screen.
|
|
15
|
+
|
|
16
|
+
## Example
|
|
17
|
+
|
|
18
|
+
`scripts/codeops-test.mjs`:
|
|
19
|
+
|
|
20
|
+
```js
|
|
21
|
+
#!/usr/bin/env node
|
|
22
|
+
import { spawnSync } from "node:child_process"
|
|
23
|
+
import { fileURLToPath } from "node:url"
|
|
24
|
+
import { dirname, join } from "node:path"
|
|
25
|
+
|
|
26
|
+
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
|
|
27
|
+
const cli = process.env.CODEOPS_TEST_CLI ?? "codeops-test"
|
|
28
|
+
const result = spawnSync(cli, process.argv.slice(2), {
|
|
29
|
+
cwd: root,
|
|
30
|
+
stdio: "inherit",
|
|
31
|
+
shell: false,
|
|
32
|
+
})
|
|
33
|
+
process.exitCode = result.status ?? 1
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Package scripts:
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"scripts": {
|
|
41
|
+
"verify:task": "node scripts/codeops-test.mjs run --gate task",
|
|
42
|
+
"verify:phase": "node scripts/codeops-test.mjs run --gate phase",
|
|
43
|
+
"test:map:check": "node scripts/codeops-test.mjs check --strict"
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The full verify command stays the project's existing command:
|
|
49
|
+
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"scripts": {
|
|
53
|
+
"verify": "npm run typecheck && npm run test"
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## When not to use a wrapper
|
|
59
|
+
|
|
60
|
+
A project may call `node bin/codeops-test.mjs` or the installed `codeops-test`
|
|
61
|
+
binary directly. The AGENTS testing block names whichever command is real. Never
|
|
62
|
+
invent a script name that does not exist.
|
|
63
|
+
|
|
64
|
+
## Verification
|
|
65
|
+
|
|
66
|
+
After adding a wrapper, run:
|
|
67
|
+
|
|
68
|
+
1. `npm run test:map:check` — the map is valid.
|
|
69
|
+
2. `npm run verify:task` with one real change — a nonzero selected set runs.
|
|
70
|
+
3. `npm run verify` — the full gate still passes.
|
|
71
|
+
|
|
72
|
+
Record the observed counts in the adoption evidence.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Shadow Adoption
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
Shadow mode proves that a proposed map selects every test the full suite would
|
|
6
|
+
fail. It is evidence, not a switch. Adoption requires zero missed failures; never enable the map from a single run.
|
|
7
|
+
|
|
8
|
+
## Procedure
|
|
9
|
+
|
|
10
|
+
1. Compute the selected set for the current diff with an explicit baseline.
|
|
11
|
+
2. Run the selected groups with `--no-fail-fast`; capture every result.
|
|
12
|
+
3. Run the full command regardless of the selected result.
|
|
13
|
+
4. Compare the two results.
|
|
14
|
+
5. Write the evidence record and report the verdict.
|
|
15
|
+
6. Never enable the map automatically; a user or the skill decides after
|
|
16
|
+
reviewing the evidence.
|
|
17
|
+
|
|
18
|
+
## Verdicts
|
|
19
|
+
|
|
20
|
+
| Selected | Full | Verdict | Action |
|
|
21
|
+
| --- | --- | --- | --- |
|
|
22
|
+
| pass | pass | `zero-missed` | Candidate for adoption after review |
|
|
23
|
+
| pass | fail | `escaped` | Refuse adoption; name the escaped failure |
|
|
24
|
+
| fail | pass | `selected-failure` | Refuse adoption; fix selection or flake |
|
|
25
|
+
| fail | fail | `both-failed` | No adoption signal; report both tails |
|
|
26
|
+
|
|
27
|
+
The skill must refuse adoption on an escaped verdict. An `escaped` verdict is the dangerous case: the selected slice was green while
|
|
28
|
+
the full suite was red. The report must name the escaping group and include the
|
|
29
|
+
full failure tail. Do not explain it away as flaky without evidence.
|
|
30
|
+
|
|
31
|
+
## Evidence shape
|
|
32
|
+
|
|
33
|
+
```json
|
|
34
|
+
{
|
|
35
|
+
"verdict": "zero-missed",
|
|
36
|
+
"gate": "pr",
|
|
37
|
+
"changed": ["src/a/dialog.ts"],
|
|
38
|
+
"mapHash": "<sha256>",
|
|
39
|
+
"selected": ["a.unit"],
|
|
40
|
+
"selectedResult": "passed",
|
|
41
|
+
"fullResult": "passed",
|
|
42
|
+
"escaped": [],
|
|
43
|
+
"enabled": false
|
|
44
|
+
}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The record contains no test output, secrets, or environment values. Failure
|
|
48
|
+
tails may appear in the human report but not in a committed map.
|
|
49
|
+
|
|
50
|
+
## Repetition
|
|
51
|
+
|
|
52
|
+
Run shadow mode on several real change sets, including:
|
|
53
|
+
|
|
54
|
+
- a single-slice change;
|
|
55
|
+
- a shared-contract change;
|
|
56
|
+
- a change that should force full;
|
|
57
|
+
- a docs-only change.
|
|
58
|
+
|
|
59
|
+
A single green shadow run is not enough to prove a selector. Adopt only after
|
|
60
|
+
the project has seen zero missed failures across representative changes.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: setup-codeops
|
|
3
3
|
description: >-
|
|
4
|
-
Sets up or upgrades CodeOps in the current git repo. It scaffolds a fresh nested codeops/ layout, auto-migrates a flat requirements/ + plans/ layout, and detects obsolete traceability.json workflow-state graphs in an existing nested project. Use when the user says "setup-codeops", "/setup-codeops", "set up codeops", "initialize codeops", "upgrade existing codeops", "migrate to the nested layout", or "scaffold the codeops structure". Supports --dry-run
|
|
4
|
+
Sets up or upgrades CodeOps in the current git repo. It scaffolds a fresh nested codeops/ layout, auto-migrates a flat requirements/ + plans/ layout, and detects obsolete traceability.json workflow-state graphs in an existing nested project. Use when the user says "setup-codeops", "/setup-codeops", "set up codeops", "initialize codeops", "upgrade existing codeops", "migrate to the nested layout", or "scaffold the codeops structure". Supports --dry-run, unattended --yes, and --enable-discovery/--disable-discovery for existing projects. Every migration previews deterministically, refuses unsafe or ambiguous apply, requires clean Git state for writes, and is idempotent. setup-codeops is the SOLE writer of the layout marker.
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
# CodeOps Layout Setup (`setup-codeops`)
|
|
@@ -21,8 +21,9 @@ it is the single source of truth for the layout. Do not re-encode paths here.
|
|
|
21
21
|
- This skill sets up the *structure*. It never authors requirements, plans, or roadmaps — that is
|
|
22
22
|
`make-requirements` / `make-plan` / `roadmap`, which then resolve paths via the convention doc.
|
|
23
23
|
- A fresh scaffold enables Discovery by default (`discovery.enabled: true`,
|
|
24
|
-
`chatMaxQuestions: 3`). Existing projects
|
|
25
|
-
`
|
|
24
|
+
`chatMaxQuestions: 3`). Existing projects can toggle it with
|
|
25
|
+
`setup-codeops --enable-discovery` / `--disable-discovery`; `analyze-project --with-discovery`
|
|
26
|
+
remains available for the same config change with guidance refresh.
|
|
26
27
|
|
|
27
28
|
## Dispatch — detect repo state, then branch
|
|
28
29
|
|
|
@@ -59,6 +60,31 @@ fresh scaffold should live in version control) and suggest `git init` first.
|
|
|
59
60
|
| *(none)* | Interactive: scaffold creates the skeleton; either migration previews then asks for one confirmation before applying. |
|
|
60
61
|
| `--dry-run` | Preview only — compute and show what would happen; change **nothing**. |
|
|
61
62
|
| `--yes` | Apply an unblocked migration without confirmation, then verify it. Safety refusals still apply. |
|
|
63
|
+
| `--enable-discovery` | Enable Discovery in an existing project without re-scaffolding. Preserves every other config key. |
|
|
64
|
+
| `--disable-discovery` | Disable Discovery, keeping `chatMaxQuestions` and every other config key. |
|
|
65
|
+
|
|
66
|
+
`--enable-discovery` and `--disable-discovery` are mutually exclusive. Validate that before the
|
|
67
|
+
layout branch so a usage error never changes the repository.
|
|
68
|
+
|
|
69
|
+
## Discovery toggle
|
|
70
|
+
|
|
71
|
+
After the layout branch (no-op, migration, or fresh scaffold), apply exactly one discovery flag
|
|
72
|
+
when present:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
node "${CODEOPS_PLUGIN_ROOT}/scripts/discovery-toggle.mjs" --root . --enable-discovery
|
|
76
|
+
node "${CODEOPS_PLUGIN_ROOT}/scripts/discovery-toggle.mjs" --root . --disable-discovery
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Add `--dry-run` to preview. The script validates the JSON, refuses symlinked `codeops/` or
|
|
80
|
+
`codeops/codeops.json`, preserves unrelated policy, and adds or removes the managed line
|
|
81
|
+
`- Discovery: enabled — question rounds open in the local Discovery page` when the
|
|
82
|
+
`CODEOPS-PROJECT` block exists. If `AGENTS.md` is absent or has no managed block, the config still
|
|
83
|
+
changes; report that guidance should be refreshed with `analyze-project`.
|
|
84
|
+
|
|
85
|
+
A fresh scaffold starts enabled. Passing `--disable-discovery` during scaffold turns it off; passing
|
|
86
|
+
`--enable-discovery` is idempotent. Without either flag, an existing project keeps its current
|
|
87
|
+
setting and a fresh scaffold stays enabled.
|
|
62
88
|
|
|
63
89
|
## Migration engines (delegation — do not re-implement)
|
|
64
90
|
|
|
@@ -86,6 +112,7 @@ deletion, and idempotency.
|
|
|
86
112
|
|
|
87
113
|
- [scaffold.md](scaffold.md) — the minimal fresh-repo skeleton.
|
|
88
114
|
- [migration.md](migration.md) — flat-layout and legacy workflow-state migration UX.
|
|
115
|
+
- `scripts/discovery-toggle.mjs` — deterministic config and managed-line toggle.
|
|
89
116
|
- [_shared/layout-convention.md](../../_shared/layout-convention.md) — the layout/path/ID/marker source of truth.
|
|
90
117
|
|
|
91
118
|
## Grounded Options & Recommendations
|
|
@@ -71,6 +71,34 @@ These apply to all code I write unless this project's `AGENTS.md` overrides a sp
|
|
|
71
71
|
|
|
72
72
|
# Testing standards
|
|
73
73
|
|
|
74
|
+
## Prime directives for verification
|
|
75
|
+
|
|
76
|
+
1. **Test-first is unchanged.** Specification tests are written before implementation and run
|
|
77
|
+
red, then green. Selection never changes this.
|
|
78
|
+
2. **The inner loop runs the smallest safe verification slice.** With a valid map, a task runs the
|
|
79
|
+
selected feature-slice `task` groups plus fast static checks. Without a map, a plan's
|
|
80
|
+
`**Task verify**` command may provide the same scoped behavior; otherwise the full verify runs
|
|
81
|
+
as today.
|
|
82
|
+
3. **The outer gate is always full.** The full verify runs at phase/plan completion, before
|
|
83
|
+
push/release, at acceptance, and in CI. A green slice is not a release signal.
|
|
84
|
+
4. **Selection fails closed.** Unknown paths, shared infrastructure, manifests/lockfiles,
|
|
85
|
+
test-runner/config changes, CI changes, stale/invalid maps, and selector errors select the
|
|
86
|
+
full suite. No silent skips.
|
|
87
|
+
5. **Spec tests are never skipped.** The feature's spec test file always runs, even if the map
|
|
88
|
+
would not select its group. The executor passes the plan's declared spec file(s) with
|
|
89
|
+
`--include`; the selector adds the owning group, or falls back to full if the spec file is
|
|
90
|
+
unowned.
|
|
91
|
+
6. **Evidence is explicit.** Verification names the groups or scoped command used, what was
|
|
92
|
+
deferred, and the baseline. CI remains authoritative.
|
|
93
|
+
|
|
94
|
+
**Conditional wording.** Run the project's full verify command before completing any task or
|
|
95
|
+
committing, unless the project has a valid `codeops/test-map.json` or `codeops/test-map/`
|
|
96
|
+
directory, or the plan records a validated `**Task verify**` scoped command. With either opt-in,
|
|
97
|
+
the task-level verify may run the selected test groups plus the project's fast static checks, or
|
|
98
|
+
the plan-scoped command. The full verify still runs at phase/plan completion, before push/merge,
|
|
99
|
+
at acceptance, and in CI. Any selection error, unknown path, shared change, or stale map falls
|
|
100
|
+
back to the full verify. The canonical protocol is `_shared/test-selection.md`.
|
|
101
|
+
|
|
74
102
|
- **Run the project's verify command (build + test) before completing any task or committing.** No code is "done" while any test fails.
|
|
75
103
|
- **Targeted vs. full:** iterate with targeted tests, but run the **full** verify before declaring completion.
|
|
76
104
|
- **Maximum, granular coverage:** happy path, edge/boundary cases, error/invalid inputs, and integration — each test focused on one thing with a clear failure message.
|