opencode-codeops 2.5.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +39 -0
- package/_shared/spec-first-ordering.md +6 -2
- package/_shared/test-selection.md +130 -0
- package/bin/codeops-test.mjs +855 -0
- package/bin/lib/test-branch-policy.d.mts +51 -0
- package/bin/lib/test-branch-policy.mjs +185 -0
- package/bin/lib/test-map-validate.d.mts +42 -0
- package/bin/lib/test-map-validate.mjs +618 -0
- package/bin/lib/test-map.d.mts +114 -0
- package/bin/lib/test-map.mjs +384 -0
- package/bin/lib/test-provenance.d.mts +73 -0
- package/bin/lib/test-provenance.mjs +188 -0
- package/bin/lib/test-runner.d.mts +72 -0
- package/bin/lib/test-runner.mjs +350 -0
- package/bin/lib/test-selection-changes.d.mts +76 -0
- package/bin/lib/test-selection-changes.mjs +272 -0
- package/bin/lib/test-selection-explain.mjs +75 -0
- package/bin/lib/test-selection-glob.d.mts +32 -0
- package/bin/lib/test-selection-glob.mjs +206 -0
- package/bin/lib/test-selection.d.mts +80 -0
- package/bin/lib/test-selection.mjs +418 -0
- package/package.json +4 -2
- package/schemas/test-map.schema.json +164 -0
- package/scripts/__pycache__/codeops_plan.cpython-314.pyc +0 -0
- package/scripts/__pycache__/install_agents.cpython-314.pyc +0 -0
- package/skills/analyze-project/SKILL.md +8 -1
- package/skills/exec-plan/SKILL.md +16 -8
- package/skills/exec-plan/execution-protocol.md +15 -2
- package/skills/git-commit/SKILL.md +1 -1
- package/skills/make-plan/SKILL.md +10 -3
- package/skills/make-plan/quality-checklist.md +4 -0
- package/skills/make-plan/templates.md +19 -1
- package/skills/organize-tests/SKILL.md +169 -0
- package/skills/organize-tests/references/agents-testing-block.md +80 -0
- package/skills/organize-tests/references/analysis-protocol.md +94 -0
- package/skills/organize-tests/references/ci-github-actions.md +178 -0
- package/skills/organize-tests/references/language-adapters.md +54 -0
- package/skills/organize-tests/references/map-health.md +47 -0
- package/skills/organize-tests/references/project-wrapper.md +72 -0
- package/skills/organize-tests/references/shadow-mode.md +60 -0
- package/standards/coding-standards-full.md +28 -0
- package/standards/coding-standards.md +29 -0
- package/scripts/__pycache__/codeops_plan.cpython-312.pyc +0 -0
- package/scripts/__pycache__/install_agents.cpython-312.pyc +0 -0
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Language Adapter Recipes
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
The selector core is language-neutral. Each adapter is only a discovery method
|
|
6
|
+
and an argv command stored in the map. Never add a language branch to the core.
|
|
7
|
+
|
|
8
|
+
| Ecosystem | Natural unit | Discovery | Subset command |
|
|
9
|
+
| --- | --- | --- | --- |
|
|
10
|
+
| Node `node --test` | test file | file globs | `node --test {tests}` |
|
|
11
|
+
| Vitest | test file | config/project | `npx vitest run {tests}` |
|
|
12
|
+
| Jest | test file | config | `npx jest --runTestsByPath {tests}` |
|
|
13
|
+
| Rust | crate or target | `cargo metadata` | `cargo test -p <crate> --lib --tests` |
|
|
14
|
+
| Python | file or node ID | `pytest --collect-only` | `python3 -m pytest {tests}` |
|
|
15
|
+
| Go | package | `go list ./...` | `go test ./pkg/...` |
|
|
16
|
+
| JVM | class or package | Gradle/Maven tasks | `./gradlew test --tests com.example.core.*` |
|
|
17
|
+
| .NET | project or class | `dotnet test --list-tests` | `dotnet test --filter FullyQualifiedName~Core` |
|
|
18
|
+
| C/C++ | binary or target | `ctest -N`, Bazel query | `ctest -R ^core_`, Bazel target |
|
|
19
|
+
| Ruby | file | runner config | `rspec {files}` |
|
|
20
|
+
| PHP | file | runner config | `phpunit {files}` |
|
|
21
|
+
| Custom | project unit | project command | project argv |
|
|
22
|
+
|
|
23
|
+
## Target-based groups
|
|
24
|
+
|
|
25
|
+
Rust unit tests are inline in source, Go tests are package-level, and C/C++
|
|
26
|
+
tests are binary/target-level. These runners discover their own unit, so a
|
|
27
|
+
target-based group may omit `tests`. Ownership is then validated through
|
|
28
|
+
`sources` plus a smoke run of the command; the group is exempt from file-glob
|
|
29
|
+
ownership checks.
|
|
30
|
+
|
|
31
|
+
A target-based command must still prove a nonzero test count. The preview records
|
|
32
|
+
that count.
|
|
33
|
+
|
|
34
|
+
## File-based groups
|
|
35
|
+
|
|
36
|
+
Node, Vitest, Jest, Python, Ruby, and PHP groups use `tests` globs. `{tests}`
|
|
37
|
+
expands to every file the group owns, independent of the change set. A
|
|
38
|
+
source-only change therefore still runs the group's whole test set.
|
|
39
|
+
|
|
40
|
+
## Custom adapter
|
|
41
|
+
|
|
42
|
+
Use `--custom` when no recipe fits. Ask for both commands:
|
|
43
|
+
|
|
44
|
+
- the full verify command;
|
|
45
|
+
- the subset command, as argv tokens, with `{tests}` where file paths belong.
|
|
46
|
+
|
|
47
|
+
Run the subset once and record the count. If the command cannot run a subset,
|
|
48
|
+
keep the project on the full gate; do not invent a selector.
|
|
49
|
+
|
|
50
|
+
## Cross-language projects
|
|
51
|
+
|
|
52
|
+
Use one fragment per ecosystem and a shared `policy.json`. The full command may
|
|
53
|
+
be a single umbrella script or an array of argv commands. Never let one
|
|
54
|
+
ecosystem's group command run another ecosystem's suite.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Map Health and Audit
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
A map can be valid JSON and still be unhealthy. The audit looks for drift and
|
|
6
|
+
ownership defects that make selection quietly less useful.
|
|
7
|
+
|
|
8
|
+
## Checks
|
|
9
|
+
|
|
10
|
+
| Finding | Meaning | Action |
|
|
11
|
+
| --- | --- | --- |
|
|
12
|
+
| `map.orphan-test` | A test under a test root belongs to no group | Add it to an owning group, or record an explicit ignore/acceptance decision; the skill never deletes a test |
|
|
13
|
+
| `map.overlap` | A test belongs to more than one group | Give it exactly one owner |
|
|
14
|
+
| `map.unknown-source` | A source path is owned, full-on, or ignored by nothing | Add ownership, a full-on rule, or a documented ignore |
|
|
15
|
+
| `map.slice-no-task` | A slice has no fast task tier | Add a task group or split the slice |
|
|
16
|
+
| `map.slice-no-release` | A slice has no release/acceptance path | Add a release group or a per-slice acceptance record |
|
|
17
|
+
| `map.group.slow` | A task group estimates above 180 s | Split it or move slow tests to a higher tier |
|
|
18
|
+
| `map.empty-group` | A file-based group matches no tracked test | Fix the glob or remove the group |
|
|
19
|
+
| never selected | A group has never run in recorded evidence | Check its sources and dependency edges |
|
|
20
|
+
| flaky candidate | A group fails intermittently | Stabilize the test; do not hide it in a higher tier |
|
|
21
|
+
|
|
22
|
+
A file is a test candidate when a group's `tests` glob claims it or its name follows a
|
|
23
|
+
test convention (`.test.`, `.spec.`, `_test.`, `test_*`, `*Test`, or a `.rs` file under a
|
|
24
|
+
test root). Fixtures and helpers under a test root are not orphan tests.
|
|
25
|
+
|
|
26
|
+
## Strict mode
|
|
27
|
+
|
|
28
|
+
`check --strict` promotes drift warnings to errors and exits 4. Malformed map
|
|
29
|
+
rules still exit 3. Release and acceptance CI runs strict mode so a stale map
|
|
30
|
+
cannot silently widen selection.
|
|
31
|
+
|
|
32
|
+
## Refresh
|
|
33
|
+
|
|
34
|
+
`--refresh` may update globs, dependency edges, and timings. It preserves manual
|
|
35
|
+
group edits, owner fields, and acceptance records. It never renames an ID; a
|
|
36
|
+
renamed group is a new group with a new owner decision.
|
|
37
|
+
|
|
38
|
+
## Slow groups
|
|
39
|
+
|
|
40
|
+
A slow task tier is a design smell, not a selector failure. Split the slice or
|
|
41
|
+
move the slow test to `phase` or `release`. Never drop the test from the map to
|
|
42
|
+
make the budget green.
|
|
43
|
+
|
|
44
|
+
## Reporting
|
|
45
|
+
|
|
46
|
+
Every audit finding names the path or group ID and the rule code. The report is
|
|
47
|
+
read-only; the user decides whether to refresh, split, or accept the finding.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Project Wrapper Guidance
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
A wrapper is optional. It gives a project short npm scripts without copying
|
|
6
|
+
selector logic. The wrapper is a thin adapter over the plugin CLI.
|
|
7
|
+
|
|
8
|
+
## Rules
|
|
9
|
+
|
|
10
|
+
- No selection logic in the wrapper.
|
|
11
|
+
- No language detection, glob parsing, or command rewriting.
|
|
12
|
+
- Forward every argument unchanged.
|
|
13
|
+
- Resolve the project root and map path once, then call the CLI.
|
|
14
|
+
- Keep the file small enough to review in one screen.
|
|
15
|
+
|
|
16
|
+
## Example
|
|
17
|
+
|
|
18
|
+
`scripts/codeops-test.mjs`:
|
|
19
|
+
|
|
20
|
+
```js
|
|
21
|
+
#!/usr/bin/env node
|
|
22
|
+
import { spawnSync } from "node:child_process"
|
|
23
|
+
import { fileURLToPath } from "node:url"
|
|
24
|
+
import { dirname, join } from "node:path"
|
|
25
|
+
|
|
26
|
+
const root = join(dirname(fileURLToPath(import.meta.url)), "..")
|
|
27
|
+
const cli = process.env.CODEOPS_TEST_CLI ?? "codeops-test"
|
|
28
|
+
const result = spawnSync(cli, process.argv.slice(2), {
|
|
29
|
+
cwd: root,
|
|
30
|
+
stdio: "inherit",
|
|
31
|
+
shell: false,
|
|
32
|
+
})
|
|
33
|
+
process.exitCode = result.status ?? 1
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Package scripts:
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"scripts": {
|
|
41
|
+
"verify:task": "node scripts/codeops-test.mjs run --gate task",
|
|
42
|
+
"verify:phase": "node scripts/codeops-test.mjs run --gate phase",
|
|
43
|
+
"test:map:check": "node scripts/codeops-test.mjs check --strict"
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The full verify command stays the project's existing command:
|
|
49
|
+
|
|
50
|
+
```json
|
|
51
|
+
{
|
|
52
|
+
"scripts": {
|
|
53
|
+
"verify": "npm run typecheck && npm run test"
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## When not to use a wrapper
|
|
59
|
+
|
|
60
|
+
A project may call `node bin/codeops-test.mjs` or the installed `codeops-test`
|
|
61
|
+
binary directly. The AGENTS testing block names whichever command is real. Never
|
|
62
|
+
invent a script name that does not exist.
|
|
63
|
+
|
|
64
|
+
## Verification
|
|
65
|
+
|
|
66
|
+
After adding a wrapper, run:
|
|
67
|
+
|
|
68
|
+
1. `npm run test:map:check` — the map is valid.
|
|
69
|
+
2. `npm run verify:task` with one real change — a nonzero selected set runs.
|
|
70
|
+
3. `npm run verify` — the full gate still passes.
|
|
71
|
+
|
|
72
|
+
Record the observed counts in the adoption evidence.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Shadow Adoption
|
|
2
|
+
|
|
3
|
+
> **CodeOps Artifact Schema**: 1
|
|
4
|
+
|
|
5
|
+
Shadow mode proves that a proposed map selects every test the full suite would
|
|
6
|
+
fail. It is evidence, not a switch. Adoption requires zero missed failures; never enable the map from a single run.
|
|
7
|
+
|
|
8
|
+
## Procedure
|
|
9
|
+
|
|
10
|
+
1. Compute the selected set for the current diff with an explicit baseline.
|
|
11
|
+
2. Run the selected groups with `--no-fail-fast`; capture every result.
|
|
12
|
+
3. Run the full command regardless of the selected result.
|
|
13
|
+
4. Compare the two results.
|
|
14
|
+
5. Write the evidence record and report the verdict.
|
|
15
|
+
6. Never enable the map automatically; a user or the skill decides after
|
|
16
|
+
reviewing the evidence.
|
|
17
|
+
|
|
18
|
+
## Verdicts
|
|
19
|
+
|
|
20
|
+
| Selected | Full | Verdict | Action |
|
|
21
|
+
| --- | --- | --- | --- |
|
|
22
|
+
| pass | pass | `zero-missed` | Candidate for adoption after review |
|
|
23
|
+
| pass | fail | `escaped` | Refuse adoption; name the escaped failure |
|
|
24
|
+
| fail | pass | `selected-failure` | Refuse adoption; fix selection or flake |
|
|
25
|
+
| fail | fail | `both-failed` | No adoption signal; report both tails |
|
|
26
|
+
|
|
27
|
+
The skill must refuse adoption on an escaped verdict. An `escaped` verdict is the dangerous case: the selected slice was green while
|
|
28
|
+
the full suite was red. The report must name the escaping group and include the
|
|
29
|
+
full failure tail. Do not explain it away as flaky without evidence.
|
|
30
|
+
|
|
31
|
+
## Evidence shape
|
|
32
|
+
|
|
33
|
+
```json
|
|
34
|
+
{
|
|
35
|
+
"verdict": "zero-missed",
|
|
36
|
+
"gate": "pr",
|
|
37
|
+
"changed": ["src/a/dialog.ts"],
|
|
38
|
+
"mapHash": "<sha256>",
|
|
39
|
+
"selected": ["a.unit"],
|
|
40
|
+
"selectedResult": "passed",
|
|
41
|
+
"fullResult": "passed",
|
|
42
|
+
"escaped": [],
|
|
43
|
+
"enabled": false
|
|
44
|
+
}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The record contains no test output, secrets, or environment values. Failure
|
|
48
|
+
tails may appear in the human report but not in a committed map.
|
|
49
|
+
|
|
50
|
+
## Repetition
|
|
51
|
+
|
|
52
|
+
Run shadow mode on several real change sets, including:
|
|
53
|
+
|
|
54
|
+
- a single-slice change;
|
|
55
|
+
- a shared-contract change;
|
|
56
|
+
- a change that should force full;
|
|
57
|
+
- a docs-only change.
|
|
58
|
+
|
|
59
|
+
A single green shadow run is not enough to prove a selector. Adopt only after
|
|
60
|
+
the project has seen zero missed failures across representative changes.
|
|
@@ -71,6 +71,34 @@ These apply to all code I write unless this project's `AGENTS.md` overrides a sp
|
|
|
71
71
|
|
|
72
72
|
# Testing standards
|
|
73
73
|
|
|
74
|
+
## Prime directives for verification
|
|
75
|
+
|
|
76
|
+
1. **Test-first is unchanged.** Specification tests are written before implementation and run
|
|
77
|
+
red, then green. Selection never changes this.
|
|
78
|
+
2. **The inner loop runs the smallest safe verification slice.** With a valid map, a task runs the
|
|
79
|
+
selected feature-slice `task` groups plus fast static checks. Without a map, a plan's
|
|
80
|
+
`**Task verify**` command may provide the same scoped behavior; otherwise the full verify runs
|
|
81
|
+
as today.
|
|
82
|
+
3. **The outer gate is always full.** The full verify runs at phase/plan completion, before
|
|
83
|
+
push/release, at acceptance, and in CI. A green slice is not a release signal.
|
|
84
|
+
4. **Selection fails closed.** Unknown paths, shared infrastructure, manifests/lockfiles,
|
|
85
|
+
test-runner/config changes, CI changes, stale/invalid maps, and selector errors select the
|
|
86
|
+
full suite. No silent skips.
|
|
87
|
+
5. **Spec tests are never skipped.** The feature's spec test file always runs, even if the map
|
|
88
|
+
would not select its group. The executor passes the plan's declared spec file(s) with
|
|
89
|
+
`--include`; the selector adds the owning group, or falls back to full if the spec file is
|
|
90
|
+
unowned.
|
|
91
|
+
6. **Evidence is explicit.** Verification names the groups or scoped command used, what was
|
|
92
|
+
deferred, and the baseline. CI remains authoritative.
|
|
93
|
+
|
|
94
|
+
**Conditional wording.** Run the project's full verify command before completing any task or
|
|
95
|
+
committing, unless the project has a valid `codeops/test-map.json` or `codeops/test-map/`
|
|
96
|
+
directory, or the plan records a validated `**Task verify**` scoped command. With either opt-in,
|
|
97
|
+
the task-level verify may run the selected test groups plus the project's fast static checks, or
|
|
98
|
+
the plan-scoped command. The full verify still runs at phase/plan completion, before push/merge,
|
|
99
|
+
at acceptance, and in CI. Any selection error, unknown path, shared change, or stale map falls
|
|
100
|
+
back to the full verify. The canonical protocol is `_shared/test-selection.md`.
|
|
101
|
+
|
|
74
102
|
- **Run the project's verify command (build + test) before completing any task or committing.** No code is "done" while any test fails.
|
|
75
103
|
- **Targeted vs. full:** iterate with targeted tests, but run the **full** verify before declaring completion.
|
|
76
104
|
- **Maximum, granular coverage:** happy path, edge/boundary cases, error/invalid inputs, and integration — each test focused on one thing with a clear failure message.
|
|
@@ -30,6 +30,35 @@ Do not duplicate these standards in `~/.config/opencode/AGENTS.md` — the plugi
|
|
|
30
30
|
containers.
|
|
31
31
|
|
|
32
32
|
# Testing standards
|
|
33
|
+
|
|
34
|
+
## Prime directives for verification
|
|
35
|
+
|
|
36
|
+
1. **Test-first is unchanged.** Specification tests are written before implementation and run
|
|
37
|
+
red, then green. Selection never changes this.
|
|
38
|
+
2. **The inner loop runs the smallest safe verification slice.** With a valid map, a task runs the
|
|
39
|
+
selected feature-slice `task` groups plus fast static checks. Without a map, a plan's
|
|
40
|
+
`**Task verify**` command may provide the same scoped behavior; otherwise the full verify runs
|
|
41
|
+
as today.
|
|
42
|
+
3. **The outer gate is always full.** The full verify runs at phase/plan completion, before
|
|
43
|
+
push/release, at acceptance, and in CI. A green slice is not a release signal.
|
|
44
|
+
4. **Selection fails closed.** Unknown paths, shared infrastructure, manifests/lockfiles,
|
|
45
|
+
test-runner/config changes, CI changes, stale/invalid maps, and selector errors select the
|
|
46
|
+
full suite. No silent skips.
|
|
47
|
+
5. **Spec tests are never skipped.** The feature's spec test file always runs, even if the map
|
|
48
|
+
would not select its group. The executor passes the plan's declared spec file(s) with
|
|
49
|
+
`--include`; the selector adds the owning group, or falls back to full if the spec file is
|
|
50
|
+
unowned.
|
|
51
|
+
6. **Evidence is explicit.** Verification names the groups or scoped command used, what was
|
|
52
|
+
deferred, and the baseline. CI remains authoritative.
|
|
53
|
+
|
|
54
|
+
**Conditional wording.** Run the project's full verify command before completing any task or
|
|
55
|
+
committing, unless the project has a valid `codeops/test-map.json` or `codeops/test-map/`
|
|
56
|
+
directory, or the plan records a validated `**Task verify**` scoped command. With either opt-in,
|
|
57
|
+
the task-level verify may run the selected groups or the scoped command; the full verify still
|
|
58
|
+
runs at phase/plan completion, before push/merge, at acceptance, and in CI. Any selection error,
|
|
59
|
+
unknown path, shared change, or stale map falls back to the full verify. The canonical detail is
|
|
60
|
+
`_shared/test-selection.md`.
|
|
61
|
+
|
|
33
62
|
- **Run the project's verify command before completing any task or committing**; full verify
|
|
34
63
|
before declaring done. No code is "done" while any test fails.
|
|
35
64
|
- Granular coverage (happy path, edges, errors, integration); E2E where feasible; real objects
|
|
Binary file
|
|
Binary file
|