@heihei0299/matt-skills 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/instance-test/SKILL.md +61 -0
- package/.agents/skills/instance-test/agents/openai.yaml +5 -0
- package/.agents/skills/instance-test/references/instances.md +45 -0
- package/bin/cli.js +1 -1
- package/package.json +3 -2
- package/template/.opencode/skills/instance-test/SKILL.md +61 -0
- package/template/.opencode/skills/instance-test/agents/openai.yaml +5 -0
- package/template/.opencode/skills/instance-test/references/instances.md +45 -0
- package/template/.pi/skills/instance-test/SKILL.md +61 -0
- package/template/.pi/skills/instance-test/agents/openai.yaml +5 -0
- package/template/.pi/skills/instance-test/references/instances.md +45 -0
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: instance-test
|
|
3
|
+
disable-model-invocation: true
|
|
4
|
+
description: "Verify project meets expected goals by running prompt instances in isolated temp dirs"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Instance Test
|
|
8
|
+
|
|
9
|
+
Run **instance** prompts to verify project meets expected goals via actual functional tests. Each **instance** is a prompt + expected outcome, executed in an isolated temp dir — no mocks, no stubs.
|
|
10
|
+
|
|
11
|
+
## Steps
|
|
12
|
+
|
|
13
|
+
### 1. Gather instances
|
|
14
|
+
|
|
15
|
+
Collect the **instance** set to run:
|
|
16
|
+
|
|
17
|
+
- User-provided instances (prompt, command, expected files/stdout/exit code), or
|
|
18
|
+
- Derived from `spec.md`/`README` acceptance criteria — extract each verifiable behavior as one instance, then confirm the list with the user before running.
|
|
19
|
+
|
|
20
|
+
Each **instance** must declare: command to run, expected files/content, expected stdout phrases, expected exit code.
|
|
21
|
+
|
|
22
|
+
Completion: instance list is fixed (prompt, expected outcome, verification command) — no instance is added mid-run.
|
|
23
|
+
|
|
24
|
+
### 2. Run instances
|
|
25
|
+
|
|
26
|
+
For each **instance** in order:
|
|
27
|
+
|
|
28
|
+
1. `mktemp -d` isolated dir (or `git worktree` / `--dest` if the project supports it).
|
|
29
|
+
2. Execute the instance's command — capture stdout/stderr and exit code.
|
|
30
|
+
3. Snapshot result files and side effects declared in expected.
|
|
31
|
+
|
|
32
|
+
Do not run instances in parallel — one **instance** at a time, so failures are isolated and artifacts do not collide.
|
|
33
|
+
|
|
34
|
+
Completion: every **instance** has a run dir with captured output and file snapshot.
|
|
35
|
+
|
|
36
|
+
### 3. Evaluate
|
|
37
|
+
|
|
38
|
+
Compare each **instance**'s actual vs expected:
|
|
39
|
+
|
|
40
|
+
- File existence/content (`test -f`, `grep -q`, `diff`).
|
|
41
|
+
- Stdout/stderr contains expected phrases.
|
|
42
|
+
- Exit code matches expected.
|
|
43
|
+
|
|
44
|
+
Mark `PASS`/`FAIL` per **instance** with evidence (file path, stdout line, or diff).
|
|
45
|
+
|
|
46
|
+
Completion: every **instance** has a `PASS` or `FAIL` with evidence — no unevaluated instance.
|
|
47
|
+
|
|
48
|
+
### 4. Report
|
|
49
|
+
|
|
50
|
+
Summarize in conversation:
|
|
51
|
+
|
|
52
|
+
- `PASS m/n` with per-instance evidence.
|
|
53
|
+
- Failures list the gap (expected vs actual) and the run dir for reproduction.
|
|
54
|
+
- Clean up temp dirs unless `--keep` is requested.
|
|
55
|
+
|
|
56
|
+
Do not write a report file (`report-*.md`) — output stays in conversation. Keep temp dirs only on failure for debugging.
|
|
57
|
+
|
|
58
|
+
## References
|
|
59
|
+
|
|
60
|
+
- Instance definitions (if any): `references/instances.md` — example set, auto-loaded only when present, not required.
|
|
61
|
+
- Project expected behavior: `spec.md`/`README`/`--help` — the source of truth for what to verify.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Instances template
|
|
2
|
+
|
|
3
|
+
Generic template for **instance** functional tests. Each **instance** is a prompt + command + expected outcome. Copy and adapt for your project; the example below is for `matt-skills`.
|
|
4
|
+
|
|
5
|
+
## Format
|
|
6
|
+
|
|
7
|
+
Each instance declares:
|
|
8
|
+
|
|
9
|
+
- Prompt: human intent (what to verify)
|
|
10
|
+
- Command: shell command to run in isolated dir
|
|
11
|
+
- Expected: files/content, stdout phrases, exit code
|
|
12
|
+
|
|
13
|
+
Verification commands are in `SKILL.md` steps.
|
|
14
|
+
|
|
15
|
+
## Example: matt-skills functional behavior
|
|
16
|
+
|
|
17
|
+
### 1. Fresh init
|
|
18
|
+
Prompt: verify fresh project initialization
|
|
19
|
+
Command: `node bin/cli.js init --dest <tmp>`
|
|
20
|
+
Expected: `AGENTS.md`, `.opencode/skills/tdd-implement/SKILL.md`, `.pi/skills/tdd-implement/SKILL.md`, `.agents/skills/tdd` (22 upstream) exist; stdout `模板:已复制` + `上游技能:已装 22`; no `.bak`; exit 0.
|
|
21
|
+
|
|
22
|
+
### 2. Init skip on existing
|
|
23
|
+
Prompt: verify idempotent init without --force
|
|
24
|
+
Command: `init` twice without `--force`, second with local edit to `AGENTS.md`
|
|
25
|
+
Expected: second stdout `模板已存在.*跳过`, `上游技能:已装 0、跳过 22`; local edit preserved; exit 0.
|
|
26
|
+
|
|
27
|
+
### 3. Init --force with backup
|
|
28
|
+
Prompt: verify forced init backs up
|
|
29
|
+
Command: `init --force --dest <tmp>` after local edit
|
|
30
|
+
Expected: stdout `已备份` + `备份 22`; `AGENTS.md.bak` exists with local edit; `AGENTS.md` restored from template; `.agents/skills/tdd.bak` exists; exit 0.
|
|
31
|
+
|
|
32
|
+
### 4. Sync on existing
|
|
33
|
+
Prompt: verify sync backs up existing project
|
|
34
|
+
Command: `sync --dest <tmp>` after local edit
|
|
35
|
+
Expected: stdout `同步` + `已备份`; `AGENTS.md.bak` exists; exit 0.
|
|
36
|
+
|
|
37
|
+
### 5. Sync --force without backup
|
|
38
|
+
Prompt: verify sync --force does not backup
|
|
39
|
+
Command: `sync --force --dest <tmp>`
|
|
40
|
+
Expected: stdout `已覆盖` without new `.bak`; exit 0.
|
|
41
|
+
|
|
42
|
+
### 6. List
|
|
43
|
+
Prompt: verify skill listing
|
|
44
|
+
Command: `list` and `list --json`
|
|
45
|
+
Expected: 27 skills, includes `tdd` with correct description; `--json` is valid JSON array; exit 0.
|
package/bin/cli.js
CHANGED
|
@@ -7,7 +7,7 @@ import prompts from 'prompts';
|
|
|
7
7
|
|
|
8
8
|
const SKILLS_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', '.agents', 'skills');
|
|
9
9
|
const TEMPLATE_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', 'template');
|
|
10
|
-
const PROPRIETARY_SKILLS = new Set(['tdd-implement', 'grill-to-spec', 'diagnose-fix', 'commit-check']);
|
|
10
|
+
const PROPRIETARY_SKILLS = new Set(['tdd-implement', 'grill-to-spec', 'diagnose-fix', 'commit-check', 'instance-test']);
|
|
11
11
|
process.stdout.on('error', (err) => {
|
|
12
12
|
if (err.code === 'EPIPE') process.exit(0);
|
|
13
13
|
throw err;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@heihei0299/matt-skills",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Agent skills + 项目配置模板:一条命令初始化 opencode / pi-agent 项目(含 mattpocock/skills 上游技能)",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -14,7 +14,8 @@
|
|
|
14
14
|
],
|
|
15
15
|
"scripts": {
|
|
16
16
|
"test": "node --test test/*.test.js",
|
|
17
|
-
"prepublishOnly": "node --test test/*.test.js"
|
|
17
|
+
"prepublishOnly": "node --test test/*.test.js",
|
|
18
|
+
"build:template": "node scripts/build-template.js"
|
|
18
19
|
},
|
|
19
20
|
"dependencies": {
|
|
20
21
|
"prompts": "^2.4.2"
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: instance-test
|
|
3
|
+
disable-model-invocation: true
|
|
4
|
+
description: "Verify project meets expected goals by running prompt instances in isolated temp dirs"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Instance Test
|
|
8
|
+
|
|
9
|
+
Run **instance** prompts to verify project meets expected goals via actual functional tests. Each **instance** is a prompt + expected outcome, executed in an isolated temp dir — no mocks, no stubs.
|
|
10
|
+
|
|
11
|
+
## Steps
|
|
12
|
+
|
|
13
|
+
### 1. Gather instances
|
|
14
|
+
|
|
15
|
+
Collect the **instance** set to run:
|
|
16
|
+
|
|
17
|
+
- User-provided instances (prompt, command, expected files/stdout/exit code), or
|
|
18
|
+
- Derived from `spec.md`/`README` acceptance criteria — extract each verifiable behavior as one instance, then confirm the list with the user before running.
|
|
19
|
+
|
|
20
|
+
Each **instance** must declare: command to run, expected files/content, expected stdout phrases, expected exit code.
|
|
21
|
+
|
|
22
|
+
Completion: instance list is fixed (prompt, expected outcome, verification command) — no instance is added mid-run.
|
|
23
|
+
|
|
24
|
+
### 2. Run instances
|
|
25
|
+
|
|
26
|
+
For each **instance** in order:
|
|
27
|
+
|
|
28
|
+
1. `mktemp -d` isolated dir (or `git worktree` / `--dest` if the project supports it).
|
|
29
|
+
2. Execute the instance's command — capture stdout/stderr and exit code.
|
|
30
|
+
3. Snapshot result files and side effects declared in expected.
|
|
31
|
+
|
|
32
|
+
Do not run instances in parallel — one **instance** at a time, so failures are isolated and artifacts do not collide.
|
|
33
|
+
|
|
34
|
+
Completion: every **instance** has a run dir with captured output and file snapshot.
|
|
35
|
+
|
|
36
|
+
### 3. Evaluate
|
|
37
|
+
|
|
38
|
+
Compare each **instance**'s actual vs expected:
|
|
39
|
+
|
|
40
|
+
- File existence/content (`test -f`, `grep -q`, `diff`).
|
|
41
|
+
- Stdout/stderr contains expected phrases.
|
|
42
|
+
- Exit code matches expected.
|
|
43
|
+
|
|
44
|
+
Mark `PASS`/`FAIL` per **instance** with evidence (file path, stdout line, or diff).
|
|
45
|
+
|
|
46
|
+
Completion: every **instance** has a `PASS` or `FAIL` with evidence — no unevaluated instance.
|
|
47
|
+
|
|
48
|
+
### 4. Report
|
|
49
|
+
|
|
50
|
+
Summarize in conversation:
|
|
51
|
+
|
|
52
|
+
- `PASS m/n` with per-instance evidence.
|
|
53
|
+
- Failures list the gap (expected vs actual) and the run dir for reproduction.
|
|
54
|
+
- Clean up temp dirs unless `--keep` is requested.
|
|
55
|
+
|
|
56
|
+
Do not write a report file (`report-*.md`) — output stays in conversation. Keep temp dirs only on failure for debugging.
|
|
57
|
+
|
|
58
|
+
## References
|
|
59
|
+
|
|
60
|
+
- Instance definitions (if any): `references/instances.md` — example set, auto-loaded only when present, not required.
|
|
61
|
+
- Project expected behavior: `spec.md`/`README`/`--help` — the source of truth for what to verify.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Instances template
|
|
2
|
+
|
|
3
|
+
Generic template for **instance** functional tests. Each **instance** is a prompt + command + expected outcome. Copy and adapt for your project; the example below is for `matt-skills`.
|
|
4
|
+
|
|
5
|
+
## Format
|
|
6
|
+
|
|
7
|
+
Each instance declares:
|
|
8
|
+
|
|
9
|
+
- Prompt: human intent (what to verify)
|
|
10
|
+
- Command: shell command to run in isolated dir
|
|
11
|
+
- Expected: files/content, stdout phrases, exit code
|
|
12
|
+
|
|
13
|
+
Verification commands are in `SKILL.md` steps.
|
|
14
|
+
|
|
15
|
+
## Example: matt-skills functional behavior
|
|
16
|
+
|
|
17
|
+
### 1. Fresh init
|
|
18
|
+
Prompt: verify fresh project initialization
|
|
19
|
+
Command: `node bin/cli.js init --dest <tmp>`
|
|
20
|
+
Expected: `AGENTS.md`, `.opencode/skills/tdd-implement/SKILL.md`, `.pi/skills/tdd-implement/SKILL.md`, `.agents/skills/tdd` (22 upstream) exist; stdout `模板:已复制` + `上游技能:已装 22`; no `.bak`; exit 0.
|
|
21
|
+
|
|
22
|
+
### 2. Init skip on existing
|
|
23
|
+
Prompt: verify idempotent init without --force
|
|
24
|
+
Command: `init` twice without `--force`, second with local edit to `AGENTS.md`
|
|
25
|
+
Expected: second stdout `模板已存在.*跳过`, `上游技能:已装 0、跳过 22`; local edit preserved; exit 0.
|
|
26
|
+
|
|
27
|
+
### 3. Init --force with backup
|
|
28
|
+
Prompt: verify forced init backs up
|
|
29
|
+
Command: `init --force --dest <tmp>` after local edit
|
|
30
|
+
Expected: stdout `已备份` + `备份 22`; `AGENTS.md.bak` exists with local edit; `AGENTS.md` restored from template; `.agents/skills/tdd.bak` exists; exit 0.
|
|
31
|
+
|
|
32
|
+
### 4. Sync on existing
|
|
33
|
+
Prompt: verify sync backs up existing project
|
|
34
|
+
Command: `sync --dest <tmp>` after local edit
|
|
35
|
+
Expected: stdout `同步` + `已备份`; `AGENTS.md.bak` exists; exit 0.
|
|
36
|
+
|
|
37
|
+
### 5. Sync --force without backup
|
|
38
|
+
Prompt: verify sync --force does not backup
|
|
39
|
+
Command: `sync --force --dest <tmp>`
|
|
40
|
+
Expected: stdout `已覆盖` without new `.bak`; exit 0.
|
|
41
|
+
|
|
42
|
+
### 6. List
|
|
43
|
+
Prompt: verify skill listing
|
|
44
|
+
Command: `list` and `list --json`
|
|
45
|
+
Expected: 27 skills, includes `tdd` with correct description; `--json` is valid JSON array; exit 0.
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: instance-test
|
|
3
|
+
disable-model-invocation: true
|
|
4
|
+
description: "Verify project meets expected goals by running prompt instances in isolated temp dirs"
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Instance Test
|
|
8
|
+
|
|
9
|
+
Run **instance** prompts to verify project meets expected goals via actual functional tests. Each **instance** is a prompt + expected outcome, executed in an isolated temp dir — no mocks, no stubs.
|
|
10
|
+
|
|
11
|
+
## Steps
|
|
12
|
+
|
|
13
|
+
### 1. Gather instances
|
|
14
|
+
|
|
15
|
+
Collect the **instance** set to run:
|
|
16
|
+
|
|
17
|
+
- User-provided instances (prompt, command, expected files/stdout/exit code), or
|
|
18
|
+
- Derived from `spec.md`/`README` acceptance criteria — extract each verifiable behavior as one instance, then confirm the list with the user before running.
|
|
19
|
+
|
|
20
|
+
Each **instance** must declare: command to run, expected files/content, expected stdout phrases, expected exit code.
|
|
21
|
+
|
|
22
|
+
Completion: instance list is fixed (prompt, expected outcome, verification command) — no instance is added mid-run.
|
|
23
|
+
|
|
24
|
+
### 2. Run instances
|
|
25
|
+
|
|
26
|
+
For each **instance** in order:
|
|
27
|
+
|
|
28
|
+
1. `mktemp -d` isolated dir (or `git worktree` / `--dest` if the project supports it).
|
|
29
|
+
2. Execute the instance's command — capture stdout/stderr and exit code.
|
|
30
|
+
3. Snapshot result files and side effects declared in expected.
|
|
31
|
+
|
|
32
|
+
Do not run instances in parallel — one **instance** at a time, so failures are isolated and artifacts do not collide.
|
|
33
|
+
|
|
34
|
+
Completion: every **instance** has a run dir with captured output and file snapshot.
|
|
35
|
+
|
|
36
|
+
### 3. Evaluate
|
|
37
|
+
|
|
38
|
+
Compare each **instance**'s actual vs expected:
|
|
39
|
+
|
|
40
|
+
- File existence/content (`test -f`, `grep -q`, `diff`).
|
|
41
|
+
- Stdout/stderr contains expected phrases.
|
|
42
|
+
- Exit code matches expected.
|
|
43
|
+
|
|
44
|
+
Mark `PASS`/`FAIL` per **instance** with evidence (file path, stdout line, or diff).
|
|
45
|
+
|
|
46
|
+
Completion: every **instance** has a `PASS` or `FAIL` with evidence — no unevaluated instance.
|
|
47
|
+
|
|
48
|
+
### 4. Report
|
|
49
|
+
|
|
50
|
+
Summarize in conversation:
|
|
51
|
+
|
|
52
|
+
- `PASS m/n` with per-instance evidence.
|
|
53
|
+
- Failures list the gap (expected vs actual) and the run dir for reproduction.
|
|
54
|
+
- Clean up temp dirs unless `--keep` is requested.
|
|
55
|
+
|
|
56
|
+
Do not write a report file (`report-*.md`) — output stays in conversation. Keep temp dirs only on failure for debugging.
|
|
57
|
+
|
|
58
|
+
## References
|
|
59
|
+
|
|
60
|
+
- Instance definitions (if any): `references/instances.md` — example set, auto-loaded only when present, not required.
|
|
61
|
+
- Project expected behavior: `spec.md`/`README`/`--help` — the source of truth for what to verify.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Instances template
|
|
2
|
+
|
|
3
|
+
Generic template for **instance** functional tests. Each **instance** is a prompt + command + expected outcome. Copy and adapt for your project; the example below is for `matt-skills`.
|
|
4
|
+
|
|
5
|
+
## Format
|
|
6
|
+
|
|
7
|
+
Each instance declares:
|
|
8
|
+
|
|
9
|
+
- Prompt: human intent (what to verify)
|
|
10
|
+
- Command: shell command to run in isolated dir
|
|
11
|
+
- Expected: files/content, stdout phrases, exit code
|
|
12
|
+
|
|
13
|
+
Verification commands are in `SKILL.md` steps.
|
|
14
|
+
|
|
15
|
+
## Example: matt-skills functional behavior
|
|
16
|
+
|
|
17
|
+
### 1. Fresh init
|
|
18
|
+
Prompt: verify fresh project initialization
|
|
19
|
+
Command: `node bin/cli.js init --dest <tmp>`
|
|
20
|
+
Expected: `AGENTS.md`, `.opencode/skills/tdd-implement/SKILL.md`, `.pi/skills/tdd-implement/SKILL.md`, `.agents/skills/tdd` (22 upstream) exist; stdout `模板:已复制` + `上游技能:已装 22`; no `.bak`; exit 0.
|
|
21
|
+
|
|
22
|
+
### 2. Init skip on existing
|
|
23
|
+
Prompt: verify idempotent init without --force
|
|
24
|
+
Command: `init` twice without `--force`, second with local edit to `AGENTS.md`
|
|
25
|
+
Expected: second stdout `模板已存在.*跳过`, `上游技能:已装 0、跳过 22`; local edit preserved; exit 0.
|
|
26
|
+
|
|
27
|
+
### 3. Init --force with backup
|
|
28
|
+
Prompt: verify forced init backs up
|
|
29
|
+
Command: `init --force --dest <tmp>` after local edit
|
|
30
|
+
Expected: stdout `已备份` + `备份 22`; `AGENTS.md.bak` exists with local edit; `AGENTS.md` restored from template; `.agents/skills/tdd.bak` exists; exit 0.
|
|
31
|
+
|
|
32
|
+
### 4. Sync on existing
|
|
33
|
+
Prompt: verify sync backs up existing project
|
|
34
|
+
Command: `sync --dest <tmp>` after local edit
|
|
35
|
+
Expected: stdout `同步` + `已备份`; `AGENTS.md.bak` exists; exit 0.
|
|
36
|
+
|
|
37
|
+
### 5. Sync --force without backup
|
|
38
|
+
Prompt: verify sync --force does not backup
|
|
39
|
+
Command: `sync --force --dest <tmp>`
|
|
40
|
+
Expected: stdout `已覆盖` without new `.bak`; exit 0.
|
|
41
|
+
|
|
42
|
+
### 6. List
|
|
43
|
+
Prompt: verify skill listing
|
|
44
|
+
Command: `list` and `list --json`
|
|
45
|
+
Expected: 27 skills, includes `tdd` with correct description; `--json` is valid JSON array; exit 0.
|