@turri-dev/cli 0.1.27 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -6
- package/dist/advisor-chat.d.ts +63 -0
- package/dist/advisor-chat.js +290 -0
- package/dist/background-skills.d.ts +53 -0
- package/dist/background-skills.js +121 -0
- package/dist/bedrock-region.d.ts +66 -0
- package/dist/bedrock-region.js +144 -0
- package/dist/btw.d.ts +1 -1
- package/dist/btw.js +1 -1
- package/dist/change-plain-cli.js +20 -13
- package/dist/chat-idle.d.ts +7 -0
- package/dist/chat-idle.js +20 -0
- package/dist/chats.d.ts +10 -0
- package/dist/chats.js +23 -0
- package/dist/cli-options.d.ts +1 -2
- package/dist/cli-options.js +2 -7
- package/dist/cli.js +11 -3
- package/dist/config-command.d.ts +14 -0
- package/dist/config-command.js +35 -15
- package/dist/config-edit.d.ts +3 -3
- package/dist/config-edit.js +16 -12
- package/dist/config.d.ts +54 -34
- package/dist/config.js +40 -44
- package/dist/conform-cli.js +12 -10
- package/dist/conformance-switch.d.ts +34 -0
- package/dist/conformance-switch.js +108 -0
- package/dist/cultivate-cli.js +15 -5
- package/dist/drift-cli.js +2 -0
- package/dist/extension.d.ts +15 -0
- package/dist/extension.js +260 -83
- package/dist/host-session.d.ts +26 -0
- package/dist/host-session.js +30 -2
- package/dist/host.d.ts +5 -2
- package/dist/host.js +7 -1
- package/dist/init-config.d.ts +1 -1
- package/dist/init-config.js +20 -11
- package/dist/loop-cli.d.ts +3 -3
- package/dist/loop-cli.js +13 -16
- package/dist/loop-tools.js +18 -10
- package/dist/model-check.d.ts +8 -3
- package/dist/model-check.js +13 -8
- package/dist/new-project.js +12 -8
- package/dist/{validate-cli.js → parse-cli.js} +11 -5
- package/dist/parse-tool.d.ts +9 -0
- package/dist/parse-tool.js +33 -0
- package/dist/pi-launch.js +1 -1
- package/dist/plain/check.d.ts +53 -6
- package/dist/plain/check.js +217 -24
- package/dist/plain/concepts.d.ts +8 -0
- package/dist/plain/concepts.js +66 -23
- package/dist/plain/cycles.d.ts +21 -0
- package/dist/plain/cycles.js +190 -0
- package/dist/plain/errors.d.ts +4 -0
- package/dist/plain/errors.js +7 -0
- package/dist/plain/files.d.ts +18 -1
- package/dist/plain/files.js +39 -2
- package/dist/plain/graph.d.ts +33 -4
- package/dist/plain/graph.js +60 -12
- package/dist/plain/highlight.d.ts +3 -2
- package/dist/plain/highlight.js +4 -3
- package/dist/plain/index.d.ts +8 -5
- package/dist/plain/index.js +8 -5
- package/dist/plain/liquid.js +1 -6
- package/dist/plain/markdown.d.ts +32 -3
- package/dist/plain/markdown.js +146 -24
- package/dist/plain/module-concepts.js +2 -2
- package/dist/plain/parser.d.ts +83 -28
- package/dist/plain/parser.js +381 -260
- package/dist/plain/plain.tmLanguage.json +29 -3
- package/dist/plain/project.d.ts +48 -0
- package/dist/plain/project.js +155 -0
- package/dist/plain/spec.d.ts +90 -28
- package/dist/plain/spec.js +107 -133
- package/dist/plain/test-plain-file.d.ts +59 -0
- package/dist/plain/test-plain-file.js +232 -0
- package/dist/regenerate/activity.d.ts +20 -9
- package/dist/regenerate/activity.js +11 -4
- package/dist/regenerate/advisor-records.d.ts +93 -0
- package/dist/regenerate/advisor-records.js +110 -0
- package/dist/regenerate/advisor-runs.d.ts +87 -0
- package/dist/regenerate/advisor-runs.js +327 -0
- package/dist/regenerate/base-folder.d.ts +68 -0
- package/dist/regenerate/base-folder.js +168 -0
- package/dist/regenerate/cases.d.ts +9 -6
- package/dist/regenerate/cases.js +70 -14
- package/dist/regenerate/change-view.d.ts +8 -3
- package/dist/regenerate/change-view.js +71 -22
- package/dist/regenerate/chunk-plan.d.ts +36 -15
- package/dist/regenerate/chunk-plan.js +165 -59
- package/dist/regenerate/conform.d.ts +3 -0
- package/dist/regenerate/conform.js +6 -5
- package/dist/regenerate/conformance-agents.d.ts +18 -39
- package/dist/regenerate/conformance-agents.js +60 -88
- package/dist/regenerate/conformance.d.ts +23 -13
- package/dist/regenerate/conformance.js +119 -82
- package/dist/regenerate/diff-view.d.ts +1 -1
- package/dist/regenerate/diff-view.js +20 -4
- package/dist/regenerate/drift.d.ts +47 -15
- package/dist/regenerate/drift.js +101 -30
- package/dist/regenerate/jobs.d.ts +2 -2
- package/dist/regenerate/jobs.js +1 -1
- package/dist/regenerate/ledger.d.ts +19 -0
- package/dist/regenerate/loop.d.ts +62 -43
- package/dist/regenerate/loop.js +472 -341
- package/dist/regenerate/paths.d.ts +4 -0
- package/dist/regenerate/paths.js +7 -1
- package/dist/regenerate/plain-palette.js +2 -0
- package/dist/regenerate/prompt.d.ts +57 -19
- package/dist/regenerate/prompt.js +157 -122
- package/dist/regenerate/records.d.ts +43 -12
- package/dist/regenerate/records.js +102 -88
- package/dist/regenerate/regenerate.js +29 -11
- package/dist/regenerate/regression.d.ts +4 -4
- package/dist/regenerate/regression.js +14 -10
- package/dist/regenerate/renames.d.ts +42 -0
- package/dist/regenerate/renames.js +185 -0
- package/dist/regenerate/requires.d.ts +1 -1
- package/dist/regenerate/requires.js +10 -3
- package/dist/regenerate/resources.d.ts +5 -3
- package/dist/regenerate/resources.js +4 -4
- package/dist/regenerate/scripts-being-written.d.ts +19 -0
- package/dist/regenerate/scripts-being-written.js +30 -0
- package/dist/regenerate/spec-map-view.d.ts +2 -2
- package/dist/regenerate/spec-map-view.js +46 -15
- package/dist/regenerate/spec-map.d.ts +27 -5
- package/dist/regenerate/spec-map.js +92 -60
- package/dist/regenerate/spec-steps.d.ts +48 -5
- package/dist/regenerate/spec-steps.js +213 -28
- package/dist/regenerate/state.d.ts +4 -3
- package/dist/regenerate/stop-report.d.ts +1 -1
- package/dist/regenerate/stop-report.js +10 -10
- package/dist/regenerate/test-advisor.d.ts +101 -0
- package/dist/regenerate/test-advisor.js +379 -0
- package/dist/regenerate/test-kinds.d.ts +71 -28
- package/dist/regenerate/test-kinds.js +110 -63
- package/dist/regenerate/test-scripts.d.ts +2 -0
- package/dist/regenerate/test-scripts.js +4 -0
- package/dist/regenerate/testing-steps.d.ts +41 -0
- package/dist/regenerate/testing-steps.js +110 -0
- package/dist/regenerate/testing.d.ts +1 -1
- package/dist/regenerate/testing.js +5 -4
- package/dist/regenerate/unit-review.d.ts +1 -1
- package/dist/regenerate/unit-review.js +3 -3
- package/dist/regenerate-cli.js +11 -12
- package/dist/release-agent.js +4 -0
- package/dist/release-cli.js +2 -0
- package/dist/release-tools.js +2 -0
- package/dist/run-report.d.ts +2 -2
- package/dist/run-report.js +4 -3
- package/dist/self-invocation.d.ts +1 -1
- package/dist/self-invocation.js +1 -1
- package/dist/sentinel.js +1 -1
- package/dist/setup.d.ts +10 -0
- package/dist/setup.js +52 -4
- package/dist/skill.d.ts +10 -0
- package/dist/skill.js +6 -0
- package/dist/skills.js +3 -1
- package/dist/spec-tools.js +21 -31
- package/dist/test-plain-files.d.ts +41 -0
- package/dist/test-plain-files.js +177 -0
- package/docs/README.md +3 -1
- package/docs/authoring-plain.md +46 -13
- package/docs/chats.md +3 -1
- package/docs/config.md +20 -15
- package/docs/deploy.md +1 -1
- package/docs/development.md +2 -1
- package/docs/headless.md +15 -18
- package/docs/hosting.md +28 -12
- package/docs/installation.md +1 -0
- package/docs/parser.md +88 -27
- package/docs/regenerating.md +87 -52
- package/docs/sentinel.md +1 -1
- package/docs/skills-and-gates.md +3 -1
- package/docs/test-advisor.md +110 -0
- package/docs/theming.md +1 -1
- package/docs/using.md +17 -15
- package/genome/rules/bullet-continuation.md +8 -8
- package/genome/rules/concise-specs-examples.md +111 -80
- package/genome/rules/concise-specs-integration-examples.md +56 -41
- package/genome/rules/concise-specs.md +17 -16
- package/genome/rules/definitions.md +5 -5
- package/genome/rules/examples/integration-embedded-testing/run_conformance_tests_java.sh +1 -1
- package/genome/rules/func-specs.md +69 -74
- package/genome/rules/impl-reqs.md +4 -4
- package/genome/rules/integration-embedded-testing.md +20 -20
- package/genome/rules/integration-embedded.md +31 -31
- package/genome/rules/integration-standalone.md +9 -9
- package/genome/rules/integrations.md +18 -18
- package/genome/rules/linked-resources.md +19 -16
- package/genome/rules/module-structure.md +21 -14
- package/genome/rules/project-model.md +18 -17
- package/genome/rules/required-concepts.md +1 -1
- package/genome/rules/requires-modules.md +6 -5
- package/genome/rules/semantic-ids.md +49 -0
- package/genome/rules/stdlib.md +4 -5
- package/genome/rules/test-plain-files.md +130 -0
- package/genome/rules/test-reqs.md +8 -8
- package/genome/rules/turri-regeneration.md +23 -8
- package/genome/skills/analyze-2-func-specs/SKILL.md +36 -21
- package/genome/skills/analyze-func-specs/SKILL.md +46 -32
- package/genome/skills/break-down-func-spec/SKILL.md +91 -67
- package/genome/skills/consolidate-concepts/SKILL.md +10 -10
- package/genome/skills/convert-old-syntax/SKILL.md +68 -0
- package/genome/skills/convert-old-syntax/references/rubric.md +11 -0
- package/genome/skills/debug-specs/SKILL.md +62 -56
- package/genome/skills/implement-conformance-testing-script/SKILL.md +19 -15
- package/genome/skills/implement-conformance-testing-script/assets/run_conformance_tests_java.sh +2 -2
- package/genome/skills/implement-conformance-testing-script/assets/run_conformance_tests_python.sh +2 -2
- package/genome/skills/implement-unit-testing-script/SKILL.md +14 -10
- package/genome/skills/refactor-module/SKILL.md +35 -35
- package/genome/skills/resolve-section-ownership/SKILL.md +9 -9
- package/genome/skills/resolve-spec-conflict/SKILL.md +35 -29
- package/package.json +7 -6
- package/dist/regenerate/choose-conformance.d.ts +0 -32
- package/dist/regenerate/choose-conformance.js +0 -117
- /package/dist/{validate-cli.d.ts → parse-cli.d.ts} +0 -0
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: debug-specs
|
|
3
3
|
description: >-
|
|
4
4
|
Investigate a bug observed in the running application by reading the generated
|
|
5
|
-
code in plain_modules/, tracing the issue back to the
|
|
5
|
+
code in plain_modules/, tracing the issue back to the ***plain, and fixing only
|
|
6
6
|
the .plain files. Generated code is never modified. Use when the user reports
|
|
7
7
|
unexpected behavior, visual glitches, crashes, or incorrect logic in the app.
|
|
8
8
|
allowed-tools: [read, write, edit, bash, ls, grep, find]
|
|
@@ -19,17 +19,17 @@ allowed-tools: [read, write, edit, bash, ls, grep, find]
|
|
|
19
19
|
|
|
20
20
|
## Guiding Principle
|
|
21
21
|
|
|
22
|
-
Generated code in `plain_modules/<module>/code/` and `plain_modules/<module>/tests/` is **read-only** — it exists solely as evidence for diagnosis. All fixes are applied exclusively to the `.plain`
|
|
22
|
+
Generated code in `plain_modules/<module>/code/` and `plain_modules/<module>/tests/` is **read-only** — it exists solely as evidence for diagnosis. All fixes are applied exclusively to the `.plain` files. The workflow is: observe → read generated code → trace to the ***plain → fix the ***plain.
|
|
23
23
|
|
|
24
24
|
## Input
|
|
25
25
|
|
|
26
26
|
1. **The module name** — identifies the `plain_modules/<module_name>/` directory (`code/` and `tests/`) and the corresponding `.plain` file(s).
|
|
27
27
|
2. **The user's observation** — what is wrong? This can be a bug description, a screenshot, a test failure, an error message, or a general "this doesn't work right."
|
|
28
|
-
3. **Optional: a specific functional spec** — if the user suspects a particular spec, start there. Otherwise, investigate broadly.
|
|
28
|
+
3. **Optional: a specific functional spec** — if the user suspects a particular functional spec, start there. Otherwise, investigate broadly.
|
|
29
29
|
|
|
30
30
|
## Phase 1 — Understand the Context
|
|
31
31
|
|
|
32
|
-
1. **Read the `.plain` file(s)** for the module — frontmatter, definitions, implementation reqs, test reqs, and all functional specs. Also read `import` and `requires` chains.
|
|
32
|
+
1. **Read the `.plain` file(s)** for the module — frontmatter, definitions, implementation reqs, test reqs, and all functional specs. Also read `import` and `requires` chains, and the module's test plain files.
|
|
33
33
|
2. **Read the user's observation carefully.** Rephrase it back to confirm understanding. Identify:
|
|
34
34
|
- What is the expected behavior?
|
|
35
35
|
- What is the actual behavior?
|
|
@@ -37,12 +37,12 @@ Generated code in `plain_modules/<module>/code/` and `plain_modules/<module>/tes
|
|
|
37
37
|
|
|
38
38
|
## Phase 2 — Investigate the Generated Code
|
|
39
39
|
|
|
40
|
-
Read files in `plain_modules/<module_name>/code/` to understand what the
|
|
40
|
+
Read files in `plain_modules/<module_name>/code/` to understand what the coding agent produced. **Do not modify any generated files.**
|
|
41
41
|
|
|
42
42
|
### 2a. Narrow the search
|
|
43
43
|
|
|
44
|
-
If the user pointed to a specific spec or area:
|
|
45
|
-
- Identify which generated files implement that spec's behavior.
|
|
44
|
+
If the user pointed to a specific functional spec or area:
|
|
45
|
+
- Identify which generated files implement that functional spec's behavior.
|
|
46
46
|
- Read those files to understand the current implementation.
|
|
47
47
|
|
|
48
48
|
If the investigation is broad:
|
|
@@ -53,60 +53,65 @@ If the investigation is broad:
|
|
|
53
53
|
|
|
54
54
|
For each relevant file:
|
|
55
55
|
1. Read the code and understand what it does.
|
|
56
|
-
2. Compare the behavior to what the spec says it should do.
|
|
57
|
-
3. Note any discrepancies — places where the code doesn't match the spec's intent.
|
|
56
|
+
2. Compare the behavior to what the functional spec says it should do.
|
|
57
|
+
3. Note any discrepancies — places where the code doesn't match the functional spec's intent.
|
|
58
58
|
|
|
59
59
|
### 2c. Check conformance tests and unit tests (if relevant)
|
|
60
60
|
|
|
61
61
|
If the bug manifests as a test failure:
|
|
62
|
-
1. Read the failing test — conformance tests in `plain_modules/<module_name>/tests/` (the module's whole suite, no per-
|
|
62
|
+
1. Read the failing test — conformance tests in `plain_modules/<module_name>/tests/` (the module's whole suite, no per-functional-spec subfolders), each labelled with the scenario it implements (`conformance scenario <scenario-id> of <module>:<func-id>`, `acceptance scenario <scenario-id> of <module>:<func-id>`), unit tests inside `plain_modules/<module_name>/code/`.
|
|
63
63
|
2. Understand what the test expects vs. what the implementation does.
|
|
64
|
-
3. Determine whether the test expectation is correct (matches the spec) or incorrect (doesn't match the spec).
|
|
64
|
+
3. Determine whether the test expectation is correct (matches the functional spec) or incorrect (doesn't match the functional spec).
|
|
65
65
|
|
|
66
66
|
## Phase 3 — Diagnose the Root Cause
|
|
67
67
|
|
|
68
|
-
Trace the issue from the generated code back to the
|
|
68
|
+
Trace the issue from the generated code back to the ***plain. There are six possible root causes — determine which one applies **before** making changes:
|
|
69
69
|
|
|
70
70
|
| Root Cause | Symptom | Fix |
|
|
71
71
|
|------------|---------|-----|
|
|
72
|
-
| **Ambiguous spec** | The spec is correct in intent, but vague enough that the
|
|
73
|
-
| **Missing spec** | The desired behavior is not covered by any spec. The
|
|
74
|
-
| **Conflicting specs** | Two specs contradict each other, causing the
|
|
75
|
-
| **Incorrect spec** | The spec explicitly describes the wrong behavior. The
|
|
76
|
-
| **
|
|
72
|
+
| **Ambiguous functional spec** | The functional spec is correct in intent, but vague enough that the coding agent interpreted it differently than intended. | Add explicit detail to the functional spec to eliminate the ambiguity. |
|
|
73
|
+
| **Missing functional spec** | The desired behavior is not covered by any functional spec. The coding agent had no guidance and either did nothing or made an arbitrary choice. | Add a new functional spec, checked as `func-specs.md` says (*Adding a functional spec*). |
|
|
74
|
+
| **Conflicting functional specs** | Two functional specs contradict each other, causing the coding agent to produce inconsistent behavior. | Use the `resolve-spec-conflict` skill. |
|
|
75
|
+
| **Incorrect functional spec** | The functional spec explicitly describes the wrong behavior. The coding agent implemented it faithfully, but the functional spec itself is wrong. | Rewrite the functional spec to describe the correct behavior. |
|
|
76
|
+
| **Incorrect scenario** | The functional spec is correct and the code matches it, but a scenario in its test plain file expects an outcome the functional spec does not state, or contradicts one it states. | Fix the scenario in the test plain file (`test-plain-files.md`). |
|
|
77
|
+
| **Missing implementation req** | The functional spec is correct, but the implementation req doesn't provide enough guidance for the coding agent to produce the right code (e.g., missing library, missing architectural constraint, missing platform detail). | Add or update the implementation req, per `impl-reqs.md`. |
|
|
77
78
|
|
|
78
79
|
### How to diagnose
|
|
79
80
|
|
|
80
|
-
1. **Read the spec that governs the buggy behavior.** Does it clearly and unambiguously describe the correct behavior?
|
|
81
|
-
- If the spec is clear and correct but the code is wrong → **ambiguous spec** (the
|
|
82
|
-
- If the spec doesn't mention the behavior at all → **missing spec**.
|
|
83
|
-
- If the spec explicitly describes the wrong behavior → **incorrect spec**.
|
|
84
|
-
2. **Check for conflicts.** Could another spec be overriding or contradicting this behavior?
|
|
81
|
+
1. **Read the functional spec that governs the buggy behavior.** Does it clearly and unambiguously describe the correct behavior?
|
|
82
|
+
- If the functional spec is clear and correct but the code is wrong → **ambiguous functional spec** (the coding agent misinterpreted it) or **missing implementation req**.
|
|
83
|
+
- If the functional spec doesn't mention the behavior at all → **missing functional spec**.
|
|
84
|
+
- If the functional spec explicitly describes the wrong behavior → **incorrect functional spec**.
|
|
85
|
+
2. **Check for conflicts.** Could another functional spec be overriding or contradicting this behavior?
|
|
85
86
|
- Read all functional specs that touch the same concepts.
|
|
86
|
-
- If two specs are in tension → **conflicting specs**.
|
|
87
|
-
3. **Check
|
|
87
|
+
- If two functional specs are in tension → **conflicting functional specs**.
|
|
88
|
+
3. **Check the scenario.** When a conformance or acceptance test fails, find its scenario by the scenario-id in the test's label and compare it with the functional spec.
|
|
89
|
+
- If the scenario contradicts or extends the functional spec → **incorrect scenario**.
|
|
90
|
+
4. **Check implementation reqs.** Is the coding agent missing guidance on how to implement correctly?
|
|
88
91
|
- Does the implementation req specify the right library, pattern, or constraint?
|
|
89
92
|
- If a platform-specific or framework-specific detail is missing → **missing implementation req**.
|
|
90
93
|
|
|
91
|
-
## Phase 4 — Fix the
|
|
94
|
+
## Phase 4 — Fix the ***plain
|
|
92
95
|
|
|
93
96
|
Apply the fix based on the diagnosed root cause:
|
|
94
97
|
|
|
95
98
|
| Root Cause | How |
|
|
96
99
|
|------------|-------------|
|
|
97
|
-
| Ambiguous spec | Edit the spec inline — add
|
|
98
|
-
| Missing spec | `func-specs.md`, *Adding a functional spec* |
|
|
99
|
-
| Conflicting specs | `resolve-spec-conflict` |
|
|
100
|
-
| Incorrect spec | Edit the spec inline — rewrite the incorrect part |
|
|
100
|
+
| Ambiguous functional spec | Edit the functional spec inline — add detail under its statement line or reword for clarity |
|
|
101
|
+
| Missing functional spec | `func-specs.md`, *Adding a functional spec* |
|
|
102
|
+
| Conflicting functional specs | `resolve-spec-conflict` |
|
|
103
|
+
| Incorrect functional spec | Edit the functional spec inline — rewrite the incorrect part |
|
|
104
|
+
| Incorrect scenario | `test-plain-files.md` - change or take out the scenario |
|
|
101
105
|
| Missing implementation req | `impl-reqs.md` |
|
|
102
106
|
|
|
103
107
|
### Fix guidelines
|
|
104
108
|
|
|
105
|
-
- **Minimal changes.** Only modify what is necessary to fix the observed bug. Avoid rewriting unrelated specs.
|
|
106
|
-
- **Preserve chronological order.** If adding a new spec, place it correctly relative to existing
|
|
109
|
+
- **Minimal changes.** Only modify what is necessary to fix the observed bug. Avoid rewriting unrelated functional specs.
|
|
110
|
+
- **Preserve chronological order.** If adding a new functional spec, place it correctly relative to the existing ones.
|
|
107
111
|
- **Stay language-agnostic.** Functional specs describe behavior, not implementation. Platform-specific guidance belongs in implementation reqs.
|
|
108
|
-
- **Respect the 200 LOC limit.** If a fix makes a spec too complex, use `break-down-func-spec` to split it.
|
|
109
|
-
- **
|
|
112
|
+
- **Respect the 200 LOC limit.** If a fix makes a functional spec too complex, use `break-down-func-spec` to split it.
|
|
113
|
+
- **Keep the test plain file in step.** A functional spec you change has its scenarios, if it has any, reviewed in the same ***plain diff, so they still follow from it (`test-plain-files.md`).
|
|
114
|
+
- **Check for new conflicts.** After editing, verify the fix doesn't conflict with other functional specs by running `analyze-func-specs` once with the edited functional spec(s) plus every existing functional spec that touches the same concepts. The batched analyzer reports all conflicting pairs in a single call; resolve each with `resolve-spec-conflict`.
|
|
110
115
|
|
|
111
116
|
## Phase 5 — Verify and Report
|
|
112
117
|
|
|
@@ -114,30 +119,30 @@ Apply the fix based on the diagnosed root cause:
|
|
|
114
119
|
2. **Summarize the findings** for the user:
|
|
115
120
|
- What was observed (the bug)
|
|
116
121
|
- What the generated code was doing (the symptom in code)
|
|
117
|
-
- Which spec(s) caused the issue (the root cause)
|
|
118
|
-
- What was changed in the
|
|
119
|
-
- What to expect after
|
|
122
|
+
- Which functional spec(s) caused the issue (the root cause)
|
|
123
|
+
- What was changed in the ***plain (the fix)
|
|
124
|
+
- What to expect after regenerating
|
|
120
125
|
3. **If uncertain**, present the diagnosis and proposed fix to the user for confirmation before making changes. Some bugs have multiple possible causes — when in doubt, explain the options and let the user decide.
|
|
121
126
|
|
|
122
127
|
## Debugging Strategies
|
|
123
128
|
|
|
124
129
|
### Strategy 1: Observation-Driven (most common)
|
|
125
130
|
|
|
126
|
-
Start from the user's observation and work backward through the code to the spec.
|
|
131
|
+
Start from the user's observation and work backward through the code to the functional spec.
|
|
127
132
|
|
|
128
133
|
```
|
|
129
134
|
User sees bug → Read generated code → Find the responsible code path →
|
|
130
|
-
Identify which spec governs that code → Diagnose why the spec produced wrong code →
|
|
131
|
-
Fix the
|
|
135
|
+
Identify which functional spec governs that code → Diagnose why the functional spec produced wrong code →
|
|
136
|
+
Fix the ***plain
|
|
132
137
|
```
|
|
133
138
|
|
|
134
139
|
### Strategy 2: Spec-Focused
|
|
135
140
|
|
|
136
|
-
The user suspects a specific spec. Compare the spec directly against the generated code.
|
|
141
|
+
The user suspects a specific functional spec. Compare the functional spec directly against the generated code.
|
|
137
142
|
|
|
138
143
|
```
|
|
139
|
-
User points to spec → Read the spec → Find the generated code that implements it →
|
|
140
|
-
Compare spec intent vs. code behavior → Diagnose the gap → Fix the
|
|
144
|
+
User points to functional spec → Read the functional spec → Find the generated code that implements it →
|
|
145
|
+
Compare functional spec intent vs. code behavior → Diagnose the gap → Fix the ***plain
|
|
141
146
|
```
|
|
142
147
|
|
|
143
148
|
### Strategy 3: Test-Failure-Driven
|
|
@@ -146,43 +151,44 @@ A test is failing. Use the test as the entry point.
|
|
|
146
151
|
|
|
147
152
|
```
|
|
148
153
|
Read failing test → Understand what it expects → Read the implementation →
|
|
149
|
-
Determine if the test or the implementation is wrong → Trace back to the
|
|
150
|
-
Fix the
|
|
154
|
+
Determine if the test or the implementation is wrong → Trace back to the ***plain →
|
|
155
|
+
Fix the ***plain (or the test reqs / the scenario in the test plain file if the test itself is wrong)
|
|
151
156
|
```
|
|
152
157
|
|
|
153
158
|
### Strategy 4: Differential
|
|
154
159
|
|
|
155
|
-
The bug appeared after a new spec was added. Compare before and after.
|
|
160
|
+
The bug appeared after a new functional spec was added. Compare before and after.
|
|
156
161
|
|
|
157
162
|
```
|
|
158
|
-
Identify the newly added spec → Read the specs it might conflict with →
|
|
159
|
-
Check if the new spec introduced the bug → Use resolve-spec-conflict or
|
|
160
|
-
revise the new spec
|
|
163
|
+
Identify the newly added functional spec → Read the functional specs it might conflict with →
|
|
164
|
+
Check if the new functional spec introduced the bug → Use resolve-spec-conflict or
|
|
165
|
+
revise the new functional spec
|
|
161
166
|
```
|
|
162
167
|
|
|
163
168
|
## Common Pitfalls
|
|
164
169
|
|
|
165
|
-
### Fixing the code instead of the
|
|
166
|
-
Never modify files in `plain_modules/<module>/code/` or `plain_modules/<module>/tests/`. Even if the fix is obvious in the code, the change must be made in the `.plain` file so it persists across
|
|
170
|
+
### Fixing the code instead of the ***plain
|
|
171
|
+
Never modify files in `plain_modules/<module>/code/` or `plain_modules/<module>/tests/`. Even if the fix is obvious in the code, the change must be made in the `.plain` file so it persists across regenerations.
|
|
167
172
|
|
|
168
173
|
### Treating symptoms instead of root causes
|
|
169
|
-
If the user says "the button is in the wrong place," don't just add positioning detail. Investigate why the
|
|
174
|
+
If the user says "the button is in the wrong place," don't just add positioning detail. Investigate why the coding agent placed it there — the root cause might be a missing layout functional spec, an ambiguous screen description, or a conflict with another functional spec.
|
|
170
175
|
|
|
171
176
|
### Over-specifying the fix
|
|
172
177
|
Adding too much implementation detail to a functional spec (e.g., specific pixel values, widget names, CSS properties) makes it brittle. Prefer behavior-level fixes. Use implementation reqs for platform-specific guidance.
|
|
173
178
|
|
|
174
179
|
### Ignoring ripple effects
|
|
175
|
-
A spec
|
|
180
|
+
A fix to one functional spec may affect others that reference the same concepts. Always check neighboring functional specs for conflicts after making a change.
|
|
176
181
|
|
|
177
182
|
## Validation Checklist
|
|
178
183
|
|
|
179
184
|
- [ ] User's observation is clearly understood (expected vs. actual behavior)
|
|
180
185
|
- [ ] Generated code has been read to understand the current implementation
|
|
181
|
-
- [ ] Root cause is diagnosed (ambiguous / missing / conflicting / incorrect / missing impl req)
|
|
186
|
+
- [ ] Root cause is diagnosed (ambiguous / missing / conflicting / incorrect / incorrect scenario / missing impl req)
|
|
182
187
|
- [ ] Fix is applied in the `.plain` file(s) only — no generated code modified
|
|
183
188
|
- [ ] Fix is minimal and targeted to the observed bug
|
|
184
|
-
- [ ] Fix does not introduce new conflicts with existing specs
|
|
185
|
-
- [ ] Modified specs respect the 200 LOC limit
|
|
186
|
-
- [ ] Modified specs remain language-agnostic
|
|
189
|
+
- [ ] Fix does not introduce new conflicts with existing functional specs
|
|
190
|
+
- [ ] Modified functional specs respect the 200 LOC limit
|
|
191
|
+
- [ ] Modified functional specs remain language-agnostic
|
|
187
192
|
- [ ] Chronological ordering is preserved
|
|
193
|
+
- [ ] Test plain file of every changed functional spec in step with it
|
|
188
194
|
- [ ] Summary of findings and fix presented to the user
|
|
@@ -10,6 +10,10 @@ description: >-
|
|
|
10
10
|
a new language (Node.js, Go, Rust, Flutter, etc.) to a ***plain project, or
|
|
11
11
|
wants to regenerate / adapt one of the existing runners.
|
|
12
12
|
allowed-tools: [read, write, edit, bash, ls, grep, find]
|
|
13
|
+
metadata:
|
|
14
|
+
# In the chat it runs in the background, on the coding model of the module it is for: its end comes back to the chat.
|
|
15
|
+
background: true
|
|
16
|
+
model: coding
|
|
13
17
|
---
|
|
14
18
|
|
|
15
19
|
# Implement Conformance Testing Script
|
|
@@ -26,9 +30,9 @@ Read both `.sh` references before writing anything — every script you produce
|
|
|
26
30
|
|
|
27
31
|
## How the harness uses the script
|
|
28
32
|
|
|
29
|
-
A module's conformance tests live in one folder, `plain_modules/<module>/tests` — the module's whole conformance suite: every
|
|
33
|
+
A module's conformance tests live in one folder, `plain_modules/<module>/tests` — the module's whole conformance suite: every functional spec's tests, plus those carried up from the modules it requires. There are no per-functional-spec subfolders.
|
|
30
34
|
|
|
31
|
-
|
|
35
|
+
turri (not the agent that writes the tests) runs the script once per check, against the whole suite:
|
|
32
36
|
|
|
33
37
|
```
|
|
34
38
|
bash run_conformance_tests_<lang>.sh <build_folder> <conformance_tests_folder>
|
|
@@ -42,7 +46,7 @@ Every conformance script is self-contained: toolchain check, argument validation
|
|
|
42
46
|
|
|
43
47
|
A conformance script is structurally very close to a unit-test script (see the sibling skill [`implement-unit-testing-script`](../implement-unit-testing-script/SKILL.md)) but with two important differences:
|
|
44
48
|
|
|
45
|
-
1. **Two positional arguments instead of one.** A conformance script takes both the **build folder** (source under test) and a **separate conformance tests folder** (the tests to execute against that build). Both arrive as **absolute paths**
|
|
49
|
+
1. **Two positional arguments instead of one.** A conformance script takes both the **build folder** (source under test) and a **separate conformance tests folder** (the tests to execute against that build). Both arrive from turri as **absolute paths** (older harnesses passed relative ones) — the script must work with both.
|
|
46
50
|
2. **Tests are loaded from outside the working folder.** The build is staged into the working folder `<temp>/<lang>_<basename>` (system temp directory + language id + basename of `$1` — see [Conventions](#conventions)) and the script `cd`s into it, but the test command is pointed at the conformance tests folder, resolved to an **absolute path** up front (`$CONFORMANCE_TESTS_FOLDER` below). Tests are never copied into the staging area.
|
|
47
51
|
|
|
48
52
|
Everything else — toolchain check, build staging, dependency isolation, exit codes — is the same.
|
|
@@ -57,12 +61,12 @@ A PowerShell (`.ps1`) copy is optional and only for people who run the script by
|
|
|
57
61
|
|
|
58
62
|
1. **Toolchain check.** Verify that the required language runtime / build tool (and the required version, if any) is installed. If not, print an error and exit with code `69`.
|
|
59
63
|
2. **Argument validation.** Require **two** positional arguments: `<build_folder>` and `<conformance_tests_folder>`. If either is missing, print usage and exit with code `69`.
|
|
60
|
-
3. **Resolve the conformance tests folder to an absolute path.**
|
|
64
|
+
3. **Resolve the conformance tests folder to an absolute path.** turri passes `$2` as an absolute path; older harnesses passed it relative to the invocation directory. Resolve it once, up front, into a variable the test command in step 8 uses:
|
|
61
65
|
- Bash: `case "$2" in /*) CONFORMANCE_TESTS_FOLDER="$2" ;; *) CONFORMANCE_TESTS_FOLDER="$(pwd)/$2" ;; esac`
|
|
62
66
|
- PowerShell: `if (-not [System.IO.Path]::IsPathRooted($ConformanceTestsFolder)) { $ConformanceTestsFolder = Join-Path (Get-Location) $ConformanceTestsFolder }`
|
|
63
67
|
|
|
64
68
|
Never concatenate `$current_dir/$2` at the point of use — with an absolute `$2` that produces a doubled path (`/project//abs/path/...`), the classic symptom being `ImportError: Start directory is not importable` pointing at a path that contains the project root twice.
|
|
65
|
-
4. **Working directory setup.** Define a working folder at `<temp>/<lang>_<basename>` (`/tmp/<lang>_$(basename "$1")` in Bash, `Join-Path ([System.IO.Path]::GetTempPath()) "<lang>_$(Split-Path $BuildFolder -Leaf)"` in PowerShell). Wipe it (`rm -rf` / `Remove-Item -Recurse -Force`) and recreate it. Register cleanup so the folder is removed when the script exits — `trap 'rm -rf "$WORKING_FOLDER"' EXIT` in Bash, `Remove-Item -Recurse -Force` in the `finally` block in PowerShell. This folder — and **only** this folder — is where every subsequent write must land. 4b. **Refuse conformance tests in the build folder.** Conformance and acceptance tests live in the conformance tests folder (`$2`) and nowhere else. Before anything is staged, look for them in `$1`: any file carrying a conformance or acceptance label (`conformance tests of
|
|
69
|
+
4. **Working directory setup.** Define a working folder at `<temp>/<lang>_<basename>` (`/tmp/<lang>_$(basename "$1")` in Bash, `Join-Path ([System.IO.Path]::GetTempPath()) "<lang>_$(Split-Path $BuildFolder -Leaf)"` in PowerShell). Wipe it (`rm -rf` / `Remove-Item -Recurse -Force`) and recreate it. Register cleanup so the folder is removed when the script exits — `trap 'rm -rf "$WORKING_FOLDER"' EXIT` in Bash, `Remove-Item -Recurse -Force` in the `finally` block in PowerShell. This folder — and **only** this folder — is where every subsequent write must land. 4b. **Refuse conformance tests in the build folder.** Conformance and acceptance tests live in the conformance tests folder (`$2`) and nowhere else. Before anything is staged, look for them in `$1`: any file carrying a conformance or acceptance label (`conformance tests of <module>:<func-id>`, `conformance scenario <scenario-id> of <module>:<func-id>`, `acceptance scenario <scenario-id> of <module>:<func-id>`), and any test file under a folder or name that says conformance or acceptance. Found any: print each path, say they belong in the conformance tests folder, and exit `1`. See [Tests in the build folder](#tests-in-the-build-folder).
|
|
66
70
|
5. **Copy the build.** Recursively copy everything from `<build_folder>` (`$1`) into the working folder. **Do not** copy the conformance tests — they stay where they are. After this step both `$1` (build folder) and `$2` (conformance tests folder) are treated as **read-only** for the rest of the script.
|
|
67
71
|
6. **Enter the working directory.** `cd` / `Set-Location` into `<temp>/<lang>_<basename>`. If that fails, exit with code `69`. All remaining steps run from inside the working folder; they must never write back to `$1` or `$2`.
|
|
68
72
|
7. **Install dependencies into an isolated environment inside `<temp>/<lang>_<basename>`.** Set up a per-working-folder dependency location (a Python venv at `./.venv`, a local `./node_modules`, a project-scoped Maven repo at `./.m2`, etc.) and install/resolve all dependencies into it. **Never** install into the source build folder (`$1`), the conformance tests folder (`$2`), the user's global cache (`~/.m2`, system-wide `pip`, `~/.cargo`, `~/.npm`, ...), or anywhere outside `<temp>/<lang>_<basename>`. If the install command fails, propagate its exit code immediately and **do not** proceed to step 8. See [Dependency isolation](#dependency-isolation). 7b. **Static checks of the conformance tests.** Once the build and the test libraries are installed - so the tests resolve against them as they do when they run - check **the conformance tests in `$CONFORMANCE_TESTS_FOLDER`, and only them**: the formatter in check mode, the linter, and the type checker where the language has one apart from its compiler. See [Static checks of the tests](#static-checks-of-the-tests). Run every check, even after one fails, print each one's whole output, remember whether any failed - and go on to step 8 whatever they said, so one run reports both.
|
|
@@ -83,7 +87,7 @@ A conformance script has **two** read-only inputs: the source build folder (`$1`
|
|
|
83
87
|
|
|
84
88
|
Why each input is read-only:
|
|
85
89
|
|
|
86
|
-
- **`$1` (build folder)** is shared with
|
|
90
|
+
- **`$1` (build folder)** is shared with turri (`plain_modules/<module>/code` by default) and downstream tooling. Writing into it corrupts turri's view of "what was generated" and breaks subsequent regenerations. The whole point of staging into the system temp directory is so the source folder stays a clean, reproducible artifact of the regeneration.
|
|
87
91
|
- **`$2` (conformance tests folder)** is the user's authored test source — typically checked into version control. Writing into it pollutes the working tree, churns git status, and (with frameworks that auto-discover) can make subsequent runs pick up generated files as if they were tests.
|
|
88
92
|
|
|
89
93
|
If you find yourself about to issue any command whose `cwd` is `$1` or `$2`, or whose target path starts with `$1/` or `$2/`, **stop**. Either move the operation into `<temp>/<lang>_<basename>`, or you're doing something the script must not do.
|
|
@@ -94,7 +98,7 @@ A conformance test written into the build folder is in the wrong place: nothing
|
|
|
94
98
|
|
|
95
99
|
```bash
|
|
96
100
|
MISPLACED=$(grep -rlE --exclude=".turri*" --exclude-dir=.venv --exclude-dir=node_modules --exclude-dir=target \
|
|
97
|
-
"conformance tests
|
|
101
|
+
"(conformance tests|(conformance|acceptance) scenario [a-z0-9-]+) of [^[:space:]:]+:[a-z0-9-]+" "$1" 2>/dev/null; \
|
|
98
102
|
find "$1" -type f \( -ipath "*conformance*" -o -ipath "*acceptance*" \) -iname "*test*" \
|
|
99
103
|
-not -name ".turri*" -not -path "*/.venv/*" -not -path "*/node_modules/*" -not -path "*/target/*" 2>/dev/null)
|
|
100
104
|
if [ -n "$MISPLACED" ]; then
|
|
@@ -148,7 +152,7 @@ Shared across both shell flavors:
|
|
|
148
152
|
- Any other non-zero code — propagated from the underlying test command.
|
|
149
153
|
- **Working folder naming:** `<temp>/<lang>_<basename>` — the system temp directory (`/tmp` in Bash, `[System.IO.Path]::GetTempPath()` in PowerShell) plus a short identifier for the language (`java`, `python`, `node`, `go`, `rust`, ...) and the **basename** of the *first* argument (the build folder — `$(basename "$1")` / `Split-Path $BuildFolder -Leaf`), never the conformance tests folder and never the raw argument (it may be absolute; concatenating it produces broken doubled paths). All dependency installs, build outputs, caches, test runner artifacts, and the test invocation itself live inside this folder. Nothing the script does should touch `$1` after step 5, or `$2` at any point.
|
|
150
154
|
- **Cleanup ownership:** the script owns its working folder and removes it on exit, success and failure alike (`trap 'rm -rf "$WORKING_FOLDER"' EXIT` in Bash; `Remove-Item -Recurse -Force` in the PowerShell `finally` block, after `Pop-Location`).
|
|
151
|
-
- **Logging — be as verbose as possible.** The script is the only thing the operator sees between a `turri`
|
|
155
|
+
- **Logging — be as verbose as possible.** The script is the only thing the operator sees between a `turri` regeneration and a green/red conformance result; when it fails, the only forensic evidence anyone has is its stdout/stderr. Treat the script like a production runbook, not a quiet helper. Concretely:
|
|
152
156
|
- **Announce every step before doing it**, with the exact value of every variable involved — the resolved build folder, the resolved conformance-tests folder, the working folder, the language, the toolchain version detected, the isolation root being set up, and the test command about to run. "Running tests" alone is useless; "Running `python -m unittest discover -b -s /abs/path/to/tests` from working folder `/tmp/python_build/` with venv activated at `./.venv` (Python 3.11.6)" is triage-ready.
|
|
153
157
|
- **Echo every non-trivial command before executing it.** In Bash, use `set -x` for the body of the script (or `echo "+ <cmd>"` immediately before each call); in PowerShell, set `$VerbosePreference = 'Continue'` and use `Write-Verbose` / `Write-Host` to print each command line with its arguments.
|
|
154
158
|
- **Print clear section banners.** Each pattern step gets a banner like `===== [7/8] Install dependencies =====` so a long log can be navigated by eye.
|
|
@@ -202,26 +206,26 @@ The harness never runs `.ps1` scripts. These notes apply only to an optional cop
|
|
|
202
206
|
|
|
203
207
|
## Workflow
|
|
204
208
|
|
|
205
|
-
1.
|
|
209
|
+
1. Take the target **language**, **conformance test framework** and **dependency manifest** (`pom.xml`, `requirements.txt` / `pyproject.toml`, `package.json`, `go.mod`, `Cargo.toml`, ...) from the module the script is for: its stack - the `stdlib/...` template it imports, and its conformance template `stdlib/<kind>/conformance/<stack>` - its ***test reqs***, and the project. You run in the background, with nobody to ask: decide what they leave open, the choice most consistent with them, and say what you decided in your result.
|
|
206
210
|
2. Read [assets/run_conformance_tests_java.sh](assets/run_conformance_tests_java.sh) and [assets/run_conformance_tests_python.sh](assets/run_conformance_tests_python.sh) to refresh the exact structure.
|
|
207
211
|
3. Translate each step into the equivalent Bash commands for the target language. The toolchain check, dependency install, and test invocation are the language-specific parts; the rest is mechanical.
|
|
208
212
|
4. Pick the right row from the [Dependency isolation](#dependency-isolation) table — use the same flag/env var in steps 7, 7b and 8 — and the checks from [Static checks of the tests](#static-checks-of-the-tests).
|
|
209
213
|
5. Add the language-appropriate "no tests discovered" guard from [No tests discovered detection](#no-tests-discovered-detection), and the guard against conformance tests in the build folder from [Tests in the build folder](#tests-in-the-build-folder).
|
|
210
214
|
6. Save the new script as `test_scripts/run_conformance_tests_<lang>.sh` and `chmod +x` it. Write a `.ps1` copy only if the user wants to run it by hand on Windows.
|
|
211
|
-
7. **Make sure turri finds the script.** Do not link it from the
|
|
215
|
+
7. **Make sure turri finds the script.** Do not link it from the ***plain - turri finds the conformance testing script in the project root (the folder that holds `plain_modules/`) and puts it, whole, into the prompts of the agents that write and review the conformance tests itself, whenever conformance tests are written for a build; a link to it from the ***plain is ignored. It looks, in order, at `config.turri.yaml` in the root (`conformance-tests-script: <path relative to the root>`) - with the chosen stack's `config.turri.<stack>.yaml` over it, then for the one `run_conformance_tests*.sh` in the root, then for the one in `test_scripts/`. Saving the script in `test_scripts/` (or the root) is enough when it is the only one there. If the project keeps it elsewhere, name it in `config.turri.yaml` as `conformance-tests-script:` - and when the project has only a `config.yaml`, copy it to `config.turri.yaml` first (and each `config.<name>.yaml` to `config.turri.<name>.yaml`) and edit the copy; `config.yaml` may be another tool's, and is never written. If the project has modules on different stacks - e.g. a Python backend and a React frontend side by side, sharing only an imported module - each part is on a stack of its own, and each stack names its own script in its own file beside `config.turri.yaml`: `conformance-tests-script:` in `config.turri.backend.yaml` for the stack `backend`, in `config.turri.frontend.yaml` for the stack `frontend`. Name each stack after the module it builds - `config.turri.backend.yaml` for `backend.plain` - and that module is built with it by itself, with nothing to choose; a stack named otherwise is passed as `stack` to `regenerate` in the chat, or `--stack <stack>` on the command line. turri never guesses between two. The script is required: without one, no conformance or acceptance test is written or run. The `***test reqs***` still describe the conformance framework and command in words; do not paraphrase the script into them.
|
|
212
216
|
|
|
213
217
|
## Anti-Patterns
|
|
214
218
|
|
|
215
|
-
- **(Hard mistake) Don't install into, build into, or otherwise write to the source build folder (`$1`) or the conformance tests folder (`$2`).** Both arguments are read-only input. Every install, cache, build artifact, log, JUnit XML, coverage report, compiled test class, and temp file must land in `<temp>/<lang>_<basename>`. This includes never running `pip install`, `npm install`, `mvn install`, or `cargo build` with `$1` or `$2` as their `cwd` or target; never letting a venv / `node_modules` / `.m2` / `.gocache` / `.cargo` directory appear inside `$1` or `$2`; and never running the test command from inside either folder. The whole point of staging into the system temp directory is so the build folder remains a clean artifact of the
|
|
216
|
-
- **(Hard mistake) Don't build any path by concatenating a raw argument.** `$1` and `$2` arrive as absolute paths
|
|
219
|
+
- **(Hard mistake) Don't install into, build into, or otherwise write to the source build folder (`$1`) or the conformance tests folder (`$2`).** Both arguments are read-only input. Every install, cache, build artifact, log, JUnit XML, coverage report, compiled test class, and temp file must land in `<temp>/<lang>_<basename>`. This includes never running `pip install`, `npm install`, `mvn install`, or `cargo build` with `$1` or `$2` as their `cwd` or target; never letting a venv / `node_modules` / `.m2` / `.gocache` / `.cargo` directory appear inside `$1` or `$2`; and never running the test command from inside either folder. The whole point of staging into the system temp directory is so the build folder remains a clean artifact of the regeneration and the conformance tests folder remains a clean tree under the user's version control — writing to either one corrupts those guarantees.
|
|
220
|
+
- **(Hard mistake) Don't build any path by concatenating a raw argument.** `$1` and `$2` arrive from turri as absolute paths. Patterns like `WORKING_FOLDER=.tmp/$1`, `WORKING_FOLDER=<lang>_$1`, or `"$current_dir/$2"` silently produce doubled paths (`/project//abs/path/...`) — the classic symptom is `ImportError: Start directory is not importable` or "folder does not exist" pointing at a path that contains the project root twice. Always derive the working folder from `$(basename "$1")` / `Split-Path -Leaf`, and use the `$CONFORMANCE_TESTS_FOLDER` resolved in step 3.
|
|
217
221
|
- **Don't rely on anything being set up before the script runs.** The script is self-contained: it stages the build and installs dependencies itself, every time. Don't look for a pre-warmed working folder or environment left by an earlier run.
|
|
218
222
|
- **Don't copy the conformance tests folder into the working folder.** Only the build folder is staged. The test folder is read in place from `$CONFORMANCE_TESTS_FOLDER`.
|
|
219
223
|
- **Don't compute the test path after `cd`.** Resolve `$2` to an absolute path in step 3; otherwise a relative `$2` will be resolved against the working folder and silently miss the tests.
|
|
220
|
-
- **Don't expect per-
|
|
224
|
+
- **Don't expect per-functional-spec subfolders in `$2`.** `$2` holds the module's whole conformance suite; point the test command at it and let the runner discover every test in it.
|
|
221
225
|
- **Don't run tests from anywhere but the conformance tests folder, and don't let conformance tests in the build folder pass unnoticed.** The build's tests are the unit testing script's; a conformance test in the build folder is misplaced, and the script fails on it (step 4b) instead of running it or ignoring it.
|
|
222
226
|
- **Don't skip the "no tests discovered" check.** A conformance suite that finds zero tests and exits `0` is the worst possible failure mode — it looks like success.
|
|
223
227
|
- **Don't skip the toolchain check**, even when "everyone has it installed" — exit code `69` is what the calling system relies on to detect a missing runtime.
|
|
224
|
-
- **Don't reuse the source folder in place.** Always copy into `<temp>/<lang>_<basename>` first;
|
|
228
|
+
- **Don't reuse the source folder in place.** Always copy into `<temp>/<lang>_<basename>` first; turri relies on this isolation.
|
|
225
229
|
- **Don't skip cleanup.** Register the `trap` / `finally` removal of the working folder right after creating it, so it is removed on success and failure alike.
|
|
226
230
|
- **Don't change the exit-code contract.** Other parts of the system branch on `69`, `1` and `80` specifically — and these codes must be identical between the Bash script and any PowerShell copy.
|
|
227
231
|
- **Don't check the tests on their own, or the code with them.** The checks run after the install, against what the tests run with, and report the tests' findings alone.
|
|
@@ -231,5 +235,5 @@ The harness never runs `.ps1` scripts. These notes apply only to an optional cop
|
|
|
231
235
|
- **Don't ship only a `.ps1`.** The harness runs `.sh` scripts only; a project with just a PowerShell runner has no conformance testing script as far as turri is concerned.
|
|
232
236
|
- **Don't install dependencies into the user's global location** (`~/.m2`, system-wide `pip`, `~/.cargo`, etc.). Always isolate inside `$WORKING_FOLDER` so concurrent runs and other projects can't interfere.
|
|
233
237
|
- **Don't run the test command without first verifying the install succeeded.** A failed install followed by a "test" run produces misleading errors that look like test failures.
|
|
234
|
-
- **Don't link the script from the
|
|
238
|
+
- **Don't link the script from the ***plain, and don't leave turri guessing.** turri finds the script in `config.turri.yaml`, the project root, or `test_scripts/`, shows it to the agents that write the conformance tests itself, and writes or runs no conformance test without it; a link to it from the ***plain is ignored. Put it where it is found, and when there are two conformance testing scripts in one folder, name the one to use as `conformance-tests-script:` - in each stack's `config.turri.<stack>.yaml` when they are for parts on different stacks (see Workflow step 7).
|
|
235
239
|
- **Don't write a terse script.** Silent steps, `>/dev/null` redirects on the install output, missing `--version` prints, absent step banners, an un-logged resolved `$2`, and a missing final summary all make the script harder to debug than the tests it is running. Follow the *Logging — be as verbose as possible* rule under [Conventions](#conventions) literally: every step announces itself, every command is echoed, the resolved `$CONFORMANCE_TESTS_FOLDER` is printed, every failure prints what was about to run and where, and the final summary names the test command, exit code, and working folder.
|
package/genome/skills/implement-conformance-testing-script/assets/run_conformance_tests_java.sh
CHANGED
|
@@ -21,7 +21,7 @@ printf "Current directory: $current_dir\n"
|
|
|
21
21
|
|
|
22
22
|
# Resolve the conformance tests folder to an absolute path so it can be used
|
|
23
23
|
# after changing into the working folder ($2 may be relative to the invocation
|
|
24
|
-
# directory on older
|
|
24
|
+
# directory on older harnesses; turri passes it as an absolute path).
|
|
25
25
|
case "$2" in
|
|
26
26
|
/*) CONFORMANCE_TESTS_SOURCE="$2" ;;
|
|
27
27
|
*) CONFORMANCE_TESTS_SOURCE="$current_dir/$2" ;;
|
|
@@ -46,7 +46,7 @@ mkdir -p "$JAVA_BUILD_SUBFOLDER"
|
|
|
46
46
|
# Conformance tests live in the conformance tests folder and nowhere else. One in the
|
|
47
47
|
# build folder is in the wrong place: nothing would run it as part of the suite. Refuse.
|
|
48
48
|
MISPLACED=$( (grep -rlE --exclude=".turri*" --exclude-dir=target --exclude-dir=.m2 \
|
|
49
|
-
"conformance tests
|
|
49
|
+
"(conformance tests|(conformance|acceptance) scenario [a-z0-9-]+) of [^[:space:]:]+:[a-z0-9-]+" "$1" 2>/dev/null; \
|
|
50
50
|
find "$1" -type f \( -ipath "*conformance*" -o -ipath "*acceptance*" \) -iname "*test*" \
|
|
51
51
|
-not -name ".turri*" -not -path "*/target/*" -not -path "*/.m2/*" 2>/dev/null) | sort -u)
|
|
52
52
|
if [ -n "$MISPLACED" ]; then
|
package/genome/skills/implement-conformance-testing-script/assets/run_conformance_tests_python.sh
CHANGED
|
@@ -28,7 +28,7 @@ fi
|
|
|
28
28
|
|
|
29
29
|
# Resolve the conformance tests folder to an absolute path so it can be used
|
|
30
30
|
# after changing into the working folder ($2 may be relative to the invocation
|
|
31
|
-
# directory on older
|
|
31
|
+
# directory on older harnesses; turri passes it as an absolute path).
|
|
32
32
|
case "$2" in
|
|
33
33
|
/*) CONFORMANCE_TESTS_FOLDER="$2" ;;
|
|
34
34
|
*) CONFORMANCE_TESTS_FOLDER="$(pwd)/$2" ;;
|
|
@@ -37,7 +37,7 @@ esac
|
|
|
37
37
|
# Conformance tests live in the conformance tests folder and nowhere else. One in the
|
|
38
38
|
# build folder is in the wrong place: nothing would run it as part of the suite. Refuse.
|
|
39
39
|
MISPLACED=$( (grep -rlE --exclude=".turri*" --exclude-dir=.venv --exclude-dir=node_modules --exclude-dir=target \
|
|
40
|
-
"conformance tests
|
|
40
|
+
"(conformance tests|(conformance|acceptance) scenario [a-z0-9-]+) of [^[:space:]:]+:[a-z0-9-]+" "$1" 2>/dev/null; \
|
|
41
41
|
find "$1" -type f \( -ipath "*conformance*" -o -ipath "*acceptance*" \) -iname "*test*" \
|
|
42
42
|
-not -name ".turri*" -not -path "*/.venv/*" -not -path "*/node_modules/*" -not -path "*/target/*" 2>/dev/null) | sort -u)
|
|
43
43
|
if [ -n "$MISPLACED" ]; then
|
|
@@ -10,6 +10,10 @@ description: >-
|
|
|
10
10
|
etc.) to a ***plain project, or wants to regenerate / adapt the existing
|
|
11
11
|
Java runner.
|
|
12
12
|
allowed-tools: [read, write, edit, bash, ls, grep, find]
|
|
13
|
+
metadata:
|
|
14
|
+
# In the chat it runs in the background, on the coding model of the module it is for: its end comes back to the chat.
|
|
15
|
+
background: true
|
|
16
|
+
model: coding
|
|
13
17
|
---
|
|
14
18
|
|
|
15
19
|
# Implement Unit Testing Script
|
|
@@ -20,7 +24,7 @@ The reference implementation is [assets/run_unittests_java.sh](assets/run_unitte
|
|
|
20
24
|
|
|
21
25
|
## Always Bash
|
|
22
26
|
|
|
23
|
-
turri requires this script and runs it itself, as `bash run_unittests_<lang>.sh <build_folder>` on the person's machine - macOS, Linux or WSL: its exit code decides whether a
|
|
27
|
+
turri requires this script and runs it itself, as `bash run_unittests_<lang>.sh <build_folder>` on the person's machine - macOS, Linux or WSL: its exit code decides whether a functional spec's unit tests pass. So the script is always **Bash (`.sh`)**, and stays portable across them - POSIX tools and flags, nothing only one of them has.
|
|
24
28
|
|
|
25
29
|
A **PowerShell (`.ps1`)** copy is written only when the user asks for one, to run the tests by hand on native Windows. turri never runs it, so a project with only a `.ps1` has no unit testing script.
|
|
26
30
|
|
|
@@ -31,7 +35,7 @@ The same eight-step pattern applies to both. Only the syntax changes.
|
|
|
31
35
|
Every testing script must implement these steps **in this order**:
|
|
32
36
|
|
|
33
37
|
1. **Toolchain check.** Verify that the required language runtime / build tool (and the required version, if any) is installed. If not, print an error and exit with code `69`.
|
|
34
|
-
2. **Argument validation.** Require exactly one positional argument: the source build folder. If missing, print usage and exit with code `1`.
|
|
38
|
+
2. **Argument validation.** Require exactly one positional argument: the source build folder. If missing, print usage and exit with code `1`. turri passes this argument as an **absolute path** (older harnesses passed a relative one) — the script must work with both, which is why the working-folder name in step 3 is built from the argument's *basename*, never from the raw argument.
|
|
35
39
|
3. **Working directory setup.** Define a working folder in the **system temp directory**: `/tmp/<lang>_$(basename "$1")` in Bash, `Join-Path ([System.IO.Path]::GetTempPath()) "<lang>_$(Split-Path $BuildFolder -Leaf)"` in PowerShell (written as `<temp>/<lang>_<basename>` below). If it exists, wipe its contents; otherwise create it. Register cleanup so the folder is removed when the script exits — `trap 'rm -rf "$WORKING_FOLDER"' EXIT` in Bash, a `Remove-Item -Recurse -Force` in the `finally` block in PowerShell. This folder — and **only** this folder — is where every subsequent write must land.
|
|
36
40
|
4. **Copy the build.** Recursively copy everything from the source folder into the working folder. After this step the source folder (`$1`) is treated as **read-only** for the rest of the script.
|
|
37
41
|
5. **Enter the working directory.** `cd` / `Set-Location` into `<temp>/<lang>_<basename>`. If that fails, exit with code `2`. All remaining steps run from inside the working folder; they must never write back to the source build folder.
|
|
@@ -48,7 +52,7 @@ The source build folder passed in as `$1` is **input only**. The script must nev
|
|
|
48
52
|
- run the test command from inside it (every test command runs from inside `<temp>/<lang>_<basename>` after the `cd` in step 5),
|
|
49
53
|
- create logs, caches, build outputs, or temp files inside it.
|
|
50
54
|
|
|
51
|
-
The build folder is shared with
|
|
55
|
+
The build folder is shared with turri (`plain_modules/<module>/code` by default) and downstream tooling. Writing into it corrupts turri's view of "what was generated" and breaks subsequent regenerations. Every write must go into `<temp>/<lang>_<basename>` — the whole point of staging into the system temp directory is so the source build folder stays a clean, reproducible artifact of the regeneration and no build debris is left inside the user's project.
|
|
52
56
|
|
|
53
57
|
If you find yourself about to issue any command whose `cwd` is the source folder, or whose target path starts with `$1/`, **stop**. Either move the operation into `<temp>/<lang>_<basename>`, or you're doing something the script must not do.
|
|
54
58
|
|
|
@@ -64,7 +68,7 @@ Shared across both shell flavors:
|
|
|
64
68
|
- Any other non-zero code — propagated from the underlying test command.
|
|
65
69
|
- **Working folder naming:** `<temp>/<lang>_<basename>` — the system temp directory (`/tmp` in Bash, `[System.IO.Path]::GetTempPath()` in PowerShell) plus a short identifier for the language (`java`, `python`, `node`, `go`, `rust`, ...) and the **basename** of the build-folder argument (`$(basename "$1")` / `Split-Path $BuildFolder -Leaf`). Never embed the raw argument in the path — it may be absolute, and concatenating it produces broken doubled paths. All dependency installs, build outputs, caches, and the test run itself live inside this folder. Nothing the script does should touch the source build folder after step 4.
|
|
66
70
|
- **Cleanup on exit:** the working folder is temporary — remove it when the script exits, on success and failure alike (`trap 'rm -rf "$WORKING_FOLDER"' EXIT` in Bash; `Remove-Item -Recurse -Force` in the PowerShell `finally` block, after `Pop-Location`). No leftover build folders may accumulate in the temp directory or the user's project.
|
|
67
|
-
- **Logging — be as verbose as possible.** The script is the only thing the operator sees between a `turri`
|
|
71
|
+
- **Logging — be as verbose as possible.** The script is the only thing the operator sees between a `turri` regeneration and a green/red test result; when it fails, the only forensic evidence anyone has is its stdout/stderr. Treat the script like a production runbook, not a quiet helper. Concretely:
|
|
68
72
|
- **Announce every step before doing it**, with the exact value of every variable involved — the resolved source folder, the working folder, the language, the toolchain version detected, the isolation root, the dependency-manifest path, the test command about to run. "Installing dependencies" alone is useless; "Installing dependencies from `./requirements.txt` into venv `./.venv` (Python 3.11.6 at `/usr/local/bin/python3`)" is triage-ready.
|
|
69
73
|
- **Echo every non-trivial command before executing it.** In Bash, use `set -x` for the body of the script (or `echo "+ <cmd>"` immediately before each call); in PowerShell, set `$VerbosePreference = 'Continue'` and use `Write-Verbose` / `Write-Host` to print each command line with its arguments.
|
|
70
74
|
- **Print clear section banners.** Each of the eight steps gets a banner like `===== [3/8] Working directory setup =====` so a long log can be navigated by eye.
|
|
@@ -134,19 +138,19 @@ Notes:
|
|
|
134
138
|
|
|
135
139
|
## Workflow
|
|
136
140
|
|
|
137
|
-
1.
|
|
141
|
+
1. Take the target **language**, **test framework** and **dependency manifest** (`pom.xml`, `requirements.txt` / `pyproject.toml`, `package.json`, `go.mod`, `Cargo.toml`, ...) from the module the script is for: its stack - the `stdlib/...` template it imports - its implementation reqs, and the project. You run in the background, with nobody to ask: decide what they leave open, the choice most consistent with them, and say what you decided in your result. The script is Bash; a PowerShell copy only when the context asks for one. When `config.turri.yaml` - or the stack's `config.turri.<stack>.yaml` - names a `base-folder:`, the module's code starts as a copy of that folder: an existing codebase to build on, or only files to use. Read it. When it holds code with its own test setup - a `pytest.ini` or `pyproject.toml` test section, `package.json` test scripts, a `pom.xml`, a `go.mod` - the script runs that setup's tests as they run there, beside the tests the coding agents write, so they keep passing; its existing dependency manifest is the one to install from. A folder of assets alone changes nothing.
|
|
138
142
|
2. Read [assets/run_unittests_java.sh](assets/run_unittests_java.sh) to refresh the exact structure.
|
|
139
143
|
3. Translate each of the eight steps above into the equivalent commands for the target language **and** shell. The toolchain check, dependency install, static checks and test invocation are the language-specific parts; the rest is mechanical translation between Bash and PowerShell syntax.
|
|
140
144
|
4. Pick the dependency-isolation mechanism from the [Dependency isolation](#dependency-isolation) table and use it consistently in steps 6, 7 and 8, and the checks from the [Static checks](#static-checks) table.
|
|
141
145
|
5. Save the new script as `test_scripts/run_unittests_<lang>.sh` (a `.ps1` copy next to it only if the user wants to run it by hand on Windows). For Bash, `chmod +x` it.
|
|
142
|
-
6. **Make sure turri finds the script.** Do not link it from the
|
|
146
|
+
6. **Make sure turri finds the script.** Do not link it from the ***plain - turri finds the unit testing script in the project root (the folder that holds `plain_modules/`) and puts it, whole, into the prompt of the coding agent itself; a link to it from the ***plain is ignored. It looks, in order, at `config.turri.yaml` in the root (`unittests-script: <path relative to the root>`) - with the chosen stack's `config.turri.<stack>.yaml` over it, then for the one `run_unittests*.sh` in the root, then for the one in `test_scripts/`. Saving the script in `test_scripts/` (or the root) is enough when it is the only one there. If the project keeps it elsewhere, name it in `config.turri.yaml` as `unittests-script:` - and when the project has only a `config.yaml`, copy it to `config.turri.yaml` first (and each `config.<name>.yaml` to `config.turri.<name>.yaml`) and edit the copy; `config.yaml` may be another tool's, and is never written. If the project has modules on different stacks - e.g. a Python backend and a React frontend side by side, sharing only an imported module - each part is on a stack of its own, and each stack names its own script in its own file beside `config.turri.yaml`: `unittests-script:` in `config.turri.backend.yaml` for the stack `backend`, in `config.turri.frontend.yaml` for the stack `frontend`. Name each stack after the module it builds - `config.turri.backend.yaml` for `backend.plain` - and that module is built with it by itself, with nothing to choose; a stack named otherwise is passed as `stack` to `regenerate` in the chat, or `--stack <stack>` on the command line. turri never guesses between two. The script is required: a build without one stops before it starts.
|
|
143
147
|
|
|
144
148
|
## Anti-Patterns
|
|
145
149
|
|
|
146
|
-
- **(Hard mistake) Don't install into, build into, or otherwise write to the source build folder.** The build folder passed as `$1` is read-only input. Every install, cache, build artifact, log, and temp file must land in `<temp>/<lang>_<basename>`. This includes never running `pip install`, `npm install`, `mvn install`, or `cargo build` with the source folder as their `cwd` or target, never letting a venv / `node_modules` / `.m2` / `.gocache` / `.cargo` directory appear inside the source folder, and never running the test command from inside it. The whole point of staging the build into the system temp directory is so the source folder remains a clean, reproducible artifact of the
|
|
147
|
-
- **(Hard mistake) Don't build any path by concatenating the raw build-folder argument.** `$1` arrives as an absolute path
|
|
150
|
+
- **(Hard mistake) Don't install into, build into, or otherwise write to the source build folder.** The build folder passed as `$1` is read-only input. Every install, cache, build artifact, log, and temp file must land in `<temp>/<lang>_<basename>`. This includes never running `pip install`, `npm install`, `mvn install`, or `cargo build` with the source folder as their `cwd` or target, never letting a venv / `node_modules` / `.m2` / `.gocache` / `.cargo` directory appear inside the source folder, and never running the test command from inside it. The whole point of staging the build into the system temp directory is so the source folder remains a clean, reproducible artifact of the regeneration — writing to it corrupts turri's view and breaks subsequent regenerations.
|
|
151
|
+
- **(Hard mistake) Don't build any path by concatenating the raw build-folder argument.** `$1` arrives from turri as an absolute path. Patterns like `WORKING_FOLDER=.tmp/$1`, `WORKING_FOLDER=<lang>_$1`, or `"$current_dir/$1"` silently produce doubled paths (`/project//abs/path/...`) — the classic symptom is an `ImportError: Start directory is not importable` or "folder does not exist" pointing at a path that contains the project root twice. Always take `$(basename "$1")` / `Split-Path -Leaf`.
|
|
148
152
|
- Don't skip the toolchain check, even when "everyone has it installed" — exit code `69` is what the calling system relies on to detect a missing runtime.
|
|
149
|
-
- Don't reuse the source folder in place. Always copy into `<temp>/<lang>_<basename>` first;
|
|
153
|
+
- Don't reuse the source folder in place. Always copy into `<temp>/<lang>_<basename>` first; turri relies on this isolation.
|
|
150
154
|
- Don't leave the working folder behind. Register the cleanup (`trap … EXIT` / `finally`) before the first write so even an early failure removes it.
|
|
151
155
|
- Don't change the exit-code contract. Other parts of the system branch on `1`, `2`, `69` and `80` specifically — and these codes must be identical between the Bash and PowerShell variants.
|
|
152
156
|
- Don't stop at the first failing check, and don't skip the tests because a check failed. Every check and the tests run each time, so one run tells the agent everything; the exit code says which failed.
|
|
@@ -155,5 +159,5 @@ Notes:
|
|
|
155
159
|
- Don't write a cross-shell hybrid (e.g. a `.sh` that detects PowerShell, or vice versa). Ship one script per shell, named with the appropriate extension.
|
|
156
160
|
- Don't install dependencies into the user's global location (`~/.m2`, system-wide `pip`, `~/.cargo`, etc.). Always isolate inside `$WORKING_FOLDER` so concurrent runs and other projects can't interfere.
|
|
157
161
|
- Don't run the test command without first verifying the install step succeeded. A failed install followed by a "test" run produces misleading errors that look like test failures.
|
|
158
|
-
- **Don't link the script from the
|
|
162
|
+
- **Don't link the script from the ***plain, and don't leave turri guessing.** turri finds the script in `config.turri.yaml`, the project root, or `test_scripts/`, and shows it to the coding agent itself; a link to it from the ***plain is ignored. Put it where it is found, and when there are two unit testing scripts in one folder, name the one to use as `unittests-script:` - in each stack's `config.turri.<stack>.yaml` when they are for parts on different stacks (see Workflow step 6).
|
|
159
163
|
- **Don't write a terse script.** Silent steps, `>/dev/null` redirects on the install output, missing `--version` prints, and absent failure banners all make the script harder to debug than the code it is testing. Follow the *Logging — be as verbose as possible* rule under [Conventions](#conventions) literally: every step announces itself, every command is echoed, every failure prints what was about to run and where.
|