plain-forge 1.0.16 → 1.0.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/forge/rules/examples/integration-embedded-testing/run_conformance_tests_java.sh +14 -3
- package/forge/rules/integration-embedded-testing.md +9 -7
- package/forge/rules/integration-embedded.md +4 -4
- package/forge/skills/implement-conformance-testing-script/SKILL.md +56 -43
- package/forge/skills/implement-conformance-testing-script/assets/run_conformance_tests_cypress.ps1 +34 -21
- package/forge/skills/implement-conformance-testing-script/assets/run_conformance_tests_golang.ps1 +16 -5
- package/forge/skills/implement-conformance-testing-script/assets/run_conformance_tests_java.sh +22 -11
- package/forge/skills/implement-conformance-testing-script/assets/run_conformance_tests_python.ps1 +13 -2
- package/forge/skills/implement-conformance-testing-script/assets/run_conformance_tests_python.sh +16 -6
- package/forge/skills/implement-prepare-environment-script/SKILL.md +23 -18
- package/forge/skills/implement-prepare-environment-script/assets/prepare_environment_java.sh +8 -4
- package/forge/skills/implement-prepare-environment-script/assets/prepare_environment_python.sh +8 -4
- package/forge/skills/implement-unit-testing-script/SKILL.md +14 -11
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_flutter.ps1 +6 -1
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_golang.ps1 +6 -1
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_java.sh +6 -3
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_python.ps1 +6 -1
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_python.sh +8 -4
- package/forge/skills/implement-unit-testing-script/assets/run_unittests_react.ps1 +6 -2
- package/forge/skills/init-config-file/SKILL.md +5 -5
- package/forge/skills/load-plain-reference/SKILL.md +97 -42
- package/forge/skills/plain-healthcheck/SKILL.md +1 -1
- package/forge/skills/run-codeplain/SKILL.md +12 -3
- package/package.json +1 -1
|
@@ -360,7 +360,7 @@ The **primary** purpose of these scripts is **automated execution by the rendere
|
|
|
360
360
|
- **Conformance tests are per-functional-spec.** Each functional spec in a module has its own folder under `conformance_tests/<module>/<functionality>/`. After the renderer finishes generating code for a new functional spec and the unit tests and refactoring passes, it runs the conformance tests of **all previous functional spec** to detect regressions — see [Conformance Test Workflow](#conformance-test-workflow). For a single module (with 0 `requires` modules) with N functional specs, the conformance script gets invoked **on the order of N times per render**, each invocation pointing at a different spec's test folder.
|
|
361
361
|
- **Each per-spec invocation is independent.** The conformance script does not know that it's the second invocation in a long sequence; from its point of view, each invocation is a cold start against a single spec's tests.
|
|
362
362
|
- **Per-spec independence is also what makes dependency installation expensive.** A naive conformance runner would re-install all of the project's runtime dependencies (Python venv + `pip install`, Maven dependency tree, `npm ci`, `cargo build`, ...) on every one of those N invocations. That's `N × install-cost` of wasted work for every render.
|
|
363
|
-
- **That is exactly why `prepare_environment_<lang>` exists.** Its **only** job is to amortize the install cost: install once at the start of a render, populate
|
|
363
|
+
- **That is exactly why `prepare_environment_<lang>` exists.** Its **only** job is to amortize the install cost: install once at the start of a render, populate the working folder in the system temp directory (`/tmp/<lang>_<basename-of-build-folder>/` on macOS/Linux, the `GetTempPath()` equivalent on Windows) with the warmed dependency cache and build artifacts, then let the conformance runner **attach** to that working folder on each of the N per-spec invocations instead of re-installing. The conformance runner's [activate-only variant](../implement-conformance-testing-script/SKILL.md#variant-decision-install-inline-vs-activate-only) does precisely that — and it **never deletes** that working folder; prepare owns its lifecycle. When no prepare script exists, the conformance runner falls back to the install-inline variant and pays the install cost N times — acceptable for tiny projects, costly for anything realistic.
|
|
364
364
|
- **The unit-test runner has a different execution model, because unit tests live in a different place.** Unit tests are part of the generated codebase itself — they sit directly inside `plain_modules/<module>/` next to the implementation they exercise — whereas conformance tests live *outside* the codebase, in their own per-spec folders under `conformance_tests/<module>/<spec>/`. As a result, the unit-test runner doesn't have a per-spec axis to iterate over: it just runs against the whole `plain_modules/<module>/` build in one go, gets invoked far fewer times per render, and has no amortization gain to chase. That's why the unit-test runner is always self-contained and there is no `prepare_environment`-equivalent for it.
|
|
365
365
|
|
|
366
366
|
Keep this framing in mind when you author or adapt any of these scripts. The decisions about working-folder paths, isolation locations, exit codes, and the activate-only-vs-install-inline split are not arbitrary house style — they are what makes the renderer's per-spec loop tractable.
|
|
@@ -369,7 +369,7 @@ Keep this framing in mind when you author or adapt any of these scripts. The dec
|
|
|
369
369
|
|
|
370
370
|
- **`run_unittests_<lang>.sh` / `.ps1`** — runs the auto-generated unit tests in `plain_modules/<module>/`. Authored by the [`implement-unit-testing-script`](../implement-unit-testing-script/SKILL.md) skill. Receives one positional argument: the source build folder name. Invoked roughly once per render. **Fully self-contained:** it installs its own dependencies inline (via `pip install -r requirements.txt`, `npm ci`, `mvn`, `cargo fetch`, etc.) and never relies on any other script having run first.
|
|
371
371
|
- **`run_conformance_tests_<lang>.sh` / `.ps1`** — runs the conformance tests in `conformance_tests/<module>/<spec>/` against the generated implementation. Authored by the [`implement-conformance-testing-script`](../implement-conformance-testing-script/SKILL.md) skill. Receives two positional arguments: the source build folder and the conformance tests folder. **Invoked once per previous functional spec on every render** — i.e. roughly N times for a module with N functional specs.
|
|
372
|
-
- **`prepare_environment_<lang>.sh` / `.ps1`** — *optional* one-time setup that runs **before** the conformance script and **only the conformance script**. Invoked **once per render** to install the build's dependencies and pre-warm build artifacts into
|
|
372
|
+
- **`prepare_environment_<lang>.sh` / `.ps1`** — *optional* one-time setup that runs **before** the conformance script and **only the conformance script**. Invoked **once per render** to install the build's dependencies and pre-warm build artifacts into the system-temp working folder (`/tmp/<lang>_<basename>/`) so the N subsequent conformance invocations can attach to the warmed environment instead of re-installing. Authored by the [`implement-prepare-environment-script`](../implement-prepare-environment-script/SKILL.md) skill. Receives one positional argument: the source build folder name. **It does not feed the unit-test script** — see [`prepare_environment` is conformance-only](#prepare_environment-is-conformance-only-common-mistake) below.
|
|
373
373
|
|
|
374
374
|
### `prepare_environment` is conformance-only (common mistake)
|
|
375
375
|
|
|
@@ -379,7 +379,7 @@ It is a **common and costly mistake** to assume that `prepare_environment_<lang>
|
|
|
379
379
|
|
|
380
380
|
Why:
|
|
381
381
|
|
|
382
|
-
- **Unit tests run against `plain_modules/<module>/`, conformance tests run against
|
|
382
|
+
- **Unit tests run against `plain_modules/<module>/`, conformance tests run against the system-temp working folder.** The two scripts stage into different places. `prepare_environment` populates `/tmp/<lang>_<basename>/` for conformance; the unit-test script does its own staging into its **own** system-temp working folder, installs its own dependencies there, and removes that folder when it exits.
|
|
383
383
|
- **The unit-test runner must work in isolation.** Users and CI systems run unit tests as a quick smoke check without ever invoking conformance. If `run_unittests_<lang>` depended on `prepare_environment` having run, those one-off unit-test invocations would silently fail (or be "fixed" by a misguided edit to make it depend on `prepare`).
|
|
384
384
|
- **The skill contract enforces it.** [`implement-unit-testing-script`](../implement-unit-testing-script/SKILL.md) emits a fully self-contained script every time: toolchain check → stage → install dependencies inline → run tests. It never emits an activate-only variant. The two-variant pattern is exclusive to the conformance runner.
|
|
385
385
|
|
|
@@ -389,11 +389,13 @@ If you find yourself authoring (or asked to author) a `prepare_environment` scri
|
|
|
389
389
|
|
|
390
390
|
Anything not listed here is documented in the individual skill file:
|
|
391
391
|
|
|
392
|
-
- **Input folders are read-only — hard rule.** The build folder (and, for conformance, the conformance tests folder too) is **input only**. Every install, build artifact, cache, log, JUnit XML, coverage report, compiled test class, and temp file must land inside
|
|
392
|
+
- **Input folders are read-only — hard rule.** The build folder (and, for conformance, the conformance tests folder too) is **input only**. Every install, build artifact, cache, log, JUnit XML, coverage report, compiled test class, and temp file must land inside the working folder in the system temp directory, never inside the input folder. The build folder is shared with the renderer and with the user's version control; writing into it corrupts both. If you find yourself about to issue a command whose `cwd` is an input folder, or whose target path starts with the input folder, **stop** — the write has to go into the working folder.
|
|
393
|
+
- **Arguments arrive as absolute paths.** Current renderers resolve the build folder and conformance-tests folder to absolute paths before invoking the scripts (older renderers passed relative ones). Scripts therefore build their working-folder name from the **basename** of the argument (`/tmp/<lang>_$(basename "$1")` in Bash, `Join-Path ([System.IO.Path]::GetTempPath()) "<lang>_$(Split-Path $arg -Leaf)"` in PowerShell) and resolve the conformance-tests argument to absolute before any `cd`. Concatenating a raw argument (`.tmp/$1`, `<lang>_$1`, `$current_dir/$2`) produces doubled paths and errors like `ImportError: Start directory is not importable`.
|
|
394
|
+
- **Cleanup ownership.** The unit-test runner and the install-inline conformance runner remove their working folder on exit (`trap 'rm -rf …' EXIT` / `finally`). `prepare_environment` and the activate-only conformance runner **never delete** the working folder — prepare wipes-and-recreates it once per render and the activate-only runner attaches to it N times; deleting it in between destroys the amortization.
|
|
393
395
|
- **Shell flavor matches the host.** `.sh` on macOS / Linux, `.ps1` on Windows. A project intended for both OSes ships both files in matching pairs (`prepare` + `conformance` for each language must agree on working-folder name and isolation paths).
|
|
394
396
|
- **Exit codes are part of the contract.** `69` for unrecoverable errors (missing toolchain, bad args, can't enter the working folder, install failed); `1` for the "no tests discovered" guard in the conformance runner (and bad usage in the unit-test runner); any other non-zero code is propagated from the underlying test command. Other skills — notably [`plain-healthcheck`](../plain-healthcheck/SKILL.md) and [`check-plain-env`](../check-plain-env/SKILL.md) — branch on these codes.
|
|
395
397
|
- **Wired in via `config.yaml`.** Each script that is actually generated must be referenced from the relevant `config.yaml` via the `unittests-script:`, `conformance-tests-script:`, and `prepare-environment-script:` keys respectively. See the [`init-config-file`](../init-config-file/SKILL.md) skill for the canonical assembly. **If `prepare-environment-script` is declared, `conformance-tests-script` must be declared too** — a prepare script only makes sense in service of conformance, and `plain-healthcheck` will hard-fail a project that violates this.
|
|
396
|
-
- **Conformance scripts come in two variants — unit-test scripts do not.** When a `prepare_environment_<lang>` script exists, the conformance script is the **activate-only** variant (it attaches to the env prepare populated in
|
|
398
|
+
- **Conformance scripts come in two variants — unit-test scripts do not.** When a `prepare_environment_<lang>` script exists, the conformance script is the **activate-only** variant (it attaches to the env prepare populated in the system-temp working folder). When no prepare exists, the conformance script is the **install-inline** variant (it stages and installs in one shot). The `implement-conformance-testing-script` skill picks the right variant automatically based on whether a prepare script is already on disk. **The unit-test script has no activate-only variant** — it is always self-contained, regardless of whether a `prepare_environment_<lang>` script exists.
|
|
397
399
|
- **Dependency isolation is project-local.** Each language's package cache / virtual env / build repo lives inside the working folder (`./.venv` for Python, `./node_modules` for Node, `./.m2` for Java, `./.gocache` for Go, `./.cargo` for Rust, `./.pub-cache` for Flutter, ...) — never in the user's home directory. The conformance script reads from the same project-local location prepare wrote to; the unit-test script uses its **own** working folder and its **own** copy of the isolated dependencies.
|
|
398
400
|
- **No language-package checks live in these scripts.** The scripts themselves install language packages via `pip install -r requirements.txt`, `npm ci`, `mvn -Dmaven.repo.local=...`, `go mod download`, `cargo fetch`, etc. They do **not** pre-verify individual packages; that's the package manager's job. The host-level checks for the toolchains and external dependencies belong in `check-plain-env`, not in these scripts.
|
|
399
401
|
- **Scripts are verbose**. They print out every step they take, including toolchain checks, dependency installations, and test results. This makes it easier to debug and understand what's going on.
|
|
@@ -569,13 +571,37 @@ GOOD — split into two independent root modules
|
|
|
569
571
|
|
|
570
572
|
## `codeplain` CLI reference
|
|
571
573
|
|
|
574
|
+
### Path parameter resolution
|
|
575
|
+
|
|
576
|
+
Every path argument the renderer accepts (`--build-folder`, `--conformance-tests-folder`, `--unittests-script`, `--conformance-tests-script`, `--prepare-environment-script`, `--log-file-name`, `--logging-config-path`, `--template-dir`, `--base-folder`, `--build-dest`, `--conformance-tests-dest`) resolves **based on where the value was written**:
|
|
577
|
+
|
|
578
|
+
- **CLI argument** → resolved against the **current working directory** of the `codeplain` invocation.
|
|
579
|
+
- **`config.yaml` value** → resolved against the **directory containing that config file**.
|
|
580
|
+
- **Default (not provided anywhere)** → resolved against the **directory containing the plain file**.
|
|
581
|
+
- **Absolute paths** and paths starting with `~` are used as-is (`~` is expanded).
|
|
582
|
+
|
|
583
|
+
Consequences worth knowing when authoring or debugging a project:
|
|
584
|
+
|
|
585
|
+
- Script paths are **validated at parse time** — the renderer fails fast with a `FileNotFoundError` if a `*-script` value doesn't resolve to an existing file. There is no fallback lookup (older renderer versions fell back to the renderer's own directory; that behavior is gone).
|
|
586
|
+
- The renderer passes the **resolved absolute paths** to the test scripts as their positional arguments — the scripts must not assume relative paths (see [Testing Scripts](#testing-scripts)).
|
|
587
|
+
- Since defaults resolve against the plain file's directory, `plain_modules/`, `conformance_tests/`, and `codeplain.log` land next to the spec regardless of where `codeplain` is invoked from — but an explicitly passed relative path lands relative to where it was written (CLI → cwd, config → config dir).
|
|
588
|
+
|
|
589
|
+
### CLI help
|
|
590
|
+
|
|
572
591
|
```txt
|
|
573
|
-
Render
|
|
592
|
+
Render plain code to target code. Path arguments resolve based on where they
|
|
593
|
+
were written: values given on the command line are resolved against the
|
|
594
|
+
current working directory, values read from config.yaml are resolved against
|
|
595
|
+
the config file's directory, and defaults are resolved against the directory
|
|
596
|
+
containing the plain file. Absolute paths (and paths starting with '~') are
|
|
597
|
+
used as-is.
|
|
574
598
|
|
|
575
599
|
positional arguments:
|
|
576
|
-
filename Path to the plain file to render. The directory
|
|
577
|
-
|
|
578
|
-
loading
|
|
600
|
+
filename Path to the plain file to render. The directory
|
|
601
|
+
containing this file has highest precedence for
|
|
602
|
+
template loading, so you can place custom templates
|
|
603
|
+
here to override the defaults. See --template-dir for
|
|
604
|
+
more details about template loading.
|
|
579
605
|
|
|
580
606
|
options:
|
|
581
607
|
-h, --help show this help message and exit
|
|
@@ -585,61 +611,90 @@ options:
|
|
|
585
611
|
--build-folder BUILD_FOLDER
|
|
586
612
|
Folder for build files
|
|
587
613
|
--log-to-file, --no-log-to-file
|
|
588
|
-
Enable logging to a file. Defaults to True. Set to
|
|
614
|
+
Enable logging to a file. Defaults to True. Set to
|
|
615
|
+
False to disable.
|
|
589
616
|
--log-file-name LOG_FILE_NAME
|
|
590
|
-
Name of the log file. Defaults to 'codeplain.log'.
|
|
591
|
-
|
|
617
|
+
Name of the log file. Defaults to 'codeplain.log'. If
|
|
618
|
+
a file already exists at the resolved path, it will be
|
|
619
|
+
overwritten by the current logs.
|
|
592
620
|
--render-range RENDER_RANGE
|
|
593
|
-
Specify a range of functionalities to render (e.g. `1`
|
|
594
|
-
|
|
621
|
+
Specify a range of functionalities to render (e.g. `1`
|
|
622
|
+
, `2`, `3`). Use comma to separate start and end IDs.
|
|
623
|
+
If only one functionality ID is provided, only that
|
|
624
|
+
functionality is rendered. Range is inclusive of both
|
|
625
|
+
start and end IDs.
|
|
595
626
|
--render-from RENDER_FROM
|
|
596
|
-
Continue generation starting from this specific
|
|
597
|
-
|
|
627
|
+
Continue generation starting from this specific
|
|
628
|
+
functionality (e.g. `2`). The functionality with this
|
|
629
|
+
ID will be included in the output. The functionality
|
|
630
|
+
ID must match one of the functionalities in your plain
|
|
631
|
+
file.
|
|
598
632
|
--force-render Force re-render of all the required modules.
|
|
599
633
|
--unittests-script UNITTESTS_SCRIPT
|
|
600
|
-
Shell script to run unit tests on generated code.
|
|
601
|
-
|
|
634
|
+
Shell script to run unit tests on generated code.
|
|
635
|
+
Receives the build folder path as its first argument
|
|
636
|
+
(default: 'plain_modules').
|
|
602
637
|
--conformance-tests-folder CONFORMANCE_TESTS_FOLDER
|
|
603
638
|
Folder for conformance test files
|
|
604
639
|
--conformance-tests-script CONFORMANCE_TESTS_SCRIPT
|
|
605
|
-
Path to conformance tests shell script. Every
|
|
606
|
-
|
|
607
|
-
|
|
640
|
+
Path to conformance tests shell script. Every
|
|
641
|
+
conformance test script should accept two arguments:
|
|
642
|
+
1) Path to a folder (e.g. `plain_modules/module_name`)
|
|
643
|
+
containing generated source code, 2) Path to a
|
|
644
|
+
subfolder of the conformance tests folder (e.g.
|
|
645
|
+
`conformance_tests/subfoldername`) containing test
|
|
646
|
+
files.
|
|
608
647
|
--prepare-environment-script PREPARE_ENVIRONMENT_SCRIPT
|
|
609
|
-
Path to a shell script that prepares the testing
|
|
610
|
-
|
|
648
|
+
Path to a shell script that prepares the testing
|
|
649
|
+
environment. The script should accept the source code
|
|
650
|
+
folder path as its first argument.
|
|
611
651
|
--test-script-timeout TEST_SCRIPT_TIMEOUT
|
|
612
|
-
Timeout for test scripts in seconds. If not provided,
|
|
613
|
-
|
|
614
|
-
--api
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
652
|
+
Timeout for test scripts in seconds. If not provided,
|
|
653
|
+
the default timeout of 120 seconds is used.
|
|
654
|
+
--api [API] Alternative base URL for the API. Default:
|
|
655
|
+
`https://api.codeplain.ai`
|
|
656
|
+
--api-key API_KEY API key used to access the API. If not provided, the
|
|
657
|
+
`CODEPLAIN_API_KEY` environment variable is used.
|
|
658
|
+
--full-plain Full preview ***plain specification before code
|
|
659
|
+
generation.Use when you want to preview context of all
|
|
660
|
+
***plain primitives that are going to be included in
|
|
661
|
+
order to render the given module.
|
|
662
|
+
--dry-run Dry run preview of the code generation (without
|
|
663
|
+
actually making any changes).
|
|
618
664
|
--replay-with REPLAY_WITH
|
|
619
665
|
--template-dir TEMPLATE_DIR
|
|
620
|
-
Path to a custom template directory. Templates are
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
666
|
+
Path to a custom template directory. Templates are
|
|
667
|
+
searched in the following order: 1) Directory
|
|
668
|
+
containing the plain file, 2) Custom template
|
|
669
|
+
directory (if provided through this argument), 3)
|
|
670
|
+
Built-in standard_template_library directory
|
|
671
|
+
--copy-build If set, copy the rendered contents of code in `--base-
|
|
672
|
+
folder` folder to `--build-dest` folder after
|
|
673
|
+
successful rendering.
|
|
625
674
|
--build-dest BUILD_DEST
|
|
626
|
-
Target folder to copy rendered contents of code to
|
|
675
|
+
Target folder to copy rendered contents of code to
|
|
676
|
+
(used only if --copy-build is set).
|
|
627
677
|
--copy-conformance-tests
|
|
628
|
-
If set, copy the conformance tests of code in
|
|
629
|
-
folder
|
|
678
|
+
If set, copy the conformance tests of code in
|
|
679
|
+
`--conformance-tests-folder` folder to `--conformance-
|
|
680
|
+
tests-dest` folder successful rendering. Requires
|
|
681
|
+
--conformance-tests-script.
|
|
630
682
|
--conformance-tests-dest CONFORMANCE_TESTS_DEST
|
|
631
|
-
Target folder to copy conformance tests of code to
|
|
683
|
+
Target folder to copy conformance tests of code to
|
|
684
|
+
(used only if --copy-conformance-tests is set).
|
|
632
685
|
--render-machine-graph
|
|
633
686
|
If set, render the state machine graph.
|
|
634
687
|
--logging-config-path LOGGING_CONFIG_PATH
|
|
635
688
|
Path to the logging configuration file.
|
|
636
|
-
--headless Run in headless mode: no TUI, no terminal output
|
|
637
|
-
|
|
689
|
+
--headless Run in headless mode: no TUI, no terminal output
|
|
690
|
+
except a single render-started message. All logs are
|
|
691
|
+
written to the log file.
|
|
638
692
|
|
|
639
693
|
configuration:
|
|
640
694
|
--config-name CONFIG_NAME
|
|
641
|
-
Name of the config file to look for. Looked up in the
|
|
642
|
-
|
|
695
|
+
Name of the config file to look for. Looked up in the
|
|
696
|
+
plain file directory and the current working
|
|
697
|
+
directory. Defaults to config.yaml.
|
|
643
698
|
|
|
644
699
|
```
|
|
645
700
|
|
|
@@ -52,7 +52,7 @@ For each `config.yaml` in the inventory, check **all** of the following. Collect
|
|
|
52
52
|
2. **At minimum `unittests-script` is present.** Every project gets a unit-test runner.
|
|
53
53
|
3. **For every script field that is present** (`unittests-script`, `conformance-tests-script`, `prepare-environment-script`):
|
|
54
54
|
- The path is a string ending in `.sh` (macOS/Linux) or `.ps1` (Windows). The extension must match the rest of the project — do not mix `.sh` and `.ps1` in a single config.
|
|
55
|
-
- The referenced file actually exists on disk under `test_scripts
|
|
55
|
+
- The referenced file actually exists on disk, resolved the way the renderer resolves it: absolute / `~` paths as-is, relative paths against the **config file's directory** (typically landing under `test_scripts/`). There is no other fallback — the renderer fails fast with a `FileNotFoundError` on a script path that doesn't resolve, so a path that only "works" from some other directory is a failure here.
|
|
56
56
|
- On Unix, the script has the executable bit set (`-x`). If not, that is a fixable failure.
|
|
57
57
|
4. **No mixed stacks per config.** Every script referenced from a single `config.yaml` must target the same language/stack. For example, `backend/config.yaml` should not reference `run_unittests_js.sh`. If a config crosses stacks, that is a failure — the project should have been split into multiple configs per the rule in `PLAIN_REFERENCE.md`.
|
|
58
58
|
5. **No dangling fields.** Any `*-script` field whose target file does not exist is a failure.
|
|
@@ -65,9 +65,9 @@ Before launching anything:
|
|
|
65
65
|
|
|
66
66
|
Do **not** prompt for flags the user did not signal interest in; the empty command line is the right default.
|
|
67
67
|
|
|
68
|
-
5. **Locate and reset the log file.**
|
|
68
|
+
5. **Locate and reset the log file.** By default it is `codeplain.log` in the same directory as the `.plain` file. If `--log-file-name` was overridden, the override resolves based on **where it was written**: a relative value on the CLI resolves against the invocation directory; a relative value in `config.yaml` resolves against the config file's directory; absolute and `~` paths are used as-is. **The file is overwritten** at the start of each run (per the `--log-file-name` help text). So:
|
|
69
69
|
- Before launch, note the **absolute path** of the log file and its current size (`wc -c <path>`) — this lets you detect the moment the new run truncates and starts writing.
|
|
70
|
-
- After launch, every log read must be against this same path. Re-derive it if the user passes `--log-file-name` or runs from a different working directory.
|
|
70
|
+
- After launch, every log read must be against this same path. Re-derive it (using the resolution rule above) if the user passes `--log-file-name`, sets `log-file-name` in `config.yaml`, or runs from a different working directory.
|
|
71
71
|
- The log is **per-run**: every byte you read during this run was written by this run. You don't have to worry about stale lines from prior runs.
|
|
72
72
|
|
|
73
73
|
6. **Read every test script referenced by the governing `config.yaml`.** This is non-negotiable. The scripts are the contract between the renderer and the project; you cannot judge what the renderer is "fixing" without knowing what `pass` and `fail` actually mean in this project. For each of `unittests-script`, `prepare-environment-script`, and `conformance-tests-script` (those that are declared), read the script end-to-end **before** launching, and write down for yourself:
|
|
@@ -75,7 +75,7 @@ Before launching anything:
|
|
|
75
75
|
- **Which framework is invoked** (e.g. `vitest`, `jest`, `pytest`, `go test`, `cargo test`, `phpunit`, etc.) and any framework flags (`--noEmit`, `--reporter=verbose`, custom configs).
|
|
76
76
|
- **What "pass" means**, exit-code-wise. Some scripts combine multiple checks (e.g. `unit_testing_typescript.sh` here runs `tsc --noEmit` *and* `vitest`, and its final exit code is the worst of the two — so a type error alone fails the unit-test gate).
|
|
77
77
|
- **What "fail" means**, including any failures that the script deliberately downgrades to warnings (e.g. `prepare_environment_typescript.sh` here treats a failed `npm run build` as a non-fatal warning). Failures the script swallows are failures the renderer will **never see**, so a spec defect that depends on them will silently slip through.
|
|
78
|
-
- **Where the scripts materialize their working environment** (e.g. `.tmp/typescript_<build_folder>/`, `build/`, `target/`). This is where the renderer's iteration actually runs; if you need to look at what is breaking, this is the directory — not `plain_modules/<module>/` directly.
|
|
78
|
+
- **Where the scripts materialize their working environment** (current-convention scripts stage into the system temp directory, e.g. `/tmp/typescript_<build_folder_basename>/`; older scripts used project-local folders like `.tmp/typescript_<build_folder>/`, `build/`, `target/`). This is where the renderer's iteration actually runs; if you need to look at what is breaking, this is the directory — not `plain_modules/<module>/` directly.
|
|
79
79
|
- **How conformance tests are discovered and staged** (glob, naming convention, alias setup, etc.). Some scripts stage tests into the source tree (this repo flattens `conformance_tests/<module>/<fname>/` into `src/<module>/`); some run them in place; some require a specific file extension. This tells you exactly which conformance test file is in play for a given functionality.
|
|
80
80
|
- **Toolchain prerequisites** the script asserts (Node version, Python version, specific binaries). If any are missing on the user's machine, stop now and tell them — the renderer will burn credits on phantom failures otherwise.
|
|
81
81
|
|
|
@@ -380,6 +380,15 @@ Renderer surfaces a conflict, or you can see two specs in the same module that c
|
|
|
380
380
|
|
|
381
381
|
Symptom: log shows the unit-test or conformance-test **script** itself failing (non-zero exit, syntax error, missing dependency) on every attempt, rather than the renderer's code failing the tests. The fix is **not** a spec change — it is a `test_scripts/` change. Stop and hand off to the matching `implement-*-testing-script` skill, or have the user repair the script directly.
|
|
382
382
|
|
|
383
|
+
**Special case — stale scripts after a renderer upgrade (doubled paths).** The renderer now passes the build folder and conformance-tests folder to the scripts as **absolute paths**; older test scripts built paths by concatenating those arguments (`.tmp/$1`, `python_$1`, `$current_dir/$2`). The tell is a test-log path that contains the project root **twice**, e.g.:
|
|
384
|
+
|
|
385
|
+
```
|
|
386
|
+
ImportError: Start directory is not importable
|
|
387
|
+
/Users/x/project//Users/x/project/conformance_tests/module/spec
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
The render then gets stuck "fixing" tests that can never pass. The fix is never a spec edit: stop the render and regenerate the affected scripts via the matching `implement-*-testing-script` skill (current convention: system-temp working folder from `basename "$1"`, `$2` resolved to absolute before any `cd`).
|
|
391
|
+
|
|
383
392
|
### Pathology F — Unit-test retry loop
|
|
384
393
|
|
|
385
394
|
Less common than the conformance loop, but the same shape: `Running unit tests script ... (attempt: <N>)` with `<N>` climbing for the same functionality. Threshold: **3 attempts**. From `ATTEMPT_COUNTER >= 2` onwards, run the same [Spec-deviation classification](#spec-deviation-classification) classifier and act on the bucket it returns. The most common root causes for unit-test loops are an implementation req that doesn't match the unit-test scaffolding (Bucket 2), or test reqs / framework usage that the renderer is satisfying by weakening assertions (Bucket 3). Hand-off is typically `add-implementation-requirement` or `debug-specs`.
|