@turri-dev/cli 0.0.0-stage → 0.1.2-next.48dacfc

package/genome/rules/import-modules.md

@@ -0,0 +1,48 @@
+---
+description: Rules for creating and using import modules in .plain files
+---
+
+# Rules for import modules
+
+When creating or editing a `.plain` file that uses `import` or is intended to be imported by other modules, always follow these rules:
+
+## What an import module is
+- An import module contains **only** `***definitions***`, `***implementation reqs***`, and/or `***test reqs***`
+- It must **not** contain `***functional specs***`
+- It must **not** use `requires` in its frontmatter
+- It may optionally `import` other import modules for layered reuse
+
+## What import does
+- `import` pulls in `***definitions***`, `***implementation reqs***`, and `***test reqs***` from the target module
+- It does **not** pull in `***functional specs***`
+- Import paths omit the `.plain` extension
+
+## Concept visibility
+- All concepts defined in the imported module become available in the importing module — including concepts the imported module itself imports (layered imports are transitive)
+- Check for concept name collisions between imports and local definitions before adding — a collision is rejected as a syntax error ("Concepts were defined multiple times")
+
+## Format
+
+```plain
+---
+import:
+  - airplain
+description: Module that imports shared definitions and reqs
+---
+```
+
+A module can import multiple modules:
+
+```plain
+---
+import:
+  - airplain
+  - shared_utils
+---
+```
+
+## Combining import with requires
+- A module can use both `import` and `requires` together
+- `import` brings in shared definitions and reqs
+- `requires` attaches the module to the build chain — accumulated generated code and previous functional specs
+- An import module itself must **not** use `requires`

package/genome/rules/integration-embedded-testing.md

@@ -0,0 +1,276 @@
+---
+description: Rules for authoring the two test scripts that an embedded ***plain integration ships
+---
+
+# Rules for embedded-integration test scripts
+
+When an embedded integration ships its two `test_scripts/` (unit, conformance), these
+rules apply on top of the contracts in the corresponding
+`implement-*-testing-script` skills. `project-model.md` covers the rendering and
+testing workflow.
+
+## Staging model (read this first)
+
+A module's code and unit tests live in `plain_modules/<module>/code`; its conformance tests live in one folder, `plain_modules/<module>/tests` — the module's whole conformance suite. turri runs both scripts on the person's machine, once per check: `bash run_unittests_<lang>.sh <build_folder>` decides whether the unit tests pass, and `bash run_conformance_tests_<lang>.sh <build_folder> <conformance_tests_folder>` (absolute paths) runs the whole conformance suite at once and decides whether the conformance tests pass. There is no prepare-environment script, so each script is self-contained.
+
+Embedded integrations need a runnable host project, so both scripts stage the generated code **inside the host codebase itself**. The conformance script additionally builds / installs the host and then runs the conformance suite from a scratch folder in the system temp directory (`/tmp/<lang>_conformance/`), because the conformance suite lives in a separate project.
+
+| Script | Where it operates | What it does to the host source tree |
+|--------|-------------------|---------------------------------------|
+| `run_unittests_<lang>` | Inside the host codebase | Copies `$1` into the module's package path under the host's source tree, then runs the module's unit tests + lint inside the host |
+| `run_conformance_tests_<lang>` | Inside the host codebase, then `/tmp/<lang>_conformance/` (system temp) | Same copy as above, then cleans and builds / installs the host project; copies `$2` (the conformance-tests folder) into the scratch directory, builds the conformance project, and runs the whole suite there against the host build it just installed |
+
+This deliberately writes into the host's `src/main/...` and `src/test/...` (or the language equivalent). Two consequences flow from that:
+
+- **The host codebase root must come from a configured env var** (`HOST_CODEBASE_ROOT` is the conventional name; declare it in the integration's configuration concept). Scripts never hardcode it
+- **The destructive-op scoping rule below becomes critical.** `rm -rf` must target only the module's own package path under the host's source tree, never the host's `src/main/`, `target/`, `node_modules/`, or `build/` at the project root
+
+This rule covers the **mechanics each script must obey** regardless of language: argument handling, exit codes, idempotency, path resolution, output parsing, and the "what NOT to put here" guard rails. The language-specific install / test commands come from the skills; the contract below is invariant.
+
+## What the `.plain` spec must declare
+
+The two scripts are generated from facts in the integration's spec. For an embedded integration, those facts cannot be inferred — they have to be **declared explicitly** in the right section, partitioned by which predefined concept they describe:
+
+- **Everything about `:UnitTests:`** — paths, approach, packages, framework, conventions, fixtures, mocking policy — lives in `***implementation reqs***`. Unit tests are part of the generated codebase, so requirements that shape them are implementation reqs (see [`impl-reqs.md`](impl-reqs.md))
+- **Everything about `:ConformanceTests:`** — paths, approach, packages, framework, execution command, pass criteria, mocking policy — lives in `***test reqs***`. Conformance tests are external to the generated codebase, so requirements that shape them are test reqs (see [`test-reqs.md`](test-reqs.md))
+
+Authors use `add-implementation-requirement` for the first group and `add-test-requirement` for the second. The two groups are parallel — each predefined concept owns a complete authoring story in its own section.
+
+### In `***implementation reqs***` — everything about `:UnitTests:`
+
+These reqs feed `run_unittests_<lang>`, and the copy step that `run_conformance_tests_<lang>` shares with it. At minimum, declare:
+
+1. **Integration source path inside the host** — where `$1/<source>/*` gets copied to. Example: `src/main/java/com/example/integrations/foo`
+2. **`:UnitTests:` source path inside the host** — where `$1/<tests>/*` gets copied to, and where the test runner discovers unit tests. Example: `src/test/java/com/example/integrations/foo`
+3. **Fully qualified `:UnitTests:` package** — the package the test runner uses to scope discovery via its filter argument. Example: `com.example.integrations.foo`
+4. **`:UnitTests:` framework and conventions** — JUnit / pytest / Jest / Go's `testing` / etc., plus naming conventions (`*Test`, `test_*`, `*.test.ts`, …), fixture / mock / assertion style, file layout inside the test package
+5. **Quality gates that run alongside `:UnitTests:`** — Checkstyle / ESLint / Pylint / Ruff / `go vet` / `cargo clippy` — whatever the host project requires
+
+Author the location facts as one tight group, phrased in terms of `:UnitTests:`:
+
+```plain
+- :UnitTests: of :Implementation: live in `src/test/java/com/example/integrations/foo` inside the host codebase.
+  - The corresponding integration source code lives in `src/main/java/com/example/integrations/foo`.
+  - The fully qualified package used for :UnitTests: discovery is `com.example.integrations.foo`.
+  - :UnitTests: use JUnit 5 with the host's Checkstyle profile applied via `mvn checkstyle:check`.
+```
+
+These facts feed directly into the unit-test script body, and into the staging step of the conformance script:
+
+```bash
+# clean existing code from the host
+rm -rf $MAIN_PROJECT_FOLDER/<integration source path>/*
+rm -rf $MAIN_PROJECT_FOLDER/<:UnitTests: source path>/*
+
+# create destinations and copy generated code into the host
+mkdir -p $MAIN_PROJECT_FOLDER/<integration source path>
+mkdir -p $MAIN_PROJECT_FOLDER/<:UnitTests: source path>
+cp -R $1/<integration source path>/* $MAIN_PROJECT_FOLDER/<integration source path>
+cp -R $1/<:UnitTests: source path>/* $MAIN_PROJECT_FOLDER/<:UnitTests: source path>
+
+# run :UnitTests: scoped to the integration's package
+mvn test -Dtest='<:UnitTests: package>.**.*Test' checkstyle:check
+```
+
+### In `***test reqs***` — everything about `:ConformanceTests:`
+
+These reqs feed `run_conformance_tests_<lang>`. At minimum, declare:
+
+1. **`:ConformanceTests:` source location** — the generated suite lives in one folder, `plain_modules/<module>/tests/`, which holds the module's whole conformance suite; turri passes its absolute path as `$2`
+2. **`:ConformanceTests:` framework and execution command** — `mvn test --no-transfer-progress`, `pytest`, `npm test`, `go test ./...`, etc., with any flags / profiles the project requires
+3. **Fully qualified `:ConformanceTests:` package** (or path / pattern) used to scope discovery, if the runner needs one
+4. **`:ConformanceTests:` network and secrets policy** — by default the suite runs against the **live provider** (see [`integrations.md`](integrations.md) → *`:ConformanceTests:` always run against the live integration*). Declare the env-var names the script reads (e.g. `<PROVIDER>_API_KEY`), whether the script loads a `.env` file before running, and any specific endpoints that are mocked because they can't be exercised live safely (429, forced 5xx)
+5. **`:ConformanceTests:` pass criteria** — the strict criteria from [*Pass criteria (strict)*](#pass-criteria-strict): at least one test ran AND zero failures / errors / skipped. Reaffirm this in the spec so the renderer knows the runner must parse the test tool's stdout
+6. **`:ConformanceTests:` build / install needs** — anything the conformance project needs before `mvn test` (or equivalent) will work: the host build / install command the script runs first, dependencies, fixtures, schema files, generated stubs
+
+Author the conformance facts as one or more entries, phrased in terms of `:ConformanceTests:`:
+
+```plain
+- :ConformanceTests: of :Implementation: live in `plain_modules/foo/tests/` and are implemented with JUnit 5 + Maven.
+  - The fully qualified package used for :ConformanceTests: discovery is `com.example.integrations.foo.conformance`.
+  - :ConformanceTests: are run via `mvn test --no-transfer-progress`; the host's Surefire plugin must be installed.
+  - :ConformanceTests: run against the live :ProviderName: sandbox — no mocking of provider calls.
+  - The conformance script reads `<PROVIDER>_API_KEY` (and any additional secrets) from the shell or from a `.env` file at the project root and fails fast (exit `69`) if any required var is missing after the optional `.env` load.
+  - The 429 (rate-limit) and forced-5xx paths use a local mock for that specific endpoint; every other path is live.
+  - :ConformanceTests: pass only when the Surefire summary line shows at least one test ran with zero failures, zero errors, and zero skipped.
+```
+
+These facts feed directly into the conformance script body:
+
+```bash
+# stage $1 into the host (same copy as run_unittests), then build / install the host
+rm -rf ./target
+mvn clean install -DskipTests
+
+# stage :ConformanceTests: source into the scratch directory
+find "$DIR" -mindepth 1 -exec rm -rf {} +
+cp -R $2/* $DIR
+cd $DIR
+
+# build the conformance project, then run :ConformanceTests:
+mvn clean install -DskipTests
+output=$(mvn test --no-transfer-progress 2>&1)
+
+# parse Surefire summary against the declared pass criteria, then exit accordingly
+```
+
+### Rules common to both sections
+
+- **The paths must agree across reqs.** The `:UnitTests:` source path, the `:UnitTests:` package, and the integration source path describe the same module from three angles. Same for the `:ConformanceTests:` location and its package. A mismatch (e.g. test path `src/test/java/com/example/foo` but test package `com.example.bar`) silently produces a green build with stale or zero tests
+- **Paths are host-relative**, not absolute. `MAIN_PROJECT_FOLDER` comes from `HOST_CODEBASE_ROOT` (declared in the configuration concept); the paths above join onto it
+- **The renderer's output folder `$1` mirrors the same layout.** `$1/<integration source path>/*` exists because the renderer emits the generated code into the same package directories it'll be copied into — the `cp -R` is a straight overlay, not a path translation
+- **Each fact lives in one place.** If two `***implementation reqs***` entries declare slightly different `:UnitTests:` paths (or two `***test reqs***` entries declare slightly different `:ConformanceTests:` packages), the renderer can't tell which to use. Author each group as a tight cluster (one bullet with sub-bullets) so a future reviewer sees them together
+- **Never duplicate a fact across sections.** `:UnitTests:` facts belong **only** in `***implementation reqs***`; `:ConformanceTests:` facts belong **only** in `***test reqs***`. The two groups never overlap — the script generators read each from its own section
+- **For non-Java languages**, the facts have language-specific equivalents — Python: `src/foo/`, `tests/foo/`, `tests.foo`; Node: `src/foo/`, `test/foo/`, `test/foo/**/*.test.ts`. The `implement-*-testing-script` skills know the mapping per language, but the spec still has to declare the host-relative paths and packages so the skill knows what to put in the script bodies
+
+## Common contract (both scripts)
+
+Both scripts are invoked by turri with positional arguments, on the person's machine. They MUST:
+
+1. **Be POSIX-bash (`.sh`) on macOS / Linux, PowerShell (`.ps1`) on Windows.** Executable, idempotent — re-running with the same inputs produces the same result
+2. **Validate every positional argument up front.** Exit with a clear `Usage:` line on bad args. Conventional exit codes (consistent with the shared testing-script rules):
+   - `1` — bad arguments / usage error (and "no tests discovered" for the conformance runner)
+   - `2` — missing or inaccessible input folder
+   - `69` — unrecoverable environment failure (missing toolchain, cannot enter working folder, install failed)
+   - Any other non-zero code — propagated verbatim from the underlying build / test tool
+
+   turri passes `$1` (and `$2` for the conformance runner) as **absolute paths** (older renderer versions passed relative ones). Never build a derived path by concatenating a raw argument (`.tmp/$1`, `<lang>_$1`, `$current_dir/$2`) — with an absolute argument that produces a doubled path (`/project//abs/path/...`). Use the argument as-is where a path is needed, take its `basename` when naming a scratch folder, and resolve `$2` to absolute (if it isn't already) before any `cd`
+3. **Echo the toolchain home at the top** — the first thing a developer checks when CI breaks. For Java: `JAVA_HOME`. For Python: the active interpreter (`python --version` / `which python`). For Node: `node --version`. For Go: `go env GOROOT`. Pick the variable whose value most often explains "why does it work locally but not on the CI runner?"
+4. **Respect `VERBOSE=1`.** Gate chatty diagnostic prints behind `if [ "${VERBOSE:-}" = "1" ]` (and the PowerShell equivalent). Errors and key step markers print unconditionally
+5. **Resolve paths relative to the script, not `$PWD`.** Use `"$(cd "$(dirname "$0")/<relative-path-to-anchor>" && pwd)"`. Hard-coded `../../` chains break the moment the renderer's `cwd` changes
+6. **Create destination directories before copying.** `mkdir -p` before any `cp -R` / `rsync` / `robocopy` — most copy commands do not create intermediate directories and fail silently in some shells
+7. **Scope destructive operations narrowly.** Any `rm -rf` (or `Remove-Item -Recurse -Force`) targets only the module's own package path inside the working folder — never the host's `src/`, `target/`, `node_modules/`, `build/`, or `dist/` at the project root. The conformance script cleans the host's build-output directory (e.g. `./target`) only as part of its own build / install step, and removes its system-temp scratch folder on exit
+8. **Print where you are before you `cd`.** `echo "Moving to: $DIR"` saves an hour of debugging when paths are wrong. PowerShell: `Write-Host "Moving to: $DIR"`
+
+## 1. `run_unittests_<lang>` — copy into the host, then run unit tests there
+
+Receives one positional argument: the renderer's build output folder (`plain_modules/<module>/code/`, absolute).
+
+Purpose: stage the generated code **into the host codebase at the module's package path** and run the module's unit tests + language-appropriate quality gates against the host project from inside the host root.
+
+Required steps:
+
+1. Validate `$1`
+2. Resolve the host codebase root from the configured env var
+3. `rm -rf` the module's main + test package directories under the host's source tree — both, because stale tests in the target package cause confusing failures
+4. `mkdir -p` the destination directories and copy `$1/src/...` (or the language-equivalent layout) into the host's source tree at the module's package path
+5. `cd` into the host codebase root, verify with an explicit exit-code check (`if [ $? -ne 0 ]` / `if (! $?) { ... }`)
+6. Run the test command **scoped to the module's package only**:
+   - Maven: `mvn test -Dtest='<fully.qualified.package>.**.*Test' <lint-check>:check`
+   - pytest: `pytest tests/<module>/` (or whatever path matches the module)
+   - Jest: `npm test -- --testPathPattern '<module>/'`
+   - Go: `go test ./<module>/...`
+7. Always include the host's lint / static-analysis gate in the same invocation when the build system has one (Checkstyle, ESLint, Pylint / Ruff, `go vet`, `cargo clippy`, …). Running tests without the lint gate produces a false-green when style violations would otherwise block the build
+
+Hard rules:
+
+- **Self-contained.** The script stages `$1` itself on every run; it never relies on another script having run first
+- **Never run the full test suite** when scoping is possible. A per-module script that runs the entire host's tests wastes minutes on every render iteration
+
+## 2. `run_conformance_tests_<lang>` — copy into the host, build it, then run the external suite
+
+Receives two positional arguments: the renderer's build output folder (`$1`, `plain_modules/<module>/code/`) and the conformance-tests folder (`$2`, `plain_modules/<module>/tests/` — the module's whole conformance suite), both absolute.
+
+Purpose: stage the generated code into the host, build / install the host, and run the whole conformance suite in `$2` against that build. The conformance suite is a separate project that consumes the integration as a dependency from a local cache (`~/.m2`, `node_modules`, the language equivalent), so the script installs the host itself first — otherwise the suite links against whatever was last built, which may not be the renderer's most recent output. turri runs it once per check, and its exit code decides whether the conformance tests pass.
+
+Required steps:
+
+1. Validate `$1` and `$2`. Resolve `$2` to an absolute path if it isn't one already (newer renderers pass it absolute; older ones passed it relative to the invocation directory)
+2. Resolve the host codebase root from the configured env var, `rm -rf` the module's main + test package directories under the host's source tree, `mkdir -p` the destination directories, and copy `$1/src/...` into the host's source tree at the module's package path — **the same staging as `run_unittests`**
+3. `cd` into the host codebase root, verify with an explicit exit-code check
+4. Clean the host's build-output directory before compiling — stale class / object files from a previous build cause confusing link-time mismatches when the contract has changed. Maven: `rm -rf ./target`. Cargo: `cargo clean -p <pkg>`. Go: `go clean -cache` (usually unnecessary, but mention if applicable). For npm projects with incremental builds disabled, wipe the relevant `dist/` / `build/` folder
+5. Run the host's standard install / build command so the conformance project can resolve the integration — **skip the fat-artifact and skip tests** when the build system supports it, because the tests run afterwards. Exit `69` if it fails:
+   - Maven: `mvn clean install -DskipTests` (plus any host-specific repackage-skip flag)
+   - npm / pnpm / yarn: `npm ci` (no test script; dependency install only)
+   - Python: `pip install -e .` (then any extra integration dependencies)
+   - Go: `go mod tidy && go build ./...`
+   - Cargo: `cargo fetch && cargo build --tests --no-run`
+6. Stage `$2/*` into a scratch directory in the system temp directory (`/tmp/<lang>_conformance/` in Bash, the `GetTempPath()` equivalent in PowerShell). Wipe it first if it exists, and remove it again on exit (`trap 'rm -rf "$DIR"' EXIT` / `finally`). Use a deletion form that handles hidden files and odd shells safely:
+   - Bash: `find "$DIR" -mindepth 1 -exec rm -rf {} +` (safer than `rm -rf "$DIR"/*`)
+   - PowerShell: `Remove-Item -Recurse -Force "$DIR\*"` after confirming the path exists
+7. `cd` into the scratch directory
+8. Compile / set up the conformance project — exit early if this fails:
+   - Maven: `mvn clean install -DskipTests`
+   - npm: `npm ci`
+   - Python: `pip install -r requirements.txt`
+   - Go / Cargo: standard build
+9. Run the whole conformance suite and **capture stdout to a variable**:
+   - Bash: `output=$(mvn test --no-transfer-progress 2>&1)` (or the language equivalent)
+   - PowerShell: `$output = & <cmd> 2>&1 | Out-String`
+10. **Parse the captured output strictly** (see *Pass criteria* below) — don't trust the test tool's exit code alone
+11. On the failure path: print the captured output unconditionally so the renderer (and the user) can see what failed
+12. On the success path: print the output only if `VERBOSE=1`
+
+### Secrets and `.env` handling
+
+`:ConformanceTests:` run against the live provider (per [`integrations.md`](integrations.md) → *`:ConformanceTests:` always run against the live integration*), so the conformance script needs the user's credentials at runtime.
+
+- **Credentials are read from environment variables** named in `***test reqs***` and the auth concept. Never from a literal, never from a file checked into the repo
+- **The script may optionally load a `.env` file** before running the suite. Typical pattern in Bash:
+
+  ```bash
+  ENV_FILE="${ENV_FILE:-$MAIN_PROJECT_FOLDER/.env}"
+  if [ -f "$ENV_FILE" ]; then
+      echo "Loading env from $ENV_FILE"
+      set -a; . "$ENV_FILE"; set +a
+  fi
+  ```
+
+  PowerShell uses the equivalent `Get-Content` + `Set-Item Env:` loop. Absence of `.env` is **not** an error — CI provides the same vars directly through the shell
+- **Verify required env vars exist after the optional `.env` load** and fail fast if any are missing:
+
+  ```bash
+  for var in PROVIDER_API_KEY PROVIDER_ACCOUNT_ID; do
+      if [ -z "${!var:-}" ]; then
+          printf "Error: %s is required for :ConformanceTests:\n" "$var" >&2
+          exit 69
+      fi
+  done
+  ```
+
+- **Export the resolved vars** (already in scope thanks to `set -a`) so child processes started by the test runner inherit them — Maven, pytest, npm, Go all read env vars from their parent process
+- **Never log credential values.** Echo the env-var **name** when reporting "loaded", not its value. Redact in any error path that dumps captured output
+- **Document the secret names twice** — once in the auth concept (for the runtime), once in `***test reqs***` for `:ConformanceTests:` (for the script). They must be the same names; a divergence means the script reads different credentials than the runtime does, and conformance silently tests the wrong account
+
+### Pass criteria (strict)
+
+These are the **only** valid pass criteria. Anything outside this list is a failure:
+
+- At least one test ran AND zero failures AND zero errors AND zero skipped. Maven example: `Tests run: [1-9][0-9]*, Failures: 0, Errors: 0, Skipped: 0`. Each language's test runner has an equivalent summary line — match it with a regex, not by substring
+- Exit with the test tool's exit code on the success path (almost always `0`)
+
+Failure cases — each must be detected explicitly:
+
+- `Tests run: 0` (or the language equivalent) → **failure**. The renderer must never be told "green" when nothing ran. Exit `1` per the shared testing-script rules (the "no tests discovered" exit code)
+- Any non-zero `Failures:`, `Errors:`, or `Skipped:` count → **failure**. Dump the captured output and propagate the test tool's exit code
+- The test tool exited non-zero but the summary line shows `Tests run: 0` → still a `1` (no-tests-discovered), not the tool's own exit code
+
+**Why parse stdout instead of trusting the test tool's exit code.** Every major test runner returns `0` when zero test classes match a filter. Maven Surefire, pytest with no collected tests, `go test ./pkg/that/has/no/_test.go/files`, Jest with an empty pattern — all green-by-default. The renderer's contract is "tests must run AND must pass" — the exit code alone cannot express that.
+
+## Cross-cutting rules
+
+- **No secrets in scripts.** Read credentials from env vars the renderer (or the user's shell) already sets. If a required env var is missing, fail fast with a clear `Error: $FOO is required for ...` and exit `69`. Never hardcode an API key, never read from a checked-in file
+- **Mirror the package / module path exactly.** A typo in the package segment (e.g. one wrong directory in `com/example/foo/...` or `host_pkg/foo/bar/...`) makes `rm -rf` silently do nothing and the subsequent copy put files in the wrong place. Both produce a **green** build with stale code. Derive the package path from the `host-codebase` concept; never retype it inline
+- **Don't add cleverness to the script that belongs in the build.** If you're tempted to `grep` the test output for more than the pass/fail summary, change the test or the test-runner config instead. The script is a thin contract — keep it that way
+- **Don't use shortcut profiles that skip quality gates.** Maven's `-Pfast`, Gradle's `-x check`, npm's `--ignore-scripts`, pytest's `--no-cov`, `-DskipTests` anywhere except the conformance script's two build / install steps (the host install and the conformance project's compile, where the tests run right after) — all defeat the purpose of these scripts. The build's full quality gate (tests + lint + coverage thresholds) must run for unit and conformance
+- **Failure messages are for humans.** `printf "Error: <what failed> at <where>\n"` then exit with the right code. No silent failures, no `|| true` to paper over errors, no `2>/dev/null` on commands whose stderr matters
+- **Use existing scripts as templates.** When you add a new embedded integration, copy from a known-good module, then search-and-replace only the package path and the test-filter argument. Resist the urge to "improve" structure mid-port — pattern-matching across many modules is a feature, not a bug
+
+## Reference implementation (Java + Maven)
+
+A working two-script set for a Java / Maven embedded integration lives under [`examples/integration-embedded-testing/`](examples/integration-embedded-testing/) next to this file:
+
+- [`run_unittests_java.sh`](examples/integration-embedded-testing/run_unittests_java.sh) — stages `$1` into the host and runs `mvn test -Dtest='<module-pkg>.**.*Test' checkstyle:check`
+- [`run_conformance_tests_java.sh`](examples/integration-embedded-testing/run_conformance_tests_java.sh) — stages `$1` into the host the same way, cleans `target/`, runs `mvn clean install -DskipTests` in the host, then copies `$2` into `/tmp/java_conformance/` (system temp, removed again on exit), builds the conformance project, runs the suite, and parses the Surefire summary line per the strict pass criteria above
+
+Use them as a template when adding a new embedded integration in Java / Maven — copy, search-and-replace only the package segment and the `-Dtest` filter, then put them in the project's `test_scripts/`. For other languages (Python, Node.js, Go, Rust, …), the `implement-*-testing-script` skills generate the equivalent two-script set following the same contract.
+
+## Adding a script for a new module
+
+1. Copy the two scripts from a working embedded-integration module into the project's `test_scripts/`
+2. Search-and-replace the package segment everywhere it appears — including the test-filter argument (the most commonly missed spot)
+3. `chmod +x` both (and verify ACLs on Windows for the `.ps1` siblings)
+4. Do not link them from the specs: turri finds them in the project root - `config.turri.yaml` (`unittests-script:`, `conformance-tests-script:`), else the one of each kind in the root, else in `test_scripts/` - and shows them to the agents itself. If `test_scripts/` ends up with two scripts of one kind, name the one to use in `config.turri.yaml`.
+5. Smoke-test by running each script manually with the renderer's output folder as `$1` (and the conformance-tests folder as `$2` for the conformance runner). Confirm exit codes match what the renderer expects
+
+For non-Java languages, the skills [`implement-unit-testing-script`](../skills/implement-unit-testing-script/SKILL.md), and [`implement-conformance-testing-script`](../skills/implement-conformance-testing-script/SKILL.md) adapt the contract above to Python, Node.js, Go, Rust, Flutter, and others. Always route script creation through these skills so the cross-cutting rules above are applied uniformly.

package/genome/rules/integration-embedded.md

@@ -0,0 +1,171 @@
+---
+description: Rules for authoring ***plain specs for REST API integrations embedded into an existing host codebase
+---
+
+# Rules for **embedded** integration specs
+
+When an integration `.plain` module is **embedded** — meaning the generated code in `plain_modules/` is consumed in-process by an existing host codebase as a library / module — these rules apply on top of the shared rules in [`integrations.md`](integrations.md). If anything below contradicts a guess made from memory, the rules here win.
+
+Embedded means: the host codebase already exists, has its own language / framework / dependency manager / packaging layout, and the integration must conform to all of that without negotiation.
+
+> **For test-script authoring**, also follow [`integration-embedded-testing.md`](integration-embedded-testing.md). It defines the per-script contract (`run_unittests_<lang>`, `run_conformance_tests_<lang>`) — staging into the host and the system temp directory, arg validation, exit codes, output parsing, the `***implementation reqs***` and `***test reqs***` entries the spec must declare so the scripts can be generated. This file (`integration-embedded.md`) only summarizes the test-script wiring; the testing rule is the source of truth.
+
+## The host codebase dictates the tech stack (hard rule)
+
+- Language, framework, dependency manager, packaging layout, coding standards, error model, logging library, and architecture are **inherited** from the host — they are **never chosen** by the integration spec
+- Do not re-ask the user about any of these in any phase — they are facts to be discovered from the host's manifest files (`pyproject.toml`, `package.json`, `go.mod`, `Cargo.toml`, `pom.xml`, …) and other integrations.
+- If a `brood` stack-phase question seems to push back on a host rule, treat the host as ground truth and rewrite the question
+- Implementation reqs added in the stack phase are **transcribed** from the host stack verbatim — host language and exact version, host framework + version, dependency manager and manifest path, packaging layout, host conventions the contract must follow, and every host-package version the contract pins
+
+## Project layout — the integration lives under `plain/` at the host root
+
+The embedded integration's `.plain` specs (and the rest of the ***plain project — `template/`, `resources/`, `test_scripts/`, generated `plain_modules/`) live in a single top-level `plain/` folder at the host repository root, alongside the host's own source tree:
+
+```
+<host repo>/
+├── docs/
+├── plain/          # ← the embedded ***plain integration lives here
+│   ├── <module>.plain
+│   ├── template/
+│   ├── resources/
+│   ├── test_scripts/
+│   ├── plain_modules/     # generated; gitignored
+│   │   └── <module>/
+│   │       ├── code/      #   implementation + unit tests
+│   │       └── tests/     #   conformance tests: the module's whole suite
+└── src/            # host source tree (Java, Python, etc.)
+```
+
+This keeps the integration self-contained in one directory the user can `cd` into to render, while leaving the host's own source layout untouched.
+
+- If the host **already** has a `plain/` directory, adopt it verbatim — see step 1 of *Discover before you ask* below
+- If the host does **not** yet have one, create `plain/` at the repo root; do not invent a different name (`specs/`, `plain_specs/`, …)
+- The two test scripts still `cd` into `$HOST_CODEBASE_ROOT` (the host repo root) to compile and run tests against the host project; the `plain/` folder is the integration's **authoring** root, not its **runtime** root
+
+## Discover before you ask
+
+Run host discovery **before** the first interview question. Treat the results as ground truth for everything that follows.
+
+1. **Locate the host's `.plain` setup (if any).** Look for an existing `.plain` file or directory, a `plain_modules/`, a `resources/`, and a `test_scripts/`. If the host already has a `.plain` setup, **adopt it verbatim**: the new integration module lands inside the existing `plain_source_dir` and `requires` the relevant base module (use `create-requires-module`). Do not create a parallel layout.
+2. **Read every existing integration in the host.** For each one, extract: intent and scope, host base class / interface / protocol the integration subclasses or implements, package path and naming convention, configuration pattern (env vars, settings module, secret manager), error / exception hierarchy, logging / metrics / tracing conventions, testing pattern (live vs recorded vs mocked, fixture location, conformance-test layout). The new integration must follow the same patterns unless the user explicitly opts out.
+3. **Capture findings in the `host-codebase` concept.** Cite the existing integration by file path so the reasoning is auditable. The concept holds, as facts:
+   - Host codebase root (absolute or project-relative path)
+   - Host language and exact version (from the manifest file)
+   - Dependency manager and manifest file path (pip + `requirements.txt`, Poetry, uv, npm, pnpm, yarn, Go modules, Cargo, Maven, Gradle, …)
+   - Package / module path inside the host where the integration will be consumed
+   - Fully qualified import path of any host class / interface / struct / protocol the integration must conform to (e.g. `host_project.integrations.base.IntegrationContract`)
+   - Host conventions (custom base classes, Pydantic major version, sync vs. async style, exception hierarchy, dependency-injection seams, logging library)
+   - Target generated-class fully qualified name under `plain_modules/` (e.g. `plain_modules.integrations.<provider>.Client`) and the host base class it should subclass
+4. **Only ask the user what the codebase cannot tell you.** The user's time goes into the third-party API itself (provider, docs, endpoints, edge cases, webhooks) and authentication / credentials. Everything else is a deduction. If a deduction is ambiguous (two existing integrations subclass two different base classes), surface the ambiguity with a single `ask_user` question that quotes both code locations — the question is about resolving a conflict the host already contains, not about asking the user to design the integration.
+
+## Reference host symbols by fully qualified import path
+
+- Every host class, interface, struct, exception, or type alias that appears in a spec must be written with its full dotted / slashed import path (e.g. `host_project.integrations.base.IntegrationContract`, `@host/integrations#Contract`) and tagged in the spec text as **"imported from the host codebase; do not redefine"**
+- The renderer is allowed to redefine **only** symbols the host does not provide and the contract schema does not capture
+- Naming the symbol by FQN is not optional decoration — it tells the renderer where the type comes from, which prevents a duplicate definition under `plain_modules/`
+
+## Link host files at their original path — never copy them into `resources/`
+
+The integration's `.plain` module lives **inside the host codebase** (per the "adopt the host's `.plain` setup verbatim" step). That means host source files are already reachable as linked resources via their host-relative paths. The integration spec references them **in place**; it never duplicates them under `resources/host/`.
+
+- Every host file the integration touches (base classes, configuration modules, registries, exception classes, lifecycle hooks) is referenced from the relevant spec using `***linked resource***` syntax with the **path as it exists in the host codebase** — e.g. `[base.py](host_project/integrations/base.py)` if the `.plain` module sits next to `host_project/`
+- **Do NOT copy host files into `resources/host/`.** A copy creates a second source of truth that drifts the moment the host file is edited; the rendered code will then disagree with whatever the host actually ships
+- **Do NOT add host files via the `add-resource` skill's default copy behavior** when that behavior would duplicate the file — point at the existing host path directly
+- **Never inline a host file's contents** into a spec
+- **Never describe a host symbol's shape from memory** — the renderer reads the linked file's bytes at its host path and that is the source of truth
+- This still obeys the broader [`linked-resources.md`](linked-resources.md) rules: a directory is not a valid link, a URL is not a valid link, a binary is not a valid link. Only the *location* changes — host files live where the host put them, not under `resources/`
+
+### What still belongs under `resources/`
+
+This rule applies to **host source code only**. Other artifacts still live under `resources/` exactly like in a non-embedded project:
+
+- **Contract schemas authored by the integration** — `resources/contract/<entry-point>.schema.json`
+- **Configuration schema** — `resources/config.schema.json`
+- **Static lookup tables** the integration owns — `resources/error-map.yaml`, `resources/retry-policy.yaml`, etc.
+
+The rule of thumb: if the host wrote it and ships it, link it where the host put it. If the integration is authoring it for the first time, it goes under `resources/`.
+
+## The contract spec declares inheritance, not duplication
+
+- The entry-point class / interface / struct in the contract spec must `subclass` / `implements` / `embeds` the host symbol by its full import path
+- The spec describes only the **integration-specific additions and overrides** — never restates the parent's fields or methods
+- The additions and overrides are themselves expressed in the linked schema under `resources/contract/` (JSON Schema or OpenAPI), with `allOf` / `$ref` extending the host's schema rather than duplicating fields
+- If a host base class adds fields the integration shouldn't redeclare, the contract schema's `allOf` chain captures that explicitly
+
+## Renderer directives go in the spec, shapes go in the schema
+
+Each contract spec carries the language-specific glue that the schema can't express:
+
+- Target generated-class fully qualified name (e.g. `plain_modules.integrations.<provider>.Client`)
+- Target file path under `plain_modules/`
+- Host base class import path to subclass / implement
+- Host-package version pins (e.g. `pydantic ~= 2.5`, `fastapi ^0.110`)
+- Framework-specific decorators or metaclasses (`model_config`, `@Depends`, …)
+
+The renderer reads the directives from the spec and the shapes from the linked schema, then emits the host-language class. The spec must **not** also contain a class body or a field list — that creates two sources of truth and they will drift.
+
+## Single source of truth for the host root
+
+- The `host-codebase` concept holds the host root path as a **fact**
+- Test scripts, configuration-loading specs, and any other spec that needs the host location reads it from **that one fact** (via the env var declared in the configuration concept)
+- Never hardcode the host path in any spec, script, or runtime config
+
+## No host-overlapping reqs
+
+- Implementation reqs added in any phase must not contradict the host codebase — same language, same dependency manager, same packaging layout, same error hierarchy, same logging library
+- If two reqs are in tension (one from the host, one newly authored), the host wins; rewrite or drop the newly authored req
+- Do not author a req that re-declares something the host already enforces — that's a maintenance burden with no benefit
+
+## Test-script wiring — copy into the host, run tests there
+
+Embedded integrations are tested **inside the host codebase itself**. A module's code and unit tests live in `plain_modules/<module>/code`; its conformance tests live in one folder, `plain_modules/<module>/tests` — the module's whole conformance suite. turri runs two self-contained scripts on the person's machine, once per check: `run_unittests_<lang>.sh <build_folder>` decides the unit tests, and `run_conformance_tests_<lang>.sh <build_folder> <conformance_tests_folder>` (absolute paths) runs the whole conformance suite at once and its exit code decides the conformance tests. There is no prepare-environment script.
+
+Both scripts copy the renderer's output (`$1`, i.e. `plain_modules/<module>/code/`) into the host's source tree at the module's package path, then compile / test the host project in place. The conformance script also uses a scratch folder in the system temp directory (`/tmp/<lang>_conformance/`), because the conformance suite is a separate project that consumes the host build as a dependency.
+
+This matters because the integration's generated code references host symbols by their full import path (e.g. `from host_project.integrations.base import IntegrationContract`). Those imports only resolve cleanly when the test process is rooted in the host's package layout — anything else creates path edge cases that bite later in conformance failures.
+
+See [`integration-embedded-testing.md`](integration-embedded-testing.md) for the full per-script contract (arg validation, exit codes, idempotency, output parsing, pass criteria, cross-cutting rules). The summary that belongs in *this* file:
+
+- **`run_unittests_<lang>`** copies `$1` into the host's source tree at the module's package path, then runs the module's unit tests + lint scoped to the module's package
+- **`run_conformance_tests_<lang>`** makes the same copy of `$1` into the host, cleans the host's build-output directory, and runs the host's install / build (e.g. `mvn clean install -DskipTests`) so the conformance project resolves this build from the local dependency cache; then it copies `$2` (the conformance-tests folder) into `/tmp/<lang>_conformance/` (system temp), `cd`s in, builds the conformance project, and runs the whole suite
+
+### Invariants the scripts must enforce
+
+- **Host root is a parameter, not a literal.** No script may hardcode an absolute host path. Read the host root from an env var (e.g. `HOST_CODEBASE_ROOT`) with a sensible default matching the user's layout (e.g. `../host_project`). Surface the env var in each script's `--help` / usage banner. Capture this env var in the integration's configuration concept so it has exactly one declared name across specs, scripts, and runtime
+- **Everything about `:UnitTests:` is declared in `***implementation reqs***`** — paths, approach, packages, framework, conventions. Both scripts derive their copy destinations from these reqs, and the unit-test script its test-filter argument. See [`integration-embedded-testing.md`](integration-embedded-testing.md) for the exact reqs the spec must author (phrased in terms of `:UnitTests:`)
+- **Everything about `:ConformanceTests:` is declared in `***test reqs***`** — paths, approach, packages, framework, execution command, pass criteria, mocking policy. The conformance script derives its build and run steps from these reqs. See [`integration-embedded-testing.md`](integration-embedded-testing.md) for the exact reqs (phrased in terms of `:ConformanceTests:`)
+- **The two groups never overlap.** `:UnitTests:` facts belong only in `***implementation reqs***`; `:ConformanceTests:` facts belong only in `***test reqs***`. Neither lives in the `host-codebase` concept
+- **Destructive ops are scoped to the module's own package path** under the host's source tree. `rm -rf` never touches the host's `src/main/`, `target/`, `node_modules/`, `build/`, or `dist/` at the project root. Only the module-specific package directories are wiped
+- **Each script is idempotent.** Re-running the same script with the same `$1` yields the same result
+
+## Embedded-specific completion checklist
+
+Before declaring an embedded integration done, in addition to the shared checklist in [`integrations.md`](integrations.md):
+
+- [ ] `host-codebase` concept records host root path, host language + version, host dependency manager + manifest, target package path, host base class import path, target generated-class FQN, and the host conventions the contract follows
+- [ ] Contract spec carries renderer directives (target language, target file path, target class name, host base class to subclass, host-package pins) and **links** to the contract schema; no class body is inlined
+- [ ] Every host symbol referenced in any spec uses its fully qualified import path and is tagged "imported from the host codebase; do not redefine"
+- [ ] Every host file the integration touches is linked at its **original host-relative path** — no host file has been copied into `resources/host/` or anywhere else, and no host file contents are inlined in any spec
+- [ ] `brood`'s stack-phase decisions are transcribed verbatim from the host (no independent stack choices)
+- [ ] Host-package version pins are copied into `***implementation reqs***`
+- [ ] `run_unittests` copies `$1` into the host's source tree at the module's package path and invokes the host's test runner scoped to the module's package
+- [ ] `run_conformance_tests` makes the same copy of `$1` into the host, cleans the host's build-output directory, runs the host's install / build so the conformance suite resolves the integration from the local dependency cache, then copies `$2` into `/tmp/<lang>_conformance/` (system temp), `cd`s in, builds the conformance project, and runs the whole suite
+- [ ] There is no `prepare_environment_<lang>` script; each test script is self-contained
+- [ ] Host codebase root is read from a named env var (default value documented in each script's usage) — never hardcoded; the env var name is captured in the integration's configuration concept
+- [ ] `***implementation reqs***` declares **everything about `:UnitTests:`** — integration source path, `:UnitTests:` source path, `:UnitTests:` package, framework + conventions, lint / static-analysis gate — per [`integration-embedded-testing.md`](integration-embedded-testing.md)
+- [ ] `***test reqs***` declares **everything about `:ConformanceTests:`** — source location, framework + execution command, package, mocking / network policy, pass criteria, build / install needs — per [`integration-embedded-testing.md`](integration-embedded-testing.md)
+- [ ] Neither group is duplicated across sections: `:UnitTests:` facts never appear in `***test reqs***`, `:ConformanceTests:` facts never appear in `***implementation reqs***`, and neither lives in the `host-codebase` concept
+- [ ] Every `rm -rf` in the scripts is scoped to the module's own package directory under the host's source tree — never targets the host's `src/main/`, `target/`, `node_modules/`, `build/`, or `dist/` at the project root
+
+## Anti-patterns specific to embedded integrations
+
+- **Choosing a different language, framework, or dependency manager than the host.** The host stack is inherited; cross-stack `requires` chains are forbidden by [`requires-modules.md`](requires-modules.md)
+- **Redefining a host class under `plain_modules/`.** Reference the host symbol by FQN; let the renderer import it
+- **Inlining a host base class body into the contract spec.** Reference the host file as a linked resource **at its original host-relative path** — do not inline its contents and do not copy it into `resources/`
+- **Copying host source files into `resources/host/` (or anywhere under `resources/`).** That creates a second source of truth that silently drifts from the host. Link the host file in place; the integration `.plain` module already lives inside the host codebase, so the path resolves naturally
+- **Hardcoding the host codebase path in any spec or script.** Read it from the env var declared in the configuration concept
+- **Asking the user to design the integration's tech stack.** Read it from the host's manifest files
+- **Authoring an integration spec that contradicts an existing integration in the same host** without first surfacing the conflict and getting explicit user confirmation
+- **Wiring tests with `PYTHONPATH` / `NODE_PATH` / Go `replace` directives instead of physically copying `$1` into the host's package tree.** The import-stitching approach is forbidden — every test script starts by copying the generated module into the host's source tree at the module's package path
+- **Letting an `rm -rf` in a test script reach the host's `src/main/`, `target/`, `node_modules/`, `build/`, or `dist/` at the project root.** Destructive ops are scoped strictly to the module's own package directory. A wrong package path in the script silently does nothing and copies files in the wrong place — both produce a green build with stale code
+- **Running conformance tests against a stale local dependency cache.** The conformance script must stage `$1` and build / install the host itself before it runs the suite, so the conformance project resolves this build from `~/.m2` (or the language equivalent). Without that step the suite tests whichever build last hit the cache — the wrong code