@turri-dev/cli 0.0.0-stage → 0.1.1

package/genome/rules/integration-standalone.md

@@ -0,0 +1,135 @@
+---
+description: Rules for authoring ***plain specs for REST API integrations deployed standalone
+---
+
+# Rules for **standalone** integration specs
+
+When an integration `.plain` module is **standalone** — meaning the generated code in `plain_modules/` is the deployable artifact itself (a service, daemon, CLI, scheduled job, container, or library published to a registry) — these rules apply on top of the shared rules in [`integrations.md`](integrations.md). If anything below contradicts a guess made from memory, the rules here win.
+
+Standalone means: the integration is a complete application in its own right, picks its own tech stack, owns its own lifecycle, and exposes its own contract surface to external consumers.
+
+## The integration owns its tech stack
+
+Unlike embedded integrations, a standalone integration has no host to inherit from. Every stack decision is **explicit**, captured during `brood`'s stack phase, and recorded in `***implementation reqs***`:
+
+- **Implementation language** and its exact version
+- **HTTP client library** (e.g. `httpx`, `requests`, `node-fetch`, `axios`, `go-resty`, `reqwest`) — pinned, with the auth/retry/timeout features the integration uses called out
+- **Application framework** (Flask, FastAPI, Express, Fiber, Gin, Spring Boot, …) when the integration exposes HTTP; **CLI framework** (Click, argparse, `cobra`, `clap`) when it exposes a CLI
+- **Data storage** (when needed): idempotency-key persistence, webhook deduplication store, OAuth token cache, response cache — pick the storage primitive (SQLite, Postgres, Redis, on-disk file, in-memory with LRU bounds) and pin it
+- **Architecture and layering** — which patterns the codebase follows (clean architecture, hexagonal, simple flat package, …)
+- **Packaging artifact** — published Python wheel / sdist, npm package, Go binary, container image, OS package, …; one or more, named explicitly
+
+Standalone integrations may freely import any third-party library that isn't constrained by an existing host. Choose the smallest set that solves the problem.
+
+## The contract schema is the public artifact
+
+In an embedded integration, the schema feeds the renderer's code generation; the host codebase consumes the *generated class*. In a standalone integration, the schema is **shipped** to external consumers and the generated implementation is internal to `plain_modules/`.
+
+- **Pick the publication surface and write it into the spec.** Examples (one functional spec per surface):
+  - HTTP service: "served at `/openapi.json` (and a Swagger UI at `/docs`)"
+  - npm-published library: "published as `<package>` to npm; `package.json` exports the schema as `./schema.json`"
+  - PyPI-published library: "published as `<package>` to PyPI; bundle ships `schema.json` as a package data file"
+  - CLI: "emitted as `--help` and as `<cli> schema` (JSON-on-stdout)"
+  - Queue worker: "published as a JSON Schema in the team's schema registry under `<topic>.v<version>.json`"
+  - Scheduled job: "published as JSON Schema in `resources/contract/job-config.schema.json` and linked from the runbook"
+- **Schema versioning is part of the contract.** Pin the version in the schema's `$id` / OpenAPI `info.version` and follow semver. Backwards-incompatible changes ship under a new file path, never as mutations to the published version
+- **The renderer ships the schema verbatim** — the spec must not restate fields from the schema, and the generated implementation must consume the same file as the published artifact (so the two cannot drift)
+
+## All lifecycle stages must be explicit
+
+A standalone integration has no host lifecycle to delegate to. Every stage that an embedded integration could skip is mandatory here, each one captured as its own functional spec:
+
+- **Initialization** — one-time setup at startup. Read configuration, validate credentials proactively (eager) or defer to first call (lazy) — pick one and write the spec accordingly. Open connection pools, register webhook handlers, initialize observability
+- **Credential refresh** — proactive (background timer / pre-expiry refresh) vs. reactive (refresh-on-401). Use a refresh lock to prevent thundering herd. Specify the lock primitive (in-process mutex, distributed lock in Redis, …) when the integration runs as more than one replica
+- **Graceful shutdown** — drain in-flight requests with a deadline; what happens to retries still queued (abandoned, persisted, returned to caller as a specific error); what signals trigger shutdown (`SIGTERM`, container-stop hook, …)
+- **Health checks** — define what "healthy" means for **this** integration: last successful provider call within a window, valid credentials, provider's own `/health` returning 2xx, dependent storage reachable. Spec the response shape (status code, body) for the health endpoint when one is exposed
+- **Background loops** — cron jobs, polling workers, queue consumers: each gets its own functional spec covering trigger, interval, leader-election (if applicable), failure handling, and observability
+
+## Entry-point types — pick deliberately, enumerate fully
+
+A standalone integration exposes one or more **entry points** to its consumers. Mixing entry-point types (e.g. HTTP + queue worker + CLI) is allowed, but every entry point must be enumerated up front and gets at least one functional spec.
+
+| Entry-point type | Public surface | Contract format |
+|------------------|----------------|-----------------|
+| HTTP service | One or more routes (path + method) | OpenAPI 3.1 (`resources/contract/<integration>.openapi.yaml`) |
+| CLI | Command + subcommands + flags | JSON Schema for config + `--help` text seeded from the schema |
+| Queue worker | Topic / partition + message envelope | JSON Schema per message type (`resources/contract/<topic>.schema.json`) |
+| Scheduled job | Job name + cron expression + config schema | JSON Schema for config |
+| Library | Public package surface (functions, classes) | JSON Schema for I/O + published-package manifest |
+| Webhook receiver | Provider-driven inbound HTTP | JSON Schema per event type under `resources/webhooks/` (shared with embedded — see [`integrations.md`](integrations.md)) |
+
+Each entry point gets its own functional spec describing what calling it does end-to-end (which earlier functional specs it composes — auth, the endpoint call, the retry policy, the error handling).
+
+## Side effects are first-class — spell them out
+
+A standalone integration usually mutates state outside the provider call itself. Each side effect is captured as a functional spec describing:
+
+- **What** gets mutated — local DB writes, file writes, cache updates, emitted domain events, metrics, log lines
+- **When** the mutation happens — before or after the provider call
+- **Is the mutation transactional with the provider call** — atomic, eventually consistent, fire-and-forget
+- **What happens on partial failure** — provider call succeeded but the side effect failed (and vice versa); rollback / compensation / retry strategy
+
+## Concurrency and backpressure
+
+Standalone integrations face concurrent load that an embedded library doesn't. The spec must pin:
+
+- **Sync vs. async** in the host-language sense (`async def`, `Promise`, goroutines, threads)
+- **Expected peak throughput** (RPS) the consumer will issue
+- **Connection pool size**
+- **Request queue depth**
+- **Backpressure behavior** when the provider's rate limit and the consumer's demand collide — drop, queue, block with timeout, or 429 the consumer
+- **Concurrency limits on mutating operations** — when the provider has its own per-resource serialization constraints
+
+Capture all of this as a concurrency concept and one functional spec for "apply backpressure when local demand exceeds rate-limit budget".
+
+## Configuration surface
+
+- Every config knob the consumer can set lives in `resources/config.schema.json` (JSON Schema): env var names, config file keys, secrets, feature flags, regional selectors, timeouts, retry counts, base URL overrides
+- The configuration concept enumerates every key by name (type, default, required vs optional, validation, where it is read — startup vs per-call)
+- Secrets are referenced by env var name only; never written into the schema as default values
+- One functional spec covers "load and validate configuration on startup" — fail-fast on missing required values
+
+## Versioning of the integration itself
+
+Separate from the provider's API version (which lives in the provider OpenAPI file), the **integration's own contract** is versioned independently:
+
+- Semver of the published library / OpenAPI `info.version` / schema `$id` URL / CLI `--version`
+- Backwards-compatibility policy stated explicitly (e.g. "minor and patch are wire-compatible; breaking changes only on major")
+- New major versions ship at a new schema path; never mutate a published version in place
+
+Capture as a contract-version concept; pin the version in every published schema.
+
+## Testing — live conformance, secrets from env, webhooks
+
+`:ConformanceTests:` for a standalone integration **run against the live provider** (see [`integrations.md`](integrations.md) → *`:ConformanceTests:` always run against the live integration*). The testing strategy is captured as `***test reqs***` entries authored via `add-test-requirement`:
+
+- **Conformance is live by default.** No VCR cassettes, no prerecorded responses for the calls under test, no mock servers. A green conformance run that never touched the provider proves nothing. Recorded responses may back unit tests — never conformance
+- **Secrets come from environment variables.** Pin every credential as an env-var name (e.g. `<PROVIDER>_API_KEY`, `<PROVIDER>_CLIENT_ID` + `<PROVIDER>_CLIENT_SECRET`) in `***test reqs***` and in the auth concept. Use the names the provider's docs use so users don't have to translate
+- **The user supplies values via `.env` or the shell.** The project ships `.env.example` (gitignored `.env` for real values). CI provides the same env-var names from its secret store. The conformance script may optionally `source` a `.env` from the project root if one exists; it must verify every required var is set after that optional load and fail fast (exit `69`) on missing vars
+- **Sandbox credentials in CI.** Name where credentials come from (CI secret store, dedicated test tenant), the rotation / leak-response policy, and the env-var names CI must set
+- **Webhook tests** (if webhooks are in scope) must cover signature verification end-to-end — including invalid signatures and replay attempts. Signing keys are env vars, like every other secret
+- **Rate-limit (429) tests.** The 429 path must **not** exhaust the live API's quota — use a local mock for that specific endpoint, document the exception in `***test reqs***`. Every other path remains live
+- **Idempotency tests.** Run the same mutating call twice (with the same idempotency key) against the live sandbox and assert the same response
+
+## Standalone-specific completion checklist
+
+Before declaring a standalone integration done, in addition to the shared checklist in [`integrations.md`](integrations.md):
+
+- [ ] Tech stack decisions are recorded in `***implementation reqs***` — language and version, HTTP client library and version, framework (HTTP / CLI / worker as applicable), data storage choice(s), architecture/layering
+- [ ] Publication surface(s) for the contract schema are enumerated in functional specs (one per surface)
+- [ ] Schema versioning strategy is captured — `$id` / `info.version` / package version pinned, backwards-compatibility policy stated
+- [ ] Every entry point is enumerated and has at least one functional spec (HTTP routes, CLI commands, queue topics, scheduled jobs, library APIs)
+- [ ] Every lifecycle stage has a functional spec — initialization, credential refresh, graceful shutdown, health check
+- [ ] Side effects are each captured as a functional spec with ordering, transactionality, and partial-failure handling
+- [ ] Concurrency model is captured (sync vs async, pool size, queue depth, backpressure strategy)
+- [ ] Configuration surface lives in `resources/config.schema.json` and is linked from the configuration concept
+- [ ] Testing strategy is recorded in `***test reqs***` — live vs recorded, sandbox credential source, webhook signature coverage, rate-limit-test isolation
+
+## Anti-patterns specific to standalone integrations
+
+- **Skipping any lifecycle stage** because it "feels small". Standalone integrations have no host to fall back on; an implicit shutdown is an undefined shutdown
+- **Inlining fields from `resources/contract/<entry-point>.schema.json` into a functional spec.** The schema is the source of truth and is **published** — duplicating fields in spec text guarantees drift
+- **Mutating a published schema's `$id` / version path** instead of shipping a new file. Consumers may already be pinning the old version
+- **Treating configuration as "whatever env vars seem useful at runtime".** Every key lives in `resources/config.schema.json` with type, default, and validation; the loader fails fast on missing required values
+- **Picking a tech stack incrementally while writing functional specs.** Decide it once in `brood`'s stack phase and transcribe into `***implementation reqs***`; don't accrete a stack across several specs
+- **Hitting the live provider in rate-limit (429) or destructive-error (500) tests.** Use a local mock; live calls are for the happy path and for sandbox-safe error paths

package/genome/rules/integrations.md

@@ -0,0 +1,137 @@
+---
+description: Rules for authoring ***plain specs for REST API integrations
+---
+
+# Rules for writing integration specs in `***plain`
+
+When writing or editing a `.plain` file that describes an **integration** against a REST API (synchronous JSON/HTTP request-response plus webhook callbacks), always follow these rules. Non-REST integrations (gRPC, GraphQL, SOAP, message brokers, raw TCP, file drops) are out of scope of these rules.
+
+## Scope of "integration" specs
+- An integration `.plain` module describes how the project talks to a **third-party or internal REST API**
+- The integration may be **embedded** (lives as a library/module inside an existing host codebase) — see [`integration-embedded.md`](integration-embedded.md) for the additional rules that apply
+- Or **standalone** (a service, daemon, CLI, scheduled job, or container) — see [`integration-standalone.md`](integration-standalone.md) for the additional rules that apply
+- The contract surface, edge-case coverage, live-API cross-check, and `resources/` layout described below apply to **both** shapes — the shape-specific rule files add to them, never replace them
+
+## Contract artifacts live in `resources/` (hard rule)
+
+**Every structural contract the integration deals with — endpoint definitions, request/response schemas, error envelopes, webhook payloads, pagination envelopes, rate-limit headers, the integration's own I/O contract — lives in `resources/` as a linked resource.** Concepts and functional specs **reference** these files; they never restate fields, types, status codes, or header names inline.
+
+Pick the right format per artifact:
+
+| Artifact | Format | Conventional path |
+|----------|--------|-------------------|
+| Provider's REST surface | OpenAPI 3.1 (YAML or JSON) | `resources/<provider>.openapi.yaml` |
+| Webhook payload (per event type) | JSON Schema Draft 2020-12 | `resources/webhooks/<event>.schema.json` |
+| Rich webhook contracts (multi-channel) | AsyncAPI 2.6+ | `resources/webhooks.asyncapi.yaml` |
+| Integration's own I/O contract | JSON Schema or OpenAPI | `resources/contract/<entry-point>.schema.json` |
+| Configuration surface | JSON Schema | `resources/config.schema.json` |
+| Error code → category mapping | YAML enum | `resources/error-map.yaml` |
+| Rate-limit header inventory | YAML enum | `resources/rate-limit-headers.yaml` |
+| Retry policy parameters | YAML enum | `resources/retry-policy.yaml` |
+| Provider behavior with no machine-readable home | Markdown | `resources/<provider>-notes.md` |
+
+Rules that flow from this:
+
+- **Provider behavior is recorded, not specified.** How the provider itself works — its quota accounting, consistency guarantees, documented quirks, and any place the live API diverges from its docs — is true and worth capturing, but it is not a requirement on the software being built. Whatever has a machine-readable home in the table above goes there; the rest goes in `resources/<provider>-notes.md`. A functional spec never states provider behavior; it states what the integration does in the face of it.
+- **Concepts carry references, not data.** An endpoint concept names the endpoint and points at its OpenAPI `paths.<path>.<method>` entry; it does not duplicate the request/response shape in concept attributes. Same for webhook concepts, error-model concepts, pagination concepts, etc.
+- **Functional specs consume linked resources.** A spec describes **behavior** ("call endpoint X, parse the response, classify errors, retry on 5xx") and links to the resource that supplies the **shape**. Field names, types, and validation rules live in the resource file.
+- **Schemas are versioned by file path.** When the provider releases a new API version, copy the new OpenAPI file to a new path (e.g. `resources/<provider>.v2.openapi.yaml`); never mutate the v1 file in place.
+- **The renderer generates language-native types from the resources.** For embedded integrations the renderer reads JSON Schema / OpenAPI components and emits Pydantic / TypeScript / Go types in `plain_modules/`. The spec declares **which** schema to generate from, **where** the generated type should land, and **what host base class** it must subclass — but never restates the schema's fields.
+
+A single `.plain` module can (and typically will) reference many resources. That is the intended pattern; do not try to collapse them into one mega-file.
+
+## Live API must be cross-checked against the documentation
+
+Documentation lies — it goes stale, omits undocumented fields, describes a different API version, papers over breaking changes. Every integration spec must be grounded in what the API really returns, not what the docs claim it returns.
+
+- **Discover the documentation links with web search first, then `fetch` all of them.** Documentation moves, gets reorganized, and is versioned — so never assume a doc URL from memory. Begin every topic by issuing **web searches with human-readable queries** to find the canonical pages, then `fetch` **every** relevant link the search surfaces — the endpoint reference, the auth guide, the webhook catalog, the error reference, the pagination/rate-limit docs, the changelog. The search step finds *which* URLs are authoritative and current; the `fetch` step retrieves their actual content. Do not stop at the first hit — fetch the full set so a topic is grounded in all of its sources, not a single page.
+  - If the environment has **no web-search tool** (only URL-based `fetch` is available), say so explicitly, construct the URL from the provider's well-known documentation root and crawl outward by `fetch`-ing that root and following its links, and ask the user for the canonical URL whenever you cannot reach the right page that way. Never substitute memory for the search-then-fetch step.
+- **Always `fetch` the provider's documentation — even if you already "know" the API.** The only acceptable source of truth for what the API looks like *today* is the provider's own live documentation, retrieved with `fetch` at spec-authoring time. This applies without exception — there is no API well-known enough to skip this step, and a spec authored from memory is a spec authored against the wrong contract. Concretely:
+  - Before authoring **any** endpoint, auth, error, pagination, or webhook concept, **web-search to locate the relevant documentation page(s), then `fetch` each one** and quote concrete details (status codes, field names, header names, error formats) directly from the fetched content into the resources under `resources/`. Never paraphrase from memory.
+  - If a documentation page is unreachable (paywall, login wall, JS-only render that `fetch` can't see), say so explicitly and ask the user for the canonical content rather than filling the gap from memory.
+  - **The fetched documentation is then cross-checked against the live API** — see the rest of this section.
+- **Validate credentials against the live API** before authoring downstream specs. A 2xx on a low-risk read-only endpoint (`/v1/me`, `/account`, `/whoami`, `/health`) is the gate. On 401/403, stop and resolve before continuing. The credentials should be validated against the live API before any downstream specs are authored.
+- **Issue the minimum cross-check coverage** with `fetch`: one discovery / schema endpoint if available, one list endpoint per primary entity in scope, one single-object retrieval per primary entity, one empty/boundary response, one 404, one 400/422, and one deliberate 401.
+- **Where the docs and the live API disagree, the live API wins** — the resource records what the API actually returns, and nothing records the disagreement itself.
+- **Only `GET` / `HEAD` / `OPTIONS` on the cross-check.** Mutating calls (`POST`, `PATCH`, `PUT`, `DELETE`) require explicit per-call user confirmation and must target a sandbox account.
+- **Credentials are never written to `.plain` files or summaries.** Reference them by env-var name only.
+
+## `:ConformanceTests:` always run against the live integration (hard rule)
+
+Integrations exist to talk to a real third-party (or internal) API. Their `:ConformanceTests:` therefore **always run against the live integration** — no VCR cassettes, no recorded fixtures, no `nock` / `WireMock` / `MSW` mocks for the calls under test. A "green" conformance run that never touched the provider proves nothing about the integration.
+
+- The integration's `:ConformanceTests:` **must** make real network calls to the provider (typically a sandbox / staging environment, occasionally production for read-only paths). Recorded responses may back unit tests, but they are **not** a substitute for live conformance
+- The only exceptions are paths that **cannot** be exercised live safely:
+  - **Rate-limit (429) tests** — must not exhaust the live quota; use a local mock for that specific endpoint
+  - **Deliberately destructive failure modes** (forced 5xx) the provider doesn't let you trigger — same; mock the specific endpoint
+  Document each exception explicitly in `***test reqs***`; everything else is live by default
+
+### Secrets come from the environment
+
+Live conformance needs credentials. The integration spec must pin every credential as **an env var name, never a literal value**, and the conformance script must read those env vars at runtime:
+
+- **Author the env-var names in `***test reqs***`** (using `:ConformanceTests:`) and again in the auth concept. Examples: `STRIPE_API_KEY`, `GITHUB_TOKEN`, `SALESFORCE_CLIENT_ID` + `SALESFORCE_CLIENT_SECRET`. Use names that match what the provider's own docs use, so a user copying values from the provider console doesn't need to translate
+- **The user supplies the values out-of-band**, either:
+  - in a `.env` file at the project root (`.env` is gitignored; the project ships `.env.example` with the names but no values), or
+  - exported in the shell that invokes the test scripts (CI uses the same names from its secret store)
+- **`run_conformance_tests_<lang>` reads the env vars at runtime.** If a required var is missing, the script must fail fast with `Error: <NAME> is required for :ConformanceTests:` and exit `69` (per the testing-script exit-code conventions). Never default to a placeholder, never use a value baked into the script
+- **The script may optionally load a `.env` file** before running the suite — typical pattern is to look for `.env` in the project root and `source` / `dotenv -e` it if present, but never fail when it's absent (since CI provides the vars directly via the shell). If a `.env` loader is used, the script must verify each required var is set **after** loading, not before
+- **The conformance suite reads the same env-var names** the integration's runtime reads — so a credential that works at runtime is the same credential that exercises the suite
+- **Credentials never appear in `.plain` files, commits, summaries, logs, or fixtures.** The cross-check (see *Live API must be cross-checked*) already requires redacting credentials from saved fixtures; the same rule applies to conformance logs
+
+The `.plain` spec should make this discoverable. A minimal `***test reqs***` block for an integration looks like:
+
+```plain
+- :ConformanceTests: run against the live :ProviderName: sandbox — no mocking of provider calls.
+  - Credentials are read from the environment, never from a file checked into the repo.
+  - The conformance script reads `<PROVIDER>_API_KEY` (and any additional secrets) from the shell or from a `.env` file at the project root.
+  - The script must verify every required env var is set after the optional `.env` load and fail fast with a clear error if any is missing.
+  - The 429 (rate-limit) and forced-5xx paths use a local mock for that specific endpoint; every other path is live.
+```
+
+## Embedded vs standalone — pick the shape early
+
+Every integration is either **embedded** (lives as a library/module inside an existing host codebase) or **standalone** (a service, daemon, CLI, scheduled job, or container). The choice is captured as a concept (`integration-shape: embedded | standalone`) so later specs can reference it.
+
+The contract artifact itself is **identical** across both shapes: a JSON Schema (or OpenAPI) file under `resources/contract/`. What changes is **what the renderer emits from it** and **what extra context the spec carries**:
+
+- **Embedded** → the renderer generates a host-language class from the schema and wires it into the host's import path. The host codebase dictates the tech stack — see [`integration-embedded.md`](integration-embedded.md) for the full ruleset
+- **Standalone** → the renderer treats the schema as the public artifact: it generates an internal implementation in `plain_modules/` and ships the schema verbatim for external consumers. The integration owns its stack — see [`integration-standalone.md`](integration-standalone.md) for the full ruleset
+
+## Edge-case coverage is a hard floor, not a stretch goal
+
+A production-ready integration spec captures every corner case the API can throw at the integration. Each of the following must be in the specs (or explicitly recorded as "not applicable" / "not in scope" with the user's acknowledgement) before the integration is considered complete:
+
+- **Provider, purpose, canonical documentation URL(s)** as concepts
+- **Endpoints in scope** in `resources/<provider>.openapi.yaml`, one `paths.<path>.<method>` entry per endpoint; endpoint concepts link to the OpenAPI entry
+- **Auth scheme**, credential source pinned by env-var name, refresh policy, scopes — with `components.securitySchemes` in the OpenAPI matching
+- **Environments** (sandbox / staging / production) and the switch mechanism — reflected in `servers` of the OpenAPI file
+- **API version pinning** strategy (URL path, header, `Accept`, query string) and the deprecation policy
+- **Request serialization** (content type, date format, numeric precision, custom headers) in `components.schemas` / `components.parameters`
+- **Pagination model** as `components.schemas.PageEnvelope` plus `resources/pagination.yaml` (style, defaults, safety cap)
+- **Rate-limit model** as `resources/rate-limit-headers.yaml` + `resources/rate-limits.yaml` + `components.schemas.RateLimitError`
+- **Error model** as `components.schemas.ErrorEnvelope` + `resources/error-map.yaml`, with one functional spec per error category
+- **Retry policy** in `resources/retry-policy.yaml`
+- **Transient vs. permanent exception classification** — every error the integration can raise is classified as either *transient* (retryable per the retry policy above) or *permanent* (surfaced to the caller immediately). The classification lives in `resources/error-map.yaml` (per-status-code) and in the integration's own exception hierarchy concept, and the two must agree. Concretely:
+  - **Transient by default**: network-level errors (`ConnectionError` / equivalent, DNS failure, socket timeout, TLS handshake failure, connection reset), HTTP 408, 425, 429, 500, 502, 503, 504, and any provider-specific application-level error codes the docs / live API mark as retryable (e.g. Stripe's `lock_timeout`, AWS `ThrottlingException`)
+  - **Permanent by default**: every other 4xx (400, 401, 403, 404, 405, 409, 410, 415, 422, …), schema-validation failures on otherwise-2xx responses, signature-verification failures on webhooks, and any provider error code the docs / live API mark as non-retryable
+  - **Integration exception hierarchy**: the integration exposes a base exception class (e.g. `<Provider>Error`) with two direct subclasses, `<Provider>TransientError` (raised after the retry policy gives up on a transient condition) and `<Provider>PermanentError` (raised immediately on a permanent condition). Every concrete exception inherits from exactly one of those two — callers branch on the base class, never on individual subclasses
+  - **The classification is data-driven**, not code-driven — `resources/error-map.yaml` is the source of truth, the renderer reads it to generate the exception hierarchy and the `is_transient(error) -> bool` predicate. Spec text never enumerates status codes inline
+  - **Discrepancies found during the live-API cross-check override the docs** — if the docs say a code is transient but the live API never recovers from it (or vice versa), the live behavior wins and the divergence is recorded in `error-map.yaml` plus a note in the error-model concept ("docs claim X retryable, live API returns Y permanently; we follow the live API")
+- **Idempotency strategy** in `resources/idempotency.yaml` + idempotency header in `components.parameters`
+- **Webhook contracts** as `resources/webhooks/<event>.schema.json` per event type + `resources/webhook-signing.yaml`
+- **Data mapping** (entity schemas + transformations / exclusions) as entity schemas + `resources/data-mapping.yaml`
+- **Compliance / data-sensitivity** constraints (PII, PHI, payment data, data residency, log redaction, audit logs)
+- **Observability** (log fields, provider request IDs, metrics, tracing propagation)
+
+## Anti-patterns (do not do these)
+
+- **Restating an OpenAPI / JSON Schema field list in a concept or functional spec.** The schema lives in `resources/`; the spec links to it
+- **Pasting a webhook payload, error envelope, or list-endpoint response inline.** Describe its shape in the applicable schema under `resources/` and link that schema
+- **Inlining a host base class body into the contract spec.** Add the host file as a linked resource under `resources/host/` and reference it by FQN
+- **Embedding credentials, tokens, or signing keys in a `.plain` file or in a summary** — credentials are referenced by env-var name only
+- **Authoring against unverified credentials.** Validate first; if the user has no credentials yet, flag it in the module's frontmatter description and re-validate once credentials arrive
+- **`requires`-ing a separate-stack module** (a Python backend `requires`-ing a React frontend, or vice versa) — see [`requires-modules.md`](requires-modules.md). Use a shared API schema in `resources/` instead
+- **Authoring the integration's functional specs from the docs first and "reconciling" with the live API later.** Probe the API as you reach each topic; the live response is the source of truth from the moment it's captured
+- **Writing any integration spec from memory of the provider's API instead of web-searching for its documentation and `fetch`-ing every relevant page first.** No matter how well-known the API (Stripe, GitHub, Slack, Salesforce, AWS, OpenAI, …), the canonical pages must be located with web search and then retrieved with `fetch` at spec-authoring time — see *Live API must be cross-checked against the documentation*. Authoring from memory bakes in whatever version of the API was current during training, which is always older than the version the integration will actually call
+- **Guessing a documentation URL from memory and `fetch`-ing it without searching first.** A remembered URL may 404, redirect to a stale version, or miss the page that actually documents the topic. Search to find the authoritative, current links, then fetch the full set

package/genome/rules/linked-resources.md

@@ -0,0 +1,69 @@
+---
+description: Rules for linking external resources from .plain specs
+---
+
+# Rules for linked resources in `.plain` files
+
+Specifications can reference external files using markdown link syntax. The renderer reads the linked file's bytes verbatim and feeds them to the model alongside the spec. That mechanism only works for a specific shape of target — violating any of the rules below is one of the most common and disruptive mistakes in `.plain` authoring.
+
+## Hard constraint: a linked resource is always a single file on disk - text, or a PNG or JPEG image
+
+A linked resource **must not** be any of the following:
+
+1. **A folder / directory.** `[integrations](src/integrations/)`, `[host project](../host_project/)` are invalid — the renderer cannot ingest a directory. Pick the single most representative file inside (a `README.md`, an exemplar source file, a manifest) and link **that**.
+2. **A URL / external location.** `[Stripe docs](https://stripe.com/docs/api)`, any `http://` / `https://` / `ftp://` / `git://` / `s3://` / `gs://` target. Linked resources are local-file only. If a URL's content is essential, fetch it once, save the response to a text file under `resources/`, and link the saved file.
+3. **A binary file other than a PNG or JPEG image.** GIF, WEBP, SVG, PDF, DOCX, XLSX, ZIP, MP3, MP4, compiled binaries (`.exe`, `.so`, `.class`, `.wasm`), and anything else that isn't human-readable text in its raw form. Binary content cannot be meaningfully consumed by the renderer — transcribe it into a text-based form first (a PDF spec becomes a Markdown extract or the underlying JSON Schema / OpenAPI, an architecture diagram becomes a Mermaid block).
+   - **PNG and JPEG images (`.png`, `.jpg`, `.jpeg`) are the exception**: a UI mockup, a screenshot of the look to match, a diagram. The coding agents are shown the image itself, so the coding model must be one that reads images - the render stops, saying so, when it is not. Each image is at most 5 MB.
+   - An image says what something looks like, not what it does: the behaviour still belongs in the spec's text. Link a mockup from a concept (`:CheckoutPage:` looks as [checkout mockup](resources/checkout.png) shows) and keep the rules - what each button does, which fields are required - in the functional specs.
+   - Where the model may not read images, or a detail must be exact (a colour code, a spacing, a label), say it in text as well; the image is the reference, not the only source.
+4. **Another `.plain` file.** `[Auth Module](auth.plain)` is invalid. To reference another module, use the `require` or `import` frontmatter directives. Do not link it as a markdown resource; the renderer will flag this as a syntax error.
+
+## Never use a URL or folder path as a *reference* in prose
+- The hard constraint above is enforced only on markdown links. A bare URL or folder path in prose (concept body, functional-spec text, implementation reqs, test reqs) passes validation and is fed to the renderer verbatim as part of the spec text
+- That is exactly why it is dangerous as a *reference*: the renderer cannot fetch the URL or open the folder, so the model sees only the address, not the content behind it. To a human reader the line looks grounded in a source; to the renderer it is a *ghost dependency*, and the spec silently drifts from whatever the URL actually says
+- A URL or path in prose is fine when the *text itself* is the complete fact:
+  - **Runtime values** the produced software uses — the base URL an integration calls, a database connection path, a CLI argument default, an output directory layout
+  - **Provenance notes** for a snapshotted resource — recording the canonical URL a file saved under `resources/` was fetched from
+- Litmus test: "Would the renderer benefit from reading the bytes at this URL / folder?" If yes, save it to a file and link the file. If no (the address itself is the fact), it can stay as plain text
+
+## Structured protocol artifacts must be linked, never transcribed
+- JSON Schema, OpenAPI / Swagger, GraphQL SDL, Protobuf `.proto`, Avro / Thrift schemas, XML XSDs, AsyncAPI specs, JSON-RPC method definitions, wire-protocol descriptions, payload examples — anything with a formal machine-readable shape — belongs in a file under `resources/` (or another directory within the project)
+- The spec line should describe the *role* of the artifact ("the request body conforms to ...", "the public API surface is defined in ...") rather than its contents
+- Reasons: one source of truth (no drift between prose and schema); the renderer and the generated code can both consume the file directly; schema changes show up cleanly as diffs
+
+```plain
+***definitions***
+
+- :TaskCreateRequest: is the JSON payload for creating a task.
+  - It is defined by [resources/task_create_request.schema.json](resources/task_create_request.schema.json).
+- :TasksAPI: is the public HTTP surface for tasks.
+  - It is defined by [resources/tasks_openapi.yaml](resources/tasks_openapi.yaml).
+
+***functional specs***
+
+- A :User: can add a :Task: by POSTing :TaskCreateRequest: to the `POST /tasks` endpoint of :TasksAPI:.
+  - The endpoint responds per :TasksAPI:.
+```
+
+## Each linked resource is referenced from exactly one place
+- Linking the same file from two functional specs (or from a functional spec **and** an implementation requirement) creates two independent sources of truth — any later edit silently diverges
+- If a resource needs to inform multiple parts of the project, **don't repeat the link** — attach the resource to a **concept** in `***definitions***` and reference the concept token (`:ConceptName:`) elsewhere
+- If you're about to paste the same `[name](path)` link a second time, **stop** — create the concept first
+
+```plain
+***definitions***
+
+- :TaskModalSpec: is the user-interface contract for the task modal.
+  - It is fully described in [task_modal_specification.yaml](task_modal_specification.yaml).
+
+***functional specs***
+
+- A :User: can add a :Task: using :TaskModalSpec:.
+
+- A :User: can edit a :Task: using :TaskModalSpec:.
+```
+
+## File location and path resolution
+- Paths are resolved relative to the directory where `turri` is run (the current working directory)
+- Relative paths may traverse into parent directories — `[resource](../../resources/resource.md)` is valid — as long as they resolve to a real text file on disk
+- The conventional location is a `resources/` directory; keep the resource inside the project so the path stays stable from wherever `turri` is run

package/genome/rules/module-structure.md

@@ -0,0 +1,105 @@
+---
+description: File-level structure of .plain modules — frontmatter, sections, ordering, and section ownership
+---
+
+# Rules for `.plain` module structure
+
+When creating a new `.plain` file or reviewing the structure of a whole file, always follow these rules. Section *content* rules live in the per-section rule files linked below — this file covers only file-level structure.
+
+## File skeleton
+- A `.plain` file is one **module**: optional YAML frontmatter between `---` markers, followed by sections marked with `***section name***` headers
+- The frontmatter, when present, must be the first thing in the file
+- Every content line inside a section must be a `- ` list item — see `bullet-continuation.md`
+- Every line must be as short and plain as its content allows — see `concise-specs.md`
+
+## Frontmatter fields
+
+| Field | Purpose | Rule file |
+|---|---|---|
+| `description` | One-line summary of the module; recommended on every module | — |
+| `import` | Pull definitions, implementation reqs, and test reqs from import modules | `import-modules.md` |
+| `requires` | Attach the module to the build chain of another root module | `requires-modules.md` |
+| `exported_concepts` | Declare which of this module's concepts are visible to modules that `require` it | `exported-concepts.md` |
+| `required_concepts` | Declare concepts an import module expects the importing module to define | `required-concepts.md` |
+
+## Sections
+- There are exactly four top-level section markers: `***definitions***`, `***implementation reqs***`, `***test reqs***`, and `***functional specs***`
+- Each section appears **at most once** per file; all are optional (which ones are allowed depends on the module kind below)
+- `***acceptance tests***` is **never a top-level section** — it appears only nested under a single functional spec (see `func-specs.md`)
+- Write sections in canonical order: `***definitions***` → `***implementation reqs***` → `***test reqs***` → `***functional specs***`
+  - Definitions come first because every concept must be defined before it is referenced
+  - Functional specs come last because they are rendered incrementally and their nested acceptance tests close the file
+
+## Module kinds
+- A **root module** lives at the repository root and carries behavior: it contains `***functional specs***`
+- To be renderable, a root module needs at least one functional spec plus the implementation reqs to build it — its own or from an `import`ed module (implementation reqs are not inherited through `requires`). A frontmatter-only file is a valid scaffold, but it cannot be rendered until a functional spec is added
+- A **requires module** is a root module that is used by another root module via `requires` directive (see `requires-modules.md`)
+- An **import module** provides shared content that other modules pull in via `import`: it contains only `***definitions***`, `***implementation reqs***`, and/or `***test reqs***` — never `***functional specs***`, never `requires` (see `import-modules.md`)
+
+## Section ownership (where a fact lives)
+- The renderer reads each kind of fact **only from its owning section** — a fact placed in the wrong section is silently ignored, not flagged
+- Before writing any requirement, place it by content:
+- To move facts that are already misplaced, or to split a bullet carrying several kinds of fact, use the `resolve-section-ownership` skill
+
+| Content | Owning section | Rule file |
+|---|---|---|
+| Concepts (`:CamelCaseToken:`) | `***definitions***` | `definitions.md` |
+| HOW the software is built — tech stack, architecture, coding standards — and **everything about `:UnitTests:`** | `***implementation reqs***` | `impl-reqs.md` |
+| **Everything about `:ConformanceTests:`** — framework, run command, mocking and network policy | `***test reqs***` | `test-reqs.md` |
+| WHAT the software does — observable, language-agnostic behavior | `***functional specs***` | `func-specs.md` |
+| End-to-end workflow verification of one functional spec | nested `***acceptance tests***` | `func-specs.md` |
+
+## Comments
+- Lines beginning with `>` are ignored when rendering
+
+```plain
+> This is a comment in ***plain.
+```
+
+- Comments explain the specification to human authors — never use a comment to carry a requirement the renderer must implement
+
+## Liquid templates
+- `.plain` files are processed with [Liquid](https://shopify.github.io/liquid/) before parsing — the full Liquid syntax (variables, filters, conditionals, loops) is available in any section
+- Expansion is **textual** and happens before the specification is interpreted — unlike `import`, which includes another module semantically (see `import-modules.md`)
+
+```plain
+{% assign max_tags = 3 %}
+
+***functional specs***
+
+- A :Widget: can be created with up to {{ max_tags }} tags.
+{% if max_tags > 1 %}
+- A :Widget: can have its tags listed.
+{% endif %}
+```
+
+## Format
+
+```plain
+---
+description: Command-line task manager.
+import:
+  - airplain
+---
+
+***definitions***
+
+- :Task: is an activity tracked by the application.
+  - Name (required)
+
+- :TaskList: is the collection of all :Task: items. Initially empty.
+
+***implementation reqs***
+
+- :Implementation: keeps :TaskList: in memory.
+
+***test reqs***
+
+- :ConformanceTests: must not access the network.
+
+***functional specs***
+
+- A :Task: can be added to the :TaskList:.
+
+- The :TaskList: is displayed.
+```

package/genome/rules/project-model.md

@@ -0,0 +1,88 @@
+---
+description: The ***plain project model - the specs as source of truth, the repository layout, generated output, template parameters, and what an earlier test failing after a render means
+---
+
+# The ***plain project model
+
+This rule covers the project as a whole. Section content is owned by the per-section rule files;
+file structure by `module-structure.md`.
+
+## Source of truth
+
+- The `.plain` files are the source of truth. They state what the software does, how it is built,
+  and how it is tested; the code is generated from them
+- Every change to the software is a change to a `.plain` file, which the next render builds. A fix
+  made anywhere else is lost the next time the code is regenerated
+
+## Repository layout
+
+```text
+*.plain                  # root modules: what the software does
+template/*.plain         # import modules: shared definitions and reqs
+resources/               # files specs link to: schemas, API specs, fixtures
+test_scripts/            # unit and conformance runners, found by turri (not linked)
+config.turri.yaml        # the project's build decisions: models, conformance tests, the scripts in use
+plain_modules/           # generated - one folder per module
+  <module>/code/         #   implementation and unit tests
+  <module>/tests/        #   the whole conformance suite of the module
+```
+
+Follow `module-structure.md`, `import-modules.md` and `linked-resources.md` for what these
+directories may contain and how a spec refers to them.
+
+## The config records the build decisions
+
+`config.turri.yaml`, at the project root beside the specs, is where the project's decisions about how it
+is built live: the models, whether conformance tests are written (`conformance-tests:`), and the scripts
+it uses (`unittests-script:`, `conformance-tests-script:`,
+`prepare-environment-script:`). turri can find a script by its name, but the config is what a person
+reads to see how the project is set up, and what they change.
+
+- Settle a decision, write it into the config in the same step - never leave it to be inferred from
+  what is on disk. A testing script you write or keep is named by its key, by path
+  from the root (`unittests-script: test_scripts/run_unittests_python.sh`); conformance tests
+  turned on or off are `conformance-tests: true` or `false`
+- Change only the keys you settled. Where the file has a key's example commented out, uncomment that
+  line and set it; keep every other line and comment as it is
+- A project whose parts are on different stacks keeps each stack's scripts in its own
+  `config.turri.<stack>.yaml` beside it; what the stacks share stays in `config.turri.yaml`
+- A project with only a `config.yaml` has it copied to `config.turri.yaml` first, and each
+  `config.<name>.yaml` to `config.turri.<name>.yaml`; `config.yaml` may be another tool's, and is never
+  written
+- Say in your summary which lines you wrote
+
+## Generated output is read-only
+
+Everything under `plain_modules/` is generated - `code/` and `tests/` alike. Read it, run it, debug
+it; never edit it. Fix the spec instead, in the section that owns the fact:
+
+- behaviour, implementation or unit-test guidance → definitions, functional specs, or implementation
+  reqs
+- conformance-test guidance → test reqs
+- acceptance-test guidance → the acceptance tests nested under their functional spec
+
+Then render again.
+
+## Template inclusion with parameters
+
+Besides frontmatter `import`, a template can be included with parameters:
+
+```plain
+{% include "python-console-app-template.plain", main_executable_file_name: "my_app.py" %}
+```
+
+Parameters are key-value pairs; the template reads them as `{{ variable_name }}`.
+
+## When an earlier test fails after a render
+
+Functional specs render in order, each knowing the ones before it and none after it - a required
+module's specs count as before. After each render every conformance test so far runs again. A test
+that passed before and fails now means one of:
+
+1. the new code does not satisfy its spec;
+2. the test does not test its spec accurately;
+3. the new spec and an earlier one conflict.
+
+The harness triages the first two and repairs the code. The third is a spec problem: resolve it in
+the `.plain` files with `resolve-spec-conflict`, and trace a misbehaving build back to its spec with
+`debug-specs` - never by editing generated output.

package/genome/rules/required-concepts.md

@@ -0,0 +1,44 @@
+---
+description: Rules for using required_concepts in .plain files
+---
+
+# Rules for `required_concepts`
+
+When adding or editing `required_concepts` in a `.plain` file's frontmatter, always follow these rules:
+
+## What required_concepts does
+- `required_concepts` declares concepts that any module importing this file **must** define
+- It creates a contract: the import module references these concepts but does not define them — the importing module is responsible for providing definitions
+- This is used exclusively on import modules
+
+## When to use it
+- Use `required_concepts` when an import module references concepts that vary per project or per importing module
+- The import module can reference these concepts in its definitions, implementation reqs, or test reqs — but their actual definitions come from whoever imports the file
+
+## Importing module must satisfy the contract
+- Every concept listed in `required_concepts` must be defined somewhere visible to the importing module — in its own `***definitions***` section or in another module it imports
+- If the importing module does not define a required concept, the spec is invalid
+- Check `required_concepts` of all imported modules before finalizing a module
+
+## Do not define required concepts locally
+- The import module that declares `required_concepts` must **not** define those concepts itself
+- The whole point is that the importing module provides the definition
+
+## Format
+
+```plain
+---
+required_concepts: [":AppName:", ":AppConfig:"]
+description: Imported module that requires AppName and AppConfig to be defined by the importer
+---
+
+***definitions***
+- :MainFile: is the entry point for :AppName:.
+
+***implementation reqs***
+- :MainFile: should load :AppConfig: on startup.
+```
+
+In this example, `:AppName:` and `:AppConfig:` are referenced but not defined — the module that imports this import module must define them.
+
+List concepts as a YAML array with each concept in `:ConceptName:` notation.