plain-forge 1.0.8 → 1.0.10

package/forge/rules/func-specs.md

@@ -28,7 +28,7 @@ When writing or editing a `***functional specs***` section in a `.plain` file, a
 - Write in terms of behavior, concepts, and domain logic
 - Avoid language-specific terminology: generics syntax, framework annotations, language-specific collection types, decorator syntax, base-class keywords, framings like "POJO" or "dataclass"
 - General technical terms that are not language-specific are fine: null values, JSON types, HTTP status codes, REST endpoints, etc.
-- **Naming concrete components is encouraged.** Functional specs can and should refer to `:CsvToJsonConverter:` and its methods `:CsvToJson:` / `:JsonToCsv:` and pin down their inputs, outputs, and error behavior — those names are part of the public contract and survive a language switch. What they must **not** do is bake in how the contract is realized (`@staticmethod`, `class Foo extends Bar`, `List<T>`, etc.)
+- **Naming concrete components is encouraged.** Functional specs can and should refer to concrete domain components, services, or entities (e.g., `:PaymentProcessor:`, `:UserRepository:`, `:DataConverter:`) and their operations (e.g., `:ChargeCard:`, `:FindById:`), pinning down their inputs, outputs, and error behavior. Those names are part of the public contract and survive a language switch. What they must **not** do is bake in how the contract is realized (`@staticmethod`, `class Foo extends Bar`, `List<T>`, `async def`, etc.)
 - **Litmus test:** if the project switched from Python to Java (or vice versa), would the functional spec read correctly with only `***implementation reqs***` updated? If yes, the spec is language-agnostic. If the spec itself would need rewording, the construct belongs in implementation reqs.
 
 ## Disambiguation

package/forge/rules/integration-embedded.md

@@ -9,12 +9,12 @@ When an integration `.plain` module is **embedded** — meaning the generated co
 
 Embedded means: the host codebase already exists, has its own language / framework / dependency manager / packaging layout, and the integration must conform to all of that without negotiation.
 
-> **For test-script authoring**, also follow [`integration-embedded-testing.md`](integration-embedded-testing.md). It defines the per-script contract (`prepare_environment_<lang>`, `run_unittests_<lang>`, `run_conformance_tests_<lang>`) — staging into the host vs `.tmp/`, arg validation, exit codes, output parsing, the three `***implementation reqs***` entries the spec must declare so the scripts can be generated, and a Java / Maven reference implementation. This file (`integration-embedded.md`) only summarizes the test-script wiring; the testing rule is the source of truth.
+> **For test-script authoring**, also follow [`integration-embedded-testing.md`](integration-embedded-testing.md). It defines the per-script contract (`prepare_environment_<lang>`, `run_unittests_<lang>`, `run_conformance_tests_<lang>`) — staging into the host vs `.tmp/`, arg validation, exit codes, output parsing, the three `***implementation reqs***` entries the spec must declare so the scripts can be generated. This file (`integration-embedded.md`) only summarizes the test-script wiring; the testing rule is the source of truth.
 
 ## The host codebase dictates the tech stack (hard rule)
 
 - Language, framework, dependency manager, packaging layout, coding standards, error model, logging library, and architecture are **inherited** from the host — they are **never chosen** by the integration spec
-- Do not re-ask the user about any of these in any phase — they are facts to be discovered from the host's manifest files (`pyproject.toml`, `package.json`, `go.mod`, `Cargo.toml`, `pom.xml`, …) and source tree
+- Do not re-ask the user about any of these in any phase — they are facts to be discovered from the host's manifest files (`pyproject.toml`, `package.json`, `go.mod`, `Cargo.toml`, `pom.xml`, …) and other integrations.
 - If a Phase 3 (`forge-plain`) tech-stack question seems to push back on a host rule, treat the host as ground truth and rewrite the question
 - Implementation reqs added in Phase 3 are **transcribed** from the host stack verbatim — host language and exact version, host framework + version, dependency manager and manifest path, packaging layout, host conventions the contract must follow, and every host-package version the contract pins
 

package/forge/rules/integrations.md

@@ -44,12 +44,14 @@ A single `.plain` module can (and typically will) reference many resources. That
 
 Documentation lies — it goes stale, omits undocumented fields, describes a different API version, papers over breaking changes. Every integration spec must be grounded in what the API really returns, not what the docs claim it returns.
 
-- **Always `fetch` the provider's documentation — even if you already "know" the API.** Training-data memory of any third-party REST API is, by definition, stale: endpoints get renamed, fields get added or deprecated, auth flows change, error envelopes shift, and rate-limit headers are renamed between releases. The only acceptable source of truth for what the API looks like *today* is the provider's own live documentation, retrieved with `fetch` at spec-authoring time. This applies without exception — there is no API well-known enough to skip this step, and a spec authored from memory is a spec authored against the wrong contract. Concretely:
-  - Before authoring **any** endpoint, auth, error, pagination, or webhook concept, `fetch` the relevant documentation page(s) and quote concrete details (status codes, field names, header names, error formats) directly from the fetched content into the resources under `resources/`. Never paraphrase from memory.
+- **Discover the documentation links with web search first, then `fetch` all of them.** Documentation moves, gets reorganized, and is versioned — so never assume a doc URL from memory. Begin every topic by issuing **web searches with human-readable queries** to find the canonical pages, then `fetch` **every** relevant link the search surfaces — the endpoint reference, the auth guide, the webhook catalog, the error reference, the pagination/rate-limit docs, the changelog. The search step finds *which* URLs are authoritative and current; the `fetch` step retrieves their actual content. Do not stop at the first hit — fetch the full set so a topic is grounded in all of its sources, not a single page.
+  - If the environment has **no web-search tool** (only URL-based `fetch` is available), say so explicitly, construct the URL from the provider's well-known documentation root and crawl outward by `fetch`-ing that root and following its links, and ask the user for the canonical URL whenever you cannot reach the right page that way. Never substitute memory for the search-then-fetch step.
+- **Always `fetch` the provider's documentation — even if you already "know" the API.** The only acceptable source of truth for what the API looks like *today* is the provider's own live documentation, retrieved with `fetch` at spec-authoring time. This applies without exception — there is no API well-known enough to skip this step, and a spec authored from memory is a spec authored against the wrong contract. Concretely:
+  - Before authoring **any** endpoint, auth, error, pagination, or webhook concept, **web-search to locate the relevant documentation page(s), then `fetch` each one** and quote concrete details (status codes, field names, header names, error formats) directly from the fetched content into the resources under `resources/`. Never paraphrase from memory.
   - Save the fetched documentation snapshot under `resources/docs/<provider>/<page>.md` (or `.html` if structure matters) so the spec has a stable doc artifact the renderer and reviewers can consult, independent of the live URL changing or going behind auth.
   - If a documentation page is unreachable (paywall, login wall, JS-only render that `fetch` can't see), say so explicitly and ask the user for the canonical content rather than filling the gap from memory.
-  - **The fetched documentation is then cross-checked against the live API** — see the rest of this section. Memory is not part of the loop at any point.
-- **Validate credentials against the live API** before authoring downstream specs. A 2xx on a low-risk read-only endpoint (`/v1/me`, `/account`, `/whoami`, `/health`) is the gate. On 401/403, stop and resolve before continuing.
+  - **The fetched documentation is then cross-checked against the live API** — see the rest of this section.
+- **Validate credentials against the live API** before authoring downstream specs. A 2xx on a low-risk read-only endpoint (`/v1/me`, `/account`, `/whoami`, `/health`) is the gate. On 401/403, stop and resolve before continuing. The credentials should be validated against the live API before any downstream specs are authored.
 - **Issue the minimum cross-check coverage** with `fetch`: one discovery / schema endpoint if available, one list endpoint per primary entity in scope, one single-object retrieval per primary entity, one empty/boundary response, one 404, one 400/422, and one deliberate 401.
 - **Save every probe response under `resources/fixtures/`** with credentials redacted. The fixtures become the seed for `resources/<provider>.openapi.yaml` and feed conformance tests later.
 - **Every discrepancy is recorded, not smoothed over.** Each finding goes into the relevant resource (the OpenAPI file, the error envelope schema, `rate-limit-headers.yaml`, …) as the source of truth, with a short note in the corresponding concept saying "docs claim X, live API returns Y; we follow the live API".
@@ -112,6 +114,12 @@ A production-ready integration spec captures every corner case the API can throw
 - **Rate-limit model** as `resources/rate-limit-headers.yaml` + `resources/rate-limits.yaml` + `components.schemas.RateLimitError`
 - **Error model** as `components.schemas.ErrorEnvelope` + `resources/error-map.yaml`, with one functional spec per error category
 - **Retry policy** in `resources/retry-policy.yaml`
+- **Transient vs. permanent exception classification** — every error the integration can raise is classified as either *transient* (retryable per the retry policy above) or *permanent* (surfaced to the caller immediately). The classification lives in `resources/error-map.yaml` (per-status-code) and in the integration's own exception hierarchy concept, and the two must agree. Concretely:
+  - **Transient by default**: network-level errors (`ConnectionError` / equivalent, DNS failure, socket timeout, TLS handshake failure, connection reset), HTTP 408, 425, 429, 500, 502, 503, 504, and any provider-specific application-level error codes the docs / live API mark as retryable (e.g. Stripe's `lock_timeout`, AWS `ThrottlingException`)
+  - **Permanent by default**: every other 4xx (400, 401, 403, 404, 405, 409, 410, 415, 422, …), schema-validation failures on otherwise-2xx responses, signature-verification failures on webhooks, and any provider error code the docs / live API mark as non-retryable
+  - **Integration exception hierarchy**: the integration exposes a base exception class (e.g. `<Provider>Error`) with two direct subclasses, `<Provider>TransientError` (raised after the retry policy gives up on a transient condition) and `<Provider>PermanentError` (raised immediately on a permanent condition). Every concrete exception inherits from exactly one of those two — callers branch on the base class, never on individual subclasses
+  - **The classification is data-driven**, not code-driven — `resources/error-map.yaml` is the source of truth, the renderer reads it to generate the exception hierarchy and the `is_transient(error) -> bool` predicate. Spec text never enumerates status codes inline
+  - **Discrepancies found during the live-API cross-check override the docs** — if the docs say a code is transient but the live API never recovers from it (or vice versa), the live behavior wins and the divergence is recorded in `error-map.yaml` plus a note in the error-model concept ("docs claim X retryable, live API returns Y permanently; we follow the live API")
 - **Idempotency strategy** in `resources/idempotency.yaml` + idempotency header in `components.parameters`
 - **Webhook contracts** as `resources/webhooks/<event>.schema.json` per event type + `resources/webhook-signing.yaml`
 - **Data mapping** (entity schemas + transformations / exclusions) as entity schemas + `resources/data-mapping.yaml`
@@ -127,4 +135,5 @@ A production-ready integration spec captures every corner case the API can throw
 - **Authoring against unverified credentials.** Validate first; if the user has no credentials yet, flag it in the module's frontmatter description and re-validate once credentials arrive
 - **`requires`-ing a separate-stack module** (a Python backend `requires`-ing a React frontend, or vice versa) — see [`requires-modules.md`](requires-modules.md). Use a shared API schema in `resources/` instead
 - **Authoring Phase 1 specs from the docs first and "reconciling" with the live API later.** Probe the API as you reach each topic; the live response is the source of truth from the moment it's captured
-- **Writing any integration spec from memory of the provider's API instead of `fetch`-ing its documentation first.** No matter how well-known the API (Stripe, GitHub, Slack, Salesforce, AWS, OpenAI, …), the documentation must be retrieved with `fetch` at spec-authoring time and saved under `resources/docs/<provider>/` — see *Live API must be cross-checked against the documentation*. Authoring from memory bakes in whatever version of the API was current during training, which is always older than the version the integration will actually call
+- **Writing any integration spec from memory of the provider's API instead of web-searching for its documentation and `fetch`-ing every relevant page first.** No matter how well-known the API (Stripe, GitHub, Slack, Salesforce, AWS, OpenAI, …), the canonical pages must be located with web search and then retrieved with `fetch` at spec-authoring time and saved under `resources/docs/<provider>/` — see *Live API must be cross-checked against the documentation*. Authoring from memory bakes in whatever version of the API was current during training, which is always older than the version the integration will actually call
+- **Guessing a documentation URL from memory and `fetch`-ing it without searching first.** A remembered URL may 404, redirect to a stale version, or miss the page that actually documents the topic. Search to find the authoritative, current links, then fetch the full set

package/forge/skills/load-plain-reference/SKILL.md

@@ -406,9 +406,9 @@ For implementation details — the exact step sequence, toolchain checks, langua
 - **Conflicting specs must be avoided at all costs.** Functional specs should be written so that no conflicts exist between them. If two specs appear to conflict, they must be clarified by adding more detail and context to the specs until all possible conflicts are resolved. Prevention is always preferable to debugging conflicts after rendering.
 - **Specs should be language-agnostic.** Avoid using programming language-specific terminology (e.g., generics syntax, framework annotations, language-specific collection types, decorator syntax, language-specific base classes or type keywords like "POJO" or "dataclass") in functional specs and definitions. Write specs in terms of behavior, concepts, and domain logic — not implementation constructs. General technical terms that are not language-specific are fine (e.g., null values, JSON types, HTTP status codes, REST api endpoints etc.). The `***implementation reqs***` section is the appropriate place for language-specific guidance.
 
-    Naming concrete *components* — classes, methods, functions, fields — is encouraged and not in conflict with this rule. A functional spec should freely refer to `:CsvToJsonConverter:` as a component with methods `:CsvToJson:` and `:JsonToCsv:`, describe their inputs, outputs, and error behavior, and treat those names as part of the public contract. What it must *not* do is bake in how that contract is realized in a particular language: no `@staticmethod` decorators, no `class Foo extends Bar` phrasing, no `List<T>` or `Optional<T>` syntax, no "POJO with static methods" framing.
+    Naming concrete *components* — classes, methods, functions, fields — is encouraged and not in conflict with this rule. A functional spec should freely refer to concrete domain components, services, or entities (e.g., `:PaymentProcessor:`, `:UserRepository:`, `:DataConverter:`) and their operations (e.g., `:ChargeCard:`, `:FindById:`), pinning down their inputs, outputs, and error behaviors, and treat those names as part of the public contract. What it must *not* do is bake in how that contract is realized in a particular language: no `@staticmethod` decorators, no `class Foo extends Bar` phrasing, no `List<T>` or `Optional<T>` syntax, no "POJO with static methods" framing.
 
-    The litmus test: if you switched the project from Python to Java (or vice versa), would the functional spec still read correctly with only `***implementation reqs***` updated? If yes, the spec is language-agnostic. If the functional spec itself would need rewording because it referenced a language-specific construct, the construct belongs in implementation reqs instead. The component name (`:CsvToJsonConverter:`) is the same across languages; the syntax used to express "static method on a class" is not.
+    The litmus test: if you switched the project from Python to Java (or vice versa), would the functional spec still read correctly with only `***implementation reqs***` updated? If yes, the spec is language-agnostic. If the functional spec itself would need rewording because it referenced a language-specific construct, the construct belongs in implementation reqs instead. The component name (`:DataConverter:`) is the same across languages; the syntax used to express "static method on a class" is not.
 - **Keep sentences short and clear — but never at the cost of ambiguity.** Spec lines should be easy to read and understand at a glance. Prefer short, direct sentences and plain words over long sentences and jargon — if a 10-cent word and a 50-cent word say the same thing, use the 10-cent one. This applies to every spec section, not only functional specs: `***definitions***`, `***implementation reqs***`, `***test reqs***`, and `***acceptance tests***` should all be as concise as they can be while staying unambiguous. The hard constraint is in the second half of that rule: **wordy-but-precise always beats terse-but-ambiguous.** If trimming a clause, a qualifier, or a sub-bullet would leave the spec open to more than one reasonable interpretation, leave it in. When a sentence starts to grow because the behavior is genuinely complex, split it into two short sentences (or into a parent line + sub-bullets) rather than dropping detail. Concision is in service of clarity, never the other way around.
 - **Specs must be deterministic enough to both *run* and *use* the software without reading the generated code.** A developer should be able to figure out, from the specs alone, two distinct things:
 
@@ -420,12 +420,12 @@ For implementation details — the exact step sequence, toolchain checks, langua
 - **Specs must define programmatic interfaces.** Any runtime or interface details must be defined in the functional specs, not in implementation reqs. This means functional specs name the concrete *components* a caller will reach for — utilities, services, methods, functions, fields — and pin down their inputs, outputs, and error behavior, so that callers can use the software without reading the generated code. For example, a spec can require:
 
     ```plain
-    - Implement :CsvToJsonConverter: as a stateless utility component exposing two operations.
-        - :CsvToJson: takes a CSV row and a list of header strings, returns a JSON object keyed by header name. Infer and convert value types. Empty CSV values must be converted to null. Null values must be preserved — keys with null values must appear in the result, not be omitted. Handle CSV escaping (quoted values, commas within quotes). Raise an error if the number of columns does not match the headers.
-        - :JsonToCsv: takes a JSON object and a list of header strings defining column order, returns a CSV row without a header row. Handle CSV escaping for values containing commas or quotes. Output an empty string for null or missing fields. Extra keys in the JSON object that are not present in the headers list should be silently ignored.
+    - Implement :DataConverter: as a stateless utility component exposing two operations.
+        - :FormatData: takes a raw data string and a format type, returns a formatted string. Infer and convert value types. Empty inputs must be converted to null. Null values must be preserved — keys with null values must appear in the result, not be omitted. Handle escaping logic. Raise an error if the format type is not supported.
+        - :ParseData: takes a formatted string and returns a structured object. Output an empty structure for null or missing fields. Unrecognized extra keys should be silently ignored.
     ```
 
-    This rule is complementary to the earlier "specs should be language-agnostic" guideline, not in conflict with it. Component names (`:CsvToJsonConverter:`, `:CsvToJson:`, `:JsonToCsv:`) and behavioral contracts belong in functional specs because they survive a language switch unchanged. Language-specific realizations of those contracts — "POJO class with static methods", "Python module with module-level functions", `@staticmethod`, `class Foo`, exception types like `IllegalArgumentException` vs `ValueError`, choice of test framework (`pytest` vs `JUnit`), mocking library, fixture style, assertion syntax — belong in `***implementation reqs***` and `***test reqs***`, because those are exactly what changes when the target language changes. Use `***implementation reqs***` for *how the production code is realized* (language, frameworks, libraries, syntax, error types) and `***test reqs***` for *how the tests are realized* (test framework, test runner, mocking and fixture conventions, parametrization style, naming conventions, file layout). The goal is that swapping languages requires editing only `***implementation reqs***` and `***test reqs***`; the functional spec for `:CsvToJsonConverter:` should read identically whether the project is in Python, Java, or anything else.
+    This rule is complementary to the earlier "specs should be language-agnostic" guideline, not in conflict with it. Component names (`:DataConverter:`, `:FormatData:`, `:ParseData:`) and behavioral contracts belong in functional specs because they survive a language switch unchanged. Language-specific realizations of those contracts — "POJO class with static methods", "Python module with module-level functions", `@staticmethod`, `class Foo`, exception types like `IllegalArgumentException` vs `ValueError`, choice of test framework (`pytest` vs `JUnit`), mocking library, fixture style, assertion syntax — belong in `***implementation reqs***` and `***test reqs***`, because those are exactly what changes when the target language changes. Use `***implementation reqs***` for *how the production code is realized* (language, frameworks, libraries, syntax, error types) and `***test reqs***` for *how the tests are realized* (test framework, test runner, mocking and fixture conventions, parametrization style, naming conventions, file layout). The goal is that swapping languages requires editing only `***implementation reqs***` and `***test reqs***`; the functional spec for `:DataConverter:` should read identically whether the project is in Python, Java, or anything else.
 
 ## Line Length Rule
 

package/forge/skills/run-codeplain/SKILL.md

@@ -212,56 +212,7 @@ What to extract from the framework output — stay framework-agnostic, look for
 - **Stack traces**: which file under the prepared working folder threw, and at which line. Map the file back to `plain_modules/<module>/src/...` to find the code the spec is supposed to be governing.
 - **"No test files found" / "No tests collected"**: the script ran but discovered nothing. Almost always a path/glob mismatch in the staged working folder, not a spec problem. Treat as Pathology F.
 
-Always set the logs to be verbose in `config.yaml` and make sure the following logging_config.yaml exists and is set:
-
-```yaml
-version: 1
-disable_existing_loggers: false
-
-root:
-  level: DEBUG
-
-loggers:
-  codeplain:
-    level: DEBUG
-    propagate: true
-  git:
-    level: DEBUG
-    propagate: true
-  transitions:
-    level: DEBUG
-    propagate: true
-  transitions.extensions.diagrams:
-    level: DEBUG
-    propagate: true
-  anthropic:
-    level: DEBUG
-    propagate: true
-  openai:
-    level: DEBUG
-    propagate: true
-  google:
-    level: DEBUG
-    propagate: true
-  google_genai:
-    level: DEBUG
-    propagate: true
-  httpx:
-    level: DEBUG
-    propagate: true
-  httpcore:
-    level: DEBUG
-    propagate: true
-  urllib3:
-    level: DEBUG
-    propagate: true
-  requests:
-    level: DEBUG
-    propagate: true
-  asyncio:
-    level: DEBUG
-    propagate: true
-```
+Always set the logs to be verbose in `config.yaml`. 
 
 ### 2. Renderer memory under `plain_modules/<module>/.memory/`
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "plain-forge",
-  "version": "1.0.8",
+  "version": "1.0.10",
   "description": "Conversational spec-writing tool for ***plain specification language",
   "type": "module",
   "engines": {