setspec 0.2.0__tar.gz → 0.4.0__tar.gz

{setspec-0.2.0 → setspec-0.4.0}/.github/workflows/ci.yml

@@ -91,6 +91,44 @@ jobs:
       - run: pip install . --no-deps
       - run: pytest -m contract
 
+  # The three published guarantees get their own jobs rather than sharing `contracts`' single
+  # red mark, because each answers a different question and the answers are acted on differently:
+  # a snapshot diff means "bump the version", a golden failure means "the artifact is wrong", and
+  # a cross-version failure means "a consumer at another version just broke".
+  schema-snapshot-diff:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.12" }
+      - run: pip install --require-hashes -r requirements/ci.lock
+      - run: pip install . --no-deps
+      # ADR-0009 rule 7: a schema change without a version bump fails CI. The test regenerates
+      # every published schema from its writer model and diffs it against the committed snapshot.
+      - run: pytest tests/contract/test_schema_snapshots.py -m contract
+
+  golden-validation:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.12" }
+      - run: pip install --require-hashes -r requirements/ci.lock
+      - run: pip install . --no-deps
+      - run: pytest tests/contract/test_goldens.py -m contract
+
+  cross-version-compatibility:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.12" }
+      - run: pip install --require-hashes -r requirements/ci.lock
+      - run: pip install . --no-deps
+      # Testing Standards §8.3, proven by the repository that makes the promise: a v1.0 reader
+      # accepts a synthetic v1.1 golden without loss and refuses a synthetic v2.0 by major.
+      - run: pytest tests/contract/test_cross_version.py -m contract
+
   security:
     runs-on: ubuntu-latest
     steps:
@@ -148,3 +186,18 @@ jobs:
           marker = pathlib.Path(setspec.__file__).parent / 'py.typed'
           assert marker.is_file(), 'py.typed missing from the installed wheel'
           "
+      # The schemas and goldens are package data (ADR-0009 rule 7), and package data is exactly
+      # what a build is most likely to leave out silently: every test in this repository passes
+      # against the source tree whether or not the files reach the wheel. This is the only job
+      # that would notice, which is why it asserts against the *installed* distribution.
+      - name: schemas and goldens ship in the wheel
+        run: |
+          python -c "
+          from setspec import PUBLISHED_SCHEMAS, golden_payloads, json_schema_for
+          for schema, versions in PUBLISHED_SCHEMAS.items():
+              for version in versions:
+                  assert json_schema_for(schema, version)['title'] == schema
+                  goldens = golden_payloads(schema, version)
+                  assert len(goldens) >= 3, f'{schema} {version} ships {len(goldens)} goldens'
+          print(f'{len(PUBLISHED_SCHEMAS)} schemas with goldens loaded from the installed wheel')
+          "

{setspec-0.2.0 → setspec-0.4.0}/CHANGELOG.md

@@ -7,7 +7,122 @@ packaging and release standards §3.
 
 ## [Unreleased]
 
+## [0.4.0] — 2026-08-29
+
+### Added
+
+- **`setspec.prompts`** (Phase 5): prompt records and prompt packs, moved from
+  `freeweight.services.prompts` — `PromptRecord`, `PromptLibrary`, `load_record()`, `load_pack()`,
+  `build_manifest()`/`write_manifest()`, and the pack's three hashes
+  (`prompt_record_hash`/`prompt_subset_hash`/`pack_hash`, ADR-0028). Rendering runs through a
+  sandboxed, loader-less Jinja2 `Environment` with `StrictUndefined` (prompt-management-standards
+  §2.1): a referenced-but-unsupplied variable is an error, a template cannot reach the filesystem,
+  and dunder attribute access is refused. `load_pack()`'s `root` parameter is required — the
+  package carries no opinion about an application's pack layout; an application wanting a default
+  location supplies its own thin wrapper (FreeWeight's `freeweight.services.prompts.load_pack`
+  keeps its old default this way).
+
+  FreeWeight's own prompt pack adopts this module in the same change (pulling its P12 forward):
+  `freeweight.services.prompts` is now a re-export shim over `setspec.prompts`, verified to
+  produce a byte-identical `pack_hash` and per-record hash for every shipped prompt before and
+  after the move, with FreeWeight's own gate green afterwards. Two applications reading the same
+  pack format from two processes is the scenario ADR-0028's hashes exist for; the FreeWeight move
+  is the first proof of it.
+
+- **JSON Schema for `prompt.record` and `prompt.manifest`**, committed as package data under
+  `setspec/schemas/prompt.record/1.0.json` and `setspec/schemas/prompt.manifest/1.0.json`, with
+  golden examples under `setspec/goldens/prompt.{record,manifest}/1.0/{full,minimal}.json`. Both
+  are hand-authored rather than generated: `PromptRecord` is a plain dataclass, not a pydantic
+  envelope payload — a prompt record is a pack file on disk, never wrapped in a `SchemaEnvelope` —
+  so neither schema is registered in `PUBLISHED_SCHEMAS`/reachable through `json_schema_for()`,
+  and the goldens are illustrative examples rather than inputs to the envelope contract-test
+  matrix in `tests/contract/test_goldens.py`. Every golden round-trips through the real
+  `load_pack()` loader (not just schema validation) and its `pack_sha256` is the module's own
+  computed hash, not a hand-typed value.
+
+### Changed
+
+- Bumped to 0.4.0 for the `setspec.prompts` addition (packaging standards §3: additive,
+  non-breaking).
+
+## [0.3.0] — 2026-08-28
+
 ### Added
+
+- **The v1.0 freeze.** `DRAFT_SCHEMAS` is now empty. Every registered payload type —
+  `model.identity`, `machine.profile`, `benchmark.result`, `benchmark.run_summary`,
+  `capability.evidence`, `benchmark.evidence_bundle`, `benchmark.goal_pack` and
+  `benchmark.calibration_report` — is a published `1.0` that may no longer be reshaped in place.
+  FreeWeight's real output (P6 onward, and P8A–8B for the two goal payloads) demanded no field
+  change, so the freeze is a promotion rather than a correction; the one field the pass did add,
+  `MetricValueFields.metric_key`, landed before it and is frozen *with* the schema.
+
+  The set survives its own emptiness deliberately: it is the mechanism a future draft uses, and
+  deleting it would leave the next provisional payload type either shipping silently provisional or
+  inventing a second way to say so.
+
+- **Generated JSON Schema for every published version**, committed as package data under
+  `setspec/schemas/<schema>/<version>.json` and reachable through `json_schema_for()`. Generated
+  from the **writer** (`extra="forbid"`) half, so a published document says
+  `additionalProperties: false` — which is the contract API standards §7 rule 5 states and the one
+  a producer's test suite asserts its output against. The reader policy stays where it belongs, in
+  the reader: `load_envelope` accepts an unknown minor by comparing majors, never by loosening a
+  version's schema to admit fields it cannot describe.
+
+  Every `$ref` resolves inside the document's own `$defs`. Nothing is fetched (spec §14).
+
+- **25 golden payloads**, three or more per version, under
+  `setspec/goldens/<schema>/<version>/<name>.json` and reachable through `golden_payloads()` and
+  `golden_names()`. Each version ships a `minimal` (only required fields), a `full` (every field
+  populated) and, wherever the payload carries a measurement at all, an `unsupported`-heavy example.
+  `capability.evidence` adds `goal`, a calibrated `user.noir_tech_voice` record with the whole
+  goal-sourced group populated; `benchmark.goal_pack` adds `starter_unforked`; and
+  `benchmark.calibration_report` adds `gate_failed` — the outcome `capability.evidence`
+  deliberately cannot express, because a goal below its gate emits no evidence record at all.
+
+  Which versions need an `unsupported` golden is derived from the published schema rather than
+  listed in the test: a payload whose document contains no `{"const": "unsupported"}` branch has no
+  measurement field to leave unsupported, so `benchmark.goal_pack` and
+  `benchmark.calibration_report` are excused by the artifact itself rather than by an exception
+  someone maintains.
+
+- **`setspec.artifacts`** — `PUBLISHED_SCHEMAS`, `json_schema_for()`, `golden_payloads()`,
+  `golden_names()`, `payload_pair()`, `build_json_schema()` and `render_schema_document()`. The
+  first four are re-exported from `setspec` directly, matching spec §7's "schema artefacts" group:
+  an accessor that *returns* a version's artifacts is not itself versioned.
+
+  `PUBLISHED_SCHEMAS` records **exact** versions, unlike `SUPPORTED_SCHEMAS`, which records
+  supported majors. An artifact is a file that either exists or does not, so "highest minor known"
+  is the wrong shape for it — and a contract test asserts the two agree, so a schema that can be
+  negotiated but not validated, or validated but not negotiated, fails the build.
+
+- **Three contract jobs, one per published guarantee.** `schema-snapshot-diff` regenerates every
+  schema from its models and diffs it against the committed snapshot — ADR-0009 rule 7, *a schema
+  change without a version bump fails CI*, made mechanical and proven by a deliberate test-only
+  mutation. `golden-validation` checks every golden against its writer model, its reader model, the
+  published JSON Schema and a canonical round trip. `cross-version-compatibility` builds a synthetic
+  `1.1` document from each `full` golden and asserts a `1.0` reader accepts it, preserves the field
+  it does not know, and re-exports it intact — then builds a synthetic `2.0` and asserts the refusal
+  names both versions. They are separate jobs rather than steps because the three failures are acted
+  on differently: bump the version, fix the artifact, or go and warn a consumer.
+
+- **`install-check` now asserts the package data reaches the wheel.** Every test in this repository
+  passes against the source tree whether or not the schemas and goldens are built into the
+  distribution, so this job is the only one that would notice. It loads all eight schemas and their
+  goldens out of the *installed* wheel.
+
+- **`docs/schemas.md`** — the human-readable catalogue: what ships and where, every payload type
+  with its required-field count and its goldens, how to consume the artifacts from a repository that
+  shares no code with this one, and a table of every cross-field rule the JSON Schema **cannot**
+  express and the models therefore still own.
+
+- **`jsonschema` and `types-jsonschema` in the `dev` extra.** Test-only, and justified by the
+  assertion they exist for: a golden is validated against the *published document* a non-Python
+  consumer receives, not only against the model that generated it. Without a real validator that
+  check would be a hand-rolled subset of draft 2020-12 maintained in this repository — the thing
+  under test also serving as the thing testing it. Nothing under `src/` imports either, and the base
+  install still pulls in no validator. `requirements/ci.lock` regenerated.
+
 - **`metric_key` on `MetricValueFields`.** The model declared value, unit, aggregation, direction,
   sample count and dispersion — and nothing saying *which metric it is*. Both
   `BenchmarkResultFields.metrics` and `BenchmarkRunSummaryFields.aggregate_metrics` carry sequences
@@ -58,6 +173,14 @@ packaging and release standards §3.
 
 ### Changed
 
+- The five payload modules' status notes read **frozen (`1.0`)** rather than **draft (`1.0`)**, and
+  say what the freeze binds them to: a new optional field is a minor bump, and a removal, rename,
+  retype or tightening is a major — never an edit in place.
+- The two tests that asserted the draft state now assert the frozen one
+  (`test_no_registered_schema_is_still_draft`, `test_the_schema_is_frozen`). They are the same
+  guarantee read from the other side, and leaving them asserting `DRAFT_SCHEMAS == SUPPORTED_SCHEMAS`
+  would have made the freeze a change the test suite refused.
+
 - A bare reserved root is now refused by `validate_capability` and reported `False` by
   `is_known_capability`. `user` is a namespace, not a capability: a payload claiming
   `capability_id: "user"` has lost the identity that is the entire point of the namespace.
@@ -71,6 +194,7 @@ packaging and release standards §3.
   declaring `1.2` or later is unaffected. Producers on `1.1` must emit roots that exist at `1.1`.
 
 ### Fixed
+
 - **Coverage measured a directory nothing imports.** `[tool.coverage.run] source` named
   `src/setspec`, the source *path*, while CI installs the built distribution — so the moment the
   jobs stopped using an editable install, coverage reported **0 %** and failed the 95 % floor for a
@@ -89,6 +213,13 @@ packaging and release standards §3.
   (`baseaicore`, `modelrack`, `sweatmeter`). Without it, `mypy --strict` in a consuming repository
   cannot see this package's types at all and treats every import from it as untyped.
 
+### Known gaps
+
+- `event.envelope` and `error.envelope` (Phase 3) and `prompt.record` / `prompt.manifest`
+  (Phase 5) are **not** part of this freeze, because they do not exist yet: a schema is frozen by
+  being published, and an unwritten one has nothing to publish. The freeze covers the eight payload
+  types that cross an application boundary today, not the eleven ADR-0009 anticipates.
+
 ## [0.2.0] — 2026-08-23
 
 Phase 2 of the [development plan](docs/packages/setspec/development-plan.md): provisional

setspec-0.4.0/PHASE4_ISSUES.md

@@ -0,0 +1,99 @@
+# Phase 4 — issues to address
+
+Written at the end of Phase 4 (freeze v1.0, publish schemas and goldens; `setspec 0.3.0`). Each
+entry is something a later phase, a docs change, a consumer or a release step has to resolve.
+Nothing here blocks Phase 4's acceptance criteria; everything here would become a defect if it were
+forgotten.
+
+---
+
+## Status — 2026-08-28
+
+| # | Issue | Status |
+|---|---|---|
+| 1 | The freeze covers eight payload types, not ADR-0009's eleven | **Open — by design.** Phases 3 and 5 own the other three. |
+| 2 | Goldens are authored inputs, but a SetSpec writer emits every declared key | **Needs a decision.** |
+| 3 | `ContributingMetricFields.metric_key` carries no pattern while `MetricValueFields.metric_key` does | **Needs a decision.** |
+| 4 | Cross-field rules are invisible in the published JSON Schema | **Open — documented.** `docs/schemas.md` §4 lists every one. |
+| 5 | `jsonschema` joined the `dev` extra; `requirements/ci.lock` regenerated | **Closed** — verify `pip-audit` in CI. |
+| 6 | The release is not tagged or published yet | **Owner: you.** Commands below. |
+
+---
+
+## 1. The freeze covers eight payload types, not eleven
+
+ADR-0009 lists eleven initial payload types. `event.envelope` and `error.envelope` (Phase 3) and
+`prompt.record` / `prompt.manifest` (Phase 5) are not written, so they are not frozen — a schema is
+frozen by being published, and an unwritten one has nothing to publish. `DRAFT_SCHEMAS` is empty
+**now**; when Phase 3 or 5 lands a new payload type it should re-enter that set until its own
+freeze, which is exactly the mechanism the set survives its emptiness for. The changelog's *Known
+gaps* section says the same. Nothing to do until those phases start, except not to read "empty"
+as "finished".
+
+## 2. Goldens are authored inputs, but a SetSpec writer emits every declared key
+
+`dump_envelope(CapabilityEvidenceOut(...))` serialises through `model_dump()`, which writes every
+declared field — a non-goal record carries `goal_hash: null`, `uncalibrated: false`, and so on. The
+`capability.evidence/1.0/full` golden was authored the other way round: it omits the goal group
+entirely, because "fully populated" was read as "every field a *non-goal* record populates". Both
+forms validate under both models, so the contract is not broken, but a producer's structural test
+("my keys match the full golden's keys") fails against the file and passes against the file *as a
+SetSpec writer would dump it*. FreeWeight's contract test compares against the latter.
+
+**Decision needed:** should goldens be committed *as written* (every declared key present, nulls
+included), so that "matches the golden structurally" is a byte-level statement? If yes, regenerate
+the 25 goldens through their `Out` models once and add a contract test that a golden equals its own
+writer-dumped form. If no, say so in `docs/schemas.md` §3 so consumers compare against the dumped
+form, as FreeWeight now does.
+
+## 3. `ContributingMetricFields.metric_key` has no pattern
+
+`MetricValueFields.metric_key` is constrained to lower snake case, dot-separable
+(`^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)*$`). `ContributingMetricFields.metric_key` — the key inside
+`capability.evidence.contributing_metrics` — is only `min_length=1`. FreeWeight writes
+`<suite_key>.<metric_key>` there (`native.tool_use.task_success`), `criterion.<key>` for a goal's
+own record and `goal.<slug>.composite_score` for a goal contributing to a shipped capability; all
+three happen to satisfy the stricter pattern. Tightening the field is a **major** change now that
+`1.0` is frozen, so it can only land as `capability.evidence 2.0`, and only if a consumer ever
+needs the guarantee. Recorded so the asymmetry is a decision rather than an accident.
+
+## 4. Cross-field rules are invisible in the published JSON Schema
+
+Pydantic renders types, ranges, patterns and required keys; it cannot render a `model_validator`.
+`runtime_profile_hash` agreeing with its profile, `score_method_mix` summing to one,
+`measured_at ≤ computed_at`, the goal group's five coherence rules — every one is enforced by the
+models only. `docs/schemas.md` §4 tables them, and every golden is validated against *both* the
+schema and the model for exactly this reason. A non-Python consumer that needs those rules needs a
+second validation step this package does not ship. Nothing to do unless such a consumer appears.
+
+## 5. `jsonschema` in the `dev` extra
+
+Added so the golden contract test validates against the *published* document with a real
+draft-2020-12 validator rather than a hand-rolled subset. Test only — nothing under `src/`
+imports it and the base install pulls in no validator. `requirements/ci.lock` was regenerated with
+`pip-compile --generate-hashes` (jsonschema 4.26.0, jsonschema-specifications, referencing,
+rpds-py, attrs, types-jsonschema). `release.lock` is unchanged. CI's `security` job audits both
+locks; the first run after this lands is the one to watch.
+
+## 6. Tagging and publishing 0.3.0
+
+Nothing was tagged or published. `__about__.py` says `0.3.0` and the changelog carries a dated
+`[0.3.0]` section. The release procedure (packaging standards §6), from the SetSpec repository:
+
+```bash
+cd ~/ai/suite/py/SetSpec
+git add -A
+git commit -m "feat(setspec): freeze v1.0, publish JSON Schema and goldens (Phase 4, 0.3.0)"
+git push origin main
+# wait for CI to be green on main, then:
+git tag -a v0.3.0 -m "setspec 0.3.0 — v1.0 contracts frozen; JSON Schema and goldens published"
+git push origin v0.3.0
+# release.yml builds, tests the wheel, publishes via Trusted Publishing and creates the release.
+# Verify:
+python -m venv /tmp/setspec-check && /tmp/setspec-check/bin/pip install setspec==0.3.0 && \
+  /tmp/setspec-check/bin/python -c "from setspec import PUBLISHED_SCHEMAS, golden_payloads, SchemaVersion; \
+  print(len(PUBLISHED_SCHEMAS), len(golden_payloads('capability.evidence', SchemaVersion(1, 0))))"
+```
+
+FreeWeight's `pyproject.toml` already pins `setspec>=0.3,<0.4`; its `install-check` job cannot
+resolve until this release is on PyPI.

{setspec-0.2.0 → setspec-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: setspec
-Version: 0.2.0
+Version: 0.4.0
 Summary: Every versioned data contract that crosses an application boundary: benchmark results, capability evidence, event/error envelopes, prompt records.
 Project-URL: Homepage, https://github.com/JPKell/SetSpec
 Project-URL: Documentation, https://github.com/JPKell/SetSpec/tree/main/docs
@@ -21,25 +21,31 @@ Requires-Dist: jinja2<4,>=3.1
 Requires-Dist: pydantic<3,>=2.9
 Provides-Extra: dev
 Requires-Dist: import-linter<3,>=2.0; extra == 'dev'
+Requires-Dist: jsonschema<5,>=4.23; extra == 'dev'
 Requires-Dist: mypy<2,>=1.11; extra == 'dev'
 Requires-Dist: pytest-cov<6,>=5; extra == 'dev'
 Requires-Dist: pytest-randomly<4,>=3; extra == 'dev'
 Requires-Dist: pytest<10,>=9.0.3; extra == 'dev'
 Requires-Dist: respx<1,>=0.21; extra == 'dev'
 Requires-Dist: ruff<1,>=0.6; extra == 'dev'
+Requires-Dist: types-jsonschema<5,>=4.23; extra == 'dev'
 Description-Content-Type: text/markdown
 
 # SetSpec
 
 Every versioned data contract that crosses an application boundary: benchmark results, capability evidence, event/error envelopes, prompt records.
 
-**Status:** `0.2.0` — Phases 1–2 complete. The envelope, version negotiation and serialization core
-are implemented and tested; `model.identity`, `machine.profile`, `benchmark.result`,
-`benchmark.run_summary`, `capability.evidence` and `benchmark.evidence_bundle` are registered in
-`SUPPORTED_SCHEMAS`, but **draft**: Phase 4 may still reshape a field once FreeWeight has produced
-real results against these payloads. Which schemas are still provisional is readable at runtime
-from `setspec.DRAFT_SCHEMAS`, not only stated here — freezing one is a deletion from that set.
-Event and error envelopes arrive in Phase 3. See the
+**Status:** `0.3.0` — Phases 1–2, 3A and 4 complete, and **the v1.0 contracts are frozen.** Eight
+payload types are published at `1.0` — `model.identity`, `machine.profile`, `benchmark.result`,
+`benchmark.run_summary`, `capability.evidence`, `benchmark.evidence_bundle`,
+`benchmark.goal_pack` and `benchmark.calibration_report` — each with generated JSON Schema and at
+least three golden payloads shipped as package data. `setspec.DRAFT_SCHEMAS` is empty, which is
+where the freeze is readable at runtime rather than only stated here; from now on a new optional
+field is a minor bump and anything else is a major, enforced by a snapshot diff in CI.
+
+The [schema catalogue](docs/schemas.md) lists every payload type, its artifacts, and the
+cross-field rules the JSON Schema cannot express. Event and error envelopes (Phase 3) and prompt
+records (Phase 5) are not yet written and are therefore not part of the freeze. See the
 [development plan](docs/packages/setspec/development-plan.md) for what each phase adds.
 
 Part of the **Local AI Suite**.

{setspec-0.2.0 → setspec-0.4.0}/README.md

@@ -2,13 +2,17 @@
 
 Every versioned data contract that crosses an application boundary: benchmark results, capability evidence, event/error envelopes, prompt records.
 
-**Status:** `0.2.0` — Phases 1–2 complete. The envelope, version negotiation and serialization core
-are implemented and tested; `model.identity`, `machine.profile`, `benchmark.result`,
-`benchmark.run_summary`, `capability.evidence` and `benchmark.evidence_bundle` are registered in
-`SUPPORTED_SCHEMAS`, but **draft**: Phase 4 may still reshape a field once FreeWeight has produced
-real results against these payloads. Which schemas are still provisional is readable at runtime
-from `setspec.DRAFT_SCHEMAS`, not only stated here — freezing one is a deletion from that set.
-Event and error envelopes arrive in Phase 3. See the
+**Status:** `0.3.0` — Phases 1–2, 3A and 4 complete, and **the v1.0 contracts are frozen.** Eight
+payload types are published at `1.0` — `model.identity`, `machine.profile`, `benchmark.result`,
+`benchmark.run_summary`, `capability.evidence`, `benchmark.evidence_bundle`,
+`benchmark.goal_pack` and `benchmark.calibration_report` — each with generated JSON Schema and at
+least three golden payloads shipped as package data. `setspec.DRAFT_SCHEMAS` is empty, which is
+where the freeze is readable at runtime rather than only stated here; from now on a new optional
+field is a minor bump and anything else is a major, enforced by a snapshot diff in CI.
+
+The [schema catalogue](docs/schemas.md) lists every payload type, its artifacts, and the
+cross-field rules the JSON Schema cannot express. Event and error envelopes (Phase 3) and prompt
+records (Phase 5) are not yet written and are therefore not part of the freeze. See the
 [development plan](docs/packages/setspec/development-plan.md) for what each phase adds.
 
 Part of the **Local AI Suite**.

{setspec-0.2.0 → setspec-0.4.0}/docs/README.md

@@ -4,5 +4,6 @@ This directory contains the maintained documentation for SetSpec.
 
 ## Documents
 
+- [Schema catalogue](schemas.md) — every payload type and version, with its artifacts
 - [Development plan](packages/setspec/development-plan.md)
 - [Specification](packages/setspec/spec.md)

{setspec-0.2.0 → setspec-0.4.0}/docs/packages/setspec/spec.md

@@ -124,9 +124,11 @@ RESERVED_ROOTS: frozenset[str]        # roots valid ONLY as a specialization; {"
 validate_capability(capability_id: str) -> CapabilityId
 is_known_capability(capability_id: str) -> bool
 
-# Schema artefacts
-json_schema_for(schema: str, version: SchemaVersion) -> dict
-golden_payloads(schema: str, version: SchemaVersion) -> list[dict]
+# Schema artefacts (package data; every published version keeps both for the life of its major)
+PUBLISHED_SCHEMAS: Mapping[str, tuple[SchemaVersion, ...]]   # exact versions with committed artefacts
+json_schema_for(schema: str, version: SchemaVersion) -> dict   # the committed JSON Schema snapshot
+golden_payloads(schema: str, version: SchemaVersion) -> list[dict]   # ≥ 3 per version, by name
+golden_names(schema: str, version: SchemaVersion) -> tuple[str, ...]  # the same order as above
 
 # Prompts (ADR-0028) — one implementation of a determinism contract, not three
 class PromptLibrary:

setspec-0.4.0/docs/prompts-adoption-checklist.md

@@ -0,0 +1,86 @@
+# `setspec.prompts` Adoption Checklist
+
+For an application that owns a prompt pack today (its own `PromptRecord`/`load_pack`-shaped code,
+or an equivalent) and wants to move onto `setspec.prompts` instead of maintaining a second copy of
+prompt-record loading, hashing and rendering. Written against the first adopter, FreeWeight P12
+(§5 below is that migration, completed); the steps are what the *next* adopter — IdeaPress or
+LoadCoach, whichever writes its first prompt pack second — should follow.
+
+## 1. What to delete outright
+
+Anything that duplicates, byte-for-byte in behaviour, what `setspec.prompts` now provides:
+`PromptRecord`/`VariableSpec`/`RenderedPrompt`/`PromptReference`/`PromptLibrary`, `load_record()`,
+`load_pack()`, `build_manifest()`/`write_manifest()`, `prompt_record_hash()`/`prompt_subset_hash()`/
+`pack_hash()`, and the sandboxed `StrictUndefined` Jinja2 environment. If your pack format matches
+prompt-management-standards §2.1–§3 (schema_version/prompt_id/version/template/purpose/metadata,
+a manifest with pack_id/pack_version/prompts/pack_sha256), there should be nothing left to keep.
+
+## 2. The one thing every adopter supplies itself: pack location
+
+`load_pack(root: Path, *, override_root: Path | None = None)` takes `root` as a required
+positional argument — the package has no opinion about where an application's pack lives on disk,
+and no default. If your call sites currently call a zero-argument `load_pack()`, write a one-line
+wrapper in your own package that supplies your pack's fixed location and forwards everything else:
+
+```python
+from pathlib import Path
+from setspec.prompts import load_pack as _setspec_load_pack, PromptLibrary
+
+PACK_ROOT = Path(__file__).resolve().parent.parent / "prompts"
+
+def load_pack(root: Path = PACK_ROOT, *, override_root: Path | None = None) -> PromptLibrary:
+    return _setspec_load_pack(root, override_root=override_root)
+```
+
+Do this even if every current call site could be edited to pass `root` explicitly — a re-export
+shim at your old import path means the migration touches one file instead of every call site, and
+is the difference between a multi-file diff and a four-file one (§5).
+
+## 3. Behaviour to re-verify, not assume
+
+* **Hashing is over `canonical_json`, not file bytes.** If you ever hashed a record file's raw
+  bytes, that is not the same value `prompt_record_hash()` produces; re-indenting a file must not
+  change its hash, and canonical-JSON hashing already guarantees that here.
+* **Declared-and-unused is a load-time error, same as undeclared-and-used.** A variable in the
+  record's `variables` block that the template never references fails validation — not just the
+  reverse. Audit existing records for this before the first `load_pack()` call in CI.
+  See [`goldens/prompt.record/1.0/full.json`](../src/setspec/goldens/prompt.record/1.0/full.json)
+  for a record where every declared variable — including a non-string one with `min`/`max` — is
+  used in the template.
+* **The Jinja2 environment has no loader.** `{% include %}` / `{% extends %}` fail at render time
+  (`PromptRenderError`), not load time — a template that used either against your old environment
+  needs rewriting, not just re-pointing.
+* **`pack_hash()` is provenance, never a fingerprint input** (ADR-0028 §1). If your evidence
+  payloads folded a pack-wide hash into a capability fingerprint, stop — only
+  `prompt_subset_hash()` over the *specific prompts a benchmark declares* belongs there.
+
+## 4. Tests that must still pass, unchanged
+
+Whatever exercises your prompt pack today — rendering determinism, unknown-variable rejection,
+sandboxing (no filesystem reach, no dunder attribute access from a template) — should still pass
+against `setspec.prompts` with no change to the test's assertions, only to its imports. A test that
+needed a new assertion to pass found a real behaviour difference; treat that as a bug to resolve,
+not a test to relax, before shipping the migration.
+
+## 5. Verification: FreeWeight P12 (completed)
+
+FreeWeight adopted `setspec.prompts` in the same change that added it to `setspec` (Phase 5 pulled
+FreeWeight's own P12 forward, rather than leaving `setspec.prompts` unused until a later phase).
+The proof this checklist rests on:
+
+1. **Before**: captured the pack hash and every individual record hash from FreeWeight's original,
+   pre-migration `freeweight.services.prompts` — `pack_hash: sha256:b1b0ffd0a5941fee5e0013d2a826732ea02a285b229bdc006ebd6dd25ff4ceb4`
+   plus 18 individual record hashes spanning `benchmarks.agent.goal` through `goals.judge.rubric`.
+2. **After, direct**: loaded the same pack through `setspec.prompts.load_pack()` directly and
+   recomputed every hash — identical.
+3. **After, through the shim**: rewrote `freeweight/services/prompts.py` as a thin re-export over
+   `setspec.prompts` (§2's pattern, `PACK_ROOT` default preserved) and recomputed through
+   FreeWeight's own public import path — identical again.
+4. **Full gate, unchanged**: FreeWeight's complete pre-PR gate (ruff, mypy strict over 265 files,
+   import-linter, pytest) passed with no test assertions changed — 2297 passed, 28 skipped, 21
+   deselected (performance). The only non-shim source edit was 3 import statements in
+   `tests/security/test_goal_pack_import.py`, which reached into the module's private
+   `_environment()` directly for a sandboxing assertion; every other call site needed no change.
+
+Final diff: 4 files changed. The next adopter should expect a diff of similar shape — one shim
+file, a dependency bump, and an edit only where a test reached past the public API.