foundry-testing-actor 0.1.1__tar.gz → 0.2.0__tar.gz

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: foundry-testing-actor
-Version: 0.1.1
+Version: 0.2.0
 Summary: Runs a headless Claude Code black-box test-authoring session against one capability's own testing repo — a papeete-actor for one use, with the capability supplied by a sidecar.
 Project-URL: Homepage, https://github.com/papeete-hub/foundry-testing-actor
 Author-email: Papeete Consulting <yoann.remy@outlook.com>
@@ -77,7 +77,7 @@ ground_in:
 and, beside it, a four-line Dockerfile:
 
 ```dockerfile
-FROM ghcr.io/papeete-hub/foundry-testing-actor:0.1.1
+FROM ghcr.io/papeete-hub/foundry-testing-actor:0.2.0
 RUN pip install --no-cache-dir kpack==2.0.1 kontract==0.1.0   # what this sidecar's ground_in names
 COPY actor-agentic-context.yaml /actor/
 RUN foundry-testing-actor render-cards /actor && foundry-testing-actor lint /actor
@@ -137,9 +137,11 @@ Both name the same engine. `Actor.judge()` hands it the door id, and it dispatch
 
 ```
 orchestration ──▶ testing:        propose-acceptance {task_id, title, definition_of_done, components, context?}
-              ◀──                 {expectations: [{id, statement, handle, component?}], open_questions: [...]}
-orchestration ──▶ implementation: assess-task {..., acceptance_surface: expectations}
+              ◀──                 {expectations: [{id, statement, handle, component?}],
+                                   datasets: [{expectation, via, …}], open_questions: [...]}
+orchestration ──▶ implementation: assess-task {..., acceptance_surface: expectations}   ← never datasets
               ◀──                 {feasible, objections, commitments}
+orchestration ──▶ testing:        test-task {..., acceptance_surface, datasets}
 ```
 
 A **query**, read-only: a clone of the testing repo, a clone of the implementation repo's **default
@@ -154,9 +156,25 @@ implementer to commit to or object to; and anything the task does not determine
 `open_questions`, **never** into an invented statement. A non-empty `open_questions` stops the round
 and reaches a human. See `adr/ADR-FTA-0002-*.md`.
 
-The reply is **projected** onto `{expectations, open_questions}` — a session's stray keys never
-reach the orchestrating actor dressed as contract — after checking that every expectation carries
-an `id`, a `statement` and a `handle`, with ids unique. `open_questions` is always a list.
+**Every expectation says how its data comes to exist** (`adr/ADR-FTA-0003-*.md`), in a separate
+`datasets` list that is **private to this actor** — the implementer never sees it. One entry per
+expectation, `via` one of:
+
+| `via` | the test's state comes from | must carry |
+|---|---|---|
+| `none` | nothing — written down, never inferred from a missing entry | — |
+| `command` | the capability's own commands, called by the test first (preferred) | `steps` |
+| `event` | upstream events the test publishes; broker address in `AMQP_URL` | `steps` |
+| `seed` | pre-existing data that is itself under test | `because`, `provided_by` (a proposed expectation) |
+
+Never a component's storage. Anything a dataset needs that the contract does not already offer — a
+seed, a command this task introduces — must also be proposed as an **expectation**, because that is
+the only thing the implementer assesses.
+
+The reply is **projected** onto `{expectations, datasets, open_questions}` — a session's stray keys
+never reach the orchestrating actor dressed as contract — after checking that every expectation
+carries an `id`, a `statement` and a `handle`, with ids unique, and has exactly one valid dataset.
+`open_questions` is always a list.
 
 ### `test-task`
 

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/README.md

@@ -54,7 +54,7 @@ ground_in:
 and, beside it, a four-line Dockerfile:
 
 ```dockerfile
-FROM ghcr.io/papeete-hub/foundry-testing-actor:0.1.1
+FROM ghcr.io/papeete-hub/foundry-testing-actor:0.2.0
 RUN pip install --no-cache-dir kpack==2.0.1 kontract==0.1.0   # what this sidecar's ground_in names
 COPY actor-agentic-context.yaml /actor/
 RUN foundry-testing-actor render-cards /actor && foundry-testing-actor lint /actor
@@ -114,9 +114,11 @@ Both name the same engine. `Actor.judge()` hands it the door id, and it dispatch
 
 ```
 orchestration ──▶ testing:        propose-acceptance {task_id, title, definition_of_done, components, context?}
-              ◀──                 {expectations: [{id, statement, handle, component?}], open_questions: [...]}
-orchestration ──▶ implementation: assess-task {..., acceptance_surface: expectations}
+              ◀──                 {expectations: [{id, statement, handle, component?}],
+                                   datasets: [{expectation, via, …}], open_questions: [...]}
+orchestration ──▶ implementation: assess-task {..., acceptance_surface: expectations}   ← never datasets
               ◀──                 {feasible, objections, commitments}
+orchestration ──▶ testing:        test-task {..., acceptance_surface, datasets}
 ```
 
 A **query**, read-only: a clone of the testing repo, a clone of the implementation repo's **default
@@ -131,9 +133,25 @@ implementer to commit to or object to; and anything the task does not determine
 `open_questions`, **never** into an invented statement. A non-empty `open_questions` stops the round
 and reaches a human. See `adr/ADR-FTA-0002-*.md`.
 
-The reply is **projected** onto `{expectations, open_questions}` — a session's stray keys never
-reach the orchestrating actor dressed as contract — after checking that every expectation carries
-an `id`, a `statement` and a `handle`, with ids unique. `open_questions` is always a list.
+**Every expectation says how its data comes to exist** (`adr/ADR-FTA-0003-*.md`), in a separate
+`datasets` list that is **private to this actor** — the implementer never sees it. One entry per
+expectation, `via` one of:
+
+| `via` | the test's state comes from | must carry |
+|---|---|---|
+| `none` | nothing — written down, never inferred from a missing entry | — |
+| `command` | the capability's own commands, called by the test first (preferred) | `steps` |
+| `event` | upstream events the test publishes; broker address in `AMQP_URL` | `steps` |
+| `seed` | pre-existing data that is itself under test | `because`, `provided_by` (a proposed expectation) |
+
+Never a component's storage. Anything a dataset needs that the contract does not already offer — a
+seed, a command this task introduces — must also be proposed as an **expectation**, because that is
+the only thing the implementer assesses.
+
+The reply is **projected** onto `{expectations, datasets, open_questions}` — a session's stray keys
+never reach the orchestrating actor dressed as contract — after checking that every expectation
+carries an `id`, a `statement` and a `handle`, with ids unique, and has exactly one valid dataset.
+`open_questions` is always a list.
 
 ### `test-task`
 

foundry_testing_actor-0.2.0/adr/ADR-FTA-0003-every-expectation-says-how-its-data-comes-to-exist.md

@@ -0,0 +1,136 @@
+---
+id: ADR-FTA-0003
+title: "Every expectation says how its data comes to exist — privately, and anything the contract cannot already provide becomes an expectation"
+status: Proposed
+date: 2026-09-14
+supersedes: []
+references:
+  - ../src/foundry_testing_actor/cards/actor-data.yaml
+  - ../src/foundry_testing_actor/cards/actor-synchronous-messaging.yaml
+  - ../src/foundry_testing_actor/engine.py
+  - ./ADR-FTA-0002-this-actors-half-of-the-three-amigos-round.md
+  - https://github.com/papeete-hub/foundry-task-orchestration-actor/blob/main/adr/ADR-FTOA-0003-datasets-travel-to-the-tester-only.md
+  - https://github.com/papeete-hub/foundry-implementation-actor/blob/main/adr/ADR-FIA-0004-the-three-amigos-round.md
+---
+
+# ADR-FTA-0003 — Every expectation says how its data comes to exist
+
+## Context
+
+ADR-FTA-0002 made this actor propose its acceptance surface before anything is built, and told it
+to **propose a concrete value** wherever a test needs one the task leaves unnamed: "a fixture's id,
+a seeded record". It said nothing about *where the data a test runs against comes from*, and so the
+default was the one the whole round exists to remove — data that already exists, found by reading.
+
+The second live run showed it. Asked for a deliberately vague task, the propose session read the
+implementation's `stub/fixtures/anchors.json`, proposed the three anchors it found there at their
+existing ids, and stopped the round on a question about how a consumer would ever learn those ids.
+Every expectation depended on pre-existing records. None needed to: the capability's own contract
+has a command that creates an anchor.
+
+Two things are true of every capability this actor will ever serve, not only of the one that
+showed it:
+
+1. **A black-box test can almost always build its own state** through the contract under test —
+   the commands it exposes, the upstream events it consumes. A test that does so needs no agreed
+   fixture, cannot be broken by someone regenerating one, and does not care what the database
+   already holds.
+2. **Where it cannot**, that is a fact the implementer has to know about: a command the task must
+   introduce, or seeded data at fixed values that someone has to put there and keep there.
+
+A first design put the dataset inside each shared expectation (`given`). It was rejected in
+review: for the ordinary case — build state through what the contract already offers — the
+implementer has no use for it, and the one case that does concern the implementer is better stated
+as an expectation of its own than buried in a private detail of another.
+
+## Decision
+
+**1. `propose-acceptance` answers a second list, `datasets`, private to this actor.** One entry per
+expectation, keyed by the expectation's `id`:
+
+```json
+{"expectation": "E3",
+ "via": "command",
+ "steps": ["POST $BACKEND_URL/anchors {valid mint payload} → internal_id",
+           "POST $BACKEND_URL/anchors/{internal_id}/archive"]}
+```
+
+`via` is one of four, and nothing else:
+
+| `via` | the test's state comes from | the entry must carry |
+|---|---|---|
+| `none` | nothing — the expectation needs no prior state | — (saying so is the point) |
+| `command` | the capability's own commands, called by the test | `steps` |
+| `event` | upstream events the test publishes on the bus | `steps` |
+| `seed` | data that exists before the test runs | `because`, and `provided_by` naming the expectation that states the seed |
+
+**2. Every expectation has exactly one dataset — and this is checked, not asked.** The engine
+refuses a proposal where an expectation has no entry, an entry names an unknown expectation or has
+two, `via` is outside the four, `command`/`event` has no `steps`, or `seed` has no `because` or a
+`provided_by` that is not a proposed expectation. The refusal happens at this door, where a
+proposal is still cheap; `none` must be written, not inferred, because "needs nothing" is a claim
+that can be wrong.
+
+**3. Anything a dataset needs that the contract does not already offer is promoted to an
+expectation.** A seed is the plainest case: `provided_by` must point at an expectation whose
+`statement` and `handle` describe the seeded data — which is what the implementer assesses and
+commits to. The same holds for a command or a subscription the task itself introduces: it is
+something the increment must deliver, so it is an expectation, and a dataset may then use it.
+`seed` is for data whose existence *is* what is being tested (a stub's canned records); the prompt
+says to prefer `command` and `event` everywhere else.
+
+**4. `datasets` never reaches the implementer.** The shared expectation stays `{id, statement,
+handle}` with optional `component`. The orchestrating actor relays `datasets` unread to
+`test-task`, and never to `assess-task` or `implement-task` (ADR-FTOA-0003).
+
+**5. `test-task` takes the datasets and builds state exactly as they say.** `test-task-cmd` gains an
+optional `datasets`. Each test arranges its state as its expectation's dataset states; it does not
+rely on records it did not create, except through a `seed` dataset. Without datasets — a caller that
+skipped round 0 — the prompt still says to build state through the contract's commands.
+
+**6. How a test reaches the bus is a convention, like `<COMPONENT>_URL`.** An `event` dataset
+publishes to the broker whose address arrives in `AMQP_URL`. The orchestrating actor sets it on the
+test Job when its use declares one (ADR-FTOA-0003).
+
+**7. No `via: database`.** A test never writes a component's storage. Doing so would bind the
+tests to the implementer's private schema — the coupling black-box testing exists to prevent — and
+a state the contract cannot produce is a question about the contract, which belongs in
+`open_questions`.
+
+## Rationale
+
+**Why a separate list rather than a field on the expectation.** The expectation is the part of the
+proposal two actors agree on. How a test sets itself up is how this actor does its job; putting it
+on the shared object would put it in front of an implementer that can only ignore it, and — worse —
+would let a genuine dev↔test dependency ride along unseen instead of being stated where it gets
+assessed.
+
+**Why keyed by id and validated here.** It is the same reason ids are unique (ADR-FTA-0002 §6): the
+orchestrating actor and a verdict both address expectations by id. A dataset that names nothing, or
+an expectation with none, is exactly the silent gap this ADR closes, and this door is the last place
+it costs nothing to refuse.
+
+**Why relay through the orchestrator instead of re-deriving at `test-task`.** The plan a test is
+written against should be the one that existed when the implementer agreed to the surface. A
+`test-task` session re-deciding how to arrange data, after the increment is built, would be free to
+read state off the build again — the *after* ADR-FIA-0004 excludes.
+
+**Why the promotion rule rather than trusting the dataset.** A private plan is only safe if it can
+not hide a demand on someone else. The rule makes every such demand public in the one shape the
+implementer already assesses. Where a session breaks it anyway — assumes a command that does not
+exist — the first attempt's tests fail against the real deployment, and remediation names them.
+
+## Consequences
+
+- **The completion shape of `propose-acceptance` changes**: `datasets` is required. A use pinned to
+  an older image keeps the older door; a use that takes this version gets it through its `FROM`
+  line and `render-cards`, with nothing written by hand. Released as `0.2.0`.
+- **A proposal can now be refused for a missing dataset**, which the orchestrating actor reports as
+  a round-0 failure — `testing did not answer`, with the reason. That is intended: a proposal with
+  no plan for its data is not a proposal.
+- **`remediation_context` gains a failure class**: a dataset that could not be built. The
+  `test-task` prompt already asks the session to judge whether a failure is the test's or the
+  implementation's; it now also names this one.
+- **Not decided here**: whether datasets should be committed beside the tests that use them (they
+  are in the tests' code, and in the transcript), and whether `open_questions` should be split into
+  blocking and non-blocking — raised by the same run, still open.

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/adr/README.md

@@ -17,6 +17,7 @@ are cited rather than restated.
 |----|-------|--------|
 | [ADR-FTA-0001](./ADR-FTA-0001-the-machinery-leaves-the-capability.md) | The machinery leaves the capability — a published testing actor any capability can instantiate | Proposed |
 | [ADR-FTA-0002](./ADR-FTA-0002-this-actors-half-of-the-three-amigos-round.md) | This actor's half of the three amigos round — propose before anything is built, read-only, and hand the unknowns to a human | Proposed |
+| [ADR-FTA-0003](./ADR-FTA-0003-every-expectation-says-how-its-data-comes-to-exist.md) | Every expectation says how its data comes to exist — privately, and anything the contract cannot already provide becomes an expectation | Proposed |
 
 ## Authoring
 

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/examples/ACME.PARTS.CAP.SUP.007.WID-testing/Dockerfile

@@ -13,8 +13,8 @@
 # The default names the product registry rather than GHCR, as the implementation actor's example
 # does, because it is the copy an in-cluster builder can pull (ADR-FIA-0006's arrangement, copied).
 # Override it if you are somewhere else:
-#     docker build --build-arg ACTOR_IMAGE=ghcr.io/papeete-hub/foundry-testing-actor:0.1.1 .
-ARG ACTOR_IMAGE=papeetefoundry.azurecr.io/foundry/foundry-testing-actor:0.1.1
+#     docker build --build-arg ACTOR_IMAGE=ghcr.io/papeete-hub/foundry-testing-actor:0.2.0 .
+ARG ACTOR_IMAGE=papeetefoundry.azurecr.io/foundry/foundry-testing-actor:0.2.0
 FROM ${ACTOR_IMAGE}
 
 # THE KNOWLEDGE TOOLS THIS SIDECAR NAMES, and the one thing the base image deliberately does not

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/examples/README.md

@@ -22,7 +22,7 @@ repo and publishes them as a runnable image.**
    definition_of_done,             ground the session                  ─▶ the knowledge tools
    components, context?            propose expectations                    the sidecar names
                                      (Read/Glob/Grep only)
-  ◀──────────────────────────────  {expectations: [{id, statement, handle}], open_questions}
+  ◀──────────────────────────────  {expectations, datasets (private), open_questions}
 
   POST /test-task           ─────▶ clone testing repo → test/TASK-NNN
    ...the same, plus               clone implementation → impl/TASK-NNN  ─▶ recompute images under test
@@ -75,8 +75,8 @@ python scripts/probe_grounding.py examples/ACME.PARTS.CAP.SUP.007.WID-testing
 **Build it** — this needs the base image, from a registry you can pull, or one you built:
 
 ```bash
-uv build && docker build -f docker/Dockerfile -t foundry-testing-actor:0.1.1 .
-docker build --build-arg ACTOR_IMAGE=foundry-testing-actor:0.1.1 -t acme-wid-tester \
+uv build && docker build -f docker/Dockerfile -t foundry-testing-actor:0.2.0 .
+docker build --build-arg ACTOR_IMAGE=foundry-testing-actor:0.2.0 -t acme-wid-tester \
   examples/ACME.PARTS.CAP.SUP.007.WID-testing
 docker run --rm acme-wid-tester foundry-testing-actor lint /actor
 ```
@@ -102,10 +102,24 @@ curl -X POST http://<actor>/propose-acceptance -H 'Content-Type: application/jso
     "handle": {"component": "stub", "base_url_env": "STUB_URL", "method": "GET",
                "path": "/widgets/{id}",
                "ids": {"ACTIVE": "w-0001", "ARCHIVED": "w-0002", "RETIRED": "w-0003"}},
-    "component": "stub"}],
+    "component": "stub"},
+   {"id": "retired-widget-refuses-update",
+    "statement": "PATCH /widgets/{id} on a retired widget answers 409 WIDGET_RETIRED",
+    "handle": "PATCH $BACKEND_URL/widgets/{id}", "component": "backend"}],
+ "datasets": [
+   {"expectation": "stub-seeded-widgets", "via": "seed",
+    "because": "the stub's canned widgets are themselves what is under test",
+    "provided_by": "stub-seeded-widgets"},
+   {"expectation": "retired-widget-refuses-update", "via": "command",
+    "steps": ["POST $BACKEND_URL/widgets {valid payload} → id",
+              "POST $BACKEND_URL/widgets/{id}/retire"]}],
  "open_questions": []}
 ```
 
+`datasets` stays with this actor: the orchestrating actor relays it to `test-task` and never to the
+implementer. The second test creates the widget it asserts on, so it needs no agreed fixture; the
+first relies on seeded data, which is why that data is an expectation the implementer assesses.
+
 The ids are **proposed**, not discovered: the implementer commits to them at its `assess-task`
 door, or objects. Then `test-task` is sent the agreed surface as `acceptance_surface`, and
 answers `{"accepted": true, "branch": "test/TASK-014", "images":

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "foundry-testing-actor"
-version = "0.1.1"
+version = "0.2.0"
 description = "Runs a headless Claude Code black-box test-authoring session against one capability's own testing repo — a papeete-actor for one use, with the capability supplied by a sidecar."
 readme = "README.md"
 requires-python = ">=3.11"

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/src/foundry_testing_actor/cards/actor-data.yaml

@@ -45,6 +45,18 @@ items:
       reaches it, and optionally the `component` it belongs to. Proposed BEFORE anything is built,
       so a concrete value in a handle is a proposal the implementer commits to or objects to, never
       a report of what was found
+  - name: datasets
+    type: list
+    description: >-
+      PRIVATE TO THIS ACTOR — how the state each proposed expectation's test runs against comes to
+      exist, one object per expectation: `expectation` (its id) and `via`, one of `none` (needs no
+      prior state), `command` (the test calls the capability's own commands first; with `steps`),
+      `event` (the test publishes upstream events, broker address in AMQP_URL; with `steps`) or
+      `seed` (pre-existing data that is itself under test; with `because`, and `provided_by`
+      naming the proposed expectation that states it). Never a component's storage. Relayed
+      unread to test-task by the orchestrating actor and never shown to the implementer: anything
+      a dataset needs that the contract does not already offer is proposed as an expectation too
+      (ADR-FTA-0003)
   - name: open_questions
     type: list
     description: >-

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/src/foundry_testing_actor/cards/actor-message.yaml

@@ -12,16 +12,17 @@ messages:
 
   - name: acceptance-proposed-result
     intent: >-
-      answer with the proposed expectations, each with a stable id, a statement and a handle, and
-      the questions the task leaves open
-    references: [expectations, open_questions]
+      answer with the proposed expectations, each with a stable id, a statement and a handle;
+      how the state each one's test runs against comes to exist (private to this actor); and the
+      questions the task leaves open
+    references: [expectations, datasets, open_questions]
     optional: [open_questions]
 
   - name: test-task-cmd
     intent: ask this actor to author black-box tests for the published image(s) of a TASK-NNN card
     references: [task_id, title, definition_of_done, components, context, remediation_context,
-                 acceptance_surface]
-    optional: [context, remediation_context, acceptance_surface]
+                 acceptance_surface, datasets]
+    optional: [context, remediation_context, acceptance_surface, datasets]
 
   - name: test-task-result
     intent: report that the tests were written, pushed, and their touched components' test images published

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/src/foundry_testing_actor/cards/actor-synchronous-messaging.yaml

@@ -19,14 +19,16 @@ actions:
     means: >-
       the door for "author black-box tests for TASK-NNN of the capability I serve, now that its
       increment is built". Send it here with the task id, title, definition of done, components,
-      and (optionally) context, remediation_context and the agreed `acceptance_surface` — the
+      and (optionally) context, remediation_context, the agreed `acceptance_surface` and the
+      `datasets` I proposed beside it — the
       caller supplies everything, no task card is looked up here. I clone my capability's testing
       repository into my own private copy, and read-only clone its implementation repository at
       impl/TASK-NNN solely to recompute, by convention, the image refs published there (never
       passed to me, and never shown to my session); ground myself in that capability's own
       standing context; and extend the persistent suite under each named component's declared
       tests root — asserting every agreed expectation by its id, through its handle, where a
-      surface was sent. I never bring up the images, hit a live endpoint, or render a verdict.
+      surface was sent, each test building its state as its dataset says, or through the
+      contract's own commands where none was sent. I never bring up the images, hit a live endpoint, or render a verdict.
       Then I commit and push to test/TASK-NNN, and build each touched component's test image in
       the cluster's shared buildkit and push it to the registry, named and versioned by
       convention. I never open a pull request — an orchestrating actor runs that image and does.
@@ -51,6 +53,10 @@ queries:
       reaches it (the component, the endpoint, the event routing key, the environment variable its
       base URL arrives in — with a concrete value proposed wherever a test needs one the task does
       not name, for the implementer to commit to or object to), and optionally its `component`;
+      for EVERY expectation, a dataset saying how the state its test needs comes to exist — `none`,
+      `command` (preferred), `event`, or `seed` only where pre-existing data is itself under test
+      and a proposed expectation states it — kept private to me: anything a dataset needs that the
+      contract does not already offer is proposed as an expectation too (ADR-FTA-0003);
       and the open questions the task and the standing context leave undetermined, each a string
       or an `about`/`question` object, never an invented answer. Any open question is meant to
       stop the round and reach a human rather than a session.

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/src/foundry_testing_actor/engine.py

@@ -86,6 +86,14 @@ DEFAULT_MAX_TURNS = 60
 DEFAULT_PROPOSE_TIMEOUT_S = 900
 DEFAULT_PROPOSE_MAX_TURNS = 30
 
+# How the state a test runs against comes to exist (ADR-FTA-0003). Closed on purpose: a fifth way
+# is a decision, not a session's improvisation — and `database` is deliberately not one of them.
+DATASET_VIA = ("none", "command", "event", "seed")
+
+# Where an `event` dataset reaches the bus: the broker address arrives in this variable, set on the
+# test Job by the orchestrating actor, the way <COMPONENT>_URL is (ADR-FTA-0003 §6, ADR-FTOA-0003).
+BROKER_URL_ENV = "AMQP_URL"
+
 TEST_TOOLS = "Bash,Read,Edit,Write,Glob,Grep"
 PROPOSE_TOOLS = "Read,Glob,Grep"
 
@@ -293,8 +301,8 @@ def _proposal(judged: dict) -> dict:
     the framework (ADR-PAS-0009), so a session's helpful `"notes"` key beside its answer would not
     be refused — it would travel to the orchestrating actor looking like part of a contract three
     packages share, which is worse. And the day this door names a second outcome, the framework
-    closes the schema and that same key refuses the whole proposal. Only the two fields the message
-    references survive; everything else a session said is in the transcript.
+    closes the schema and that same key refuses the whole proposal. Only the three fields the
+    message references survive; everything else a session said is in the transcript.
 
     WHAT IS CHECKED, AND WHY IT IS NOT MORE. `expectations` must be a list of objects, each with an
     `id` and a `statement`, a `handle` key, and ids unique within the proposal — because a later
@@ -328,7 +336,81 @@ def _proposal(judged: dict) -> dict:
                 f"expectation by its id, so an id two expectations share names neither"
             )
         seen.add(identifier)
-    return {"expectations": expectations, "open_questions": _as_list(judged.get("open_questions"))}
+    datasets = _datasets(judged.get("datasets"), [str(e["id"]) for e in expectations])
+    return {"expectations": expectations, "datasets": datasets,
+            "open_questions": _as_list(judged.get("open_questions"))}
+
+
+def _datasets(raw, expectation_ids: list[str]) -> list:
+    """Every expectation's dataset, checked — how the state its test runs against comes to exist.
+
+    PRIVATE TO THIS ACTOR (ADR-FTA-0003). The implementer never sees this list; the orchestrating
+    actor relays it to `test-task` unread. So the checks that matter are the ones that stop a
+    private plan from hiding a gap:
+
+    - exactly one entry per proposed expectation, none for an expectation nobody proposed — "needs
+      nothing" is `via: none`, written down, never an absent entry;
+    - `via` is one of `DATASET_VIA`, and nothing else — in particular never a component's database;
+    - `command` and `event` say how, as `steps`;
+    - `seed` says why (`because`) and points `provided_by` at a PROPOSED expectation that states
+      the seeded data — because data that must exist before a test runs is something the
+      implementer has to put there, and the only place the implementer looks is the expectations.
+
+    What a step SAYS is not checked, for the reason a handle's content is not: it depends on what
+    is being addressed, and a test failing against the real deployment is the check that can.
+    """
+    if not expectation_ids and raw in (None, []):
+        return []
+    if raw is None:
+        raise EngineError(
+            "the proposal carries no `datasets` — every expectation says how the state its test "
+            "runs against comes to exist, even when that is `via: none` (ADR-FTA-0003)"
+        )
+    if not isinstance(raw, list):
+        raise EngineError(f"`datasets` is not a list: {raw!r}")
+    proposed = set(expectation_ids)
+    covered: set[str] = set()
+    for index, entry in enumerate(raw):
+        where = f"datasets[{index}]"
+        if not isinstance(entry, dict):
+            raise EngineError(f"{where} is not an object: {entry!r}")
+        expectation = str(entry.get("expectation") or "")
+        if expectation not in proposed:
+            raise EngineError(
+                f"{where} is for expectation '{expectation}', which this proposal does not "
+                f"propose — a dataset belongs to exactly one proposed expectation")
+        if expectation in covered:
+            raise EngineError(
+                f"expectation '{expectation}' has two datasets — its test builds its state one way")
+        covered.add(expectation)
+        via = entry.get("via")
+        if via not in DATASET_VIA:
+            raise EngineError(
+                f"{where} (for {expectation}) says via {via!r}; a test's state comes from one of "
+                f"{', '.join(DATASET_VIA)} — never a component's own storage")
+        if via in ("command", "event"):
+            steps = entry.get("steps")
+            if not isinstance(steps, list) or not steps:
+                raise EngineError(
+                    f"{where} (for {expectation}) is via {via} but lists no `steps` — say which "
+                    f"{'commands the test calls' if via == 'command' else 'events it publishes'}")
+        if via == "seed":
+            if not entry.get("because"):
+                raise EngineError(
+                    f"{where} (for {expectation}) relies on seeded data without saying `because` — "
+                    f"a seed is for data whose existence is itself under test; everything else is "
+                    f"built through the contract")
+            if str(entry.get("provided_by") or "") not in proposed:
+                raise EngineError(
+                    f"{where} (for {expectation}) relies on seeded data but `provided_by` names no "
+                    f"proposed expectation — data that must exist before a test runs is something "
+                    f"the implementer has to provide, so it has to be an expectation of its own")
+    missing = [i for i in expectation_ids if i not in covered]
+    if missing:
+        raise EngineError(
+            f"no dataset for expectation(s) {', '.join(missing)} — say how the state each test "
+            f"runs against comes to exist, even when that is `via: none`")
+    return raw
 
 
 class ClaudeCodeTesterEngine:
@@ -529,6 +611,7 @@ class ClaudeCodeTesterEngine:
         proposal = _proposal(_extract_json(answer))
         correlation.event("acceptance-proposed",
                           expectations=[e["id"] for e in proposal["expectations"]],
+                          datasets={d["expectation"]: d["via"] for d in proposal["datasets"]},
                           open_questions=len(proposal["open_questions"]))
         return proposal
 
@@ -679,11 +762,35 @@ class ClaudeCodeTesterEngine:
                 "agreed expectation rather than at a line of output.\n\n"
                 + json.dumps(payload["acceptance_surface"], indent=2, ensure_ascii=False)
             )
+        if payload.get("datasets"):
+            # Private to this actor, and relayed unread by the orchestrating actor: the plan this
+            # actor proposed for each expectation's data BEFORE the build, so the tests are not
+            # free to read state off the build now (ADR-FTA-0003).
+            sections.append(
+                "## How each test builds the data it runs against\n"
+                "You planned this yourself before anything was built. Each test arranges its "
+                "state exactly as its expectation's dataset says: `none` needs nothing; "
+                "`command` calls the listed commands first and asserts on what they returned; "
+                f"`event` publishes the listed events to the broker at `{BROKER_URL_ENV}`; "
+                "`seed` relies on the seeded data its `provided_by` expectation states. Do not "
+                "rely on any record the test did not create, except through a `seed` dataset, and "
+                "never write a component's storage directly.\n\n"
+                + json.dumps(payload["datasets"], indent=2, ensure_ascii=False)
+            )
+        else:
+            sections.append(
+                "## The data each test runs against\n"
+                "Build each test's state through this capability's own contract — call its "
+                "commands, or publish the upstream events it consumes (broker address in "
+                f"`{BROKER_URL_ENV}`) — and assert on what the test created. Do not rely on "
+                "records that happen to exist, and never write a component's storage directly."
+            )
         if payload.get("remediation_context"):
             sections.append(
                 "## Remediation — the prior attempt's failing criteria\n"
                 "A criterion failing could mean the test itself was wrong (bad expectation, "
-                "wrong endpoint, wrong shape) or that the implementation was wrong. Review each "
+                "wrong endpoint, wrong shape, a dataset it could not build) or that the "
+                "implementation was wrong. Review each "
                 "one: fix the test here if the fault is in it; if the test looks correct against "
                 "this capability's standing context and the agreed acceptance surface, leave it "
                 "as-is (the implementation side should fix it instead) and say so in your "
@@ -748,10 +855,33 @@ class ClaudeCodeTesterEngine:
             f"(method and path), the event routing key, the environment variable carrying its "
             f"base URL (<COMPONENT>_URL unless you need another)\n"
             f"  - `component` (optional): which of the components below it belongs to\n\n"
-            f"Where a test needs a concrete value the task does not name — a fixture's id, a "
-            f"seeded record, a path parameter, a routing key — PROPOSE A CONCRETE VALUE in the "
-            f"handle. The implementer will accept it and commit to it, or object. Do not leave it "
-            f"for the implementation to choose and for a test to discover afterwards.\n\n"
+            f"Where a test needs a concrete value the task does not name — a path parameter, a "
+            f"routing key, a field value — PROPOSE A CONCRETE VALUE in the handle. The implementer "
+            f"will accept it and commit to it, or object. Do not leave it for the implementation "
+            f"to choose and for a test to discover afterwards.\n\n"
+            f"## The data each test runs against\n"
+            f"For EVERY expectation, say how the state its test needs comes to exist, in a "
+            f"separate `datasets` list — one entry per expectation. This list is yours: the "
+            f"implementer never sees it. Each entry is `{{\"expectation\": <id>, \"via\": ...}}`, "
+            f"with `via` one of:\n"
+            f"  - `none`: the expectation needs no prior state (e.g. a health check). Write it "
+            f"down; do not leave the entry out.\n"
+            f"  - `command` (PREFERRED): the test builds its own state by calling this "
+            f"capability's own commands first — add `steps`, the calls in order. A test that "
+            f"creates what it asserts on needs no agreed fixture and breaks when nobody "
+            f"regenerates one.\n"
+            f"  - `event`: the test publishes the upstream events this capability consumes — add "
+            f"`steps`. The broker's address arrives in `{BROKER_URL_ENV}`.\n"
+            f"  - `seed`: ONLY when pre-existing data is itself what is under test (a stub's "
+            f"canned records, say) — add `because`, and `provided_by`: the id of a proposed "
+            f"expectation stating that seeded data, so the implementer assesses it.\n"
+            f"Never a component's database: a test does not write the implementation's storage.\n"
+            f"Existing fixtures in the implementation repository are NOT a reason to use `seed` — "
+            f"prefer building the state through the contract.\n"
+            f"RULE: anything a dataset needs that this capability's contract does not ALREADY "
+            f"offer — a command or subscription this task introduces, seeded data at fixed values "
+            f"— must also be proposed as an expectation of its own, because the implementer only "
+            f"reads the expectations.\n\n"
             f"Anything the task and the standing context do not determine, and that no proposed "
             f"value can settle because it is a question of what the behaviour should BE, goes in "
             f"`open_questions` — never into an invented statement. An open question is not a "
@@ -773,8 +903,9 @@ class ClaudeCodeTesterEngine:
             + (f" conforming to:\n\n```json\n{json.dumps(schema, indent=2)}\n```"
                if schema else
                " with `expectations` (a list of objects, each with `id`, `statement` and "
-               "`handle`) and `open_questions` (a list of strings, or of objects with `about` and "
-               "`question`; empty when the task determines everything).")
+               "`handle`), `datasets` (one object per expectation, with `expectation` and `via`, "
+               "as described above) and `open_questions` (a list of strings, or of objects with "
+               "`about` and `question`; empty when the task determines everything).")
         )
         return "\n\n".join(sections)
 

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/tests/test_cards.py

@@ -78,19 +78,21 @@ def test_propose_acceptance_accepts_exactly_the_contracted_payload():
     assert offer.request_schema["properties"]["components"]["type"] == "array"
 
 
-def test_propose_acceptance_replies_with_expectations_and_open_questions():
+def test_propose_acceptance_replies_with_expectations_datasets_and_open_questions():
+    """`datasets` is required: every proposal says how each test's state comes to exist
+    (ADR-FTA-0003)."""
     offer = card.load(cards_path()).queries["propose-acceptance"]
     (completion,) = offer.completion_schema
     fields, required = _fields(completion)
-    assert fields == {"expectations", "open_questions"}
-    assert required == {"expectations"}
+    assert fields == {"expectations", "datasets", "open_questions"}
+    assert required == {"expectations", "datasets"}
 
 
 def test_test_task_accepts_exactly_the_contracted_payload():
     offer = card.load(cards_path()).actions["test-task"]
     fields, required = _fields(offer.request_schema)
     assert fields == {"task_id", "title", "definition_of_done", "components", "context",
-                      "remediation_context", "acceptance_surface"}
+                      "remediation_context", "acceptance_surface", "datasets"}
     assert required == {"task_id", "title", "definition_of_done", "components"}
 
 
@@ -114,6 +116,7 @@ def test_the_propose_door_is_answered_by_its_engine_with_no_handler_registered()
         def judge(self, *, system, prompt, schema=None):
             assert prompt.splitlines()[1] == "door: propose-acceptance"
             return {"expectations": [{"id": "E1", "statement": "s", "handle": {}}],
+                    "datasets": [{"expectation": "E1", "via": "none"}],
                     "open_questions": []}
 
     actor = Actor.from_card(cards_path(), engines={"claude-code": StubEngine()})

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/tests/test_engine.py

@@ -17,7 +17,7 @@ from papeete_actor_synchronous_messaging.engine import Engine, EngineError
 
 from foundry_testing_actor import engine as engine_module
 from foundry_testing_actor.engine import (
-    LINE_BUDGET, PROPOSE_TOOLS, TEST_TOOLS, ClaudeCodeTesterEngine, _door_from_prompt,
+    BROKER_URL_ENV, DATASET_VIA, LINE_BUDGET, PROPOSE_TOOLS, TEST_TOOLS, ClaudeCodeTesterEngine, _door_from_prompt,
     _extract_json, _line, _payload_from_prompt, _project, _proposal,
 )
 
@@ -270,13 +270,88 @@ def test_open_questions_always_arrive_as_a_list(given, expected):
 
 def test_a_sessions_extra_keys_do_not_reach_the_caller():
     proposal = _proposal({"expectations": [], "notes": "I also looked at the stub", "x": 1})
-    assert set(proposal) == {"expectations", "open_questions"}
+    assert set(proposal) == {"expectations", "datasets", "open_questions"}
 
 
 def test_extra_keys_inside_an_expectation_survive():
     """The surface is typed only as a list; an entry's own extra keys are the contract's to allow."""
     entry = {"id": "E1", "statement": "s", "handle": {}, "component": "backend", "why": "DoD 1"}
-    assert _proposal({"expectations": [entry]})["expectations"] == [entry]
+    proposal = _proposal({"expectations": [entry], "datasets": [NONE("E1")]})
+    assert proposal["expectations"] == [entry]
+
+
+# ── datasets: how each test's state comes to exist (ADR-FTA-0003) ───────────────────────────
+
+def NONE(expectation):
+    return {"expectation": expectation, "via": "none"}
+
+
+TWO = [{"id": "E1", "statement": "an archived anchor refuses an update", "handle": "PATCH"},
+       {"id": "E2", "statement": "the stub serves anchor 0001", "handle": "GET"}]
+
+
+def test_every_via_the_contract_allows_is_accepted():
+    datasets = [
+        {"expectation": "E1", "via": "command", "steps": ["POST /anchors", "POST …/archive"]},
+        {"expectation": "E2", "via": "seed", "because": "the canned record IS the subject",
+         "provided_by": "E2"},
+    ]
+    assert _proposal({"expectations": TWO, "datasets": datasets})["datasets"] == datasets
+    assert set(DATASET_VIA) == {"none", "command", "event", "seed"}
+
+
+def test_a_proposal_with_expectations_and_no_datasets_is_refused():
+    with pytest.raises(EngineError, match="no `datasets`"):
+        _proposal({"expectations": TWO})
+
+
+def test_an_expectation_without_a_dataset_is_refused_even_when_it_needs_nothing():
+    """`none` is a claim that can be wrong, so it is written — never inferred from an absence."""
+    with pytest.raises(EngineError, match="no dataset for expectation.*E2"):
+        _proposal({"expectations": TWO, "datasets": [NONE("E1")]})
+
+
+@pytest.mark.parametrize("datasets, match", [
+    ([NONE("E1"), NONE("E2"), NONE("E9")], "does not propose"),
+    ([NONE("E1"), NONE("E1"), NONE("E2")], "two datasets"),
+    ([NONE("E1"), {"expectation": "E2", "via": "database"}], "never a component's own storage"),
+    ([NONE("E1"), {"expectation": "E2", "via": "command"}], "lists no `steps`"),
+    ([NONE("E1"), {"expectation": "E2", "via": "event", "steps": []}], "lists no `steps`"),
+    ([NONE("E1"), {"expectation": "E2", "via": "seed", "provided_by": "E2"}], "without saying"),
+    ([NONE("E1"), {"expectation": "E2", "via": "seed", "because": "canned"}], "provided_by"),
+    ([NONE("E1"), {"expectation": "E2", "via": "seed", "because": "canned",
+                   "provided_by": "E7"}], "provided_by"),
+    ([NONE("E1"), "E2: none"], "not an object"),
+])
+def test_a_dataset_that_hides_a_gap_is_refused(datasets, match):
+    with pytest.raises(EngineError, match=match):
+        _proposal({"expectations": TWO, "datasets": datasets})
+
+
+def test_an_empty_proposal_needs_no_datasets():
+    assert _proposal({"expectations": []})["datasets"] == []
+
+
+def test_the_proposal_prompt_asks_for_a_dataset_per_expectation_and_prefers_commands(engine,
+                                                                                    tmp_path):
+    prompt = engine._proposal_prompt(PAYLOAD, None, tmp_path)
+    assert "`datasets`" in prompt and "one entry per expectation" in prompt
+    assert "`command` (PREFERRED)" in prompt
+    assert "the implementer never sees it" in prompt
+    assert "Never a component's database" in prompt
+    assert "must also be proposed as an expectation of its own" in prompt
+    assert BROKER_URL_ENV in prompt
+
+
+def test_the_test_prompt_builds_state_as_the_datasets_say(engine, config):
+    datasets = [{"expectation": "EXP-001", "via": "command", "steps": ["POST /widgets"]}]
+    with_plan = _prompt(engine, config, {**PAYLOAD, "datasets": datasets})
+    assert "How each test builds the data it runs against" in with_plan
+    assert '"POST /widgets"' in with_plan
+
+    without = _prompt(engine, config, PAYLOAD)
+    assert "Build each test's state through this capability's own contract" in without
+    assert "never write a component's storage directly" in without
 
 
 # ── the propose door end to end, without a session ──────────────────────────────────────────
@@ -307,7 +382,8 @@ def proposed(engine, monkeypatch):
     return _run
 
 
-ANSWER = '```json\n{"expectations": [{"id": "E1", "statement": "s", "handle": {}}]}\n```'
+ANSWER = ('```json\n{"expectations": [{"id": "E1", "statement": "s", "handle": {}}],'
+          ' "datasets": [{"expectation": "E1", "via": "none"}]}\n```')
 
 
 def test_the_propose_session_is_given_no_tool_that_writes(proposed):
@@ -363,8 +439,10 @@ def test_the_propose_door_removes_both_clones_on_failure(proposed):
 def test_the_proposal_comes_back_as_the_reply(proposed):
     reply, _ = proposed(
         '```json\n{"expectations": [{"id": "E1", "statement": "s", "handle": {"path": "/w"}}],'
+        ' "datasets": [{"expectation": "E1", "via": "command", "steps": ["POST /w"]}],'
         ' "open_questions": "which status codes?"}\n```')
     assert reply == {"expectations": [{"id": "E1", "statement": "s", "handle": {"path": "/w"}}],
+                     "datasets": [{"expectation": "E1", "via": "command", "steps": ["POST /w"]}],
                      "open_questions": ["which status codes?"]}
 
 

{foundry_testing_actor-0.1.1 → foundry_testing_actor-0.2.0}/uv.lock

@@ -18,7 +18,7 @@ wheels = [
 
 [[package]]
 name = "foundry-testing-actor"
-version = "0.1.1"
+version = "0.2.0"
 source = { editable = "." }
 dependencies = [
     { name = "papeete-actor-synchronous-messaging" },