pl-e2e-test 0.1.4__tar.gz → 0.1.6__tar.gz

{pl_e2e_test-0.1.4/src/pl_e2e_test.egg-info → pl_e2e_test-0.1.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pl-e2e-test
-Version: 0.1.4
+Version: 0.1.6
 Summary: Fuzz PrairieLearn question lifecycles against the real Python element runtime
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/PrairieLearn/pl-e2e-test
@@ -266,24 +266,105 @@ nonGradableSelectors = ['pl-sketch[read-only="true"]', 'pl-sketch[read-only="1"]
 ```
 
 These are the defaults. `elementSolutionSelectors` identifies elements that provide
-solutions without a `correct_answers` entry. `nonGradableSelectors` excludes
-display-only answers from grading checks. Each supplied list replaces its default;
-omitted settings keep their defaults, and `[]` disables a list.
+solutions without a `correct_answers` entry. It only relaxes that entry requirement;
+it does not generate a test submission. `nonGradableSelectors` excludes display-only
+answers from grading checks. Each supplied list replaces its default; omitted settings
+keep their defaults, and `[]` disables a list.
 
 Use `--config path/to/settings.toml` to select another TOML file with the same
 `[tool.pl-e2e-test]` table. This works with normal runs, `diff`, and arguments after
 `parallel run` or `parallel diff`. Explicit paths are relative to the working directory.
 
+### Lifecycle JSON boundary
+
+PrairieLearn crosses a JSON boundary after processing the elements in a lifecycle phase
+and again after calling the question's `server.py` hook. The harness mirrors those
+boundaries. It first checks the in-process result for legal field mutations, then
+serializes and deserializes `data`; the normalized result is what the next hook or phase
+receives.
+
+This accepts the same Python values as PrairieLearn's JSON encoder rather than requiring
+literal JSON-domain Python types. For example, tuples normalize to arrays (Python lists),
+and supported non-string object keys such as numeric drawing IDs normalize to strings.
+This conversion can be lossy: if two keys normalize to the same string, the last key in
+serialized order wins, matching PrairieLearn's boundary. Unsupported objects, non-finite
+floats, and integers outside JavaScript's safe range are rejected with the failing data
+path.
+
+### Correct submissions for custom elements
+
+The grade check uses PrairieLearn's `test` lifecycle to obtain a deterministic correct
+submission. PrairieLearn first calls `test(element_html, data)` on the elements in the
+question, then the harness calls `test(data)` in the question's `server.py`. A reusable
+course element should normally implement the element hook. The question hook can fill
+in a submission when the correct state is question-specific or when a custom element
+does not provide one itself.
+
+For every gradable `answers-name`, the correct test case must provide:
+
+- a non-null value in `data["raw_submitted_answers"]` under the answer name, or one or
+  more compound input keys prefixed with `answers-name-`; and
+- `data["partial_scores"][answers_name]["score"] = 1`.
+
+Raw values must survive PrairieLearn's JSON boundary and have the same browser transport
+shape that the element's `parse` hook expects. Structured widgets commonly serialize
+their state as a JSON string. The hook must derive its output only from deterministic
+variant state such as `params` and `correct_answers`, mutate `data` in place, and return
+`None`. The normal phase mutation checks and JSON-boundary normalization still apply.
+
+For example, a question can supply the correct state for a drag-and-drop element with:
+
+```python
+import json
+
+
+def test(data):
+    if data["test_type"] != "correct":
+        return
+
+    answers_name = "ordering"
+    state = {
+        "solution": data["params"]["correct_order"],
+        "starter": [],
+    }
+    data["raw_submitted_answers"][answers_name] = json.dumps(state)
+    data["partial_scores"][answers_name] = {"score": 1}
+```
+
+Compound elements keep the declared answer name as their grading identity while using
+namespaced child fields for browser inputs. For example, an element with
+`answers-name="match"` may test and parse `match-dropdown-0` and `match-dropdown-1`,
+then store its score in `partial_scores["match"]`; it does not need to invent a
+`submitted_answers["match"]` value. After parsing, every raw field must either remain
+non-null under its child key or produce a non-null value under its declared parent
+answer. This also supports elements that normalize `answer-input` into `answer`.
+
+This contract keeps element-specific synthesis with the element or question that owns
+the data. PrairieLearn's tuple-based `pl-matching` state normalizes to arrays at the JSON
+boundary, and its existing namespaced test fields require no harness adapter. A question
+using `pl-sketch` without an element-owned solution can supply its deterministic
+`answers-name-sketchresponse-submission` payload in `server.py`. A course-owned
+drag-and-drop element such as `pl-faded-parsons` can implement one reusable element
+test hook that serializes its prepared correct tray and blank state.
+
 ### Question-local configuration
 
-Store saved concrete seeds and narrow phase skips in `.question-e2e.json` beside the
-question's `info.json`:
+Store saved concrete seeds, narrow phase skips, and expected upstream errors in
+`.question-e2e.json` beside the question's `info.json`:
 
 ```json
 {
   "regression_seeds": [1843927501],
   "skip_methods": {
     "grade": "Temporarily blocked by an upstream issue"
+  },
+  "expected_errors": {
+    "render": [
+      {
+        "match": "^render: known upstream failure$",
+        "reason": "https://github.com/PrairieLearn/PrairieLearn/issues/12345"
+      }
+    ]
   }
 }
 ```
@@ -292,6 +373,21 @@ Saved regression seeds run before newly generated seeds. Skips require a nonempt
 reason and may name only `generate`, `prepare`, `render`, or `grade`. Manual-grading
 questions automatically skip only the grade check.
 
+Prefer `expected_errors` to a skip when the lifecycle can run far enough to expose a
+known upstream defect. Each rule requires a human-readable reason or issue URL and
+exactly one matcher: either `type`, containing a fully qualified exception type, or
+`match`, containing a Python regular expression searched against the exception message.
+A matching exception is reported as `XFAIL`; a different exception fails normally,
+while seeds that do not raise an exception pass normally. A method may not be both
+skipped and expected to fail.
+
+Rules under one method are alternatives. At least one rule must match across the selected
+seeds for that question and method, or an otherwise-successful run fails an audit asking
+you to remove or update the stale expectation. Parallel runs perform this audit after
+merging all shards. Because one upstream error can prevent later code from running,
+obsolete individual alternatives cannot be detected while another alternative still
+matches.
+
 Record the concrete seed shown in a failing pytest case with:
 
 ```sh

{pl_e2e_test-0.1.4 → pl_e2e_test-0.1.6}/README.md

@@ -242,24 +242,105 @@ nonGradableSelectors = ['pl-sketch[read-only="true"]', 'pl-sketch[read-only="1"]
 ```
 
 These are the defaults. `elementSolutionSelectors` identifies elements that provide
-solutions without a `correct_answers` entry. `nonGradableSelectors` excludes
-display-only answers from grading checks. Each supplied list replaces its default;
-omitted settings keep their defaults, and `[]` disables a list.
+solutions without a `correct_answers` entry. It only relaxes that entry requirement;
+it does not generate a test submission. `nonGradableSelectors` excludes display-only
+answers from grading checks. Each supplied list replaces its default; omitted settings
+keep their defaults, and `[]` disables a list.
 
 Use `--config path/to/settings.toml` to select another TOML file with the same
 `[tool.pl-e2e-test]` table. This works with normal runs, `diff`, and arguments after
 `parallel run` or `parallel diff`. Explicit paths are relative to the working directory.
 
+### Lifecycle JSON boundary
+
+PrairieLearn crosses a JSON boundary after processing the elements in a lifecycle phase
+and again after calling the question's `server.py` hook. The harness mirrors those
+boundaries. It first checks the in-process result for legal field mutations, then
+serializes and deserializes `data`; the normalized result is what the next hook or phase
+receives.
+
+This accepts the same Python values as PrairieLearn's JSON encoder rather than requiring
+literal JSON-domain Python types. For example, tuples normalize to arrays (Python lists),
+and supported non-string object keys such as numeric drawing IDs normalize to strings.
+This conversion can be lossy: if two keys normalize to the same string, the last key in
+serialized order wins, matching PrairieLearn's boundary. Unsupported objects, non-finite
+floats, and integers outside JavaScript's safe range are rejected with the failing data
+path.
+
+### Correct submissions for custom elements
+
+The grade check uses PrairieLearn's `test` lifecycle to obtain a deterministic correct
+submission. PrairieLearn first calls `test(element_html, data)` on the elements in the
+question, then the harness calls `test(data)` in the question's `server.py`. A reusable
+course element should normally implement the element hook. The question hook can fill
+in a submission when the correct state is question-specific or when a custom element
+does not provide one itself.
+
+For every gradable `answers-name`, the correct test case must provide:
+
+- a non-null value in `data["raw_submitted_answers"]` under the answer name, or one or
+  more compound input keys prefixed with `answers-name-`; and
+- `data["partial_scores"][answers_name]["score"] = 1`.
+
+Raw values must survive PrairieLearn's JSON boundary and have the same browser transport
+shape that the element's `parse` hook expects. Structured widgets commonly serialize
+their state as a JSON string. The hook must derive its output only from deterministic
+variant state such as `params` and `correct_answers`, mutate `data` in place, and return
+`None`. The normal phase mutation checks and JSON-boundary normalization still apply.
+
+For example, a question can supply the correct state for a drag-and-drop element with:
+
+```python
+import json
+
+
+def test(data):
+    if data["test_type"] != "correct":
+        return
+
+    answers_name = "ordering"
+    state = {
+        "solution": data["params"]["correct_order"],
+        "starter": [],
+    }
+    data["raw_submitted_answers"][answers_name] = json.dumps(state)
+    data["partial_scores"][answers_name] = {"score": 1}
+```
+
+Compound elements keep the declared answer name as their grading identity while using
+namespaced child fields for browser inputs. For example, an element with
+`answers-name="match"` may test and parse `match-dropdown-0` and `match-dropdown-1`,
+then store its score in `partial_scores["match"]`; it does not need to invent a
+`submitted_answers["match"]` value. After parsing, every raw field must either remain
+non-null under its child key or produce a non-null value under its declared parent
+answer. This also supports elements that normalize `answer-input` into `answer`.
+
+This contract keeps element-specific synthesis with the element or question that owns
+the data. PrairieLearn's tuple-based `pl-matching` state normalizes to arrays at the JSON
+boundary, and its existing namespaced test fields require no harness adapter. A question
+using `pl-sketch` without an element-owned solution can supply its deterministic
+`answers-name-sketchresponse-submission` payload in `server.py`. A course-owned
+drag-and-drop element such as `pl-faded-parsons` can implement one reusable element
+test hook that serializes its prepared correct tray and blank state.
+
 ### Question-local configuration
 
-Store saved concrete seeds and narrow phase skips in `.question-e2e.json` beside the
-question's `info.json`:
+Store saved concrete seeds, narrow phase skips, and expected upstream errors in
+`.question-e2e.json` beside the question's `info.json`:
 
 ```json
 {
   "regression_seeds": [1843927501],
   "skip_methods": {
     "grade": "Temporarily blocked by an upstream issue"
+  },
+  "expected_errors": {
+    "render": [
+      {
+        "match": "^render: known upstream failure$",
+        "reason": "https://github.com/PrairieLearn/PrairieLearn/issues/12345"
+      }
+    ]
   }
 }
 ```
@@ -268,6 +349,21 @@ Saved regression seeds run before newly generated seeds. Skips require a nonempt
 reason and may name only `generate`, `prepare`, `render`, or `grade`. Manual-grading
 questions automatically skip only the grade check.
 
+Prefer `expected_errors` to a skip when the lifecycle can run far enough to expose a
+known upstream defect. Each rule requires a human-readable reason or issue URL and
+exactly one matcher: either `type`, containing a fully qualified exception type, or
+`match`, containing a Python regular expression searched against the exception message.
+A matching exception is reported as `XFAIL`; a different exception fails normally,
+while seeds that do not raise an exception pass normally. A method may not be both
+skipped and expected to fail.
+
+Rules under one method are alternatives. At least one rule must match across the selected
+seeds for that question and method, or an otherwise-successful run fails an audit asking
+you to remove or update the stale expectation. Parallel runs perform this audit after
+merging all shards. Because one upstream error can prevent later code from running,
+obsolete individual alternatives cannot be detected while another alternative still
+matches.
+
 Record the concrete seed shown in a failing pytest case with:
 
 ```sh

{pl_e2e_test-0.1.4 → pl_e2e_test-0.1.6}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pl-e2e-test"
-version = "0.1.4"
+version = "0.1.6"
 description = "Fuzz PrairieLearn question lifecycles against the real Python element runtime"
 readme = "README.md"
 requires-python = ">=3.13"
@@ -39,6 +39,12 @@ where = ["src"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]
+markers = [
+  "generate: question generate-phase lifecycle checks",
+  "prepare: question prepare-phase lifecycle checks",
+  "render: question render-phase lifecycle checks",
+  "grade: question grading lifecycle checks",
+]
 
 [tool.pyright]
 include = ["src/prairielearn_e2e", "tests"]

{pl_e2e_test-0.1.4 → pl_e2e_test-0.1.6/src/pl_e2e_test.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pl-e2e-test
-Version: 0.1.4
+Version: 0.1.6
 Summary: Fuzz PrairieLearn question lifecycles against the real Python element runtime
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/PrairieLearn/pl-e2e-test
@@ -266,24 +266,105 @@ nonGradableSelectors = ['pl-sketch[read-only="true"]', 'pl-sketch[read-only="1"]
 ```
 
 These are the defaults. `elementSolutionSelectors` identifies elements that provide
-solutions without a `correct_answers` entry. `nonGradableSelectors` excludes
-display-only answers from grading checks. Each supplied list replaces its default;
-omitted settings keep their defaults, and `[]` disables a list.
+solutions without a `correct_answers` entry. It only relaxes that entry requirement;
+it does not generate a test submission. `nonGradableSelectors` excludes display-only
+answers from grading checks. Each supplied list replaces its default; omitted settings
+keep their defaults, and `[]` disables a list.
 
 Use `--config path/to/settings.toml` to select another TOML file with the same
 `[tool.pl-e2e-test]` table. This works with normal runs, `diff`, and arguments after
 `parallel run` or `parallel diff`. Explicit paths are relative to the working directory.
 
+### Lifecycle JSON boundary
+
+PrairieLearn crosses a JSON boundary after processing the elements in a lifecycle phase
+and again after calling the question's `server.py` hook. The harness mirrors those
+boundaries. It first checks the in-process result for legal field mutations, then
+serializes and deserializes `data`; the normalized result is what the next hook or phase
+receives.
+
+This accepts the same Python values as PrairieLearn's JSON encoder rather than requiring
+literal JSON-domain Python types. For example, tuples normalize to arrays (Python lists),
+and supported non-string object keys such as numeric drawing IDs normalize to strings.
+This conversion can be lossy: if two keys normalize to the same string, the last key in
+serialized order wins, matching PrairieLearn's boundary. Unsupported objects, non-finite
+floats, and integers outside JavaScript's safe range are rejected with the failing data
+path.
+
+### Correct submissions for custom elements
+
+The grade check uses PrairieLearn's `test` lifecycle to obtain a deterministic correct
+submission. PrairieLearn first calls `test(element_html, data)` on the elements in the
+question, then the harness calls `test(data)` in the question's `server.py`. A reusable
+course element should normally implement the element hook. The question hook can fill
+in a submission when the correct state is question-specific or when a custom element
+does not provide one itself.
+
+For every gradable `answers-name`, the correct test case must provide:
+
+- a non-null value in `data["raw_submitted_answers"]` under the answer name, or one or
+  more compound input keys prefixed with `answers-name-`; and
+- `data["partial_scores"][answers_name]["score"] = 1`.
+
+Raw values must survive PrairieLearn's JSON boundary and have the same browser transport
+shape that the element's `parse` hook expects. Structured widgets commonly serialize
+their state as a JSON string. The hook must derive its output only from deterministic
+variant state such as `params` and `correct_answers`, mutate `data` in place, and return
+`None`. The normal phase mutation checks and JSON-boundary normalization still apply.
+
+For example, a question can supply the correct state for a drag-and-drop element with:
+
+```python
+import json
+
+
+def test(data):
+    if data["test_type"] != "correct":
+        return
+
+    answers_name = "ordering"
+    state = {
+        "solution": data["params"]["correct_order"],
+        "starter": [],
+    }
+    data["raw_submitted_answers"][answers_name] = json.dumps(state)
+    data["partial_scores"][answers_name] = {"score": 1}
+```
+
+Compound elements keep the declared answer name as their grading identity while using
+namespaced child fields for browser inputs. For example, an element with
+`answers-name="match"` may test and parse `match-dropdown-0` and `match-dropdown-1`,
+then store its score in `partial_scores["match"]`; it does not need to invent a
+`submitted_answers["match"]` value. After parsing, every raw field must either remain
+non-null under its child key or produce a non-null value under its declared parent
+answer. This also supports elements that normalize `answer-input` into `answer`.
+
+This contract keeps element-specific synthesis with the element or question that owns
+the data. PrairieLearn's tuple-based `pl-matching` state normalizes to arrays at the JSON
+boundary, and its existing namespaced test fields require no harness adapter. A question
+using `pl-sketch` without an element-owned solution can supply its deterministic
+`answers-name-sketchresponse-submission` payload in `server.py`. A course-owned
+drag-and-drop element such as `pl-faded-parsons` can implement one reusable element
+test hook that serializes its prepared correct tray and blank state.
+
 ### Question-local configuration
 
-Store saved concrete seeds and narrow phase skips in `.question-e2e.json` beside the
-question's `info.json`:
+Store saved concrete seeds, narrow phase skips, and expected upstream errors in
+`.question-e2e.json` beside the question's `info.json`:
 
 ```json
 {
   "regression_seeds": [1843927501],
   "skip_methods": {
     "grade": "Temporarily blocked by an upstream issue"
+  },
+  "expected_errors": {
+    "render": [
+      {
+        "match": "^render: known upstream failure$",
+        "reason": "https://github.com/PrairieLearn/PrairieLearn/issues/12345"
+      }
+    ]
   }
 }
 ```
@@ -292,6 +373,21 @@ Saved regression seeds run before newly generated seeds. Skips require a nonempt
 reason and may name only `generate`, `prepare`, `render`, or `grade`. Manual-grading
 questions automatically skip only the grade check.
 
+Prefer `expected_errors` to a skip when the lifecycle can run far enough to expose a
+known upstream defect. Each rule requires a human-readable reason or issue URL and
+exactly one matcher: either `type`, containing a fully qualified exception type, or
+`match`, containing a Python regular expression searched against the exception message.
+A matching exception is reported as `XFAIL`; a different exception fails normally,
+while seeds that do not raise an exception pass normally. A method may not be both
+skipped and expected to fail.
+
+Rules under one method are alternatives. At least one rule must match across the selected
+seeds for that question and method, or an otherwise-successful run fails an audit asking
+you to remove or update the stale expectation. Parallel runs perform this audit after
+merging all shards. Because one upstream error can prevent later code from running,
+obsolete individual alternatives cannot be detected while another alternative still
+matches.
+
 Record the concrete seed shown in a failing pytest case with:
 
 ```sh

{pl_e2e_test-0.1.4 → pl_e2e_test-0.1.6}/src/pl_e2e_test.egg-info/SOURCES.txt

@@ -17,6 +17,7 @@ src/prairielearn_e2e/config.py
 src/prairielearn_e2e/contract.py
 src/prairielearn_e2e/discovery.py
 src/prairielearn_e2e/errors.py
+src/prairielearn_e2e/expected_errors.py
 src/prairielearn_e2e/fuzz.py
 src/prairielearn_e2e/harness.py
 src/prairielearn_e2e/html_checks.py
@@ -45,6 +46,7 @@ tests/test_metadata.py
 tests/test_parallel.py
 tests/test_publish_version.py
 tests/test_pytest_plugin.py
+tests/test_question_tests.py
 tests/test_regression.py
 tests/test_reporting.py
 tests/test_repository.py

{pl_e2e_test-0.1.4 → pl_e2e_test-0.1.6}/src/prairielearn_e2e/contract.py

@@ -164,35 +164,117 @@ def validate_mutations(
             raise HarnessError(f"{phase}: data[{name!r}] was illegally modified")
 
 
-def validate_serializable(data: dict[str, Any], phase: Phase) -> None:
-    def check_json(value: Any, path: str) -> None:
-        if value is None or isinstance(value, (str, bool, int)):
-            return
-        if isinstance(value, float):
-            if not math.isfinite(value):
-                raise TypeError(f"{path} contains NaN or infinity")
-            return
-        if isinstance(value, list):
+_JSON_ERRORS = (TypeError, ValueError, OverflowError, RecursionError)
+
+
+def _validate_integer_limits(
+    value: Any, path: str, active_containers: set[int]
+) -> None:
+    """Apply PrairieLearn's integer limit to every JSON-supported container."""
+
+    if isinstance(value, int):
+        try:
+            assert_all_integers_within_limits(value)
+        except ValueError as exc:
+            raise ValueError(f"{path} contains oversized integer: {value}") from exc
+        return
+
+    if not isinstance(value, (dict, list, tuple)):
+        return
+
+    container_id = id(value)
+    if container_id in active_containers:
+        return
+    active_containers.add(container_id)
+    try:
+        if isinstance(value, dict):
+            for key, item in value.items():
+                if isinstance(key, int):
+                    _validate_integer_limits(
+                        key, f"{path} object key {key!r}", active_containers
+                    )
+                _validate_integer_limits(item, f"{path}[{key!r}]", active_containers)
+        else:
             for index, item in enumerate(value):
-                check_json(item, f"{path}[{index}]")
-            return
+                _validate_integer_limits(item, f"{path}[{index}]", active_containers)
+    finally:
+        active_containers.remove(container_id)
+
+
+def _json_fails(value: Any) -> bool:
+    try:
+        json.dumps(value, allow_nan=False)
+    except _JSON_ERRORS:
+        return True
+    return False
+
+
+def _json_failure_path(value: Any, path: str, active_containers: set[int]) -> str:
+    """Locate an encoder failure after json.dumps has already rejected the value."""
+
+    if not isinstance(value, (dict, list, tuple)):
+        return path
+
+    container_id = id(value)
+    if container_id in active_containers:
+        return path
+    active_containers.add(container_id)
+    try:
         if isinstance(value, dict):
             for key, item in value.items():
-                if not isinstance(key, str):
-                    raise TypeError(f"{path} has a non-string object key")
-                check_json(item, f"{path}[{key!r}]")
-            return
-        raise TypeError(f"{path} contains {type(value).__name__}, which is not JSON")
+                if _json_fails({key: None}):
+                    return f"{path} object key {key!r}"
+                if _json_fails(item):
+                    return _json_failure_path(
+                        item, f"{path}[{key!r}]", active_containers
+                    )
+        else:
+            for index, item in enumerate(value):
+                if _json_fails(item):
+                    return _json_failure_path(
+                        item, f"{path}[{index}]", active_containers
+                    )
+    finally:
+        active_containers.remove(container_id)
+    return path
+
+
+def validate_serializable(data: dict[str, Any], phase: Phase) -> None:
+    """Validate and normalize data at PrairieLearn's Python-to-JSON boundary."""
 
     try:
-        check_json(data, "data")
-        assert_all_integers_within_limits(data)
-        json.dumps(data, allow_nan=False)
-    except (TypeError, ValueError) as exc:
+        _validate_integer_limits(data, "data", set())
+    except ValueError as exc:
         raise HarnessError(
             f"{phase}: data is not PrairieLearn JSON serializable: {exc}"
         ) from exc
 
+    try:
+        encoded = json.dumps(data, allow_nan=False)
+    except _JSON_ERRORS as exc:
+        try:
+            path = _json_failure_path(data, "data", set())
+        except RecursionError:
+            path = "data"
+        raise HarnessError(
+            f"{phase}: data is not PrairieLearn JSON serializable: {path}: {exc}"
+        ) from exc
+
+    try:
+        normalized = json.loads(encoded)
+    except _JSON_ERRORS as exc:
+        raise HarnessError(
+            f"{phase}: data is not PrairieLearn JSON serializable: data: {exc}"
+        ) from exc
+    if not isinstance(normalized, dict):
+        raise HarnessError(
+            f"{phase}: data is not PrairieLearn JSON serializable: "
+            "data did not normalize to an object"
+        )
+
+    data.clear()
+    data.update(normalized)
+
 
 def snapshot(data: dict[str, Any], phase: Phase) -> dict[str, Any]:
     validate_shape(data, phase)