pl-e2e-test 0.1.7__tar.gz → 0.2.0__tar.gz

{pl_e2e_test-0.1.7/src/pl_e2e_test.egg-info → pl_e2e_test-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pl-e2e-test
-Version: 0.1.7
+Version: 0.2.0
 Summary: Fuzz PrairieLearn question lifecycles against the real Python element runtime
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/PrairieLearn/pl-e2e-test
@@ -38,6 +38,8 @@ The package installs the reusable `prairielearn_e2e` Python API and a single
 - `pl-e2e-test parallel` runs local shards and merges their results.
 - `pl-e2e-test add-seed` records a failing concrete variant seed.
 - `pl-e2e-test reports` merges parallel shard results into terminal and JUnit reports.
+- `pl-e2e-test instances` shows configured, installed, cached, and local PrairieLearn
+  revisions.
 - `pl-e2e-test version` prints the installed version.
 - `pl-e2e-test help [COMMAND]` prints top-level help, or one command's help.
 
@@ -45,6 +47,7 @@ The package installs the reusable `prairielearn_e2e` Python API and a single
 
 - Python 3.13 or newer
 - Git 2.30 or newer
+- uv 0.9 or newer
 
 The harness depends on pytest, Pyright, Ruff, Chevron, JSON Schema, lxml, NumPy,
 and PrairieLearn. PrairieLearn's Python package is currently consumed from its Git
@@ -241,19 +244,34 @@ jobs:
 ```
 
 The course must provide `.prairielearn/schemas/infoQuestion.json`. The harness checks
-question metadata against that schema and uses the exact PrairieLearn revision
-installed in the uv environment. It stores a bare, partial PrairieLearn repository in
-the platform-standard user cache directory. The first use of a new Git source or
-revision requires network access; later runs reuse that exact revision offline. The
-uv lockfile, rather than the cache, decides which revision is tested.
+question metadata against that schema and resolves the configured PrairieLearn version
+before each run. The default, `latest`, resolves the Git source's remote `HEAD`; a full
+commit, branch, or unambiguous tag can be selected instead. Moving selectors require
+network access and never silently fall back to stale cache state.
+
+The selected revision's Python package and core elements are always used together. The
+harness keeps a blob-filtered bare repository in the platform-standard user cache and
+hydrates only `apps/prairielearn/python` and `apps/prairielearn/elements`. It launches
+the selected Python package through a frozen uv overlay, without changing the course's
+`pyproject.toml`, `uv.lock`, or virtual environment. A previously hydrated exact commit
+can be reused offline when its uv dependencies are also cached.
 
 To use an existing PrairieLearn worktree or bare repository instead, pass
 `--prairielearn-path /path/to/PrairieLearn`. The repository must already contain the
-installed revision. An explicit path is authoritative: the harness does not fetch into
-it or fall back to the managed cache. In either mode, only the pinned revision's Python
-element controllers are archived into the test run's temporary directory. Standard
-cache environment settings such as `XDG_CACHE_HOME` can relocate the managed cache;
-CI systems can persist that directory between runs if desired.
+selected revision. An explicit path is authoritative: the harness resolves selectors
+against its local refs, does not fetch into it, and does not fall back to the managed
+cache. For a local repository, `latest` means its current `HEAD`.
+
+Inspect available revisions without contacting remotes:
+
+```sh
+uv run pl-e2e-test instances
+```
+
+Add `--refresh` to resolve the configured moving selector and verify remote access.
+The table distinguishes fully hydrated runtime revisions from legacy element-only cache
+entries. Standard cache environment settings such as `XDG_CACHE_HOME` can relocate the
+managed cache; CI systems can persist that directory between runs if desired.
 
 ### Harness configuration
 
@@ -261,11 +279,17 @@ Selector settings are read automatically from the course's `pyproject.toml`:
 
 ```toml
 [tool.pl-e2e-test]
+prairielearnVersion = "latest"
 elementSolutionSelectors = ["pl-sketch"]
 nonGradableSelectors = ['pl-sketch[read-only="true"]', 'pl-sketch[read-only="1"]']
 ```
 
-These are the defaults. `elementSolutionSelectors` identifies elements that provide
+These are the defaults. Omitting `prairielearnVersion` is equivalent to `"latest"`.
+Use a full commit for reproducible or offline runs, or a branch/tag when a named moving
+target is intentional. `--prairielearn-version SELECTOR` overrides the TOML value for
+one invocation, and replay commands always print the resolved full commit.
+
+`elementSolutionSelectors` identifies elements that provide
 solutions without a `correct_answers` entry. It only relaxes that entry requirement;
 it does not generate a test submission. `nonGradableSelectors` excludes display-only
 answers from grading checks. Each supplied list replaces its default; omitted settings
@@ -303,7 +327,7 @@ does not provide one itself.
 For every gradable `answers-name`, the correct test case must provide:
 
 - a non-null value in `data["raw_submitted_answers"]` under the answer name, or one or
-  more compound input keys prefixed with `answers-name-`; and
+  more compound input keys prefixed with `answers-name-` or `answers-name.`; and
 - `data["partial_scores"][answers_name]["score"] = 1`.
 
 Raw values must survive PrairieLearn's JSON boundary and have the same browser transport
@@ -332,11 +356,12 @@ def test(data):
 ```
 
 Compound elements keep the declared answer name as their grading identity while using
-namespaced child fields for browser inputs. For example, an element with
-`answers-name="match"` may test and parse `match-dropdown-0` and `match-dropdown-1`,
-then store its score in `partial_scores["match"]`; it does not need to invent a
-`submitted_answers["match"]` value. After parsing, every raw field associated with a
-declared gradable answer must either remain non-null under its child key or produce a
+namespaced child fields for browser inputs. Hyphen and dot namespace boundaries are
+supported. For example, an element with `answers-name="match"` may test and parse
+`match-dropdown-0` and `match-dropdown-1`, while `answers-name="average"` may use
+`average.main`. The element stores its score under the declared answer name and does not
+need to invent a direct raw value for it. After parsing, every raw field associated with
+a declared gradable answer must either remain non-null under its child key or produce a
 non-null value under its declared parent answer. This also supports elements that
 normalize `answer-input` into `answer`. Unassociated transport fields from file-producing
 elements may instead be consumed into `submitted_answers["_files"]`; they are not answer
@@ -454,12 +479,16 @@ course's TOML settings, or `config=load_config("settings.toml")` for an explicit
 `load_config` and `HarnessConfig` are exported from `prairielearn_e2e`. Omitting
 `config` uses the built-in defaults.
 
+The command-line runner and high-level `run_parallel` API perform version resolution
+and uv bootstrapping. Manually constructed `PrairieLearnBackend` and `QuestionHarness`
+objects continue to use the PrairieLearn package already loaded by their interpreter.
+
 ## Development
 
 This project uses uv 0.9 or newer:
 
 ```sh
-uv sync
+make install
 make test
 make format
 make smoke-dist
@@ -471,44 +500,36 @@ into an isolated target, and imports it using the locked development dependencie
 
 ## Publishing
 
-There are three supported publishing paths. Choose one for a release; do not run the
-local and GitHub paths for the same version because PyPI versions are immutable.
-
-### GitHub-hosted trusted publishing
-
 Configure the PyPI project to trust the `publish.yml` workflow in the GitHub `pypi`
-environment. From a clean branch, run:
+environment. Releases must start from a clean `main` that is synchronized with
+`origin/main`, with `gh` authenticated and the publishing workflow available. To bump
+the current version, run one of:
 
 ```sh
-make publish-version VERSION=0.2.0
+make publish patch
+make publish minor
+make publish major
 ```
 
-This installs Python 3.13, performs the same locked dependency sync used in CI, runs
-the full checks, updates `pyproject.toml` and `uv.lock`, builds and smoke-tests the
-distributions, creates a release commit and annotated `v0.2.0` tag, and atomically
-pushes the branch and tag. The tag starts the workflow on `ubuntu-latest`; the workflow
-rechecks the tag/version match, tests, builds, generates attestations, and publishes
-with PyPI trusted-publishing OIDC credentials. Use `REMOTE=name` for a different Git
-remote.
-
-If an unpushed commit already contains the requested version, use the guarded bypass:
+If the synchronized commit already contains the intended, not-yet-tagged version, use:
 
 ```sh
-make publish-version VERSION=0.2.0 ALLOW_PREBUMPED_VERSION=1
+make publish current
 ```
 
-### Direct local publishing
-
-To publish the version already present in `pyproject.toml` directly from this machine,
-configure a uv-supported PyPI credential such as `UV_PUBLISH_TOKEN`, then run:
+The release command runs the shared `make release-check` gate, updates
+`pyproject.toml` and `uv.lock` when needed, creates a release commit and annotated tag,
+atomically pushes `main` and the tag, and watches the trusted-publishing workflow to
+completion. Preview the full process without changing files, refs, remotes, or workflows
+with:
 
 ```sh
-make publish-local
+make publish patch DRY_RUN=1
 ```
 
-This runs all checks, builds and smoke-tests the current wheel and source archive, and
-passes only those two version-matched files to `uv publish`. It does not create a Git
-commit or tag.
+If the release tag exists but trusted publishing needs to be dispatched again, run
+`make publish-pypi`. This validates that the tag matches the current version and is
+reachable from `origin/main`, then dispatches and watches a new workflow run.
 
 ### Local self-hosted GitHub runner
 
@@ -517,11 +538,11 @@ inside Docker. The container has no host mounts and does not receive the Docker
 Authenticate `gh` with repository administration permission, then run:
 
 ```sh
-make runner-start
-make publish-self-hosted RELEASE_TAG=v0.2.0
+make publish-self-hosted
 ```
 
-The tag must already exist. This dispatches `publish.yml` from `main` with the
-`self-hosted` runner choice. `make runner-ensure` starts Docker Desktop on macOS when
-needed and repairs the runner before dispatch. Other lifecycle commands are
-`runner-status`, `runner-stop`, `runner-remove`, `runner-logs`, and `clean-runner`.
+The current version's tag must already exist. This ensures the local runner is online,
+dispatches `publish.yml` from `main` with the `self-hosted` runner choice, and watches
+the run. `make runner-ensure` starts Docker Desktop on macOS when needed and repairs the
+runner before dispatch. Other lifecycle commands are `runner-status`, `runner-stop`,
+`runner-remove`, `runner-logs`, and `clean-runner`.

{pl_e2e_test-0.1.7 → pl_e2e_test-0.2.0}/README.md

@@ -14,6 +14,8 @@ The package installs the reusable `prairielearn_e2e` Python API and a single
 - `pl-e2e-test parallel` runs local shards and merges their results.
 - `pl-e2e-test add-seed` records a failing concrete variant seed.
 - `pl-e2e-test reports` merges parallel shard results into terminal and JUnit reports.
+- `pl-e2e-test instances` shows configured, installed, cached, and local PrairieLearn
+  revisions.
 - `pl-e2e-test version` prints the installed version.
 - `pl-e2e-test help [COMMAND]` prints top-level help, or one command's help.
 
@@ -21,6 +23,7 @@ The package installs the reusable `prairielearn_e2e` Python API and a single
 
 - Python 3.13 or newer
 - Git 2.30 or newer
+- uv 0.9 or newer
 
 The harness depends on pytest, Pyright, Ruff, Chevron, JSON Schema, lxml, NumPy,
 and PrairieLearn. PrairieLearn's Python package is currently consumed from its Git
@@ -217,19 +220,34 @@ jobs:
 ```
 
 The course must provide `.prairielearn/schemas/infoQuestion.json`. The harness checks
-question metadata against that schema and uses the exact PrairieLearn revision
-installed in the uv environment. It stores a bare, partial PrairieLearn repository in
-the platform-standard user cache directory. The first use of a new Git source or
-revision requires network access; later runs reuse that exact revision offline. The
-uv lockfile, rather than the cache, decides which revision is tested.
+question metadata against that schema and resolves the configured PrairieLearn version
+before each run. The default, `latest`, resolves the Git source's remote `HEAD`; a full
+commit, branch, or unambiguous tag can be selected instead. Moving selectors require
+network access and never silently fall back to stale cache state.
+
+The selected revision's Python package and core elements are always used together. The
+harness keeps a blob-filtered bare repository in the platform-standard user cache and
+hydrates only `apps/prairielearn/python` and `apps/prairielearn/elements`. It launches
+the selected Python package through a frozen uv overlay, without changing the course's
+`pyproject.toml`, `uv.lock`, or virtual environment. A previously hydrated exact commit
+can be reused offline when its uv dependencies are also cached.
 
 To use an existing PrairieLearn worktree or bare repository instead, pass
 `--prairielearn-path /path/to/PrairieLearn`. The repository must already contain the
-installed revision. An explicit path is authoritative: the harness does not fetch into
-it or fall back to the managed cache. In either mode, only the pinned revision's Python
-element controllers are archived into the test run's temporary directory. Standard
-cache environment settings such as `XDG_CACHE_HOME` can relocate the managed cache;
-CI systems can persist that directory between runs if desired.
+selected revision. An explicit path is authoritative: the harness resolves selectors
+against its local refs, does not fetch into it, and does not fall back to the managed
+cache. For a local repository, `latest` means its current `HEAD`.
+
+Inspect available revisions without contacting remotes:
+
+```sh
+uv run pl-e2e-test instances
+```
+
+Add `--refresh` to resolve the configured moving selector and verify remote access.
+The table distinguishes fully hydrated runtime revisions from legacy element-only cache
+entries. Standard cache environment settings such as `XDG_CACHE_HOME` can relocate the
+managed cache; CI systems can persist that directory between runs if desired.
 
 ### Harness configuration
 
@@ -237,11 +255,17 @@ Selector settings are read automatically from the course's `pyproject.toml`:
 
 ```toml
 [tool.pl-e2e-test]
+prairielearnVersion = "latest"
 elementSolutionSelectors = ["pl-sketch"]
 nonGradableSelectors = ['pl-sketch[read-only="true"]', 'pl-sketch[read-only="1"]']
 ```
 
-These are the defaults. `elementSolutionSelectors` identifies elements that provide
+These are the defaults. Omitting `prairielearnVersion` is equivalent to `"latest"`.
+Use a full commit for reproducible or offline runs, or a branch/tag when a named moving
+target is intentional. `--prairielearn-version SELECTOR` overrides the TOML value for
+one invocation, and replay commands always print the resolved full commit.
+
+`elementSolutionSelectors` identifies elements that provide
 solutions without a `correct_answers` entry. It only relaxes that entry requirement;
 it does not generate a test submission. `nonGradableSelectors` excludes display-only
 answers from grading checks. Each supplied list replaces its default; omitted settings
@@ -279,7 +303,7 @@ does not provide one itself.
 For every gradable `answers-name`, the correct test case must provide:
 
 - a non-null value in `data["raw_submitted_answers"]` under the answer name, or one or
-  more compound input keys prefixed with `answers-name-`; and
+  more compound input keys prefixed with `answers-name-` or `answers-name.`; and
 - `data["partial_scores"][answers_name]["score"] = 1`.
 
 Raw values must survive PrairieLearn's JSON boundary and have the same browser transport
@@ -308,11 +332,12 @@ def test(data):
 ```
 
 Compound elements keep the declared answer name as their grading identity while using
-namespaced child fields for browser inputs. For example, an element with
-`answers-name="match"` may test and parse `match-dropdown-0` and `match-dropdown-1`,
-then store its score in `partial_scores["match"]`; it does not need to invent a
-`submitted_answers["match"]` value. After parsing, every raw field associated with a
-declared gradable answer must either remain non-null under its child key or produce a
+namespaced child fields for browser inputs. Hyphen and dot namespace boundaries are
+supported. For example, an element with `answers-name="match"` may test and parse
+`match-dropdown-0` and `match-dropdown-1`, while `answers-name="average"` may use
+`average.main`. The element stores its score under the declared answer name and does not
+need to invent a direct raw value for it. After parsing, every raw field associated with
+a declared gradable answer must either remain non-null under its child key or produce a
 non-null value under its declared parent answer. This also supports elements that
 normalize `answer-input` into `answer`. Unassociated transport fields from file-producing
 elements may instead be consumed into `submitted_answers["_files"]`; they are not answer
@@ -430,12 +455,16 @@ course's TOML settings, or `config=load_config("settings.toml")` for an explicit
 `load_config` and `HarnessConfig` are exported from `prairielearn_e2e`. Omitting
 `config` uses the built-in defaults.
 
+The command-line runner and high-level `run_parallel` API perform version resolution
+and uv bootstrapping. Manually constructed `PrairieLearnBackend` and `QuestionHarness`
+objects continue to use the PrairieLearn package already loaded by their interpreter.
+
 ## Development
 
 This project uses uv 0.9 or newer:
 
 ```sh
-uv sync
+make install
 make test
 make format
 make smoke-dist
@@ -447,44 +476,36 @@ into an isolated target, and imports it using the locked development dependencie
 
 ## Publishing
 
-There are three supported publishing paths. Choose one for a release; do not run the
-local and GitHub paths for the same version because PyPI versions are immutable.
-
-### GitHub-hosted trusted publishing
-
 Configure the PyPI project to trust the `publish.yml` workflow in the GitHub `pypi`
-environment. From a clean branch, run:
+environment. Releases must start from a clean `main` that is synchronized with
+`origin/main`, with `gh` authenticated and the publishing workflow available. To bump
+the current version, run one of:
 
 ```sh
-make publish-version VERSION=0.2.0
+make publish patch
+make publish minor
+make publish major
 ```
 
-This installs Python 3.13, performs the same locked dependency sync used in CI, runs
-the full checks, updates `pyproject.toml` and `uv.lock`, builds and smoke-tests the
-distributions, creates a release commit and annotated `v0.2.0` tag, and atomically
-pushes the branch and tag. The tag starts the workflow on `ubuntu-latest`; the workflow
-rechecks the tag/version match, tests, builds, generates attestations, and publishes
-with PyPI trusted-publishing OIDC credentials. Use `REMOTE=name` for a different Git
-remote.
-
-If an unpushed commit already contains the requested version, use the guarded bypass:
+If the synchronized commit already contains the intended, not-yet-tagged version, use:
 
 ```sh
-make publish-version VERSION=0.2.0 ALLOW_PREBUMPED_VERSION=1
+make publish current
 ```
 
-### Direct local publishing
-
-To publish the version already present in `pyproject.toml` directly from this machine,
-configure a uv-supported PyPI credential such as `UV_PUBLISH_TOKEN`, then run:
+The release command runs the shared `make release-check` gate, updates
+`pyproject.toml` and `uv.lock` when needed, creates a release commit and annotated tag,
+atomically pushes `main` and the tag, and watches the trusted-publishing workflow to
+completion. Preview the full process without changing files, refs, remotes, or workflows
+with:
 
 ```sh
-make publish-local
+make publish patch DRY_RUN=1
 ```
 
-This runs all checks, builds and smoke-tests the current wheel and source archive, and
-passes only those two version-matched files to `uv publish`. It does not create a Git
-commit or tag.
+If the release tag exists but trusted publishing needs to be dispatched again, run
+`make publish-pypi`. This validates that the tag matches the current version and is
+reachable from `origin/main`, then dispatches and watches a new workflow run.
 
 ### Local self-hosted GitHub runner
 
@@ -493,11 +514,11 @@ inside Docker. The container has no host mounts and does not receive the Docker
 Authenticate `gh` with repository administration permission, then run:
 
 ```sh
-make runner-start
-make publish-self-hosted RELEASE_TAG=v0.2.0
+make publish-self-hosted
 ```
 
-The tag must already exist. This dispatches `publish.yml` from `main` with the
-`self-hosted` runner choice. `make runner-ensure` starts Docker Desktop on macOS when
-needed and repairs the runner before dispatch. Other lifecycle commands are
-`runner-status`, `runner-stop`, `runner-remove`, `runner-logs`, and `clean-runner`.
+The current version's tag must already exist. This ensures the local runner is online,
+dispatches `publish.yml` from `main` with the `self-hosted` runner choice, and watches
+the run. `make runner-ensure` starts Docker Desktop on macOS when needed and repairs the
+runner before dispatch. Other lifecycle commands are `runner-status`, `runner-stop`,
+`runner-remove`, `runner-logs`, and `clean-runner`.

{pl_e2e_test-0.1.7 → pl_e2e_test-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pl-e2e-test"
-version = "0.1.7"
+version = "0.2.0"
 description = "Fuzz PrairieLearn question lifecycles against the real Python element runtime"
 readme = "README.md"
 requires-python = ">=3.13"