phigrade 3.4.0__tar.gz → 3.4.2__tar.gz

{phigrade-3.4.0 → phigrade-3.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: phigrade
-Version: 3.4.0
+Version: 3.4.2
 Summary: A lightweight autograder where tests run locally and data is stored remotely
 Project-URL: Repository, https://github.com/mld-instructors/phigrade
 Author-email: Matt Gormley <mgormley@cs.cmu.edu>, Jacob Rast <jrast@andrew.cmu.edu>
@@ -533,16 +533,29 @@ but a Colab notebook raises `PhiGradeUsageError`, naming the config key and sayi
 set it `false` for local runs, rather than silently submitting without the notebook the
 student believes they sent.
 
-> **Restart the session before the run you want recorded.** phigrade caches
-> the notebook it attaches, the submission it creates, and `phigrade.yaml`
-> itself for the life of the kernel — a plain `pytest` process is fresh every
-> time, but a notebook kernel is not. The first `%%ipytest` cell you run in a
-> session captures the notebook and the submission at that moment; editing
-> code and re-running `%%ipytest` afterwards submits again, but the attached
-> `.ipynb` and the config are still the ones from that first run, not your
-> latest edits. Before the run you actually want graded, use **Runtime →
-> Restart session and run all** so the notebook, the submission, and
-> `phigrade.yaml` are all captured fresh.
+> **Each `%%ipytest` run starts fresh, even without restarting the
+> kernel — with one caveat for the notebook itself.** phigrade caches the
+> notebook it attaches, the submission it creates, and `phigrade.yaml` at
+> module level — a plain `pytest` process is fresh every time, but a notebook
+> kernel is not, and this used to mean a second `%%ipytest` run reused the
+> first run's submission and never noticed later edits. A
+> `pytest_sessionstart` hook now clears that cache at the start of every
+> pytest session, so editing code or `phigrade.yaml` and re-running
+> `%%ipytest` submits your latest edits, not the first run's. What this
+> guarantees for the notebook itself is narrower: each run *re-asks Colab*
+> for it (`google.colab._message`'s `get_ipynb`) rather than replaying a
+> cached copy, but not that Colab answers with unsaved editor state. **Save
+> the notebook (Ctrl/Cmd+S) before the run you want graded.**
+>
+> **Exception: switching between teacher and student mode against the local
+> server needs a restart.** `use_local_server: true` starts one server
+> process for the life of the kernel, and that process is locked to whichever
+> mode — and `local_server_db_file` — it first saw: `app._setup_db` uses
+> `teacher_mode` at that moment to decide between the real reference file and
+> a throwaway temp copy, so a later, disagreeing config could otherwise read
+> or write the wrong one. phigrade refuses instead of doing that silently. If
+> you flip `PHIGRADE_TEACHER_MODE` or `teacher_mode` mid-kernel and hit that
+> error, use **Runtime → Restart session and run all**.
 
 > **Publish the answer key from the notebook, not from a `.py` file.** A
 > test defined in a cell belongs to module `__main__`, and the server
@@ -561,15 +574,20 @@ student believes they sent.
 > ```
 >
 > or set `teacher_mode: true` in the `%%writefile phigrade.yaml` cell.
-> Either way, use **Runtime → Restart session and run all** (see the
-> stale-notebook warning above) so the setting is in effect before the
-> tests run.
+> Either way, re-run the `%%ipytest` cell after setting it — the next pytest
+> session reloads the environment and `phigrade.yaml` fresh (see above). If
+> you are running against the local server and this switches modes from an
+> earlier run in the same kernel, restart the session first (see the
+> exception above), or the run raises rather than publishing.
 
 ## Configuration
 
 The only configuration source is a YAML file named `phigrade.yaml`, located **in the
 directory containing the test module** that defines the `@weight` function — not the
-current working directory. It is loaded once and cached for the life of the process.
+current working directory. It is loaded once per pytest session and cached for
+the rest of that session — one session per `pytest` process outside a notebook,
+and, since the [Colab notebooks](#colab-notebooks) section above, one session per
+`%%ipytest` run in a single kernel.
 
 Five keys can be overridden from the environment — see
 [Environment overrides](#environment-overrides) below. No other key can.
@@ -628,7 +646,8 @@ Local answer key: slot_a_key.json (PHIGRADE_LOCAL_DB_FILE)
 
 That line is the whole point of announcing: a variable exported once in a shell profile
 must not silently republish an answer key, or redirect a submission, weeks later. The
-config is loaded once per process, so each line appears at most once per `pytest` run.
+config is loaded once per pytest session, so each line appears at most once per
+session (see [Configuration](#configuration) above).
 
 ### Submission Slots
 
@@ -667,9 +686,10 @@ Either way the resolved slot is announced — `Submitting to: Human only` — so
 submissions. On the environment-variable path that line goes to stdout, which
 pytest captures and prints on failure (or always, under `-s`).
 
-The prompt reads and writes `/dev/tty` directly rather than stdin/stdout, so it
-works under a plain `pytest` run **with no `-s` flag** — pytest's output capture
-never sees it.
+The prompt reads and writes the terminal device directly rather than stdin/stdout
+— `/dev/tty` on macOS and Linux, and `CONIN$`/`CONOUT$` on Windows, which has no
+single duplex device — so it works under a plain `pytest` run **with no `-s`
+flag**: pytest's output capture never sees it.
 
 Some runs have no terminal that can be prompted on, and they **must** set
 `PHIGRADE_SLOT` to one of the slot names:
@@ -678,14 +698,18 @@ Some runs have no terminal that can be prompted on, and they **must** set
 export PHIGRADE_SLOT="Human only"
 ```
 
-That covers non-interactive environments — a Gradescope autograder, CI — and two
-cases that would otherwise hang: `pytest -n auto` (each xdist worker is its own
-process, so N workers would interleave N prompts on one terminal) and any
-background job (`nohup pytest &` under a job-control shell, where reading the
-terminal raises `SIGTTIN` and suspends the process). Both are detected and turned
-into the same error naming the valid slot names.
-
-Because the config is loaded once and cached for the life of the process (see
+That covers non-interactive environments — a Gradescope autograder, CI, or a
+Windows process with no console at all (a `pythonw` run, or native Windows
+Python under Git Bash/mintty) — and three cases that would otherwise hang:
+`pytest -n auto` (each xdist worker is its own process, so N workers would
+interleave N prompts on one terminal), any background job (`nohup pytest &`
+under a job-control shell, where reading the terminal raises `SIGTTIN` and
+suspends the process — POSIX only), and a notebook kernel, which on Windows
+can inherit the console of whatever launched it and would prompt into a window
+the student is not looking at. Each is detected and turned into the same error
+naming the valid slot names.
+
+Because the config is loaded once and cached for the rest of the session (see
 above), a `pytest` run resolves the slot — and therefore prompts, if it prompts at
 all — at most once, no matter how many tests run.
 
@@ -698,7 +722,7 @@ gets a `Sub-unit-check reference not found` error on every checkpoint, naming th
 assessment, module, function and checkpoint it looked for.
 
 So an instructor must publish once per slot. Because the config is cached per
-process, that means one process per slot:
+session, that means one `pytest` process (or one `%%ipytest` run) per slot:
 
 ```bash
 PHIGRADE_TEACHER_MODE=true PHIGRADE_SLOT="Human only" pytest test_mysolution.py

{phigrade-3.4.0 → phigrade-3.4.2}/README.md

@@ -508,16 +508,29 @@ but a Colab notebook raises `PhiGradeUsageError`, naming the config key and sayi
 set it `false` for local runs, rather than silently submitting without the notebook the
 student believes they sent.
 
-> **Restart the session before the run you want recorded.** phigrade caches
-> the notebook it attaches, the submission it creates, and `phigrade.yaml`
-> itself for the life of the kernel — a plain `pytest` process is fresh every
-> time, but a notebook kernel is not. The first `%%ipytest` cell you run in a
-> session captures the notebook and the submission at that moment; editing
-> code and re-running `%%ipytest` afterwards submits again, but the attached
-> `.ipynb` and the config are still the ones from that first run, not your
-> latest edits. Before the run you actually want graded, use **Runtime →
-> Restart session and run all** so the notebook, the submission, and
-> `phigrade.yaml` are all captured fresh.
+> **Each `%%ipytest` run starts fresh, even without restarting the
+> kernel — with one caveat for the notebook itself.** phigrade caches the
+> notebook it attaches, the submission it creates, and `phigrade.yaml` at
+> module level — a plain `pytest` process is fresh every time, but a notebook
+> kernel is not, and this used to mean a second `%%ipytest` run reused the
+> first run's submission and never noticed later edits. A
+> `pytest_sessionstart` hook now clears that cache at the start of every
+> pytest session, so editing code or `phigrade.yaml` and re-running
+> `%%ipytest` submits your latest edits, not the first run's. What this
+> guarantees for the notebook itself is narrower: each run *re-asks Colab*
+> for it (`google.colab._message`'s `get_ipynb`) rather than replaying a
+> cached copy, but not that Colab answers with unsaved editor state. **Save
+> the notebook (Ctrl/Cmd+S) before the run you want graded.**
+>
+> **Exception: switching between teacher and student mode against the local
+> server needs a restart.** `use_local_server: true` starts one server
+> process for the life of the kernel, and that process is locked to whichever
+> mode — and `local_server_db_file` — it first saw: `app._setup_db` uses
+> `teacher_mode` at that moment to decide between the real reference file and
+> a throwaway temp copy, so a later, disagreeing config could otherwise read
+> or write the wrong one. phigrade refuses instead of doing that silently. If
+> you flip `PHIGRADE_TEACHER_MODE` or `teacher_mode` mid-kernel and hit that
+> error, use **Runtime → Restart session and run all**.
 
 > **Publish the answer key from the notebook, not from a `.py` file.** A
 > test defined in a cell belongs to module `__main__`, and the server
@@ -536,15 +549,20 @@ student believes they sent.
 > ```
 >
 > or set `teacher_mode: true` in the `%%writefile phigrade.yaml` cell.
-> Either way, use **Runtime → Restart session and run all** (see the
-> stale-notebook warning above) so the setting is in effect before the
-> tests run.
+> Either way, re-run the `%%ipytest` cell after setting it — the next pytest
+> session reloads the environment and `phigrade.yaml` fresh (see above). If
+> you are running against the local server and this switches modes from an
+> earlier run in the same kernel, restart the session first (see the
+> exception above), or the run raises rather than publishing.
 
 ## Configuration
 
 The only configuration source is a YAML file named `phigrade.yaml`, located **in the
 directory containing the test module** that defines the `@weight` function — not the
-current working directory. It is loaded once and cached for the life of the process.
+current working directory. It is loaded once per pytest session and cached for
+the rest of that session — one session per `pytest` process outside a notebook,
+and, since the [Colab notebooks](#colab-notebooks) section above, one session per
+`%%ipytest` run in a single kernel.
 
 Five keys can be overridden from the environment — see
 [Environment overrides](#environment-overrides) below. No other key can.
@@ -603,7 +621,8 @@ Local answer key: slot_a_key.json (PHIGRADE_LOCAL_DB_FILE)
 
 That line is the whole point of announcing: a variable exported once in a shell profile
 must not silently republish an answer key, or redirect a submission, weeks later. The
-config is loaded once per process, so each line appears at most once per `pytest` run.
+config is loaded once per pytest session, so each line appears at most once per
+session (see [Configuration](#configuration) above).
 
 ### Submission Slots
 
@@ -642,9 +661,10 @@ Either way the resolved slot is announced — `Submitting to: Human only` — so
 submissions. On the environment-variable path that line goes to stdout, which
 pytest captures and prints on failure (or always, under `-s`).
 
-The prompt reads and writes `/dev/tty` directly rather than stdin/stdout, so it
-works under a plain `pytest` run **with no `-s` flag** — pytest's output capture
-never sees it.
+The prompt reads and writes the terminal device directly rather than stdin/stdout
+— `/dev/tty` on macOS and Linux, and `CONIN$`/`CONOUT$` on Windows, which has no
+single duplex device — so it works under a plain `pytest` run **with no `-s`
+flag**: pytest's output capture never sees it.
 
 Some runs have no terminal that can be prompted on, and they **must** set
 `PHIGRADE_SLOT` to one of the slot names:
@@ -653,14 +673,18 @@ Some runs have no terminal that can be prompted on, and they **must** set
 export PHIGRADE_SLOT="Human only"
 ```
 
-That covers non-interactive environments — a Gradescope autograder, CI — and two
-cases that would otherwise hang: `pytest -n auto` (each xdist worker is its own
-process, so N workers would interleave N prompts on one terminal) and any
-background job (`nohup pytest &` under a job-control shell, where reading the
-terminal raises `SIGTTIN` and suspends the process). Both are detected and turned
-into the same error naming the valid slot names.
-
-Because the config is loaded once and cached for the life of the process (see
+That covers non-interactive environments — a Gradescope autograder, CI, or a
+Windows process with no console at all (a `pythonw` run, or native Windows
+Python under Git Bash/mintty) — and three cases that would otherwise hang:
+`pytest -n auto` (each xdist worker is its own process, so N workers would
+interleave N prompts on one terminal), any background job (`nohup pytest &`
+under a job-control shell, where reading the terminal raises `SIGTTIN` and
+suspends the process — POSIX only), and a notebook kernel, which on Windows
+can inherit the console of whatever launched it and would prompt into a window
+the student is not looking at. Each is detected and turned into the same error
+naming the valid slot names.
+
+Because the config is loaded once and cached for the rest of the session (see
 above), a `pytest` run resolves the slot — and therefore prompts, if it prompts at
 all — at most once, no matter how many tests run.
 
@@ -673,7 +697,7 @@ gets a `Sub-unit-check reference not found` error on every checkpoint, naming th
 assessment, module, function and checkpoint it looked for.
 
 So an instructor must publish once per slot. Because the config is cached per
-process, that means one process per slot:
+session, that means one `pytest` process (or one `%%ipytest` run) per slot:
 
 ```bash
 PHIGRADE_TEACHER_MODE=true PHIGRADE_SLOT="Human only" pytest test_mysolution.py

{phigrade-3.4.0 → phigrade-3.4.2}/examples/colab/README.md

@@ -42,11 +42,21 @@ are half of each unit check's identity, so the two must not drift. The notebooks
 generated from one shared definition for exactly this reason — if you edit the tests,
 edit both.
 
-**Restart the session before the run that counts.** phigrade captures the attached
-notebook, the submission, and `phigrade.yaml` on the first `%%ipytest` cell of a
-kernel session and reuses them until the kernel restarts. Edit-and-re-run still
-re-grades, but the attached `.ipynb` remains the one from that first run. Both
-notebooks say so at the point where it matters.
+**Each `%%ipytest` run starts fresh — save before the run that counts.** A
+`pytest_sessionstart` hook resets phigrade's cached submission and config at the
+start of every pytest session, so editing code or `phigrade.yaml` and re-running
+`%%ipytest` submits your latest edits, not the first run's — no restart needed for
+that. The guarantee is narrower for the notebook itself: each run re-asks Colab for
+it rather than replaying a cached copy, not that Colab returns unsaved editor
+state, so save the notebook (Ctrl/Cmd+S) before the run you want graded. A restart
+is still needed if you run this example against a local server
+(`use_local_server: true`, see "Trying it without Colab" below): the server
+process is locked to whichever mode it first saw, so phigrade now refuses a
+mid-kernel switch between teacher and student mode rather than misroute a
+submission, and re-publishing the answer key from `lab1_instructor.ipynb` in the
+same kernel still needs a restart too, since the local server's answer-key file
+accumulates duplicate rows across sessions otherwise. Both notebooks say so at the
+point where it matters.
 
 ## Trying it without Colab
 

{phigrade-3.4.0 → phigrade-3.4.2}/examples/colab/lab1_instructor.ipynb

@@ -176,10 +176,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "> **Re-publishing?** Use **Runtime -> Restart session and run all**.\n",
-    "> phigrade caches the config and the reference attempt for the life of\n",
-    "> the kernel, so a second `%%ipytest` run in the same session does not\n",
-    "> start a clean publish."
+    "> **Re-publishing?** Use **Runtime -> Restart session and run all** if\n",
+    "> you're running against a local server (`use_local_server: true`).\n",
+    "> phigrade's cached config and reference attempt now reset automatically\n",
+    "> at the start of every `%%ipytest` run, so a second run in the same\n",
+    "> *hosted* session does start a clean publish. Against a local server,\n",
+    "> restart anyway: the server process is locked to whichever mode it first\n",
+    "> saw, so phigrade now refuses a mode switch mid-kernel rather than\n",
+    "> misroute a submission, and a second publish in the same kernel still\n",
+    "> appends to the local answer-key file instead of replacing it."
    ]
   }
  ],

{phigrade-3.4.0 → phigrade-3.4.2}/examples/colab/lab1_student.ipynb

@@ -150,13 +150,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "> **Before the run you want graded: Runtime -> Restart session and run all.**\n",
+    "> **Save before the run you want graded.**\n",
     ">\n",
-    "> phigrade captures the notebook it attaches, the submission it creates, and\n",
-    "> `phigrade.yaml` on the *first* `%%ipytest` cell of a session, and reuses them\n",
-    "> for the life of the kernel. If you edit your code and re-run, the grade\n",
-    "> updates but the attached notebook is still the one from that first run.\n",
-    "> Restarting makes all three fresh."
+    "> phigrade resets its cached submission and config at the start of every\n",
+    "> `%%ipytest` run, so editing your code and re-running submits your latest\n",
+    "> work -- no restart needed for that. The guarantee is narrower for the\n",
+    "> notebook itself: each run re-asks Colab for it rather than reusing an old\n",
+    "> copy, not that Colab returns unsaved editor state. Save the notebook\n",
+    "> (Ctrl/Cmd+S) before the run you want graded."
    ]
   }
  ],

phigrade-3.4.2/examples/group/README.md

@@ -0,0 +1,71 @@
+# A group submission
+
+A worked example of the `group_members` key from
+[the client README's configuration table](../../README.md#configuration): one
+student runs the tests, and every student named in the config gets the grade.
+
+| File | What it is |
+| --- | --- |
+| `phigrade.yaml` | A complete group configuration, commented. The example *is* the config — there is no test file here, because nothing about writing tests changes when a submission is shared. |
+
+Copy it beside the test module that defines your `@weight` functions —
+`phigrade.yaml` is looked up next to that file, not in the working directory —
+fill in `course_id`, `assessment_id` and `server_url`, and put both group members'
+addresses in `group_members`.
+
+Your instructor has to allow it first: **Group Size** on the assessment settings
+page is `1` by default, which means students submit individually, and a submission
+carrying a group is refused with a message saying so.
+
+## Three things this example exists to show
+
+**List the whole group, including yourself.** Not "your partners" — everyone. Both
+students work in the same repository, so both commit the same file, and both
+machines tag every attempt with the same group. That symmetry is what makes the
+resolution agree for both of you: your grade comes from the most recently
+submitted active attempt among the attempts you are on, and if the two of you
+described the group differently, that sentence would resolve differently for each
+of you. A list that omits your own address is refused rather than quietly guessed
+at.
+
+**Read the line it prints.** A group submission announces itself:
+
+```
+Created submission 8f2c1a4e-…
+Submitting as a group: alice@andrew.cmu.edu, bob@andrew.cmu.edu
+```
+
+That is the only guard against a silent solo submission, and the failure it
+guards against — believing you submitted together and discovering otherwise when
+grades come out — has no undo. In Colab, use `%%ipytest -s`; plain `%%ipytest`
+captures stdout and the line never reaches you. The client README's
+[Colab notebooks](../../README.md#colab-notebooks) section says the same thing
+about submission slots, for the same reason.
+
+**Only one of you needs to run the tests.** The partner who never ran anything
+still sees the submission and its per-unit-test breakdown in the web app, on both
+the Attempts page and the assessment Overview, labelled with who submitted it.
+That is the whole feature: the work is graded once and counts for everyone on it.
+
+## What this does not do
+
+There are no persistent teams. The group is named at submission time and restated
+by submitting again — change `group_members` and re-run, and the new list is what
+counts. There is no invitation or acceptance step either, so a submission can name
+you without asking; it is visible to you and to staff the moment it lands, and
+submitting your own work makes yours the one that counts again.
+
+Group submissions also work on the rubric-graded PDF side, the same rule
+applying to a scanned upload as to a code submission. The group is named in a
+different place on each side: `group_members` in `phigrade.yaml` here, versus
+a "Partners" field on the PDF upload form in the web application.
+
+One PDF-specific wrinkle to know before you rely on it: if your partner
+uploads a PDF alone after you two submitted together, that solo upload
+replaces your shared submission for **both** of you, and you are left with
+no active submission at all until you submit again yourself. On the coding
+side this does not happen — a partner's solo re-submission only replaces
+their own attempt, and yours keeps counting. So on the PDF side, treat a
+group submission as something either partner can end for both of you by
+submitting alone; if that happens, re-upload to have your own work count
+again.

phigrade-3.4.2/examples/group/phigrade.yaml

@@ -0,0 +1,61 @@
+# A group submission: one student runs the tests, the whole group gets the grade.
+#
+# Drop this beside the test module that defines your `@weight` functions --
+# `phigrade.yaml` is looked up next to that file, not in the working directory.
+# Replace course_id, assessment_id and server_url with your own.
+
+course_id: "your-course-uuid"
+assessment_id: "your-assessment-uuid"
+
+use_local_server: false
+server_url: "https://phigrade.example.org"
+
+# Every student this submission is for, INCLUDING YOURSELF.
+#
+# Not "your partners" -- everyone. Both of you work in the same repository, so
+# both of you commit this same file, and both machines then tag every attempt
+# with the same group. That symmetry is the point: the server resolves your
+# grade to the most recently submitted active attempt among the attempts you
+# are on, so listing the group identically on both sides is what makes it give
+# both of you the same answer.
+#
+# Leaving this out entirely is an ordinary solo submission.
+group_members:
+  - alice@andrew.cmu.edu
+  - bob@andrew.cmu.edu
+
+teacher_mode: false
+submission_files:
+  - solution.py
+
+# ---------------------------------------------------------------------------
+# What to expect when you run this
+#
+# Every group submission prints its group, right after the submission id:
+#
+#     Created submission 8f2c1a4e-...
+#     Submitting as a group: alice@andrew.cmu.edu, bob@andrew.cmu.edu
+#
+# Read that line. It is the only thing standing between you and a silent solo
+# submission -- believing you submitted as a group and finding out at grade
+# release has no undo. In a Colab notebook use `%%ipytest -s`: plain
+# `%%ipytest` captures stdout and swallows the line.
+#
+# Four things are refused, each with 422 and a message naming the problem:
+#
+#   * the assessment does not allow groups -- "Group submissions are not
+#     enabled for this assessment." Your instructor sets Group Size in the
+#     assessment settings; it is 1 by default, which means submit individually.
+#   * the group is larger than that setting.
+#   * an address is not an active student on this course. Staff are refused
+#     here too, by their own address: group submissions are for students.
+#   * the list does not contain your own address (see above -- list everyone).
+#
+# Addresses are stripped, lower-cased and de-duplicated before they are sent,
+# so spacing and capitalisation here do not matter.
+#
+# `PHIGRADE_GROUP_MEMBERS="alice@andrew.cmu.edu,bob@andrew.cmu.edu"` overrides
+# this key for one run, which is what a CI job or a scripted submission wants.
+# It announces itself on stdout when it takes effect. The file is the normal
+# path -- in Colab as much as on a laptop, since a Colab session writes this
+# file from a `%%writefile phigrade.yaml` cell.