verifaied 0.32.0.dev80__tar.gz → 0.33.0.dev81__tar.gz

{verifaied-0.32.0.dev80 → verifaied-0.33.0.dev81}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: verifaied
-Version: 0.32.0.dev80
+Version: 0.33.0.dev81
 Summary: Find what's untested in your code — locally, no account required
 Project-URL: Homepage, https://pypi.org/project/verifaied/
 Author: Kyle Richards
@@ -479,6 +479,10 @@ verifaied test run "Settings/API tokens" --video watch
 # On a busy CI machine: replay a failed test once more, from a fresh reset.
 verifaied test run --all --retries 1
 
+# Replay four at a time — one per copy of the app the environment
+# declares (see "Running several at once" below).
+verifaied test run --all --parallel 4
+
 # What's recorded, and how each last went on this branch.
 verifaied test list
 
@@ -820,6 +824,45 @@ origin — scheme and host, no path — and a spec with a relative location
 in a repo that names none is refused up front, rather than dying on
 Playwright's "invalid URL" halfway through a run.
 
+#### Running several at once
+
+A reset before every run is what makes a flow repeatable, and it is also
+what stops two runs sharing one app: each would wipe the other's data
+mid-flow. So running side by side needs one copy of the app per extra
+run, each with its own reset — a **lane**:
+
+```toml
+[browser-tests.env.local]
+base_url = "http://localhost:3000"
+reset = "make db-reset"
+
+# One more copy of the app, for a second run beside the first.
+[[browser-tests.env.local.lanes]]
+base_url = "http://localhost:3100"
+reset = "make db-reset SLOT=1"
+```
+
+That environment runs two at once. An environment with **no** reset (a
+deployed one using `unique` variables) shares its one app between up to
+eight runs; any other runs one at a time. Lanes are checked when the file
+is read: each needs its own `base_url`, a `reset` exactly when its
+environment has one, and no two may point at the same app.
+
+How many to run is asked for separately, because it is about the
+batch rather than the machine:
+
+- **From the web**, set **Tests run at once** on the repository's
+  Settings page (Browser tests → Running tests), or `parallel_runs = 4`
+  under `[browser-tests]`. A studio then replays a Run all or a folder's
+  Run that many at a time, headless, up to what its environment can
+  isolate — and the Browser Tests page says so when it is fewer.
+  Recordings and single runs always take turns.
+- **From the terminal**, `verifaied test run --all --parallel 4` (or
+  `--folder`). Only the flag does this: the web setting never changes how
+  a scripted run behaves. Asking for more than the environment has lanes
+  for runs what it can and says why; `--parallel` with a single test or
+  with `--proxy` is refused.
+
 ### Variables — the data that differs, and the data that must be fresh
 
 A login is one flow with three passwords. A "create a token called
@@ -1054,12 +1097,23 @@ cloud can stop it. A request that fails is reported with the reason and
 the studio carries on — one bad pre-step chain is not a reason for it to
 die.
 
+**Several at once.** With **Tests run at once** above 1 on the
+repository's Settings page and an environment that can isolate them
+(see *Running several at once* above), the studio starts each headless
+run of a batch on a lane of its own and goes straight back for the next.
+A recording, an edit or a headed run queued behind the batch waits until
+the lanes are empty, and is never overtaken by the rest of the batch.
+The pill says how many it runs at once, and the page shows the studio's
+own sentence when the machine runs fewer than asked.
+
 **Stopping a run from the web.** While a run is replaying, the spinning
 Run button on the page turns into **Stop**. The studio hears it on its next
 beat, ends the replay's whole process group (SIGTERM, then SIGKILL), and
 records the run as failed with "Stopped from the web"; its video and trace
-are not kept. This uses POSIX process groups (`os.killpg`), so it needs
-macOS or Linux — the studio does not support Windows.
+are not kept. **Stop runs** in the page's header ends the whole batch at
+once — every run in progress, and everything still queued. This uses POSIX
+process groups (`os.killpg`), so it needs macOS or Linux — the studio does
+not support Windows.
 
 **Branches.** The branch is re-detected on every beat, so switching
 branches under a running studio moves what the web shows and what the next
@@ -1154,7 +1208,11 @@ run; the **last** one is the verdict, the coverage and the video and trace
 the test keeps (a failed attempt that is retried keeps its verdict and its
 failure text but uploads no video or trace). A refusal — a missing spec, a
 variable this machine cannot supply, a reset that failed — is never
-retried, and neither is a clean run whose coverage upload failed (`2`). `studio` takes `--no-replay` and takes no test name at all —
+retried, and neither is a clean run whose coverage upload failed (`2`).
+`--parallel <n>` (1–8, default 1, `--all`/`--folder` only) replays up to
+`n` at once, one per lane of the environment — see *Running several at
+once*; retries stay in the lane that owns the test, and Ctrl-C stops every
+lane and exits `130`. `studio` takes `--no-replay` and takes no test name at all —
 what it records is whatever the web asks for.
 
 `sync` takes `--repo`, `--branch` (the branch a new test is stamped with),

{verifaied-0.32.0.dev80 → verifaied-0.33.0.dev81}/README.md

@@ -433,6 +433,10 @@ verifaied test run "Settings/API tokens" --video watch
 # On a busy CI machine: replay a failed test once more, from a fresh reset.
 verifaied test run --all --retries 1
 
+# Replay four at a time — one per copy of the app the environment
+# declares (see "Running several at once" below).
+verifaied test run --all --parallel 4
+
 # What's recorded, and how each last went on this branch.
 verifaied test list
 
@@ -774,6 +778,45 @@ origin — scheme and host, no path — and a spec with a relative location
 in a repo that names none is refused up front, rather than dying on
 Playwright's "invalid URL" halfway through a run.
 
+#### Running several at once
+
+A reset before every run is what makes a flow repeatable, and it is also
+what stops two runs sharing one app: each would wipe the other's data
+mid-flow. So running side by side needs one copy of the app per extra
+run, each with its own reset — a **lane**:
+
+```toml
+[browser-tests.env.local]
+base_url = "http://localhost:3000"
+reset = "make db-reset"
+
+# One more copy of the app, for a second run beside the first.
+[[browser-tests.env.local.lanes]]
+base_url = "http://localhost:3100"
+reset = "make db-reset SLOT=1"
+```
+
+That environment runs two at once. An environment with **no** reset (a
+deployed one using `unique` variables) shares its one app between up to
+eight runs; any other runs one at a time. Lanes are checked when the file
+is read: each needs its own `base_url`, a `reset` exactly when its
+environment has one, and no two may point at the same app.
+
+How many to run is asked for separately, because it is about the
+batch rather than the machine:
+
+- **From the web**, set **Tests run at once** on the repository's
+  Settings page (Browser tests → Running tests), or `parallel_runs = 4`
+  under `[browser-tests]`. A studio then replays a Run all or a folder's
+  Run that many at a time, headless, up to what its environment can
+  isolate — and the Browser Tests page says so when it is fewer.
+  Recordings and single runs always take turns.
+- **From the terminal**, `verifaied test run --all --parallel 4` (or
+  `--folder`). Only the flag does this: the web setting never changes how
+  a scripted run behaves. Asking for more than the environment has lanes
+  for runs what it can and says why; `--parallel` with a single test or
+  with `--proxy` is refused.
+
 ### Variables — the data that differs, and the data that must be fresh
 
 A login is one flow with three passwords. A "create a token called
@@ -1008,12 +1051,23 @@ cloud can stop it. A request that fails is reported with the reason and
 the studio carries on — one bad pre-step chain is not a reason for it to
 die.
 
+**Several at once.** With **Tests run at once** above 1 on the
+repository's Settings page and an environment that can isolate them
+(see *Running several at once* above), the studio starts each headless
+run of a batch on a lane of its own and goes straight back for the next.
+A recording, an edit or a headed run queued behind the batch waits until
+the lanes are empty, and is never overtaken by the rest of the batch.
+The pill says how many it runs at once, and the page shows the studio's
+own sentence when the machine runs fewer than asked.
+
 **Stopping a run from the web.** While a run is replaying, the spinning
 Run button on the page turns into **Stop**. The studio hears it on its next
 beat, ends the replay's whole process group (SIGTERM, then SIGKILL), and
 records the run as failed with "Stopped from the web"; its video and trace
-are not kept. This uses POSIX process groups (`os.killpg`), so it needs
-macOS or Linux — the studio does not support Windows.
+are not kept. **Stop runs** in the page's header ends the whole batch at
+once — every run in progress, and everything still queued. This uses POSIX
+process groups (`os.killpg`), so it needs macOS or Linux — the studio does
+not support Windows.
 
 **Branches.** The branch is re-detected on every beat, so switching
 branches under a running studio moves what the web shows and what the next
@@ -1108,7 +1162,11 @@ run; the **last** one is the verdict, the coverage and the video and trace
 the test keeps (a failed attempt that is retried keeps its verdict and its
 failure text but uploads no video or trace). A refusal — a missing spec, a
 variable this machine cannot supply, a reset that failed — is never
-retried, and neither is a clean run whose coverage upload failed (`2`). `studio` takes `--no-replay` and takes no test name at all —
+retried, and neither is a clean run whose coverage upload failed (`2`).
+`--parallel <n>` (1–8, default 1, `--all`/`--folder` only) replays up to
+`n` at once, one per lane of the environment — see *Running several at
+once*; retries stay in the lane that owns the test, and Ctrl-C stops every
+lane and exits `130`. `studio` takes `--no-replay` and takes no test name at all —
 what it records is whatever the web asks for.
 
 `sync` takes `--repo`, `--branch` (the branch a new test is stamped with),

{verifaied-0.32.0.dev80 → verifaied-0.33.0.dev81}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "verifaied"
-version = "0.32.0.dev80"
+version = "0.33.0.dev81"
 description = "Find what's untested in your code — locally, no account required"
 readme = "README.md"
 requires-python = ">=3.10"

{verifaied-0.32.0.dev80 → verifaied-0.33.0.dev81}/src/verifaied/cli.py

@@ -48,6 +48,8 @@ from verifaied.record import DEFAULT_INTERVAL, DEFAULT_PORT, run_record
 from verifaied.repo_config import (
     CONFIG_PATH,
     ENV_KEY,
+    MAX_PARALLEL_RUNS,
+    MIN_PARALLEL_RUNS,
     SECTION,
     RepoConfigError,
     VideoProfile,
@@ -2064,6 +2066,18 @@ def test_run_command(
             "is the verdict and coverage recorded. Refusals are never retried."
         ),
     ),
+    parallel: int = typer.Option(
+        1,
+        "--parallel",
+        min=MIN_PARALLEL_RUNS,
+        max=MAX_PARALLEL_RUNS,
+        help=(
+            "Replay up to N tests at once (--all/--folder only). Needs an "
+            "environment with no reset, or one that declares "
+            "[[browser-tests.env.<name>.lanes]]; otherwise runs one at a time "
+            "and says why."
+        ),
+    ),
 ) -> None:
     """Replay a recorded test, capturing video, trace and coverage.
 
@@ -2093,6 +2107,11 @@ def test_run_command(
     each from a fresh reset. The last attempt is the verdict and the
     coverage the test keeps; a refusal is never retried.
 
+    ``--parallel N`` replays up to N tests of an ``--all`` or ``--folder``
+    run at once, one per lane of the environment — each lane its own copy
+    of the app with its own reset. Only this flag turns it on: the
+    Settings page's "Tests run at once" is for the studio.
+
     Exit codes:
 
       0  every test replayed cleanly
@@ -2125,6 +2144,7 @@ def test_run_command(
             proxy_port=proxy_port,
             video=video.value if video is not None else None,
             retries=retries,
+            parallel=parallel,
         )
     except (RecordedTestError, UploaderError) as e:
         err_console.print(f"[red]VerifAIed error[/red]: {e}")

{verifaied-0.32.0.dev80 → verifaied-0.33.0.dev81}/src/verifaied/client.py

@@ -723,6 +723,10 @@ def post_recorder_heartbeat(
     environment: str | None = None,
     busy: bool = False,
     working: UUID | None = None,
+    working_ids: list[UUID] | None = None,
+    take_kinds: list[str] | None = None,
+    lanes: int | None = None,
+    lanes_note: str | None = None,
     timeout: float = 30.0,
 ) -> dict[str, Any]:
     """Say a studio is listening, and collect what to record next.
@@ -761,6 +765,15 @@ def post_recorder_heartbeat(
     becoming a status code because presence must keep working when the
     branch allowance is spent. A service that predates any of this sends
     none of the three, which reads as nothing to say.
+
+    The lane fields are a studio replaying several headless runs at once:
+    ``working_ids`` names every run in flight, ``take_kinds`` narrows what
+    this beat may be handed to another headless run, and ``lanes`` /
+    ``lanes_note`` say how many it runs at once and why not more. Each is
+    sent only when given, so a studio running one at a time beats exactly
+    as it always did. The answer's ``parallel_runs`` is what the
+    repository's Settings page asks for; a service that predates lanes
+    leaves the key out.
     """
     request_url = f"{api_url.rstrip('/')}/repositories/{repo_id}/recorder-heartbeat"
     body: dict[str, Any] = {"branch": branch, "closing": closing}
@@ -774,6 +787,14 @@ def post_recorder_heartbeat(
         # Which run this busy studio is on, so the service does not stop it
         # as a run whose studio went away. Sent only when named.
         body["working"] = str(working)
+    if working_ids:
+        body["working_ids"] = [str(i) for i in working_ids]
+    if take_kinds is not None:
+        body["take_kinds"] = take_kinds
+    if lanes is not None:
+        body["lanes"] = lanes
+    if lanes_note is not None:
+        body["lanes_note"] = lanes_note
     try:
         response = _http().post(
             request_url,

verifaied-0.33.0.dev81/src/verifaied/lanes.py

@@ -0,0 +1,108 @@
+"""Several headless replays at once, one per lane.
+
+A lane is one copy of the app a replay can run against without another
+replay's reset wiping it (``lane_capacity`` / ``lane_env`` in
+``repo_config.py`` decide how many there are and where each points).
+This module is only the bookkeeping: which lane is free, which request
+each lane is on, and how a stop reaches the one thread that can act on
+it. It knows nothing about replays, so the studio and ``test run
+--parallel`` share it and its tests need no browser.
+
+Each lane runs on a daemon thread of its own, and removes itself from
+the pool when its work returns or raises — a lane that died must never
+read as busy forever.
+"""
+
+from __future__ import annotations
+
+import threading
+from collections.abc import Callable, Iterable
+from typing import Protocol
+from uuid import UUID
+
+
+class LaneStop(Protocol):
+    """What a lane's work is handed to be stopped by (``RunStop``)."""
+
+    def set(self) -> None: ...
+
+    def interrupt(self, message: str) -> None: ...
+
+
+class _Lane:
+    def __init__(self, request_id: UUID, stop: LaneStop) -> None:
+        self.request_id = request_id
+        self.stop = stop
+        self.thread: threading.Thread | None = None
+
+
+class LanePool:
+    """The lanes in flight, keyed by lane number."""
+
+    def __init__(self, make_stop: Callable[[], LaneStop]) -> None:
+        self._make_stop = make_stop
+        self._lanes: dict[int, _Lane] = {}
+        self._lock = threading.Lock()
+
+    def in_flight(self) -> list[UUID]:
+        """The requests being worked on, in lane order."""
+        with self._lock:
+            return [self._lanes[i].request_id for i in sorted(self._lanes)]
+
+    def free_lane(self, size: int) -> int | None:
+        """The lowest free lane below ``size``, or None when all are busy."""
+        with self._lock:
+            return next((i for i in range(size) if i not in self._lanes), None)
+
+    def start(
+        self,
+        request_id: UUID,
+        lane: int,
+        work: Callable[[LaneStop, int], None],
+    ) -> None:
+        """Run ``work(stop, lane)`` on a thread of its own in ``lane``."""
+        entry = _Lane(request_id, self._make_stop())
+
+        def run() -> None:
+            try:
+                work(entry.stop, lane)
+            finally:
+                with self._lock:
+                    del self._lanes[lane]
+
+        with self._lock:
+            if lane in self._lanes:
+                raise ValueError(f"lane {lane} is already running")
+            self._lanes[lane] = entry
+        entry.thread = threading.Thread(
+            target=run, name=f"verifaied-lane-{lane}", daemon=True
+        )
+        entry.thread.start()
+
+    def stop(self, ids: Iterable[object]) -> None:
+        """Stop the lanes working on any of ``ids``; others are left alone."""
+        wanted = {str(i) for i in ids}
+        with self._lock:
+            lanes = list(self._lanes.values())
+        for entry in lanes:
+            if str(entry.request_id) in wanted:
+                entry.stop.set()
+
+    def interrupt_all(self, message: str) -> None:
+        """Stop every lane, closing each with ``message``."""
+        with self._lock:
+            lanes = list(self._lanes.values())
+        for entry in lanes:
+            entry.stop.interrupt(message)
+
+    def join_all(self, poll: float = 0.5) -> None:
+        """Wait for every lane in flight to finish.
+
+        In short waits rather than one, so a Ctrl-C reaches the waiting
+        thread at once instead of after the slowest lane.
+        """
+        with self._lock:
+            threads = [e.thread for e in self._lanes.values() if e.thread]
+        for thread in threads:
+            while thread.is_alive():
+                thread.join(poll)