fastws-cli 0.0.32__tar.gz → 0.0.33__tar.gz

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.32
+Version: 0.0.33
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -200,7 +200,7 @@ ws-build --project solveit  # only this package and its transitive workspace dep
 
 ### `cargo develop`
 
-Rebuild a mixed Rust/Python project's extension and replace it in the editable source package:
+Build a mixed Rust/Python project's extension and opted-in binaries, keep them with its Python package as one build, and install that build in the editable source package and the venv:
 
 ```bash
 cargo develop
@@ -221,7 +221,13 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 Set `post-develop-module = "package.generator"` under `[tool.fastws]` to run a generator after installing the native extension and binaries, before copying wheel data. It runs with the current Python in a fresh process from the project root, including when Cargo reuses its build. A failure stops `cargo develop` or `ws-sync`. Use it for generated files that need the rebuilt extension's metadata.
 
-On Unix, each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes. Windows builds and tests run without this timing coordination.
+`cargo develop` keeps each build it installs in `~/.cache/fastws/builds/<project>/<id>/`, read-only. Files are copied first, then hashed, and hard-linked to blobs named by those captured bytes' sha256. The build id hashes these artifacts and the current digests of inputs listed by Cargo's dep-info. Input digests are observations at capture time, not proof of what the compiler read. The record's `started` gives the build's start time; `edited` flags inputs **possibly modified since build started**, by mtime, including inputs of a first build or newly discovered dependencies. Identical inputs and artifacts reuse their first capture's record.
+
+The installed extension and binaries are symbolic links through `<project>/installed`, which `cargo develop` points at the new build in one step. `fastws.store.builds(root)` gives the newest build records, with input changes since the preceding build, and `fastws.store.resolve(root, id)` resolves a build-id prefix. Each develop deletes builds that are neither installed, among the project's last 20, nor from the last 2 days, then old unlinked blobs.
+
+On Unix, cooperating `cargo develop` runs take a per-project lock through compilation, capture, installation, the post-develop generator and wheel data. Store readers and capture/install operations hold shared store access; collection takes it exclusively. Ordinary Cargo commands do not take the project lock: agents must not run competing raw Cargo builds against that project during develop. Windows installs copies and does not take these locks.
+
+On Unix, every rustc call holds a shared `flock` on `/tmp/aai-ws-timing.lock`, through the rustc wrapper that sync writes (see `ws-sync`). A speed test that takes that lock exclusively stays apart from participating builds and test runs. A timing also holds `/tmp/aai-ws-timing.gate` while it waits for the lock and holds it, and a shared hold passes that file first, so new rustc calls and test runs wait behind the timing. Both shared and exclusive acquisition fail after 5 minutes; work never proceeds unlocked. Python code takes the same lock with `fastws.timelock.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. Shared-inside-shared in one thread reuses its hold; each thread otherwise takes its own hold. Any nesting involving exclusive raises `RuntimeError` immediately. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. Windows builds and tests run without this timing coordination.
 
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.29` alongside `maturin>=1.13,<2.0` in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
@@ -245,7 +251,22 @@ cargo wait
 cargo wait --path /path/to/workspace
 ```
 
-Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
+### Rust tests from Python
+
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+
+Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+
+```python
+from fastws.cargo import run_tests
+
+native = run_tests('.', '--no-fail-fast')
+focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'))
+```
+
+Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
 ### `ws-sync`
 
@@ -266,9 +287,11 @@ Existing globs, entries for missing directories, and entries for checkouts that
 
 Each sync regenerates Cargo overrides in the workspace's `.cargo/config.toml`. `[patch.crates-io]` gets an entry for each local crate, including nested Cargo workspace members, except a crate marked `publish = false`, which never comes from crates.io. Git dependencies that name local crates get entries under the matching URL. Builds under the workspace root use these local checkouts. Entries pointing outside the root and other configuration sections are preserved.
 
+Sync also defaults `[term] quiet = true` in that file to suppress routine Cargo output and warnings. An explicit `quiet = false` is preserved.
+
 Do not commit a `Cargo.lock` generated under these patches. Its source-less local entries cannot be resolved on another machine.
 
-When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved. If neither a wrapper nor `sccache` is present, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it.
+Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/README.md

@@ -172,7 +172,7 @@ ws-build --project solveit  # only this package and its transitive workspace dep
 
 ### `cargo develop`
 
-Rebuild a mixed Rust/Python project's extension and replace it in the editable source package:
+Build a mixed Rust/Python project's extension and opted-in binaries, keep them with its Python package as one build, and install that build in the editable source package and the venv:
 
 ```bash
 cargo develop
@@ -193,7 +193,13 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 Set `post-develop-module = "package.generator"` under `[tool.fastws]` to run a generator after installing the native extension and binaries, before copying wheel data. It runs with the current Python in a fresh process from the project root, including when Cargo reuses its build. A failure stops `cargo develop` or `ws-sync`. Use it for generated files that need the rebuilt extension's metadata.
 
-On Unix, each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes. Windows builds and tests run without this timing coordination.
+`cargo develop` keeps each build it installs in `~/.cache/fastws/builds/<project>/<id>/`, read-only. Files are copied first, then hashed, and hard-linked to blobs named by those captured bytes' sha256. The build id hashes these artifacts and the current digests of inputs listed by Cargo's dep-info. Input digests are observations at capture time, not proof of what the compiler read. The record's `started` gives the build's start time; `edited` flags inputs **possibly modified since build started**, by mtime, including inputs of a first build or newly discovered dependencies. Identical inputs and artifacts reuse their first capture's record.
+
+The installed extension and binaries are symbolic links through `<project>/installed`, which `cargo develop` points at the new build in one step. `fastws.store.builds(root)` gives the newest build records, with input changes since the preceding build, and `fastws.store.resolve(root, id)` resolves a build-id prefix. Each develop deletes builds that are neither installed, among the project's last 20, nor from the last 2 days, then old unlinked blobs.
+
+On Unix, cooperating `cargo develop` runs take a per-project lock through compilation, capture, installation, the post-develop generator and wheel data. Store readers and capture/install operations hold shared store access; collection takes it exclusively. Ordinary Cargo commands do not take the project lock: agents must not run competing raw Cargo builds against that project during develop. Windows installs copies and does not take these locks.
+
+On Unix, every rustc call holds a shared `flock` on `/tmp/aai-ws-timing.lock`, through the rustc wrapper that sync writes (see `ws-sync`). A speed test that takes that lock exclusively stays apart from participating builds and test runs. A timing also holds `/tmp/aai-ws-timing.gate` while it waits for the lock and holds it, and a shared hold passes that file first, so new rustc calls and test runs wait behind the timing. Both shared and exclusive acquisition fail after 5 minutes; work never proceeds unlocked. Python code takes the same lock with `fastws.timelock.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. Shared-inside-shared in one thread reuses its hold; each thread otherwise takes its own hold. Any nesting involving exclusive raises `RuntimeError` immediately. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. Windows builds and tests run without this timing coordination.
 
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.29` alongside `maturin>=1.13,<2.0` in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
@@ -217,7 +223,22 @@ cargo wait
 cargo wait --path /path/to/workspace
 ```
 
-Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
+### Rust tests from Python
+
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+
+Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+
+```python
+from fastws.cargo import run_tests
+
+native = run_tests('.', '--no-fail-fast')
+focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'))
+```
+
+Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
 ### `ws-sync`
 
@@ -238,9 +259,11 @@ Existing globs, entries for missing directories, and entries for checkouts that
 
 Each sync regenerates Cargo overrides in the workspace's `.cargo/config.toml`. `[patch.crates-io]` gets an entry for each local crate, including nested Cargo workspace members, except a crate marked `publish = false`, which never comes from crates.io. Git dependencies that name local crates get entries under the matching URL. Builds under the workspace root use these local checkouts. Entries pointing outside the root and other configuration sections are preserved.
 
+Sync also defaults `[term] quiet = true` in that file to suppress routine Cargo output and warnings. An explicit `quiet = false` is preserved.
+
 Do not commit a `Cargo.lock` generated under these patches. Its source-less local entries cannot be resolved on another machine.
 
-When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved. If neither a wrapper nor `sccache` is present, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it.
+Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/fastws/__init__.py

@@ -1,4 +1,5 @@
-__version__ = "0.0.32"
+__version__ = "0.0.33"
+
 
 
 

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/fastws/cargo.py

@@ -1,17 +1,15 @@
 r"Build, test and install Cargo artifacts for mixed Rust/Python projects."
 import json, os, re, shutil, subprocess, sys, sysconfig, time
 from collections import Counter
-from contextlib import contextmanager
 from dataclasses import dataclass
 from pathlib import Path
 from typing import NamedTuple
+from . import store
+from .procs import bounded
 
 try: import tomllib
 except ModuleNotFoundError: import tomli as tomllib
 
-# The file whose lock keeps timings and builds apart on this machine. Timings take it exclusively. Builds and test runs take it shared.
-_TIMING_LOCK = '/tmp/aai-ws-timing.lock'
-
 
 class ConfigError(ValueError):
     "A project's `pyproject.toml` lacks a setting that a command needs."
@@ -32,52 +30,11 @@ def members(root):
     return {Path(p['manifest_path']) for p in json.loads(cargo(root, 'metadata', '--no-deps', '--format-version', '1'))['packages']}
 
 
-def _lock_users():
-    "The other processes that have the timing lock's file open, as `pid command` strings from `lsof`. On macOS, where `lsof` doesn't show `flock` holds, the list also includes any process waiting for the lock."
-    users, pid = [], ''
-    for line in subprocess.run(['lsof', '+c', '0', '-F', 'pc', _TIMING_LOCK], capture_output=True, text=True).stdout.splitlines():
-        if line[:1] == 'p': pid = line[1:]
-        elif line[:1] == 'c' and pid != str(os.getpid()): users.append(f'{pid} {line[1:]}')
-    return users
-
-
-@contextmanager
-def timing_lock(
-    exclusive=False, # Take the lock exclusively, as a timing does; False: take it shared, as a build or a test run does
-):
-    "Hold the Unix lock that keeps this machine's timings apart from its builds and test runs; a no-op on Windows. A hold waits up to 5 minutes for the lock. After that, a shared hold goes ahead without it. An exclusive hold raises `TimeoutError`. An exclusive hold that has waited 30 seconds prints, once, the other processes that have the lock's file open."
-    if os.name == 'nt':
-        yield
-        return
-    import fcntl
-    with open(_TIMING_LOCK, 'a') as f:
-        start, waiting, reported = time.monotonic(), False, False
-        while True:
-            try:
-                fcntl.flock(f, (fcntl.LOCK_EX if exclusive else fcntl.LOCK_SH) | fcntl.LOCK_NB)
-                break
-            except BlockingIOError: pass
-            waited = time.monotonic() - start
-            if waited > 300:
-                if exclusive: raise TimeoutError(f'a build or another timing has held {_TIMING_LOCK} for 5 minutes')
-                print(f'going ahead although a timing still holds {_TIMING_LOCK}', file=sys.stderr)
-                break
-            if not waiting: print(f'waiting for {"a build or another timing" if exclusive else "a timing"} to release {_TIMING_LOCK}', file=sys.stderr)
-            waiting = True
-            if exclusive and not reported and waited > 30:
-                print(f'{_TIMING_LOCK} is still taken after 30 seconds. Processes with it open: {", ".join(_lock_users()) or "none"}', file=sys.stderr)
-                reported = True
-            time.sleep(0.25)
-        yield
-
-
 def check(root):
-    "Check every target of the workspace with Cargo, holding the timing lock shared. Cargo's messages are in the result's `stderr`, which `diagnostics` reads."
-    with timing_lock(): return subprocess.run(['cargo', 'check', '--all-targets', '--message-format=short'], cwd=Path(root).expanduser(), stderr=subprocess.PIPE, text=True)
+    "Check every target of the workspace with Cargo. Cargo's messages are in the result's `stderr`, which `diagnostics` reads."
+    return subprocess.run(['cargo', 'check', '--all-targets', '--message-format=short'], cwd=Path(root).expanduser(), stderr=subprocess.PIPE, text=True)
 
 
-# Cargo's line that starts a test binary: `Running` with its target and executable, or `Doc-tests` with its package.
-_SUITE = re.compile(r'^\s+(?:Running (.+) \([^)]*\)|(Doc-tests .+))$')
 _TEST = re.compile(r'^test (.+) \.\.\. (ok|FAILED|ignored)')
 _FAILURE = re.compile(r'^---- (.+) std(?:out|err) ----$')
 _DIAGNOSTIC = re.compile(r'^(\S.*?):(\d+):(\d+): (warning|error)(?:\[\w+\])?: (.*)$')
@@ -105,17 +62,17 @@ def _lines(text, n=20, tail=False):
 
 @dataclass
 class Suite:
-    "What one test binary printed when `cargo test` ran it."
-    name:str # Cargo's name for the binary, such as `tests/core.rs`, `unittests src/lib.rs` or `Doc-tests basedpl`
-    output:str # Everything printed after Cargo's line that starts the binary, until the next binary starts
+    "What one native test binary printed."
+    name:str # The target source, such as `tests/core.rs` or `unittests src/lib.rs`
+    output:str # Everything the binary printed, standard output and standard error merged
     tests:dict # Each test's outcome: `ok`, `FAILED` or `ignored`
     failures:dict # Each failing test's captured output
     summary:str # libtest's `test result:` line, empty for a binary that crashed or has its own harness
-    failed:bool # Did Cargo report the binary as failing?
+    failed:bool # Did the binary exit unsuccessfully?
 
     @classmethod
-    def read(cls, name, lines):
-        "Read one binary's lines."
+    def read(cls, name, lines, returncode):
+        "Read one binary's lines and exit status."
         failures, current = {}, None
         for line in lines:
             if m := _FAILURE.match(line): current = failures.setdefault(m[1], [])
@@ -123,7 +80,7 @@ class Suite:
             elif current is not None: current.append(line)
         return cls(name, '\n'.join(lines), {m[1]: m[2] for l in lines if (m := _TEST.match(l))},
                    {k: '\n'.join(v).strip('\n') for k,v in failures.items()},
-                   next((l for l in lines if l.startswith('test result: ')), ''), any(l.startswith('error: test failed') for l in lines))
+                   next((l for l in lines if l.startswith('test result: ')), ''), bool(returncode))
 
     def __str__(self):
         n = Counter(self.tests.values())
@@ -135,11 +92,12 @@ class Suite:
 
 @dataclass
 class Results:
-    "What `cargo test` printed, read into the build's compiler messages and each test binary's results. It displays as a summary: the compiler messages, each binary that ran tests, and each failure's output."
+    "A build's compiler messages and each native test binary's results. It displays as a summary: the compiler messages, each binary that ran tests, and each failure's output."
     returncode:int
-    text:str # Everything Cargo and the tests printed, standard output and standard error in the order they were written
+    text:str # The build's messages and each binary's merged standard output and standard error
     diagnostics:list # Each `Diagnostic` from the build, before the first binary ran
-    suites:list # Each binary's `Suite`, in the order Cargo ran them
+    suites:list # Each binary's `Suite`, in execution order
+    seconds:float=0 # Wall time spent executing native test binaries, excluding compilation
 
     @property
     def ok(self): return self.returncode == 0
@@ -159,27 +117,38 @@ class Results:
         return '\n'.join(res)
 
 
-def _blocks(text):
-    "Split `text` at Cargo's lines that start a test binary, into `(name, lines)` pairs. The first pair, named `None`, holds the build's lines."
-    res = [(None, [])]
-    for line in text.splitlines():
-        if m := _SUITE.match(line): res.append((m[1] or m[2], []))
-        else: res[-1][1].append(line)
-    return res
-
-
 def run_tests(
     root, # A Cargo package or workspace (expands `~`)
-    *args, # Arguments for `cargo test`, such as `'--test', 'core'`, a test name to filter by, or `'--', '--nocapture'`
+    *args, # Cargo build/selection options, such as `'--test', 'core'` or `'--profile', 'checked'`; `--no-fail-fast` runs every binary
+    test_args=(), # Arguments passed directly to each test binary, such as `('name', '--nocapture')`
     env:dict=None, # Environment variables to set for the run, such as `{'BASEDPL_COSTS': '1'}`
-    timeout:float=None # Seconds before the run stops, raising `subprocess.TimeoutExpired`
+    timeout:float=30 # Seconds shared by all native test binaries after building, raising `TimeoutError` on expiry; None: no limit
 )->Results:
-    "Run `cargo test` in `root` and read what it prints. Compiler messages come in Cargo's short format. It doesn't hold the timing lock as `check` and `build` do: tests that time take the lock exclusively themselves, and a shared hold here would block them."
-    # A quiet Cargo config hides the `Running` lines that divide the binaries' output.
-    p = subprocess.run(['cargo', 'test', '--message-format=short', *args], cwd=Path(root).expanduser(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=timeout,
-                       env=os.environ | {'CARGO_TERM_QUIET': 'false', 'CARGO_TERM_COLOR': 'never'} | (env or {}))
-    (_, build), *runs = _blocks(p.stdout)
-    return Results(p.returncode, p.stdout, diagnostics('\n'.join(build)), [Suite.read(*b) for b in runs])
+    "Build native tests once, with no time limit, then run Cargo's reported executables directly from their package directories. Source edits after building cannot trigger another build; results describe the compiled build, not the latest working tree. `timeout` covers the binaries together, excluding compilation. Doctests are not run. This takes no timing lock: rustc's wrapper and tests that time own their holds."
+    root, env = Path(root).expanduser().resolve(), os.environ | {'CARGO_TERM_QUIET': 'false', 'CARGO_TERM_COLOR': 'never'} | (env or {})
+    built = bounded(['cargo', 'test', '--no-run', '--message-format=json-diagnostic-short', *args], seconds=None, merge=True, cwd=root, env=env)
+    text, artifacts = [], []
+    for line in built.stdout.splitlines(keepends=True):
+        if not line.startswith('{'): text.append(line); continue
+        m = json.loads(line)
+        if m['reason'] == 'compiler-message': text.append(m['message']['rendered'])
+        elif m['reason'] == 'compiler-artifact' and m['profile']['test'] and m.get('executable'): artifacts.append(m)
+    text = ''.join(text)
+    found = list(dict.fromkeys(diagnostics(text)))
+    if built.returncode: return Results(built.returncode, text, found, [])
+    suites, code, start = [], 0, time.monotonic()
+    for m in artifacts:
+        package = Path(m['manifest_path']).parent
+        name = Path(m['target']['src_path']).relative_to(package).as_posix()
+        if 'test' not in m['target']['kind']: name = 'unittests ' + name
+        left = None if timeout is None else max(0, timeout - (time.monotonic() - start))
+        p = bounded([m['executable'], *test_args], left, merge=True, cwd=package, env=env | {'CARGO_MANIFEST_DIR': str(package)})
+        suites.append(Suite.read(name, p.stdout.splitlines(), p.returncode))
+        text += f'     Running {name} ({m["executable"]})\n' + p.stdout
+        if p.returncode:
+            code = p.returncode
+            if '--no-fail-fast' not in args: break
+    return Results(code, text, found, suites, time.monotonic() - start)
 
 
 def sources(root):
@@ -236,7 +205,7 @@ def build(
     for flag, active in [('all-features', all_features), ('no-default-features', no_default_features)]:
         if active or cfg.get(flag): cmd.append('--' + flag)
     extra, env = _maturin_flags(target, os.environ) if for_maturin else ([], None)
-    with timing_lock(): result = subprocess.run(['cargo', *cmd, *extra], cwd=root, stdout=subprocess.PIPE, text=True, env=env)
+    result = subprocess.run(['cargo', *cmd, *extra], cwd=root, stdout=subprocess.PIPE, text=True, env=env)
     messages = [json.loads(line) for line in result.stdout.splitlines()]
     for m in messages:
         if m.get('reason') != 'compiler-message': continue
@@ -257,30 +226,37 @@ def replace(source, destination):
 
 
 def develop(root):
-    "Install a Python extension, opted-in binaries and wheel data, running the project's post-develop module before wheel data."
+    "Serialize this project's cooperating develop runs through build, artifact capture, installation, generator and wheel data. Raw Cargo commands do not take this lock."
     root = Path(root).expanduser()
-    tool = settings(root)
-    cfg = tool['maturin']
-    if 'python-source' not in cfg: raise ConfigError(r'cargo develop requires tool.maturin.python-source')
-    binaries = tool.get('fastws', {}).get('native-binaries', False)
-    artifacts = build(root, binaries=binaries, profile='test')
-    artifact, = [m for m in artifacts if 'cdylib' in m['target']['crate_types']]
-    lib, = [f for f in artifact['filenames'] if Path(f).suffix in ('.dylib', '.so', '.dll')]
-    module = cfg.get('module-name', artifact['target']['name'])
-    dest = root/cfg['python-source']/Path(*module.split('.'))
-    replace(lib, dest.with_name(dest.name + sysconfig.get_config_var('EXT_SUFFIX')))
-    env = Path(os.environ.get('VIRTUAL_ENV', sys.prefix))
-    if binaries:
+    with store.building(root) as started:
+        tool = settings(root)
+        cfg = tool['maturin']
+        if 'python-source' not in cfg: raise ConfigError(r'cargo develop requires tool.maturin.python-source')
+        binaries = tool.get('fastws', {}).get('native-binaries', False)
+        artifacts = build(root, binaries=binaries, profile='test')
+        artifact, = [m for m in artifacts if 'cdylib' in m['target']['crate_types']]
+        lib, = [Path(f) for f in artifact['filenames'] if Path(f).suffix in ('.dylib', '.so', '.dll')]
+        module = cfg.get('module-name', artifact['target']['name'])
+        src = root/cfg['python-source']
+        ext = src/Path(*module.split('.'))
+        ext = ext.with_name(ext.name + sysconfig.get_config_var('EXT_SUFFIX'))
+        env = Path(os.environ.get('VIRTUAL_ENV', sys.prefix))
         scripts = env/('Scripts' if os.name == 'nt' else 'bin')
-        for m in artifacts:
-            if m.get('executable'): replace(m['executable'], scripts/Path(m['executable']).name)
-    if generator := tool.get('fastws', {}).get('post-develop-module'):
-        subprocess.run([sys.executable, '-m', generator], cwd=root, check=True)
-    # Editable builds through `build_backend` don't install the wheel's data files, which a wheel install puts under the prefix.
-    if 'data' in cfg:
-        data = root/cfg['data']/'data'
-        for f in data.rglob('*'):
-            if f.is_file(): replace(f, env/f.relative_to(data))
+        exes = [Path(m['executable']) for m in artifacts if binaries and m.get('executable')]
+        package = src/module.split('.')[0]
+        files = {f'python/{f.relative_to(src)}': f for f in package.rglob('*') if f.is_file() and f != ext and '__pycache__' not in f.parts}
+        files |= {f'python/{ext.relative_to(src)}': lib} | {f'bin/{e.name}': e for e in exes}
+        with store.access():
+            kept = store.keep(root, files, dict.fromkeys(f for a in [lib, *exes] for f in store.dep_files(a)), started)
+            store.install(root, kept, {ext: f'python/{ext.relative_to(src)}'} | {scripts/e.name: f'bin/{e.name}' for e in exes})
+        store.collect()
+        if generator := tool.get('fastws', {}).get('post-develop-module'):
+            subprocess.run([sys.executable, '-m', generator], cwd=root, check=True)
+        # Editable builds through `build_backend` don't install the wheel's data files, which a wheel install puts under the prefix.
+        if 'data' in cfg:
+            data = root/cfg['data']/'data'
+            for f in data.rglob('*'):
+                if f.is_file(): replace(f, env/f.relative_to(data))
 
 
 def stage_binaries(root, profile=None, target=None, features=None, all_features=False, no_default_features=False):

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/fastws/core.py

@@ -726,7 +726,7 @@ def _entry_changes(current, new):
     return [k for k in new if k not in current], [k for k in current if k not in new]
 
 def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
-    """Regenerate `[patch]` entries in the workspace `.cargo/config.toml` and return (added, removed).
+    """Regenerate workspace Cargo patches, default to quiet output, and return (added, removed).
 
     Every local crate gets a `[patch.crates-io]` entry, except one marked `publish = false`, which never comes
     from crates.io. A member's git dep on a local crate gets an entry under that URL. Builds anywhere under `root`
@@ -739,8 +739,13 @@ def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     desired = {"crates-io": {pkg["name"]: str(d) for pkg, d in pkgs if pkg.get("publish") is not False}}
     for url, entries in _git_dep_tables(root, crates).items(): desired[url] = {name: str(d) for name, d in entries.items()}
     config = root/".cargo"/"config.toml"
-    content = config.read_text() if config.exists() else ""
-    old = tomllib.loads(content).get("patch", {}) if content else {}
+    original = content = config.read_text() if config.exists() else ""
+    data = tomllib.loads(content) if content else {}
+    old = data.get("patch", {})
+    if "quiet" not in data.get("term", {}):
+        if match := re.search(r"(?m)^\[term\][^\n]*(?:\n|$)", content):
+            content = content[:match.end()].rstrip("\n") + "\nquiet = true\n" + content[match.end():]
+        else: content = _replace_table(content, "term", "quiet = true").lstrip()
     def inside(spec):
         if not isinstance(spec, dict) or not (p := spec.get("path")): return False
         p = Path(p).expanduser()
@@ -756,7 +761,7 @@ def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
         + "".join(f"{name} = {_fmt_source(spec)}\n" for name, spec in sorted(entries.items()))
         for url, entries in tables.items() if entries)
     new = (_strip_patch_tables(content).rstrip() + "\n\n" + body).lstrip() + "\n" if body else _strip_patch_tables(content)
-    if new == content: return [], []
+    if new == original: return [], []
     tomllib.loads(new)  # never write an unparseable config
     config.parent.mkdir(exist_ok=True)
     config.write_text(new)
@@ -765,23 +770,41 @@ def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     added, removed = _entry_changes(before, after)
     return sorted(added), sorted(removed)
 
-def _sync_cargo_wrapper(root: Path) -> bool:
-    "Add sccache to the generated Cargo config when installed, without overriding another wrapper"
-    config = root/".cargo"/"config.toml"
+# The rustc wrapper that `_sync_cargo_wrapper` writes. It imports only `fastws.timelock`, by its file, so that each rustc call pays for little more than the interpreter's start.
+_RUSTC_LOCK = r"""#!{python} -IS
+# Written by ws-sync. Cargo runs each rustc call through this script, which holds the machine's timing lock shared until rustc ends (`fastws.timelock`){via}.
+import sys
+sys.path.insert(0, {dir!r})
+from timelock import exec_shared
+INNER = {inner!r}  # The wrapper that each rustc call runs through inside the lock, kept by later syncs; None: rustc alone
+exec_shared([INNER, *sys.argv[1:]] if INNER else sys.argv[1:])
+"""
+
+def _sync_cargo_wrapper(root: Path):
+    "Point the generated Cargo config's `rustc-wrapper` at `.cargo/rustc-lock`, written here, which runs each rustc call holding the machine's timing lock shared (`fastws.timelock`). Inside the lock it runs the wrapper that the config named before, where that's another tool's, or else sccache where it's installed. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names sccache."
+    config, script = root/".cargo"/"config.toml", root/".cargo"/"rustc-lock"
     content = config.read_text() if config.exists() else ""
-    data = tomllib.loads(content) if content else {}
-    if data.get("build", {}).get("rustc-wrapper"): return False
-    if not (wrapper := shutil.which("sccache")):
+    current = (tomllib.loads(content) if content else {}).get("build", {}).get("rustc-wrapper")
+    if not (sccache := shutil.which("sccache")):
         print("⚠️  sccache not found; Rust builds won't use compiler caching.\n"
               "Install it with: cargo install sccache --locked\nThen rerun ws-sync to enable it.")
-        return False
+    recorded = script.exists() and current == str(script) and re.search(r"(?m)^INNER = (.+?)  #", script.read_text())
+    previous = ast.literal_eval(recorded[1]) if recorded else current
+    inner = previous if previous not in (None, str(script)) else sccache
+    if os.name == "nt": wrapper = inner
+    else:
+        script.parent.mkdir(exist_ok=True)
+        script.write_text(_RUSTC_LOCK.format(python=sys.executable, dir=str(Path(__file__).parent), via=f", through {Path(inner).name}" if inner else "", inner=inner))
+        script.chmod(0o755)
+        wrapper = str(script)
+    if not wrapper or current == wrapper: return
+    content = re.sub(r"(?m)^rustc-wrapper = .*\n", "", content)
     line = f'rustc-wrapper = {json.dumps(wrapper)}\n'
     if match := re.search(r"(?m)^\[build\][^\n]*\n", content): new = content[:match.end()] + line + content[match.end():]
     else: new = content.rstrip() + ("\n\n" if content.strip() else "") + "[build]\n" + line
     tomllib.loads(new)
     config.parent.mkdir(exist_ok=True)
     config.write_text(new)
-    return True
 
 def _cargo_dep_tables(data):
     for name in "dependencies","build-dependencies": yield data.get(name, {})
@@ -968,11 +991,10 @@ async def ws_sync(
         projects = _build_projects(root, repos_file)
         _sync_build_isolation(pyproject_path, projects)
     
-        wrapper_added = _sync_cargo_wrapper(root)
+        _sync_cargo_wrapper(root)
         added_p, removed_p = _sync_cargo_patches(root)
         if added_p: print(f"Cargo patches added: {', '.join(added_p)}")
         if removed_p: print(f"Cargo patches removed: {', '.join(removed_p)}")
-        if wrapper_added: print("Cargo builds now use sccache")
     
         js_members = _npm_dirs(root)
         if _native_js(js_members): _check_rustup()

fastws_cli-0.0.33/fastws/procs.py

@@ -0,0 +1,46 @@
+r"Bounded commands, with one process-group owner for each Unix job."
+import os, signal, subprocess
+from contextlib import suppress
+from pathlib import Path
+
+__all__ = ['bounded', 'kill_group', 'JOB_ENV']
+JOB_ENV = 'FASTWS_JOB' # Owners mark the environment of their job; subordinate commands inherit its process group
+
+
+def kill_group(p):
+    "Kill the process group that `p` owns, then reap its leader, even if the leader already exited. Only use for a process launched with `start_new_session=True`."
+    with suppress(ProcessLookupError): os.killpg(p.pid, signal.SIGKILL)
+    p.wait()
+
+
+def bounded(
+    cmd, # The command and its arguments, as `subprocess.Popen` takes them
+    seconds:float=None, # Seconds that it may run; None: no limit
+    input:str=None, # Text written to its standard input
+    merge:bool=False, # Merge standard error into standard output, in the order written?
+    **kwargs # Other `subprocess.Popen` arguments, such as `cwd` and `env`
+)->subprocess.CompletedProcess:
+    "Run a command with captured text and a wall-clock limit. Without `FASTWS_JOB`, own a new session and kill its process group on completion, timeout or error. Within a marked job, inherit its group; a timeout/error kills only the direct child and raises. Let that error end the job so its owner cleans up descendants, even after its leader exits. Environment overrides retain the marker. Commands must not detach from the job's group."
+    owner = not os.environ.get(JOB_ENV)
+    env = kwargs.pop('env', None)
+    env = dict(os.environ if env is None else env) | {JOB_ENV: '1'}
+    with subprocess.Popen(cmd, stdin=None if input is None else subprocess.PIPE, stdout=subprocess.PIPE,
+                          stderr=subprocess.STDOUT if merge else subprocess.PIPE, text=True, start_new_session=owner, env=env, **kwargs) as p:
+        try:
+            try: out, err = p.communicate(input, timeout=seconds)
+            except BaseException as e:
+                if owner:
+                    kill_group(p)
+                    out, err = p.communicate()
+                else:
+                    p.kill()
+                    p.wait()
+                    out, err = (s.decode(errors='replace') if isinstance(s, bytes) else s or ''
+                                for s in (getattr(e, 'output', ''), getattr(e, 'stderr', '')))
+                if not isinstance(e, subprocess.TimeoutExpired): raise
+                tail = '\n'.join(f'    {line}' for line in (out + (err or '')).splitlines()[-20:])
+                cleanup = 'its process group was killed' if owner else 'its child was killed; the job owner must clean up descendants'
+                raise TimeoutError(f"`{Path(str(cmd[0])).name}` didn't finish in {seconds:.3g} s: {cleanup}. Pass a larger `timeout`, or run less at once." + (f" Its last lines:\n{tail}" if tail else '')) from None
+        finally:
+            if owner: kill_group(p)
+    return subprocess.CompletedProcess(cmd, p.returncode, out, err)

fastws_cli-0.0.33/fastws/store.py

@@ -0,0 +1,166 @@
+r"""Builds that `cargo develop` installs, each kept once, read-only, and named by the hash of its inputs and files. A project's installed build is a pointer to one of them, which installing replaces in one step, so a process or a timing that resolves a build never sees it change.
+
+A build is a directory under `STORE/<project>/`, named by its id, that holds `build.json` and a hard link to a blob for each file: `python/…` for the Python package with its extension, `bin/…` for binaries and `data/…` for wheel data. Each blob, under `STORE/blobs/`, is named by its contents' sha256, so files that don't change between builds take no more space. Installed paths are symbolic links through `STORE/<project>/installed`, a link to the installed build's directory. Windows installs copies."""
+import hashlib, json, os, re, shutil, time, tomllib
+if os.name != 'nt': import fcntl
+from contextlib import contextmanager
+from datetime import datetime
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+STORE = Path(os.environ.get('XDG_CACHE_HOME') or '~/.cache').expanduser()/'fastws'/'builds' # Each project's builds, and the blobs that their files link to
+KEEP, KEEP_DAYS = 20, 2 # `collect` keeps each project's installed build, its last `KEEP` builds and its builds of the last `KEEP_DAYS` days
+
+
+def _project(root): return STORE/tomllib.loads((Path(root)/'pyproject.toml').read_text())['project']['name']
+
+@contextmanager
+def building(root):
+    "On Unix, serialize cooperating `cargo develop` runs through build, capture and installation; yield the build's start time. Other Cargo commands do not take this lock."
+    project = _project(root)
+    project.mkdir(parents=True, exist_ok=True)
+    with open(project/'build.lock', 'a') as lock:
+        if os.name != 'nt': fcntl.flock(lock, fcntl.LOCK_EX)
+        yield time.time()
+
+@contextmanager
+def access(exclusive=False):
+    "On Unix, keep collection apart from readers and capture/install. Hold this shared across `keep` and `install`; `collect` takes it exclusively."
+    STORE.mkdir(parents=True, exist_ok=True)
+    with open(STORE/'store.lock', 'a') as lock:
+        if os.name != 'nt': fcntl.flock(lock, fcntl.LOCK_EX if exclusive else fcntl.LOCK_SH)
+        yield
+
+def _digest(path):
+    with open(path, 'rb') as f: return hashlib.file_digest(f, 'sha256').hexdigest()
+
+def _blob(path):
+    "The blob that holds `path`'s contents, read-only and executable where `path` is, added to the store where it's new."
+    directory = STORE/'blobs'
+    directory.mkdir(parents=True, exist_ok=True)
+    with TemporaryDirectory(dir=directory, suffix='.tmp') as tmp:
+        captured = Path(tmp)/'file'
+        shutil.copy2(path, captured)
+        blob = directory/_digest(captured)
+        captured.chmod(0o555 if captured.stat().st_mode & 0o111 else 0o444)
+        try: os.link(captured, blob)
+        except FileExistsError: pass
+    return blob
+
+
+def dep_files(artifact):
+    "The files that Cargo's dep-info lists for `artifact`, the `.d` file beside it, with the files under each directory it lists, such as one that a build script watches."
+    deps = re.split(r'(?<!\\)\s+', Path(artifact).with_suffix('.d').read_text().split(': ', 1)[1].strip())
+    for p in (Path(d.replace('\\ ', ' ')) for d in deps):
+        yield from (sorted(f for f in p.rglob('*') if f.is_file()) if p.is_dir() else [p])
+
+
+def _kept(d):
+    "The builds in project directory `d`, leaving out its `installed` link and builds still being written."
+    return [p for p in d.iterdir() if not p.is_symlink() and not p.name.endswith('.tmp') and (p/'build.json').exists()] if d.exists() else []
+
+
+def _metas(d):
+    "The `build.json` records of the builds in project directory `d`, oldest first."
+    return sorted((json.loads((p/'build.json').read_text()) for p in _kept(d)), key=lambda m: m['time'])
+
+
+def keep(
+    root, # The project directory, whose `pyproject.toml` names the project
+    files:dict, # Each path in the build, such as `bin/bpl`, to the file to keep there
+    inputs, # The files the build was made from
+    started:float, # The build's start time, as `building` gives it
+)->Path:
+    "Keep captured artifacts and observed input digests. Hold `building` and `access` through capture/install. `edited` flags inputs possibly modified since build start, by mtime, not compiler provenance. Identical inputs and artifacts reuse their first capture's record."
+    sources = {str(p): _digest(p) for p in inputs}
+    blobs = {rel: _blob(src) for rel, src in files.items()}
+    key = json.dumps([sorted(sources.items()), sorted((rel, b.name) for rel, b in blobs.items())], sort_keys=True)
+    build = _project(root)/hashlib.sha256(key.encode()).hexdigest()[:16]
+    if build.exists(): return build
+    with TemporaryDirectory(dir=build.parent, suffix='.tmp') as tmp:
+        tmp = Path(tmp)
+        for rel, blob in blobs.items():
+            (tmp/rel).parent.mkdir(parents=True, exist_ok=True)
+            os.link(blob, tmp/rel)
+        edited = sorted(f for f in sources if Path(f).stat().st_mtime >= started)
+        (tmp/'build.json').write_text(json.dumps(dict(id=build.name, time=time.time(), started=started, author=os.environ.get('EXHASH_AUTHOR'), inputs=sources, edited=edited), indent=1))
+        os.replace(tmp, build)
+    return build
+
+
+def _link(path, target):
+    "Make `path` a symbolic link to `target`, replacing what was there in one step."
+    if path.is_symlink() and os.readlink(path) == str(target): return
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_name(f'{path.name}.{os.getpid()}.tmp')
+    tmp.unlink(missing_ok=True)
+    tmp.symlink_to(target)
+    os.replace(tmp, path)
+
+
+def install(
+    root, # The project directory
+    build:Path, # A build that `keep` gave
+    paths:dict, # Each installed path, such as the extension in the project's Python package, to its path in the build
+):
+    "Install a captured build while holding `access`. Each path links through the project's `installed` link, which then points at `build`. Windows copies the files instead."
+    installed = _project(root)/'installed'
+    for path, rel in paths.items():
+        if os.name == 'nt': shutil.copy2(build/rel, path)
+        else: _link(Path(path), installed/rel)
+    if os.name != 'nt': _link(installed, build.name)
+
+
+def resolve(
+    root, # The project directory
+    build:str=None, # A prefix of a kept build's id; None: the installed build
+)->Path:
+    "The directory of the build of the project at `root` that `build` names, as it is now."
+    with access():
+        d = _project(root)
+        if build is None: return (d/'installed').resolve()
+        found = [p for p in _kept(d) if p.name.startswith(build)]
+        if len(found) != 1: raise ValueError(f'{len(found)} builds of {d.name} have ids starting {build!r}')
+        return found[0]
+
+
+class Builds(list):
+    "A project's first capture of each build, oldest first: `id`, `time`, `started`, `author`, capture-time `inputs` digests, `edited` and `changes`. `edited` flags inputs possibly modified since build start, by mtime, not what the compiler read. `changes` lists inputs whose digest differs from the build before's, or None for the first. `installed` is the installed build's id."
+    def __init__(self, rows=(), installed=None):
+        super().__init__(rows)
+        self.installed = installed
+    def __repr__(self): return '\n'.join(map(self._line, self))
+    def __getitem__(self, i): return Builds(super().__getitem__(i), self.installed) if isinstance(i, slice) else super().__getitem__(i)
+    def _line(self, m):
+        names = ['(first)'] if m['changes'] is None else sorted(Path(f).name for f in m['changes'])
+        edited = f"  possibly modified since build start: {', '.join(sorted(Path(f).name for f in m['edited']))}" if m['edited'] else ''
+        return f"{'*' if m['id'] == self.installed else ' '} {m['id']}  {datetime.fromtimestamp(m['time']):%m-%d %H:%M:%S}  {m['author'] or '-':<8} {', '.join(names[:8]) or '(no input changed)'}{' …' if len(names) > 8 else ''}{edited}"
+
+
+def builds(
+    root, # The project directory
+    n:int=10, # How many of the newest builds to give; None: every kept build
+)->Builds:
+    "The newest `n` builds, oldest first, with their input changes and diagnostic mtime flags. `*` marks the installed build."
+    with access():
+        d = _project(root)
+        metas = _metas(d)
+        for prev, m in zip([None, *metas], metas):
+            m['changes'] = None if prev is None else sorted(f for f, h in m['inputs'].items() if prev['inputs'].get(f) != h)
+            m.setdefault('edited', [])
+        return Builds(metas[-n:] if n else metas, (d/'installed').resolve().name if (d/'installed').exists() else None)
+
+
+def collect():
+    "Delete builds neither installed, among the last `KEEP`, nor from the last `KEEP_DAYS` days, then unlinked blobs at least an hour old. Exclusive store access keeps this apart from readers and capture/install operations. Call outside `access`."
+    if not STORE.exists(): return
+    with access(exclusive=True):
+        now = time.time()
+        for d in (p for p in STORE.iterdir() if p.is_dir() and p.name != 'blobs'):
+            installed = (d/'installed').resolve() if (d/'installed').exists() else None
+            kept = sorted(_kept(d), key=lambda p: (p/'build.json').stat().st_mtime)
+            for p in kept[:-KEEP]:
+                if p != installed and now - (p/'build.json').stat().st_mtime > KEEP_DAYS*86400: shutil.rmtree(p)
+        for blob in (STORE/'blobs').iterdir() if (STORE/'blobs').exists() else []:
+            st = blob.stat()
+            if st.st_nlink == 1 and now - st.st_mtime > 3600: blob.unlink()

fastws_cli-0.0.33/fastws/timelock.py

@@ -0,0 +1,76 @@
+r"The machine's timing lock, which keeps timings apart from builds and test runs, as `rustymath::timing` describes: a timing locks `LOCK` exclusively, holding `GATE` exclusively from the start of its wait, and anything else passes `GATE` and then locks `LOCK` shared. It imports only the standard library, because the rustc wrapper that `ws-sync` writes loads it for every rustc call."
+import os, sys, threading, time
+from contextlib import contextmanager
+if os.name != 'nt': import fcntl
+
+LOCK = '/tmp/aai-ws-timing.lock' # Timings lock it exclusively; builds and test runs lock it shared
+GATE = '/tmp/aai-ws-timing.gate' # A timing holds it exclusively while it waits for `LOCK` and holds it; a shared hold locks it for a moment first
+_holds = threading.local() # This thread's hold mode, as `exclusive`; nested shared holds reuse it
+
+
+def _users():
+    "The other processes that have `LOCK` open, as `pid command` strings from `lsof`. On macOS, where `lsof` doesn't show `flock` holds, the list also includes any process waiting for the lock."
+    import subprocess
+    users, pid = [], ''
+    for line in subprocess.run(['lsof', '+c', '0', '-F', 'pc', LOCK], capture_output=True, text=True).stdout.splitlines():
+        if line[:1] == 'p': pid = line[1:]
+        elif line[:1] == 'c' and pid != str(os.getpid()): users.append(f'{pid} {line[1:]}')
+    return users
+
+
+class _Wait:
+    "One hold's wait for `GATE` and then `LOCK`, against one deadline of 5 minutes."
+    def __init__(self, exclusive, quiet): self.exclusive, self.quiet, self.start, self.waiting, self.reported = exclusive, quiet, time.monotonic(), False, False
+
+    def take(self, f):
+        "Lock `f` as the hold needs it, waiting up to 5 minutes before raising `TimeoutError`. An exclusive hold that has waited 30 seconds prints, once, the other processes that have `LOCK` open."
+        others = 'a build or another timing' if self.exclusive else 'a timing'
+        while True:
+            try: return fcntl.flock(f, (fcntl.LOCK_EX if self.exclusive else fcntl.LOCK_SH) | fcntl.LOCK_NB)
+            except BlockingIOError: pass
+            waited = time.monotonic() - self.start
+            if waited > 300: raise TimeoutError(f'{others} has held {LOCK} for 5 minutes')
+            if not self.waiting and not self.quiet: print(f'waiting for {others} to release {LOCK}', file=sys.stderr)
+            if self.exclusive and not self.reported and waited > 30:
+                print(f'{LOCK} is still taken after 30 seconds. Processes with it open: {", ".join(_users()) or "none"}', file=sys.stderr)
+                self.reported = True
+            self.waiting = True
+            time.sleep(0.25)
+
+
+@contextmanager
+def _held(exclusive, quiet=False):
+    "Hold `LOCK`, and `GATE` too when exclusive. `quiet` leaves out the message that the wait has started."
+    with open(GATE, 'a') as gate, open(LOCK, 'a') as lock:
+        wait = _Wait(exclusive, quiet)
+        wait.take(gate)
+        wait.take(lock)
+        if not exclusive: gate.close()
+        yield lock
+
+
+@contextmanager
+def timing_lock(
+    exclusive=False, # Take the lock exclusively, as a timing does; False: take it shared, as a build or a test run does
+):
+    "Hold the Unix lock that keeps this machine's timings apart from participating builds and test runs; a no-op on Windows. Acquisition raises `TimeoutError` after 5 minutes. Shared-inside-shared in the same thread reuses its hold. Any nesting involving exclusive raises `RuntimeError`. Each thread takes its own shared lock; one thread's release never ends another's hold."
+    if os.name == 'nt':
+        yield
+        return
+    held = getattr(_holds, 'exclusive', None)
+    if held is not None:
+        if held or exclusive: raise RuntimeError('timing locks can only nest shared inside shared')
+        yield
+        return
+    with _held(exclusive):
+        _holds.exclusive = exclusive
+        try: yield
+        finally: del _holds.exclusive
+
+
+def exec_shared(argv):
+    "Replace this process with `argv`, which then holds the lock shared until it ends, as the rustc wrapper does for each rustc call. It prints nothing while it waits, since Cargo shows each rustc call's output."
+    if os.name == 'nt': os.execv(argv[0], argv)
+    with _held(False, quiet=True) as lock:
+        os.set_inheritable(lock.fileno(), True)
+        os.execv(argv[0], argv)

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/fastws_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.32
+Version: 0.0.33
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -200,7 +200,7 @@ ws-build --project solveit  # only this package and its transitive workspace dep
 
 ### `cargo develop`
 
-Rebuild a mixed Rust/Python project's extension and replace it in the editable source package:
+Build a mixed Rust/Python project's extension and opted-in binaries, keep them with its Python package as one build, and install that build in the editable source package and the venv:
 
 ```bash
 cargo develop
@@ -221,7 +221,13 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 Set `post-develop-module = "package.generator"` under `[tool.fastws]` to run a generator after installing the native extension and binaries, before copying wheel data. It runs with the current Python in a fresh process from the project root, including when Cargo reuses its build. A failure stops `cargo develop` or `ws-sync`. Use it for generated files that need the rebuilt extension's metadata.
 
-On Unix, each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes. Windows builds and tests run without this timing coordination.
+`cargo develop` keeps each build it installs in `~/.cache/fastws/builds/<project>/<id>/`, read-only. Files are copied first, then hashed, and hard-linked to blobs named by those captured bytes' sha256. The build id hashes these artifacts and the current digests of inputs listed by Cargo's dep-info. Input digests are observations at capture time, not proof of what the compiler read. The record's `started` gives the build's start time; `edited` flags inputs **possibly modified since build started**, by mtime, including inputs of a first build or newly discovered dependencies. Identical inputs and artifacts reuse their first capture's record.
+
+The installed extension and binaries are symbolic links through `<project>/installed`, which `cargo develop` points at the new build in one step. `fastws.store.builds(root)` gives the newest build records, with input changes since the preceding build, and `fastws.store.resolve(root, id)` resolves a build-id prefix. Each develop deletes builds that are neither installed, among the project's last 20, nor from the last 2 days, then old unlinked blobs.
+
+On Unix, cooperating `cargo develop` runs take a per-project lock through compilation, capture, installation, the post-develop generator and wheel data. Store readers and capture/install operations hold shared store access; collection takes it exclusively. Ordinary Cargo commands do not take the project lock: agents must not run competing raw Cargo builds against that project during develop. Windows installs copies and does not take these locks.
+
+On Unix, every rustc call holds a shared `flock` on `/tmp/aai-ws-timing.lock`, through the rustc wrapper that sync writes (see `ws-sync`). A speed test that takes that lock exclusively stays apart from participating builds and test runs. A timing also holds `/tmp/aai-ws-timing.gate` while it waits for the lock and holds it, and a shared hold passes that file first, so new rustc calls and test runs wait behind the timing. Both shared and exclusive acquisition fail after 5 minutes; work never proceeds unlocked. Python code takes the same lock with `fastws.timelock.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. Shared-inside-shared in one thread reuses its hold; each thread otherwise takes its own hold. Any nesting involving exclusive raises `RuntimeError` immediately. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. Windows builds and tests run without this timing coordination.
 
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.29` alongside `maturin>=1.13,<2.0` in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
@@ -245,7 +251,22 @@ cargo wait
 cargo wait --path /path/to/workspace
 ```
 
-Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
+### Rust tests from Python
+
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+
+Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+
+```python
+from fastws.cargo import run_tests
+
+native = run_tests('.', '--no-fail-fast')
+focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'))
+```
+
+Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
 ### `ws-sync`
 
@@ -266,9 +287,11 @@ Existing globs, entries for missing directories, and entries for checkouts that
 
 Each sync regenerates Cargo overrides in the workspace's `.cargo/config.toml`. `[patch.crates-io]` gets an entry for each local crate, including nested Cargo workspace members, except a crate marked `publish = false`, which never comes from crates.io. Git dependencies that name local crates get entries under the matching URL. Builds under the workspace root use these local checkouts. Entries pointing outside the root and other configuration sections are preserved.
 
+Sync also defaults `[term] quiet = true` in that file to suppress routine Cargo output and warnings. An explicit `quiet = false` is preserved.
+
 Do not commit a `Cargo.lock` generated under these patches. Its source-less local entries cannot be resolved on another machine.
 
-When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved. If neither a wrapper nor `sccache` is present, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it.
+Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/fastws_cli.egg-info/SOURCES.txt

@@ -6,7 +6,10 @@ fastws/build_backend.py
 fastws/cargo.py
 fastws/core.py
 fastws/news.py
+fastws/procs.py
 fastws/releases.py
+fastws/store.py
+fastws/timelock.py
 fastws_cli.egg-info/PKG-INFO
 fastws_cli.egg-info/SOURCES.txt
 fastws_cli.egg-info/dependency_links.txt
@@ -16,6 +19,9 @@ fastws_cli.egg-info/top_level.txt
 tests/test_build.py
 tests/test_build_backend.py
 tests/test_news.py
+tests/test_procs.py
 tests/test_releases.py
 tests/test_setup.py
-tests/test_sync.py
+tests/test_store.py
+tests/test_sync.py
+tests/test_timelock.py

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/pyproject.toml

@@ -60,3 +60,6 @@ version = { attr = "fastws.__version__" }
 
 [tool.setuptools.packages.find]
 include = ["fastws"]
+
+[tool.pytest.ini_options]
+asyncio_default_fixture_loop_scope = "function"

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/tests/test_build.py

@@ -48,14 +48,14 @@ def test_run_tests(tmp_path):
     (tmp_path/'Cargo.toml').write_text('[package]\nname = "app"\nversion = "0.1.0"\nedition = "2021"\n')
     (tmp_path/'src').mkdir()
     (tmp_path/'src/lib.rs').write_text('''pub fn unused() { let x = 1; }
-/// ```
-/// assert_eq!(app::two(), 2);
-/// ```
 pub fn two() -> i32 { 2 }
 #[cfg(test)] mod t {
     #[test] fn passes() {}
     #[test] fn fails() { panic!("broken"); }
-    #[test] #[ignore] fn skipped() {}
+    #[test] #[ignore] fn skipped() {
+        println!("started slow test");
+        std::thread::sleep(std::time::Duration::from_secs(60));
+    }
 }
 ''')
     (tmp_path/'tests').mkdir()
@@ -65,9 +65,11 @@ pub fn two() -> i32 { 2 }
     r = run_tests(tmp_path, '--no-fail-fast')
     assert not r.ok
     assert [(d.path, d.line, d.level) for d in r.diagnostics] == [('src/lib.rs', 1, 'warning')]
-    unit, more, docs = r.suites
-    assert [s.name for s in r.suites] == ['unittests src/lib.rs', 'tests/more.rs', 'Doc-tests app']
+    suites = {s.name: s for s in r.suites}
+    unit, more = suites['unittests src/lib.rs'], suites['tests/more.rs']
+    assert len(suites) == 2
     assert unit.tests == {'t::passes': 'ok', 't::fails': 'FAILED', 't::skipped': 'ignored'} and unit.failed
     assert 'broken' in unit.failures['t::fails'] and list(unit.failures) == ['t::fails']
     assert more.tests == {'other': 'ok'} and not more.failed
-    assert list(docs.tests.values()) == ['ok']
+    with pytest.raises(TimeoutError, match='started slow test'):
+        run_tests(tmp_path, '--lib', test_args=('skipped', '--ignored', '--nocapture'), timeout=.5)

fastws_cli-0.0.33/tests/test_procs.py

@@ -0,0 +1,35 @@
+import fcntl, os, subprocess, sys, pytest
+from fastws.procs import JOB_ENV, kill_group
+
+
+def test_nested_timeout_leaves_cleanup_to_job_owner(tmp_path):
+    lock = tmp_path/'held'
+    descendant = '''import fcntl, os, sys, time
+f = open(sys.argv[1], 'a')
+fcntl.flock(f, fcntl.LOCK_SH)
+print('descendant group', os.getpgrp(), flush=True)
+time.sleep(60)
+'''
+    child = f'''import os, subprocess, sys, time
+print('child group', os.getpgrp(), os.environ.get('FASTWS_JOB'), flush=True)
+subprocess.Popen([sys.executable, '-c', {descendant!r}, {str(lock)!r}])
+time.sleep(60)
+'''
+    job = f'''import os, sys, time
+from fastws.procs import bounded
+print('job group', os.getpgrp(), flush=True)
+try: bounded([sys.executable, '-c', {child!r}], .5, merge=True, env={{}})
+except TimeoutError as e: print(e, flush=True)
+print('job survived inner timeout', flush=True)
+time.sleep(60)
+'''
+    with subprocess.Popen([sys.executable, '-c', job], start_new_session=True, env=os.environ | {JOB_ENV: '1'},
+                          stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) as p:
+        try:
+            with pytest.raises(subprocess.TimeoutExpired): p.communicate(timeout=.8)
+        finally: kill_group(p)
+        output, _ = p.communicate(timeout=2)
+    group = output.split('job group ')[1].split()[0]
+    assert f'child group {group} 1' in output and f'descendant group {group}' in output
+    assert 'job survived inner timeout' in output
+    with lock.open() as f: fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)

fastws_cli-0.0.33/tests/test_store.py

@@ -0,0 +1,33 @@
+import hashlib, os, time, pytest
+from fastws import store
+
+pytestmark = pytest.mark.skipif(os.name == 'nt', reason='Unix store locks and symbolic-link installation')
+
+
+def test_capture_install_and_collect(tmp_path, monkeypatch):
+    monkeypatch.setattr(store, 'STORE', tmp_path/'store')
+    root = tmp_path/'project'
+    root.mkdir()
+    (root/'pyproject.toml').write_text('[project]\nname = "example"\n')
+    source, artifact, installed = root/'source', root/'artifact', root/'installed'
+    with store.building(root) as started, store.access():
+        source.write_text('new input')
+        artifact.write_bytes(b'first')
+        first = store.keep(root, {'bin/tool': artifact, 'python/data': artifact}, [source], started)
+        assert store.keep(root, {'bin/tool': artifact, 'python/data': artifact}, [source], started) == first
+        store.install(root, first, {installed: 'bin/tool'})
+    assert installed.read_bytes() == b'first' and store.resolve(root) == first
+    assert (first/'bin/tool').stat().st_ino == (first/'python/data').stat().st_ino
+    assert store.builds(root)[0]['edited'] == [str(source)]
+    kept = []
+    with store.building(root) as started, store.access():
+        for i in range(store.KEEP+1):
+            artifact.write_text(str(i))
+            kept.append(store.keep(root, {'bin/tool': artifact}, [source], started))
+        old = time.time() - (store.KEEP_DAYS+1)*86400
+        for i, path in enumerate([first, *kept]): os.utime(path/'build.json', (old+i, old+i))
+        for blob in (store.STORE/'blobs').iterdir(): os.utime(blob, (old, old))
+    store.collect()
+    assert first.exists() and kept[-1].exists() and not kept[0].exists()
+    assert installed.read_bytes() == b'first'
+    assert not (store.STORE/'blobs'/hashlib.sha256(b'0').hexdigest()).exists()

{fastws_cli-0.0.32 → fastws_cli-0.0.33}/tests/test_sync.py

@@ -91,6 +91,16 @@ def test_sync_extras_preserves_constraints_markers_and_unrelated_settings(tmp_pa
         assert other == 'other[keep]' and len(data['project']['dependencies']) == 2
         assert data['tool']['uv']['sources'] == {'My_Pkg': {'path': '../somewhere', 'editable': True}}
 
+@pytest.mark.skipif(os.name == 'nt', reason='Windows has no timing lock, so no rustc-lock script')
+def test_rustc_wrapper_keeps_another_tools_wrapper(tmp_path):
+    config, script = tmp_path/'.cargo'/'config.toml', tmp_path/'.cargo'/'rustc-lock'
+    config.parent.mkdir()
+    config.write_text('[build]\nrustc-wrapper = "/opt/custom/wrap"\n')
+    for _ in range(2):
+        core._sync_cargo_wrapper(tmp_path)
+        assert f'rustc-wrapper = "{script}"' in config.read_text()
+        assert "INNER = '/opt/custom/wrap'" in script.read_text()
+
 
 def test_remove_from_pyproject(tmp_path):
     pyproject = tmp_path/'pyproject.toml'

fastws_cli-0.0.33/tests/test_timelock.py

@@ -0,0 +1,21 @@
+import os, threading, pytest
+from fastws import timelock
+if os.name != 'nt': import fcntl
+
+@pytest.mark.skipif(os.name == 'nt', reason='Windows has no timing lock')
+def test_shared_hold_outlives_other_thread(tmp_path, monkeypatch):
+    "One thread's release leaves the other thread's shared hold in place."
+    monkeypatch.setattr(timelock, 'LOCK', str(tmp_path/'lock'))
+    monkeypatch.setattr(timelock, 'GATE', str(tmp_path/'gate'))
+    held, release = threading.Event(), threading.Event()
+    def hold():
+        with timelock.timing_lock(): held.set(); assert release.wait(3)
+    t = threading.Thread(target=hold, daemon=True)
+    try:
+        with timelock.timing_lock():
+            t.start(); assert held.wait(3)
+        with open(timelock.LOCK) as f, pytest.raises(BlockingIOError): fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
+    finally:
+        release.set()
+        if t.ident is not None: t.join(3)
+    assert not t.is_alive()