fastws-cli 0.0.25__tar.gz → 0.0.27__tar.gz

{fastws_cli-0.0.25/fastws_cli.egg-info → fastws_cli-0.0.27}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.25
+Version: 0.0.27
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -217,6 +217,8 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 `cargo develop` also copies the `data/` subdirectory of `[tool.maturin].data` into the active venv, as a wheel install does. For example, `data/share/jupyter/kernels/<name>/kernel.json` is installed as `share/jupyter/kernels/<name>/kernel.json`, where Jupyter finds the kernel. `cargo develop` installs the binaries it builds, not the staged copies in `scripts/`.
 
+Each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes.
+
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.25` alongside maturin in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
 For distribution wheels containing the binaries, also set `[tool.maturin].data` to a generated wheel-data directory. With `native-binaries = true`, wheel builds through the backend replace its `scripts/` subdirectory with the current Python-free binaries, then delegate packaging to maturin. Explicit profile and target arguments from maturin's build settings also apply to the binaries. Without `native-binaries`, wheel builds go straight to maturin, and `[tool.maturin].data` is optional.
@@ -228,6 +230,17 @@ cargo stage --profile dist
 maturin build --profile dist
 ```
 
+### `cargo wait`
+
+Wait until a Cargo workspace compiles:
+
+```bash
+cargo wait
+cargo wait --path /path/to/workspace
+```
+
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
 ### `ws-sync`
 
 Syncing proceeds in this order:

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/README.md

@@ -191,6 +191,8 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 `cargo develop` also copies the `data/` subdirectory of `[tool.maturin].data` into the active venv, as a wheel install does. For example, `data/share/jupyter/kernels/<name>/kernel.json` is installed as `share/jupyter/kernels/<name>/kernel.json`, where Jupyter finds the kernel. `cargo develop` installs the binaries it builds, not the staged copies in `scripts/`.
 
+Each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes.
+
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.25` alongside maturin in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
 For distribution wheels containing the binaries, also set `[tool.maturin].data` to a generated wheel-data directory. With `native-binaries = true`, wheel builds through the backend replace its `scripts/` subdirectory with the current Python-free binaries, then delegate packaging to maturin. Explicit profile and target arguments from maturin's build settings also apply to the binaries. Without `native-binaries`, wheel builds go straight to maturin, and `[tool.maturin].data` is optional.
@@ -202,6 +204,17 @@ cargo stage --profile dist
 maturin build --profile dist
 ```
 
+### `cargo wait`
+
+Wait until a Cargo workspace compiles:
+
+```bash
+cargo wait
+cargo wait --path /path/to/workspace
+```
+
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
 ### `ws-sync`
 
 Syncing proceeds in this order:

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/fastws/__init__.py

@@ -1,4 +1,6 @@
-__version__ = "0.0.25"
+__version__ = "0.0.27"
+
+
 
 
 

fastws_cli-0.0.27/fastws/cargo.py

@@ -0,0 +1,261 @@
+r"Build, test and install Cargo artifacts for mixed Rust/Python projects."
+import fcntl, json, os, re, shutil, subprocess, sys, sysconfig, time
+from collections import Counter
+from contextlib import contextmanager
+from dataclasses import dataclass
+from pathlib import Path
+from typing import NamedTuple
+
+try: import tomllib
+except ModuleNotFoundError: import tomli as tomllib
+
+# The file whose lock keeps timings and builds apart on this machine. Timings take it exclusively. Builds and test runs take it shared.
+_TIMING_LOCK = '/tmp/aai-ws-timing.lock'
+
+
+class ConfigError(ValueError):
+    "A project's `pyproject.toml` lacks a setting that a command needs."
+
+
+def settings(root):
+    "Read the project's tool configuration."
+    return tomllib.loads((Path(root).expanduser()/'pyproject.toml').read_text()).get('tool', {})
+
+
+def cargo(root, *args):
+    "Run Cargo in the project and return its standard output."
+    return subprocess.run(['cargo', *args], cwd=Path(root).expanduser(), stdout=subprocess.PIPE, text=True, check=True).stdout
+
+
+def members(root):
+    "The manifests of the workspace's member packages."
+    return {Path(p['manifest_path']) for p in json.loads(cargo(root, 'metadata', '--no-deps', '--format-version', '1'))['packages']}
+
+
+def _lock_users():
+    "The other processes that have the timing lock's file open, as `pid command` strings from `lsof`. On macOS, where `lsof` doesn't show `flock` holds, the list also includes any process waiting for the lock."
+    users, pid = [], ''
+    for line in subprocess.run(['lsof', '+c', '0', '-F', 'pc', _TIMING_LOCK], capture_output=True, text=True).stdout.splitlines():
+        if line[:1] == 'p': pid = line[1:]
+        elif line[:1] == 'c' and pid != str(os.getpid()): users.append(f'{pid} {line[1:]}')
+    return users
+
+
+@contextmanager
+def timing_lock(
+    exclusive=False, # Take the lock exclusively, as a timing does; False: take it shared, as a build or a test run does
+):
+    "Hold the lock that keeps this machine's timings apart from its builds and test runs. A hold waits up to 5 minutes for the lock. After that, a shared hold goes ahead without it. An exclusive hold raises `TimeoutError`. An exclusive hold that has waited 30 seconds prints, once, the other processes that have the lock's file open."
+    with open(_TIMING_LOCK, 'a') as f:
+        start, waiting, reported = time.monotonic(), False, False
+        while True:
+            try:
+                fcntl.flock(f, (fcntl.LOCK_EX if exclusive else fcntl.LOCK_SH) | fcntl.LOCK_NB)
+                break
+            except BlockingIOError: pass
+            waited = time.monotonic() - start
+            if waited > 300:
+                if exclusive: raise TimeoutError(f'a build or another timing has held {_TIMING_LOCK} for 5 minutes')
+                print(f'going ahead although a timing still holds {_TIMING_LOCK}', file=sys.stderr)
+                break
+            if not waiting: print(f'waiting for {"a build or another timing" if exclusive else "a timing"} to release {_TIMING_LOCK}', file=sys.stderr)
+            waiting = True
+            if exclusive and not reported and waited > 30:
+                print(f'{_TIMING_LOCK} is still taken after 30 seconds. Processes with it open: {", ".join(_lock_users()) or "none"}', file=sys.stderr)
+                reported = True
+            time.sleep(0.25)
+        yield
+
+
+def check(root):
+    "Check every target of the workspace with Cargo, holding the timing lock shared. Cargo's messages are in the result's `stderr`, which `diagnostics` reads."
+    with timing_lock(): return subprocess.run(['cargo', 'check', '--all-targets', '--message-format=short'], cwd=Path(root).expanduser(), stderr=subprocess.PIPE, text=True)
+
+
+# Cargo's line that starts a test binary: `Running` with its target and executable, or `Doc-tests` with its package.
+_SUITE = re.compile(r'^\s+(?:Running (.+) \([^)]*\)|(Doc-tests .+))$')
+_TEST = re.compile(r'^test (.+) \.\.\. (ok|FAILED|ignored)')
+_FAILURE = re.compile(r'^---- (.+) std(?:out|err) ----$')
+_DIAGNOSTIC = re.compile(r'^(\S.*?):(\d+):(\d+): (warning|error)(?:\[\w+\])?: (.*)$')
+
+
+class Diagnostic(NamedTuple):
+    "A compiler message, read from Cargo's short format."
+    path:str; line:int; col:int; level:str; message:str
+    def __str__(self): return f'{self.level}: {self.path}:{self.line}: {self.message}'
+
+
+def diagnostics(text):
+    "Each `Diagnostic` in `text`, which holds Cargo's short-format messages, such as `check`'s `stderr`."
+    return [Diagnostic(m[1], int(m[2]), int(m[3]), m[4], m[5]) for l in text.splitlines() if (m := _DIAGNOSTIC.match(l))]
+
+
+def _lines(text, n=20, tail=False):
+    "Up to `n` lines of `text`, indented, from its end if `tail`, with a count of the lines left out."
+    lines = text.splitlines()
+    shown = [f'    {l}' for l in (lines[-n:] if tail else lines[:n])]
+    if len(lines) <= n: return shown
+    more = f'    … {len(lines)-n} more lines'
+    return [more, *shown] if tail else [*shown, more]
+
+
+@dataclass
+class Suite:
+    "What one test binary printed when `cargo test` ran it."
+    name:str # Cargo's name for the binary, such as `tests/core.rs`, `unittests src/lib.rs` or `Doc-tests basedpl`
+    output:str # Everything printed after Cargo's line that starts the binary, until the next binary starts
+    tests:dict # Each test's outcome: `ok`, `FAILED` or `ignored`
+    failures:dict # Each failing test's captured output
+    summary:str # libtest's `test result:` line, empty for a binary that crashed or has its own harness
+    failed:bool # Did Cargo report the binary as failing?
+
+    @classmethod
+    def read(cls, name, lines):
+        "Read one binary's lines."
+        failures, current = {}, None
+        for line in lines:
+            if m := _FAILURE.match(line): current = failures.setdefault(m[1], [])
+            elif line == 'failures:': current = None
+            elif current is not None: current.append(line)
+        return cls(name, '\n'.join(lines), {m[1]: m[2] for l in lines if (m := _TEST.match(l))},
+                   {k: '\n'.join(v).strip('\n') for k,v in failures.items()},
+                   next((l for l in lines if l.startswith('test result: ')), ''), any(l.startswith('error: test failed') for l in lines))
+
+    def __str__(self):
+        n = Counter(self.tests.values())
+        status = ', '.join(f'{n[o]} {w}' for o,w in [('ok', 'passed'), ('FAILED', 'failed'), ('ignored', 'ignored')] if n[o])
+        if self.failed and not n['FAILED']: status = f'{status}, then failed' if status else 'failed'
+        time = f' in {m[1]}' if (m := re.search(r'finished in (\S+)$', self.summary)) else ''
+        return f'{self.name}: {status or "ok"}{time}'
+
+
+@dataclass
+class Results:
+    "What `cargo test` printed, read into the build's compiler messages and each test binary's results. It displays as a summary: the compiler messages, each binary that ran tests, and each failure's output."
+    returncode:int
+    text:str # Everything Cargo and the tests printed, standard output and standard error in the order they were written
+    diagnostics:list # Each `Diagnostic` from the build, before the first binary ran
+    suites:list # Each binary's `Suite`, in the order Cargo ran them
+
+    @property
+    def ok(self): return self.returncode == 0
+
+    def __repr__(self):
+        res = ['cargo test passed' if self.ok else f'cargo test failed with exit status {self.returncode}']
+        diags = sorted(self.diagnostics, key=lambda d: d.level != 'error')
+        res += [str(d) for d in diags[:20]]
+        if len(diags) > 20: res.append(f'{len(diags)-20} more compiler messages in `diagnostics`')
+        ran = [s for s in self.suites if s.tests or not s.summary]
+        for s in ran:
+            res.append(str(s))
+            for t,out in s.failures.items(): res += [f'  FAILED {t}', *_lines(out)]
+            if s.failed and not s.failures: res += _lines(s.output, tail=True)
+        if idle := len(self.suites) - len(ran): res.append(f'{idle} test binaries ran no tests')
+        if not (self.ok or self.suites or diags): res += _lines(self.text, tail=True)
+        return '\n'.join(res)
+
+
+def _blocks(text):
+    "Split `text` at Cargo's lines that start a test binary, into `(name, lines)` pairs. The first pair, named `None`, holds the build's lines."
+    res = [(None, [])]
+    for line in text.splitlines():
+        if m := _SUITE.match(line): res.append((m[1] or m[2], []))
+        else: res[-1][1].append(line)
+    return res
+
+
+def run_tests(
+    root, # A Cargo package or workspace (expands `~`)
+    *args, # Arguments for `cargo test`, such as `'--test', 'core'`, a test name to filter by, or `'--', '--nocapture'`
+    env:dict=None, # Environment variables to set for the run, such as `{'BASEDPL_COSTS': '1'}`
+    timeout:float=None # Seconds before the run stops, raising `subprocess.TimeoutExpired`
+)->Results:
+    "Run `cargo test` in `root` and read what it prints. Compiler messages come in Cargo's short format. It doesn't hold the timing lock as `check` and `build` do: tests that time take the lock exclusively themselves, and a shared hold here would block them."
+    # A quiet Cargo config hides the `Running` lines that divide the binaries' output.
+    p = subprocess.run(['cargo', 'test', '--message-format=short', *args], cwd=Path(root).expanduser(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, timeout=timeout,
+                       env=os.environ | {'CARGO_TERM_QUIET': 'false', 'CARGO_TERM_COLOR': 'never'} | (env or {}))
+    (_, build), *runs = _blocks(p.stdout)
+    return Results(p.returncode, p.stdout, diagnostics('\n'.join(build)), [Suite.read(*b) for b in runs])
+
+
+def sources(root):
+    "The manifests, and the target directories or files, of the local packages that the workspace builds: its members and its path dependencies."
+    res = set()
+    for p in json.loads(cargo(root, 'metadata', '--format-version', '1'))['packages']:
+        if p['source'] is not None: continue
+        manifest = Path(p['manifest_path'])
+        res.add(manifest)
+        for t in p['targets']:
+            src = Path(t['src_path'])
+            res.add(src if src.parent == manifest.parent else src.parent)
+    return res
+
+
+def build(root, *, library=True, binaries=False, profile=None, target=None, no_default_features=False):
+    "Build with Cargo and return the compiler artifacts of the workspace's packages."
+    root = Path(root).expanduser()
+    cfg = settings(root)['maturin']
+    cmd = ['build', '--manifest-path', str((root/'Cargo.toml').resolve()), '--message-format=json-diagnostic-short']
+    if library: cmd.append('--lib')
+    if binaries: cmd.append('--bins')
+    if profile := profile or cfg.get('profile'): cmd += ['--profile', profile]
+    if target: cmd += ['--target', target]
+    if no_default_features: cmd.append('--no-default-features')
+    else:
+        if features := cfg.get('features'): cmd += ['--features', ','.join(features)]
+        for flag in ('all-features', 'no-default-features'):
+            if cfg.get(flag): cmd.append('--' + flag)
+    with timing_lock(): result = subprocess.run(['cargo', *cmd], cwd=root, stdout=subprocess.PIPE, text=True)
+    messages = [json.loads(line) for line in result.stdout.splitlines()]
+    for m in messages:
+        if m.get('reason') != 'compiler-message': continue
+        name = tomllib.loads(Path(m['manifest_path']).read_text())['package']['name']
+        for line in m['message']['rendered'].splitlines(): print(f'[{name}] {line}', file=sys.stderr)
+    result.check_returncode()
+    workspace = members(root)
+    return [m for m in messages if m.get('reason') == 'compiler-artifact' and Path(m['manifest_path']) in workspace]
+
+
+def replace(source, destination):
+    "Replace an installed artifact without changing the file held by running processes."
+    destination.parent.mkdir(parents=True, exist_ok=True)
+    tmp = destination.with_name(f'{destination.name}.{os.getpid()}.tmp')
+    shutil.copy2(source, tmp)
+    os.replace(tmp, destination)
+    return destination
+
+
+def develop(root):
+    "Install a Python extension, opted-in binaries and wheel data files from one Cargo build."
+    root = Path(root).expanduser()
+    tool = settings(root)
+    cfg = tool['maturin']
+    if 'python-source' not in cfg: raise ConfigError(r'cargo develop requires tool.maturin.python-source')
+    binaries = tool.get('fastws', {}).get('native-binaries', False)
+    artifacts = build(root, binaries=binaries, profile='test')
+    artifact, = [m for m in artifacts if 'cdylib' in m['target']['crate_types']]
+    lib, = [f for f in artifact['filenames'] if Path(f).suffix in ('.dylib', '.so', '.dll')]
+    module = cfg.get('module-name', artifact['target']['name'])
+    dest = root/cfg['python-source']/Path(*module.split('.'))
+    replace(lib, dest.with_name(dest.name + sysconfig.get_config_var('EXT_SUFFIX')))
+    env = Path(os.environ.get('VIRTUAL_ENV', sys.prefix))
+    if binaries:
+        scripts = env/('Scripts' if os.name == 'nt' else 'bin')
+        for m in artifacts:
+            if m.get('executable'): replace(m['executable'], scripts/Path(m['executable']).name)
+    # Editable builds through `build_backend` don't install the wheel's data files, which a wheel install puts under the prefix.
+    if 'data' in cfg:
+        data = root/cfg['data']/'data'
+        for f in data.rglob('*'):
+            if f.is_file(): replace(f, env/f.relative_to(data))
+
+
+def stage_binaries(root, profile=None, target=None):
+    "Stage Python-free Cargo binaries in maturin's wheel-data directory."
+    root = Path(root).expanduser()
+    cfg = settings(root)['maturin']
+    if 'data' not in cfg: raise ConfigError(r'staging native binaries requires tool.maturin.data')
+    dest = root/cfg['data']/'scripts'
+    artifacts = build(root, library=False, binaries=True, profile=profile, target=target, no_default_features=True)
+    if dest.exists(): shutil.rmtree(dest)
+    return [replace(m['executable'], dest/Path(m['executable']).name) for m in artifacts if m.get('executable')]

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/fastws/core.py

@@ -1,9 +1,10 @@
 "Fast workspace tools for multi-repo management."
 
-__all__ = ["ws_setup", "ws_clone", "ws_pull", "ws_status", "ws_branches", "ws_build", "ws_sync", "ws_add", "ws_remove", "cargo_develop", "cargo_stage"]
+__all__ = ["ws_setup", "ws_clone", "ws_pull", "ws_status", "ws_branches", "ws_build", "ws_sync", "ws_add", "ws_remove", "cargo_develop", "cargo_stage", "cargo_wait"]
 
 import ast, fnmatch, glob, hashlib, json, os, re, shlex, shutil, subprocess, sys, time
 from pathlib import Path
+from contextlib import contextmanager
 from concurrent.futures import ThreadPoolExecutor
 
 from fastcore.parallel import parallel_async_gen
@@ -556,6 +557,16 @@ def _build_projects(root: Path, repos_file: str, project: str = None) -> list[tu
     dirs = dep_closure(key, graph)
     return [(n,d) for n,d in res if d in dirs]
 
+@contextmanager
+def _cli_errors():
+    "Exit without a traceback on command failures or missing settings, printing captured stderr and preserving command exit status."
+    from .cargo import ConfigError
+    try: yield
+    except subprocess.CalledProcessError as e:
+        if e.stderr: print(e.stderr, file=sys.stderr, end='')
+        raise SystemExit(e.returncode) from None
+    except ConfigError as e: raise SystemExit(str(e)) from None
+
 @call_parse(pos=['command'])
 def cargo_develop(
     command: str = 'develop', # Subcommand name supplied by Cargo
@@ -563,7 +574,7 @@ def cargo_develop(
 ):
     r"Build and install a mixed Rust/Python project's extension and opted-in binaries."
     from .cargo import develop
-    develop(Path(path).resolve())
+    with _cli_errors(): develop(Path(path).resolve())
 
 @call_parse(pos=['command'])
 def cargo_stage(
@@ -574,7 +585,33 @@ def cargo_stage(
 ):
     r"Stage Python-free native binaries for maturin wheel builds."
     from .cargo import stage_binaries
-    for dest in stage_binaries(Path(path).resolve(), profile or None, target or None): print(f'Staged {dest}')
+    with _cli_errors():
+        for dest in stage_binaries(Path(path).resolve(), profile or None, target or None): print(f'Staged {dest}')
+
+def _newest(paths):
+    "Newest file mtime among `paths`, walking each directory as `_src_mtime` does"
+    return max((_src_mtime(p) if p.is_dir() else p.stat().st_mtime) for p in paths if p.exists())
+
+@call_parse(pos=['command'])
+def cargo_wait(
+    command: str = 'wait', # Subcommand name supplied by Cargo
+    path: str = '.', # Cargo workspace directory
+    patience: int = 1800, # Seconds to wait before giving up
+):
+    r"Wait until a Cargo workspace compiles, checking again after each edit to its local packages' sources."
+    from .cargo import check, diagnostics, sources
+    root, start = Path(path).resolve(), time.monotonic()
+    with _cli_errors(): srcs = sources(root)
+    while (res := check(root)).returncode:
+        if time.monotonic() - start > patience: raise SystemExit(f'{res.stderr}gave up after {patience} s')
+        print(next((str(d) for d in diagnostics(res.stderr) if d.level == 'error'), res.stderr.strip() or 'cargo check failed'), file=sys.stderr)
+        before = _newest(srcs)
+        while (now := _newest(srcs)) == before and time.monotonic() - start < patience: time.sleep(1)
+        while True:
+            time.sleep(2)
+            if (later := _newest(srcs)) == now: break
+            now = later
+    print('compiles')
 
 @call_parse
 def ws_build(
@@ -898,58 +935,60 @@ async def ws_sync(
     upgrade: bool = False,  # Force the once-daily dependency upgrade pass
 ):
     "Pull the workspace baseline, clone missing shared/local repos, then pull projects and sync; upgrade dependencies daily."
-    root = _ws_root(workspace, repos_file, pyproject_file, template_file)
-    repos_path = _resolve_path(root, repos_file)
-    pyproject_path = _resolve_path(root, pyproject_file)
-    template_path = _resolve_path(root, template_file)
-    await _pull_workspace(root)
-    entries = _load_repo_entries(repos_path, root)
-    await _clone_repos(entries, workers)
-    repos = await _discover_ws_repos(root)
-
-    if missing_repos := _update_repos_file(repos_path, repos): print(f"Added local repos: {', '.join(missing_repos)}")
-    entries = _load_repo_entries(repos_path, root)
-    ext_dirs = [d for _,d,_ in entries if d.resolve().parent != root.resolve()]
-    dirs = [d for _,d,_ in entries]
-    try: dirs = await _changed_dirs(dirs)
-    except Exception: pass
-    await _pull(dirs, workers=workers)
-
-    if not pyproject_path.exists(): _sync_ws_pyproject(pyproject_path, template_path, [])
-    added_ex, removed_ex = _sync_ws_excludes(pyproject_path, root, {d.name for _,d,_ in entries if d.resolve().parent == root.resolve()})
-    if added_ex: print(f"Auto-excluded from the workspace: {', '.join(added_ex)}")
-    if removed_ex: print(f"No longer excluded from the workspace: {', '.join(removed_ex)}")
-
-    missing_projects = _sync_ws_pyproject(pyproject_path, template_path, _ws_projects(root), _external_projects(root, ext_dirs),
-        _repo_extras(root, entries))
-    if missing_projects: print(f"Added workspace projects: {', '.join(missing_projects)}")
-    projects = _build_projects(root, repos_file)
-    _sync_build_isolation(pyproject_path, projects)
-
-    wrapper_added = _sync_cargo_wrapper(root)
-    added_p, removed_p = _sync_cargo_patches(root)
-    if added_p: print(f"Cargo patches added: {', '.join(added_p)}")
-    if removed_p: print(f"Cargo patches removed: {', '.join(removed_p)}")
-    if wrapper_added: print("Cargo builds now use sccache")
-
-    js_members = _npm_dirs(root)
-    if _native_js(js_members): _check_rustup()
-    added_j, removed_j = _sync_ws_package_json(root, js_members)
-    if added_j: print(f"JS workspace packages added: {', '.join(added_j)}")
-    if removed_j: print(f"JS workspace packages removed: {', '.join(removed_j)}")
-
-    if bad := _pending_dirs(root):
-        print(f"⚠️  Skipping uv sync, not Python projects yet (scaffold with e.g. nbdev-new or ship-new, or remove): {', '.join(bad)}")
-        return
-    up = upgrade or _should_upgrade(root)
-    if up: _cargo_update(root, workers=workers)
-    _sync_cargo_keys(root)
-    subprocess.run(["uv", "sync", "-U"] if up else ["uv", "sync"], check=True, cwd=root)
-    for _,d in projects:
-        cfg = tomllib.loads((d/'pyproject.toml').read_text()).get('tool', {}).get('maturin', {})
-        if 'python-source' in cfg and cfg.get('bindings') != 'bin': cargo_develop(path=str(d))
-    if up: _upgrade_stamp(root).touch()
-    if js_members: _sync_js(root, js_members)
+    with _cli_errors():
+        root = _ws_root(workspace, repos_file, pyproject_file, template_file)
+        repos_path = _resolve_path(root, repos_file)
+        pyproject_path = _resolve_path(root, pyproject_file)
+        template_path = _resolve_path(root, template_file)
+        await _pull_workspace(root)
+        entries = _load_repo_entries(repos_path, root)
+        await _clone_repos(entries, workers)
+        repos = await _discover_ws_repos(root)
+    
+        if missing_repos := _update_repos_file(repos_path, repos): print(f"Added local repos: {', '.join(missing_repos)}")
+        entries = _load_repo_entries(repos_path, root)
+        ext_dirs = [d for _,d,_ in entries if d.resolve().parent != root.resolve()]
+        dirs = [d for _,d,_ in entries]
+        try: dirs = await _changed_dirs(dirs)
+        except Exception: pass
+        await _pull(dirs, workers=workers)
+    
+        if not pyproject_path.exists(): _sync_ws_pyproject(pyproject_path, template_path, [])
+        added_ex, removed_ex = _sync_ws_excludes(pyproject_path, root, {d.name for _,d,_ in entries if d.resolve().parent == root.resolve()})
+        if added_ex: print(f"Auto-excluded from the workspace: {', '.join(added_ex)}")
+        if removed_ex: print(f"No longer excluded from the workspace: {', '.join(removed_ex)}")
+    
+        missing_projects = _sync_ws_pyproject(pyproject_path, template_path, _ws_projects(root), _external_projects(root, ext_dirs),
+            _repo_extras(root, entries))
+        if missing_projects: print(f"Added workspace projects: {', '.join(missing_projects)}")
+        projects = _build_projects(root, repos_file)
+        _sync_build_isolation(pyproject_path, projects)
+    
+        wrapper_added = _sync_cargo_wrapper(root)
+        added_p, removed_p = _sync_cargo_patches(root)
+        if added_p: print(f"Cargo patches added: {', '.join(added_p)}")
+        if removed_p: print(f"Cargo patches removed: {', '.join(removed_p)}")
+        if wrapper_added: print("Cargo builds now use sccache")
+    
+        js_members = _npm_dirs(root)
+        if _native_js(js_members): _check_rustup()
+        added_j, removed_j = _sync_ws_package_json(root, js_members)
+        if added_j: print(f"JS workspace packages added: {', '.join(added_j)}")
+        if removed_j: print(f"JS workspace packages removed: {', '.join(removed_j)}")
+    
+        if bad := _pending_dirs(root):
+            print(f"⚠️  Skipping uv sync, not Python projects yet (scaffold with e.g. nbdev-new or ship-new, or remove): {', '.join(bad)}")
+            return
+        up = upgrade or _should_upgrade(root)
+        if up: _cargo_update(root, workers=workers)
+        _sync_cargo_keys(root)
+        subprocess.run(["uv", "sync", "-U"] if up else ["uv", "sync"], check=True, cwd=root)
+        from .cargo import develop
+        for _,d in projects:
+            cfg = tomllib.loads((d/'pyproject.toml').read_text()).get('tool', {}).get('maturin', {})
+            if 'python-source' in cfg and cfg.get('bindings') != 'bin': develop(d.resolve())
+        if up: _upgrade_stamp(root).touch()
+        if js_members: _sync_js(root, js_members)
 
 @call_parse
 async def ws_add(

{fastws_cli-0.0.25 → fastws_cli-0.0.27/fastws_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.25
+Version: 0.0.27
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -217,6 +217,8 @@ Set `native-binaries = true` under `[tool.fastws]` to build and install the bina
 
 `cargo develop` also copies the `data/` subdirectory of `[tool.maturin].data` into the active venv, as a wheel install does. For example, `data/share/jupyter/kernels/<name>/kernel.json` is installed as `share/jupyter/kernels/<name>/kernel.json`, where Jupyter finds the kernel. `cargo develop` installs the binaries it builds, not the staged copies in `scripts/`.
 
+Each Cargo build that fastws runs holds a shared `flock` on `/tmp/aai-ws-timing.lock`. A speed test that takes that lock exclusively then never times code while a build runs. A build waits up to 5 minutes for such a timing to finish, then goes ahead without the lock. Python code takes the same lock with `fastws.cargo.timing_lock`: `with timing_lock(exclusive=True):` around a timing, and `with timing_lock():` around a test run. An exclusive hold that has waited 30 seconds prints the processes that have the lock's file open. It raises `TimeoutError` after 5 minutes.
+
 Use `fastws.build_backend` as the build backend of a mixed project, and include `fastws-cli>=0.0.25` alongside maturin in `[build-system].requires`. Its editable builds install only Python metadata and the source path, and compile no Rust. `ws-sync` then runs `cargo develop` to build and install the extension, the binaries and the wheel data files. After standalone `uv sync`, run `cargo develop` yourself.
 
 For distribution wheels containing the binaries, also set `[tool.maturin].data` to a generated wheel-data directory. With `native-binaries = true`, wheel builds through the backend replace its `scripts/` subdirectory with the current Python-free binaries, then delegate packaging to maturin. Explicit profile and target arguments from maturin's build settings also apply to the binaries. Without `native-binaries`, wheel builds go straight to maturin, and `[tool.maturin].data` is optional.
@@ -228,6 +230,17 @@ cargo stage --profile dist
 maturin build --profile dist
 ```
 
+### `cargo wait`
+
+Wait until a Cargo workspace compiles:
+
+```bash
+cargo wait
+cargo wait --path /path/to/workspace
+```
+
+Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets` while holding the timing lock shared, as builds do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
+
 ### `ws-sync`
 
 Syncing proceeds in this order:

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/fastws_cli.egg-info/entry_points.txt

@@ -1,6 +1,7 @@
 [console_scripts]
 cargo-develop = fastws.core:cargo_develop
 cargo-stage = fastws.core:cargo_stage
+cargo-wait = fastws.core:cargo_wait
 ws-add = fastws.core:ws_add
 ws-branches = fastws.core:ws_branches
 ws-build = fastws.core:ws_build

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/pyproject.toml

@@ -36,6 +36,7 @@ dev = [
 [project.scripts]
 cargo-develop = "fastws.core:cargo_develop"
 cargo-stage = "fastws.core:cargo_stage"
+cargo-wait = "fastws.core:cargo_wait"
 ws-setup = "fastws.core:ws_setup"
 ws-clone = "fastws.core:ws_clone"
 ws-pull = "fastws.core:ws_pull"

fastws_cli-0.0.27/tests/test_build.py

@@ -0,0 +1,75 @@
+import subprocess
+import pytest
+import fastws.core as core
+from fastws.cargo import build, run_tests
+
+
+def _mk_proj(root, name, extra=''):
+    d = root/name
+    d.mkdir()
+    (d/'pyproject.toml').write_text(fr'''[project]
+name = "{name}"
+{extra}
+''')
+    return d
+
+
+def test_build_dependency_selection(tmp_path):
+    root = tmp_path/'ws'
+    root.mkdir()
+    _mk_proj(root, 'app', r'''dependencies = ["My.Lib>=1"]''')
+    _mk_proj(root, 'my-lib', r'''[build-system]
+requires = ["external"]''')
+    external = _mk_proj(tmp_path, 'external', r'''dependencies = ["app", "published-only"]''')
+    _mk_proj(root, 'unrelated')
+    (root/'repos-local.txt').write_text(f'owner/external {external}')
+    assert {n for n,d in core._build_projects(root, 'repos.txt', 'APP')} == {'app', 'my-lib', 'external'}
+    with pytest.raises(SystemExit, match='No workspace project'): core._build_projects(root, 'repos.txt', 'missing')
+
+
+@pytest.mark.parametrize('body, diagnostic', [('', ''), ('let unused = 1;', 'warning'), ('compile_error!("broken");', 'error')])
+def test_build_diagnostics(tmp_path, capfd, body, diagnostic):
+    (tmp_path/'pyproject.toml').write_text('[tool.maturin]\n')
+    (tmp_path/'Cargo.toml').write_text('[package]\nname = "app"\nversion = "0.1.0"\n[dependencies]\ndep = { path = "dep" }\n')
+    (tmp_path/'src').mkdir()
+    (tmp_path/'src/lib.rs').write_text('extern crate dep; pub use dep::run;')
+    (tmp_path/'dep/src').mkdir(parents=True)
+    (tmp_path/'dep/Cargo.toml').write_text('[package]\nname = "dep"\nversion = "0.1.0"\n')
+    (tmp_path/'dep/src/lib.rs').write_text(f'pub fn run() {{ {body} }}')
+    (tmp_path/'.cargo').mkdir()
+    (tmp_path/'.cargo/config.toml').write_text('[term]\nquiet = true\n')
+    if diagnostic == 'error':
+        with pytest.raises(subprocess.CalledProcessError): build(tmp_path)
+    else: assert [m['target']['name'] for m in build(tmp_path)] == ['app']
+    stderr = capfd.readouterr().err
+    if diagnostic: assert f'[dep] dep/src/lib.rs:1:' in stderr and f': {diagnostic}:' in stderr
+    else: assert not stderr
+
+
+def test_run_tests(tmp_path):
+    (tmp_path/'Cargo.toml').write_text('[package]\nname = "app"\nversion = "0.1.0"\nedition = "2021"\n')
+    (tmp_path/'src').mkdir()
+    (tmp_path/'src/lib.rs').write_text('''pub fn unused() { let x = 1; }
+/// ```
+/// assert_eq!(app::two(), 2);
+/// ```
+pub fn two() -> i32 { 2 }
+#[cfg(test)] mod t {
+    #[test] fn passes() {}
+    #[test] fn fails() { panic!("broken"); }
+    #[test] #[ignore] fn skipped() {}
+}
+''')
+    (tmp_path/'tests').mkdir()
+    (tmp_path/'tests/more.rs').write_text('#[test] fn other() { assert_eq!(app::two(), 2); }\n')
+    (tmp_path/'.cargo').mkdir()
+    (tmp_path/'.cargo/config.toml').write_text('[term]\nquiet = true\n')
+    r = run_tests(tmp_path, '--no-fail-fast')
+    assert not r.ok
+    assert [(d.path, d.line, d.level) for d in r.diagnostics] == [('src/lib.rs', 1, 'warning')]
+    unit, more, docs = r.suites
+    assert [s.name for s in r.suites] == ['unittests src/lib.rs', 'tests/more.rs', 'Doc-tests app']
+    assert unit.tests == {'t::passes': 'ok', 't::fails': 'FAILED', 't::skipped': 'ignored'} and unit.failed
+    assert 'broken' in unit.failures['t::fails'] and list(unit.failures) == ['t::fails']
+    assert more.tests == {'other': 'ok'} and not more.failed
+    assert list(docs.tests.values()) == ['ok']

{fastws_cli-0.0.25 → fastws_cli-0.0.27}/tests/test_sync.py

@@ -24,6 +24,17 @@ def mk_repo(d, origin=None):
     return g
 
 
+@pytest.mark.asyncio
+async def test_sync_reports_workspace_pull_failure(tmp_path, capsys):
+    g = mk_repo(tmp_path, origin=tmp_path/'origin')
+    g.config('pull.rebase', 'true')
+    g.config('rebase.autoStash', 'false')
+    (tmp_path/'f.txt').write_text('dirty')
+    with pytest.raises(SystemExit) as e: await core.ws_sync(workspace=str(tmp_path))
+    assert e.value.code != 0
+    assert 'cannot pull with rebase' in capsys.readouterr().err
+
+
 def test_repos_file_roundtrip(tmp_path):
     repos_path = tmp_path/'repos.txt'
     repos_path.write_text('# header\nAnswerDotAI/fastws[dev]\njph00/private[docs] ~/private\nfastai/fastai sub/dir\n')

fastws_cli-0.0.25/fastws/cargo.py

@@ -1,81 +0,0 @@
-r"Build and install Cargo artifacts for mixed Rust/Python projects."
-import json, os, shutil, subprocess, sys, sysconfig
-from pathlib import Path
-
-try: import tomllib
-except ModuleNotFoundError: import tomli as tomllib
-
-
-def settings(root):
-    "Read the project's tool configuration."
-    return tomllib.loads((root/'pyproject.toml').read_text()).get('tool', {})
-
-
-def cargo(root, *args):
-    "Run Cargo in the project and return its standard output."
-    return subprocess.run(['cargo', *args], cwd=root, stdout=subprocess.PIPE, text=True, check=True).stdout
-
-
-def members(root):
-    "The manifests of the workspace's member packages."
-    return {Path(p['manifest_path']) for p in json.loads(cargo(root, 'metadata', '--no-deps', '--format-version', '1'))['packages']}
-
-
-def build(root, *, library=True, binaries=False, profile=None, target=None, no_default_features=False):
-    "Build with Cargo and return the compiler artifacts of the workspace's packages."
-    cfg = settings(root)['maturin']
-    cmd = ['build', '--manifest-path', str((root/'Cargo.toml').resolve()), '--message-format=json-render-diagnostics']
-    if library: cmd.append('--lib')
-    if binaries: cmd.append('--bins')
-    if profile := profile or cfg.get('profile'): cmd += ['--profile', profile]
-    if target: cmd += ['--target', target]
-    if no_default_features: cmd.append('--no-default-features')
-    else:
-        if features := cfg.get('features'): cmd += ['--features', ','.join(features)]
-        for flag in ('all-features', 'no-default-features'):
-            if cfg.get(flag): cmd.append('--' + flag)
-    messages = [json.loads(line) for line in cargo(root, *cmd).splitlines()]
-    workspace = members(root)
-    return [m for m in messages if m.get('reason') == 'compiler-artifact' and Path(m['manifest_path']) in workspace]
-
-
-def replace(source, destination):
-    "Replace an installed artifact without changing the file held by running processes."
-    destination.parent.mkdir(parents=True, exist_ok=True)
-    tmp = destination.with_name(destination.name + '.tmp')
-    shutil.copy2(source, tmp)
-    os.replace(tmp, destination)
-    return destination
-
-
-def develop(root):
-    "Install a Python extension, opted-in binaries and wheel data files from one Cargo build."
-    tool = settings(root)
-    cfg = tool['maturin']
-    if 'python-source' not in cfg: raise SystemExit(r'cargo develop requires tool.maturin.python-source')
-    binaries = tool.get('fastws', {}).get('native-binaries', False)
-    artifacts = build(root, binaries=binaries, profile='test')
-    artifact, = [m for m in artifacts if 'cdylib' in m['target']['crate_types']]
-    lib, = [f for f in artifact['filenames'] if Path(f).suffix in ('.dylib', '.so', '.dll')]
-    module = cfg.get('module-name', artifact['target']['name'])
-    dest = root/cfg['python-source']/Path(*module.split('.'))
-    replace(lib, dest.with_name(dest.name + sysconfig.get_config_var('EXT_SUFFIX')))
-    env = Path(os.environ.get('VIRTUAL_ENV', sys.prefix))
-    if binaries:
-        scripts = env/('Scripts' if os.name == 'nt' else 'bin')
-        for m in artifacts:
-            if m.get('executable'): replace(m['executable'], scripts/Path(m['executable']).name)
-    # Editable builds through `build_backend` don't install the wheel's data files, which a wheel install puts under the prefix.
-    if 'data' in cfg:
-        data = root/cfg['data']/'data'
-        for f in data.rglob('*'):
-            if f.is_file(): replace(f, env/f.relative_to(data))
-
-
-def stage_binaries(root, profile=None, target=None):
-    "Stage Python-free Cargo binaries in maturin's wheel-data directory."
-    cfg = settings(root)['maturin']
-    dest = root/cfg['data']/'scripts'
-    artifacts = build(root, library=False, binaries=True, profile=profile, target=target, no_default_features=True)
-    if dest.exists(): shutil.rmtree(dest)
-    return [replace(m['executable'], dest/Path(m['executable']).name) for m in artifacts if m.get('executable')]

fastws_cli-0.0.25/tests/test_build.py

@@ -1,25 +0,0 @@
-import pytest
-import fastws.core as core
-
-
-def _mk_proj(root, name, extra=''):
-    d = root/name
-    d.mkdir()
-    (d/'pyproject.toml').write_text(fr'''[project]
-name = "{name}"
-{extra}
-''')
-    return d
-
-
-def test_build_dependency_selection(tmp_path):
-    root = tmp_path/'ws'
-    root.mkdir()
-    _mk_proj(root, 'app', r'''dependencies = ["My.Lib>=1"]''')
-    _mk_proj(root, 'my-lib', r'''[build-system]
-requires = ["external"]''')
-    external = _mk_proj(tmp_path, 'external', r'''dependencies = ["app", "published-only"]''')
-    _mk_proj(root, 'unrelated')
-    (root/'repos-local.txt').write_text(f'owner/external {external}')
-    assert {n for n,d in core._build_projects(root, 'repos.txt', 'APP')} == {'app', 'my-lib', 'external'}
-    with pytest.raises(SystemExit, match='No workspace project'): core._build_projects(root, 'repos.txt', 'missing')