fastws-cli 0.0.33__tar.gz → 0.0.34__tar.gz

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.33
+Version: 0.0.34
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -205,9 +205,10 @@ Build a mixed Rust/Python project's extension and opted-in binaries, keep them w
 ```bash
 cargo develop
 cargo develop --path /path/to/project
+cargo develop --profile quick
 ```
 
-Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. It always builds with Cargo's `test` profile, matching bare `cargo test`. The project must produce a `cdylib` and set `python-source`. To use release settings without command-line flags, add `[profile.test]` with `inherits = "release"` to `Cargo.toml`. Keep incremental compilation enabled for the local crate in that profile.
+Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. `[tool.fastws] dev-profile = "quick"` selects a project's default for both `cargo develop` and `fastws.cargo.run_tests`. Without that setting they use `test`. An explicit `--profile NAME` overrides the project default. The project must produce a `cdylib` and set `python-source`. Define the selected profile in `Cargo.toml` and keep incremental compilation enabled for the local crate.
 
 Make the initial editable installation through `ws-sync` or maturin. Repeat that installation when package metadata or entry points change. Running Python processes keep the extension they loaded.
 
@@ -253,11 +254,11 @@ cargo wait --path /path/to/workspace
 
 Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
 
-### Rust tests from Python
+### Rust builds and tests from Python
 
-`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Each binary gets its own `timeout`, 30 seconds by default, excluding compilation. A binary still running when its `timeout` ends is killed and counts as a failed binary, with its suite's `timed_out` set. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
 
-Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+The default profile is `[tool.fastws].dev-profile`, or `test` when unset. Explicit `--profile NAME`, `--profile=NAME`, `--release` and `-r` override it. Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
 
 ```python
 from fastws.cargo import run_tests
@@ -268,6 +269,34 @@ focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'
 
 Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
+`fastws.cargo.check` runs any Cargo subcommand, `check --all-targets` by default, and returns the same `Results` without test suites. A failing run is a result, not an exception. Each `Diagnostic` in its `diagnostics` holds the message's path, line, column, level and text, and its error code or lint as `code`, such as `E0308` or `clippy::too_many_lines`. The summary lists errors first. `env` sets environment variables for the run, as for `run_tests`. A subcommand that rejects `--message-format`, such as `cargo llvm-lines`, runs with `diagnostics=False`: its result holds the exit status and output, with no diagnostics.
+
+```python
+from fastws.cargo import check
+
+lints = check('.', 'clippy', '--all-targets', '--', '-W', 'clippy::too_many_lines', env={'CLIPPY_CONF_DIR': 'review'})
+errors = [d for d in lints.diagnostics if d.level == 'error']
+sizes = check('.', 'llvm-lines', '--lib', diagnostics=False).text
+```
+
+### Integration snapshots
+
+`fastws.integrate` builds several repos together while people or agents keep editing them. `integrate` snapshots each repo's working tree as a commit (untracked files included, ignored files not), checks each commit out as a detached worktree under one root, and writes the root's Cargo config. `HEAD` and the index of each live repo don't change. The config is the workspace's, with the patches for crates under the root pointing at the root's worktrees, so a build there reads the snapshots and every other local crate from the workspace. Keep the root outside the workspace, where `ws-sync` and uv don't treat its worktrees as members.
+
+`errors` checks each snapshot in turn and keeps the errors in that repo's own files. A repo's check also reports its dependencies' errors, under paths outside the repo, and each dependency's own check already covers those. `owners` reads each owner's path globs from the bullets under a Markdown file's `Files` heading, one ``- Owner: `glob` `glob` `` line each, in order. `by_owner` groups each repo's compiler messages by owner, then by file, and the first owner whose globs match a path owns it. It prefixes each path with its repo's name, so a path into a sibling repo, such as `../rustymath/src/x.rs` from basedpl's check, groups as `rustymath/src/x.rs`. `run_tests` with `--no-fail-fast` runs every test binary, each within its own `timeout`, so a slow or crashing binary doesn't stop the others. `code_count` counts the Rust code lines in a sequence of lines, leaving out blanks and comments. `code_lines` applies it to each Rust file under a directory. `exhash`'s `Journal.stat(count=code_count)` counts the code lines that edits add and remove. `record` writes a run's messages, each repo's test `Results` with each failure's output, and any sizes, as a Markdown record, and returns each owner's message count.
+
+```python
+from fastws.cargo import run_tests
+from fastws.integrate import by_owner, code_lines, errors, integrate, owners, record
+
+root = '~/git/integration'
+commits = integrate({'basedpl': '~/aai-ws/basedpl', 'rustymath': '~/aai-ws/rustymath'}, root)
+errs = errors(root, ['rustymath', 'basedpl'])
+groups = by_owner(errs, owners('~/aai-ws/basedpl/meta/owners.md'))
+tests = {r: run_tests(f'{root}/{r}', '--no-fail-fast') for r in errs if not errs[r]}
+sizes = {r: code_lines(f'{root}/{r}/src') for r in errs}
+counts = record(f'{root}/run8.md', 'Run 8', commits, groups, tests=tests, sizes=sizes)
+```
 ### `ws-sync`
 
 Syncing proceeds in this order:
@@ -293,6 +322,8 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
+For an isolated build workspace, `fastws.core.cargo_setup(root)` regenerates these Cargo patches and the same compiler wrapper without pulling, syncing Python or building.
+
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 
 Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/README.md

@@ -177,9 +177,10 @@ Build a mixed Rust/Python project's extension and opted-in binaries, keep them w
 ```bash
 cargo develop
 cargo develop --path /path/to/project
+cargo develop --profile quick
 ```
 
-Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. It always builds with Cargo's `test` profile, matching bare `cargo test`. The project must produce a `cdylib` and set `python-source`. To use release settings without command-line flags, add `[profile.test]` with `inherits = "release"` to `Cargo.toml`. Keep incremental compilation enabled for the local crate in that profile.
+Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. `[tool.fastws] dev-profile = "quick"` selects a project's default for both `cargo develop` and `fastws.cargo.run_tests`. Without that setting they use `test`. An explicit `--profile NAME` overrides the project default. The project must produce a `cdylib` and set `python-source`. Define the selected profile in `Cargo.toml` and keep incremental compilation enabled for the local crate.
 
 Make the initial editable installation through `ws-sync` or maturin. Repeat that installation when package metadata or entry points change. Running Python processes keep the extension they loaded.
 
@@ -225,11 +226,11 @@ cargo wait --path /path/to/workspace
 
 Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
 
-### Rust tests from Python
+### Rust builds and tests from Python
 
-`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Each binary gets its own `timeout`, 30 seconds by default, excluding compilation. A binary still running when its `timeout` ends is killed and counts as a failed binary, with its suite's `timed_out` set. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
 
-Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+The default profile is `[tool.fastws].dev-profile`, or `test` when unset. Explicit `--profile NAME`, `--profile=NAME`, `--release` and `-r` override it. Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
 
 ```python
 from fastws.cargo import run_tests
@@ -240,6 +241,34 @@ focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'
 
 Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
+`fastws.cargo.check` runs any Cargo subcommand, `check --all-targets` by default, and returns the same `Results` without test suites. A failing run is a result, not an exception. Each `Diagnostic` in its `diagnostics` holds the message's path, line, column, level and text, and its error code or lint as `code`, such as `E0308` or `clippy::too_many_lines`. The summary lists errors first. `env` sets environment variables for the run, as for `run_tests`. A subcommand that rejects `--message-format`, such as `cargo llvm-lines`, runs with `diagnostics=False`: its result holds the exit status and output, with no diagnostics.
+
+```python
+from fastws.cargo import check
+
+lints = check('.', 'clippy', '--all-targets', '--', '-W', 'clippy::too_many_lines', env={'CLIPPY_CONF_DIR': 'review'})
+errors = [d for d in lints.diagnostics if d.level == 'error']
+sizes = check('.', 'llvm-lines', '--lib', diagnostics=False).text
+```
+
+### Integration snapshots
+
+`fastws.integrate` builds several repos together while people or agents keep editing them. `integrate` snapshots each repo's working tree as a commit (untracked files included, ignored files not), checks each commit out as a detached worktree under one root, and writes the root's Cargo config. `HEAD` and the index of each live repo don't change. The config is the workspace's, with the patches for crates under the root pointing at the root's worktrees, so a build there reads the snapshots and every other local crate from the workspace. Keep the root outside the workspace, where `ws-sync` and uv don't treat its worktrees as members.
+
+`errors` checks each snapshot in turn and keeps the errors in that repo's own files. A repo's check also reports its dependencies' errors, under paths outside the repo, and each dependency's own check already covers those. `owners` reads each owner's path globs from the bullets under a Markdown file's `Files` heading, one ``- Owner: `glob` `glob` `` line each, in order. `by_owner` groups each repo's compiler messages by owner, then by file, and the first owner whose globs match a path owns it. It prefixes each path with its repo's name, so a path into a sibling repo, such as `../rustymath/src/x.rs` from basedpl's check, groups as `rustymath/src/x.rs`. `run_tests` with `--no-fail-fast` runs every test binary, each within its own `timeout`, so a slow or crashing binary doesn't stop the others. `code_count` counts the Rust code lines in a sequence of lines, leaving out blanks and comments. `code_lines` applies it to each Rust file under a directory. `exhash`'s `Journal.stat(count=code_count)` counts the code lines that edits add and remove. `record` writes a run's messages, each repo's test `Results` with each failure's output, and any sizes, as a Markdown record, and returns each owner's message count.
+
+```python
+from fastws.cargo import run_tests
+from fastws.integrate import by_owner, code_lines, errors, integrate, owners, record
+
+root = '~/git/integration'
+commits = integrate({'basedpl': '~/aai-ws/basedpl', 'rustymath': '~/aai-ws/rustymath'}, root)
+errs = errors(root, ['rustymath', 'basedpl'])
+groups = by_owner(errs, owners('~/aai-ws/basedpl/meta/owners.md'))
+tests = {r: run_tests(f'{root}/{r}', '--no-fail-fast') for r in errs if not errs[r]}
+sizes = {r: code_lines(f'{root}/{r}/src') for r in errs}
+counts = record(f'{root}/run8.md', 'Run 8', commits, groups, tests=tests, sizes=sizes)
+```
 ### `ws-sync`
 
 Syncing proceeds in this order:
@@ -265,6 +294,8 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
+For an isolated build workspace, `fastws.core.cargo_setup(root)` regenerates these Cargo patches and the same compiler wrapper without pulling, syncing Python or building.
+
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 
 Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws/__init__.py

@@ -1,4 +1,5 @@
-__version__ = "0.0.33"
+__version__ = "0.0.34"
+
 
 
 

fastws_cli-0.0.34/fastws/_toml.py

@@ -0,0 +1,4 @@
+__all__ = ['tomllib']
+
+try: import tomllib
+except ModuleNotFoundError: import tomli as tomllib

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws/cargo.py

@@ -1,14 +1,12 @@
 r"Build, test and install Cargo artifacts for mixed Rust/Python projects."
-import json, os, re, shutil, subprocess, sys, sysconfig, time
+import json, os, re, shutil, signal, subprocess, sys, sysconfig, time
 from collections import Counter
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from pathlib import Path
 from typing import NamedTuple
 from . import store
 from .procs import bounded
-
-try: import tomllib
-except ModuleNotFoundError: import tomli as tomllib
+from ._toml import tomllib
 
 
 class ConfigError(ValueError):
@@ -20,9 +18,13 @@ def settings(root):
     return tomllib.loads((Path(root).expanduser()/'pyproject.toml').read_text()).get('tool', {})
 
 
+def _dev_profile(root):
+    return settings(root).get('fastws', {}).get('dev-profile', 'test') if (root/'pyproject.toml').exists() else 'test'
+
+
 def cargo(root, *args):
-    "Run Cargo in the project and return its standard output."
-    return subprocess.run(['cargo', *args], cwd=Path(root).expanduser(), stdout=subprocess.PIPE, text=True, check=True).stdout
+    "Run Cargo in the project and return its standard output. A failure raises `CalledProcessError`, holding Cargo's standard error."
+    return subprocess.run(['cargo', *args], cwd=Path(root).expanduser(), capture_output=True, text=True, check=True).stdout
 
 
 def members(root):
@@ -30,32 +32,39 @@ def members(root):
     return {Path(p['manifest_path']) for p in json.loads(cargo(root, 'metadata', '--no-deps', '--format-version', '1'))['packages']}
 
 
-def check(root):
-    "Check every target of the workspace with Cargo. Cargo's messages are in the result's `stderr`, which `diagnostics` reads."
-    return subprocess.run(['cargo', 'check', '--all-targets', '--message-format=short'], cwd=Path(root).expanduser(), stderr=subprocess.PIPE, text=True)
-
-
 _TEST = re.compile(r'^test (.+) \.\.\. (ok|FAILED|ignored)')
 _FAILURE = re.compile(r'^---- (.+) std(?:out|err) ----$')
-_DIAGNOSTIC = re.compile(r'^(\S.*?):(\d+):(\d+): (warning|error)(?:\[\w+\])?: (.*)$')
 
 
 class Diagnostic(NamedTuple):
-    "A compiler message, read from Cargo's short format."
+    "A compiler message at its primary span, read from Cargo's JSON messages."
     path:str; line:int; col:int; level:str; message:str
-    def __str__(self): return f'{self.level}: {self.path}:{self.line}: {self.message}'
+    code:str|None=None # The error code or lint, such as `E0308` or `clippy::too_many_lines`
+    def __str__(self): return f'{self.level}{f"[{self.code}]" if self.code else ""}: {self.path}:{self.line}: {self.message}'
 
 
-def diagnostics(text):
-    "Each `Diagnostic` in `text`, which holds Cargo's short-format messages, such as `check`'s `stderr`."
-    return [Diagnostic(m[1], int(m[2]), int(m[3]), m[4], m[5]) for l in text.splitlines() if (m := _DIAGNOSTIC.match(l))]
+def _run(root, args, env=None, diagnostics=True):
+    "Run the Cargo subcommand and options `args` in `root`, with the environment variables `env` set: its exit status, its output, each `Diagnostic` once, and Cargo's other JSON messages, such as artifacts. With `diagnostics`, Cargo writes JSON messages and the output renders each compiler message short; without, the output is as Cargo printed it."
+    flags = ('--message-format=json-diagnostic-short',) if diagnostics else ()
+    p = bounded(['cargo', args[0], *flags, *args[1:]], seconds=None, merge=True, cwd=root, env=os.environ | (env or {}))
+    if not diagnostics: return p.returncode, p.stdout, [], []
+    text, found, rest = [], [], []
+    for line in p.stdout.splitlines(keepends=True):
+        if not line.startswith('{'): text.append(line); continue
+        m = json.loads(line)
+        if m['reason'] != 'compiler-message': rest.append(m); continue
+        msg = m['message']
+        text.append(msg['rendered'])
+        if s := next((s for s in msg['spans'] if s['is_primary']), None):
+            found.append(Diagnostic(s['file_name'], s['line_start'], s['column_start'], msg['level'], msg['message'], (msg['code'] or {}).get('code')))
+    return p.returncode, ''.join(text), list(dict.fromkeys(found)), rest
 
 
 def _lines(text, n=20, tail=False):
-    "Up to `n` lines of `text`, indented, from its end if `tail`, with a count of the lines left out."
+    "Up to `n` lines of `text` (None: all), indented, from its end if `tail`, with a count of the lines left out."
     lines = text.splitlines()
+    if n is None or len(lines) <= n: return [f'    {l}' for l in lines]
     shown = [f'    {l}' for l in (lines[-n:] if tail else lines[:n])]
-    if len(lines) <= n: return shown
     more = f'    … {len(lines)-n} more lines'
     return [more, *shown] if tail else [*shown, more]
 
@@ -69,52 +78,69 @@ class Suite:
     failures:dict # Each failing test's captured output
     summary:str # libtest's `test result:` line, empty for a binary that crashed or has its own harness
     failed:bool # Did the binary exit unsuccessfully?
+    timed_out:bool=False # Was the binary killed for running out of time?
 
     @classmethod
-    def read(cls, name, lines, returncode):
-        "Read one binary's lines and exit status."
-        failures, current = {}, None
+    def read(cls, name, lines, returncode, timed_out=False):
+        "Read one binary's lines and exit status, and whether it was killed for running out of time."
+        failures, current, in_failures = {}, None, False
         for line in lines:
-            if m := _FAILURE.match(line): current = failures.setdefault(m[1], [])
-            elif line == 'failures:': current = None
+            if line in ('successes:', 'failures:'): current, in_failures = None, line == 'failures:'
+            elif in_failures and (m := _FAILURE.match(line)): current = failures.setdefault(m[1], [])
             elif current is not None: current.append(line)
         return cls(name, '\n'.join(lines), {m[1]: m[2] for l in lines if (m := _TEST.match(l))},
                    {k: '\n'.join(v).strip('\n') for k,v in failures.items()},
-                   next((l for l in lines if l.startswith('test result: ')), ''), bool(returncode))
+                   next((l for l in lines if l.startswith('test result: ')), ''), bool(returncode), timed_out)
 
     def __str__(self):
         n = Counter(self.tests.values())
         status = ', '.join(f'{n[o]} {w}' for o,w in [('ok', 'passed'), ('FAILED', 'failed'), ('ignored', 'ignored')] if n[o])
-        if self.failed and not n['FAILED']: status = f'{status}, then failed' if status else 'failed'
+        if self.failed and not n['FAILED']:
+            end = 'ran out of time' if self.timed_out else 'failed'
+            status = f'{status}, then {end}' if status else end
         time = f' in {m[1]}' if (m := re.search(r'finished in (\S+)$', self.summary)) else ''
         return f'{self.name}: {status or "ok"}{time}'
 
 
 @dataclass
 class Results:
-    "A build's compiler messages and each native test binary's results. It displays as a summary: the compiler messages, each binary that ran tests, and each failure's output."
+    "A Cargo run's outcome, its compiler messages and, for tests, each native test binary's results. It displays as a summary: the outcome, the compiler messages with errors first, each binary that ran tests, and each failure's output."
+    command:str # The Cargo subcommand, such as `test` or `clippy`
     returncode:int
     text:str # The build's messages and each binary's merged standard output and standard error
     diagnostics:list # Each `Diagnostic` from the build, before the first binary ran
-    suites:list # Each binary's `Suite`, in execution order
+    suites:list=field(default_factory=list) # Each test binary's `Suite`, in execution order
     seconds:float=0 # Wall time spent executing native test binaries, excluding compilation
 
     @property
     def ok(self): return self.returncode == 0
 
-    def __repr__(self):
-        res = ['cargo test passed' if self.ok else f'cargo test failed with exit status {self.returncode}']
+    def summary(self, lines:int|None=20):
+        "The outcome, compiler messages, each binary's result and each failure's output, with up to `lines` lines of each output (None: all)"
+        res = [f'cargo {self.command} passed' if self.ok else f'cargo {self.command} failed with exit status {self.returncode}']
         diags = sorted(self.diagnostics, key=lambda d: d.level != 'error')
-        res += [str(d) for d in diags[:20]]
-        if len(diags) > 20: res.append(f'{len(diags)-20} more compiler messages in `diagnostics`')
+        res += [str(d) for d in diags[:lines]]
+        if lines is not None and len(diags) > lines: res.append(f'{len(diags)-lines} more compiler messages in `diagnostics`')
         ran = [s for s in self.suites if s.tests or not s.summary]
         for s in ran:
             res.append(str(s))
-            for t,out in s.failures.items(): res += [f'  FAILED {t}', *_lines(out)]
-            if s.failed and not s.failures: res += _lines(s.output, tail=True)
+            for t,out in s.failures.items(): res += [f'  FAILED {t}', *_lines(out, lines)]
+            if s.failed and not s.failures: res += _lines(s.output, lines, tail=True)
         if idle := len(self.suites) - len(ran): res.append(f'{idle} test binaries ran no tests')
-        if not (self.ok or self.suites or diags): res += _lines(self.text, tail=True)
+        if not (self.ok or self.suites or diags): res += _lines(self.text, lines, tail=True)
         return '\n'.join(res)
+    __repr__ = summary
+
+
+def check(
+    root, # A Cargo package or workspace (expands `~`)
+    *args, # The Cargo subcommand and its options, such as `'clippy', '--all-targets', '--', '-W', 'clippy::pedantic'`; none: `check --all-targets`
+    env:dict=None, # Environment variables to set for the run, such as `{'CLIPPY_CONF_DIR': '/path/to/review'}`
+    diagnostics:bool=True, # Ask Cargo for JSON compiler messages? False runs a subcommand that rejects `--message-format`, such as `llvm-lines`, with no diagnostics
+)->Results:
+    "Run a Cargo subcommand and return its `Results`, with each compiler message as a `Diagnostic`. A failing run is a result, not an exception."
+    args = args or ('check', '--all-targets')
+    return Results(args[0], *_run(Path(root).expanduser(), args, env, diagnostics)[:3])
 
 
 def run_tests(
@@ -122,33 +148,28 @@ def run_tests(
     *args, # Cargo build/selection options, such as `'--test', 'core'` or `'--profile', 'checked'`; `--no-fail-fast` runs every binary
     test_args=(), # Arguments passed directly to each test binary, such as `('name', '--nocapture')`
     env:dict=None, # Environment variables to set for the run, such as `{'BASEDPL_COSTS': '1'}`
-    timeout:float=30 # Seconds shared by all native test binaries after building, raising `TimeoutError` on expiry; None: no limit
+    timeout:float=30 # Seconds that each native test binary may run after building; None: no limit
 )->Results:
-    "Build native tests once, with no time limit, then run Cargo's reported executables directly from their package directories. Source edits after building cannot trigger another build; results describe the compiled build, not the latest working tree. `timeout` covers the binaries together, excluding compilation. Doctests are not run. This takes no timing lock: rustc's wrapper and tests that time own their holds."
-    root, env = Path(root).expanduser().resolve(), os.environ | {'CARGO_TERM_QUIET': 'false', 'CARGO_TERM_COLOR': 'never'} | (env or {})
-    built = bounded(['cargo', 'test', '--no-run', '--message-format=json-diagnostic-short', *args], seconds=None, merge=True, cwd=root, env=env)
-    text, artifacts = [], []
-    for line in built.stdout.splitlines(keepends=True):
-        if not line.startswith('{'): text.append(line); continue
-        m = json.loads(line)
-        if m['reason'] == 'compiler-message': text.append(m['message']['rendered'])
-        elif m['reason'] == 'compiler-artifact' and m['profile']['test'] and m.get('executable'): artifacts.append(m)
-    text = ''.join(text)
-    found = list(dict.fromkeys(diagnostics(text)))
-    if built.returncode: return Results(built.returncode, text, found, [])
+    "Build native tests once, with no time limit, then run Cargo's reported executables directly from their package directories. Source edits after building cannot trigger another build; results describe the compiled build, not the latest working tree. `timeout` bounds each binary on its own, excluding compilation. A binary that runs out of time is killed and reported as a failed binary whose suite is `timed_out`. An omitted profile uses tool.fastws.dev-profile, or test. Doctests are not run. This takes no timing lock: rustc's wrapper and tests that time own their holds."
+    root, env = Path(root).expanduser().resolve(), {'CARGO_TERM_QUIET': 'false', 'CARGO_TERM_COLOR': 'never'} | (env or {})
+    has_profile = any(a in ('-r', '--release', '--profile') or isinstance(a, str) and a.startswith('--profile=') for a in args)
+    if not has_profile: args += ('--profile', _dev_profile(root))
+    code, text, found, rest = _run(root, ('test', '--no-run', *args), env)
+    if code: return Results('test', code, text, found)
+    artifacts = [m for m in rest if m['reason'] == 'compiler-artifact' and m['profile']['test'] and m.get('executable')]
     suites, code, start = [], 0, time.monotonic()
     for m in artifacts:
         package = Path(m['manifest_path']).parent
         name = Path(m['target']['src_path']).relative_to(package).as_posix()
         if 'test' not in m['target']['kind']: name = 'unittests ' + name
-        left = None if timeout is None else max(0, timeout - (time.monotonic() - start))
-        p = bounded([m['executable'], *test_args], left, merge=True, cwd=package, env=env | {'CARGO_MANIFEST_DIR': str(package)})
-        suites.append(Suite.read(name, p.stdout.splitlines(), p.returncode))
+        try: p, late = bounded([m['executable'], *test_args], timeout, merge=True, cwd=package, env=os.environ | env | {'CARGO_MANIFEST_DIR': str(package)}), False
+        except TimeoutError as e: p, late = subprocess.CompletedProcess(m['executable'], -signal.SIGKILL, f'{e}\n'), True
+        suites.append(Suite.read(name, p.stdout.splitlines(), p.returncode, late))
         text += f'     Running {name} ({m["executable"]})\n' + p.stdout
         if p.returncode:
             code = p.returncode
             if '--no-fail-fast' not in args: break
-    return Results(code, text, found, suites, time.monotonic() - start)
+    return Results('test', code, text, found, suites, time.monotonic() - start)
 
 
 def sources(root):
@@ -225,15 +246,15 @@ def replace(source, destination):
     return destination
 
 
-def develop(root):
-    "Serialize this project's cooperating develop runs through build, artifact capture, installation, generator and wheel data. Raw Cargo commands do not take this lock."
+def develop(root, *, profile=None):
+    "Serialize this project's cooperating develop runs through build, artifact capture, installation, generator and wheel data. An omitted profile uses tool.fastws.dev-profile, or test. Raw Cargo commands do not take this lock."
     root = Path(root).expanduser()
     with store.building(root) as started:
         tool = settings(root)
         cfg = tool['maturin']
         if 'python-source' not in cfg: raise ConfigError(r'cargo develop requires tool.maturin.python-source')
         binaries = tool.get('fastws', {}).get('native-binaries', False)
-        artifacts = build(root, binaries=binaries, profile='test')
+        artifacts = build(root, binaries=binaries, profile=profile or _dev_profile(root))
         artifact, = [m for m in artifacts if 'cdylib' in m['target']['crate_types']]
         lib, = [Path(f) for f in artifact['filenames'] if Path(f).suffix in ('.dylib', '.so', '.dll')]
         module = cfg.get('module-name', artifact['target']['name'])

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws/core.py

@@ -1,6 +1,6 @@
 "Fast workspace tools for multi-repo management."
 
-__all__ = ["ws_setup", "ws_clone", "ws_pull", "ws_status", "ws_branches", "ws_build", "ws_sync", "ws_add", "ws_remove", "cargo_develop", "cargo_stage", "cargo_wait"]
+__all__ = ["ws_setup", "ws_clone", "ws_pull", "ws_status", "ws_branches", "ws_build", "ws_sync", "ws_add", "ws_remove", "cargo_develop", "cargo_stage", "cargo_wait", "cargo_setup"]
 
 import ast, fnmatch, glob, hashlib, json, os, re, shlex, shutil, subprocess, sys, time
 from pathlib import Path
@@ -14,8 +14,7 @@ from ghapi.core import dep_key, dep_closure
 from packaging.requirements import Requirement
 from packaging.utils import canonicalize_name
 
-try: import tomllib
-except ModuleNotFoundError: import tomli as tomllib
+from ._toml import tomllib
 
 def _repo_lines(repos_file) -> list[str]:
     p = Path(repos_file)
@@ -571,10 +570,11 @@ def _cli_errors():
 def cargo_develop(
     command: str = 'develop', # Subcommand name supplied by Cargo
     path: str = '.', # Rust/Python project directory
+    profile: str = None, # Cargo profile; default: tool.fastws.dev-profile, or test
 ):
     r"Build and install a mixed Rust/Python project's extension and opted-in binaries."
     from .cargo import develop
-    with _cli_errors(): develop(Path(path).resolve())
+    with _cli_errors(): develop(Path(path).resolve(), profile=profile)
 
 @call_parse(pos=['command'])
 def cargo_stage(
@@ -603,12 +603,12 @@ def cargo_wait(
     patience: int = 1800, # Seconds to wait before giving up
 ):
     r"Wait until a Cargo workspace compiles, checking again after each edit to its local packages' sources."
-    from .cargo import check, diagnostics, sources
+    from .cargo import check, sources
     root, start = Path(path).resolve(), time.monotonic()
     with _cli_errors(): srcs = sources(root)
     while (res := check(root)).returncode:
-        if time.monotonic() - start > patience: raise SystemExit(f'{res.stderr}gave up after {patience} s')
-        print(next((str(d) for d in diagnostics(res.stderr) if d.level == 'error'), res.stderr.strip() or 'cargo check failed'), file=sys.stderr)
+        if time.monotonic() - start > patience: raise SystemExit(f'{res.text}gave up after {patience} s')
+        print(next((str(d) for d in res.diagnostics if d.level == 'error'), res.text.strip() or 'cargo check failed'), file=sys.stderr)
         before = _newest(srcs)
         while (now := _newest(srcs)) == before and time.monotonic() - start < patience: time.sleep(1)
         while True:
@@ -721,10 +721,23 @@ def _strip_patch_tables(content: str) -> str:
         content = content[:m.start()] + content[end:]
     return content
 
+def _patch_text(tables: dict) -> str:
+    "TOML `[patch.*]` tables for `tables`, which maps each source URL to its crates' source specs"
+    return "\n".join(
+        (f"[patch.{url}]" if url == "crates-io" else f'[patch."{url}"]') + "\n"
+        + "".join(f"{name} = {_fmt_source(spec)}\n" for name, spec in sorted(entries.items()))
+        for url, entries in tables.items() if entries)
+
 def _entry_changes(current, new):
     "Added and removed keys, in the order supplied."
     return [k for k in new if k not in current], [k for k in current if k not in new]
 
+def cargo_setup(root):
+    "Regenerate local Cargo patches and the compile-lock wrapper without pulling or building."
+    root = Path(root).expanduser().resolve()
+    _sync_cargo_wrapper(root)
+    return _sync_cargo_patches(root)
+
 def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     """Regenerate workspace Cargo patches, default to quiet output, and return (added, removed).
 
@@ -756,10 +769,7 @@ def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     for url, entries in desired.items():
         cur = tables.setdefault(url, {})
         for name, path in entries.items(): cur.setdefault(name, {"path": path})
-    body = "\n".join(
-        (f"[patch.{url}]" if url == "crates-io" else f'[patch."{url}"]') + "\n"
-        + "".join(f"{name} = {_fmt_source(spec)}\n" for name, spec in sorted(entries.items()))
-        for url, entries in tables.items() if entries)
+    body = _patch_text(tables)
     new = (_strip_patch_tables(content).rstrip() + "\n\n" + body).lstrip() + "\n" if body else _strip_patch_tables(content)
     if new == original: return [], []
     tomllib.loads(new)  # never write an unparseable config
@@ -991,8 +1001,7 @@ async def ws_sync(
         projects = _build_projects(root, repos_file)
         _sync_build_isolation(pyproject_path, projects)
     
-        _sync_cargo_wrapper(root)
-        added_p, removed_p = _sync_cargo_patches(root)
+        added_p, removed_p = cargo_setup(root)
         if added_p: print(f"Cargo patches added: {', '.join(added_p)}")
         if removed_p: print(f"Cargo patches removed: {', '.join(removed_p)}")
     

fastws_cli-0.0.34/fastws/integrate.py

@@ -0,0 +1,136 @@
+r"""Build several repos' working trees together as one snapshot, apart from the live trees, and group its compiler messages by owner.
+
+Each run snapshots every repo's working tree as a commit, checks the commits out as worktrees under one root, and gives that root the workspace's Cargo config with the root's own crates patched in. Builds there never see a file halfway through an edit, and never touch the files that people and agents are editing."""
+import fnmatch, os, posixpath, re, subprocess, tempfile
+from pathlib import Path
+from ._toml import tomllib
+from .core import _local_crates, _patch_text, _strip_patch_tables, _sync_cargo_patches
+from .cargo import check
+
+__all__ = ["snapshot", "integrate", "by_owner", "owners", "errors", "code_count", "code_lines", "record"]
+
+
+def _git(repo, *args, env=None):
+    return subprocess.run(["git", *args], cwd=repo, env=env, stdout=subprocess.PIPE, text=True, check=True).stdout.strip()
+
+
+def snapshot(repo):
+    "A commit of `repo`'s working tree on top of `HEAD`, with untracked files and without ignored ones. `HEAD` and the index don't change."
+    repo = Path(repo).expanduser()
+    with tempfile.TemporaryDirectory() as d:
+        env = {**os.environ, "GIT_INDEX_FILE": str(Path(d)/"index")}
+        _git(repo, "read-tree", "HEAD", env=env)
+        _git(repo, "add", "-A", env=env)
+        tree = _git(repo, "write-tree", env=env)
+    return _git(repo, "commit-tree", tree, "-p", "HEAD", "-m", "integration snapshot")
+
+
+def _checkout(repo, dest, commit):
+    "Check `commit` out at `dest`, a detached worktree of `repo` that this adds if it's missing, removing the last run's untracked files."
+    if not dest.exists(): return _git(repo, "worktree", "add", "--detach", str(dest), commit)
+    _git(dest, "checkout", "--detach", "-f", commit)
+    _git(dest, "clean", "-fd")
+
+
+def _config(root, workspace):
+    "Write `root`'s Cargo config: `workspace`'s, with the patches for crates under `root` pointing there."
+    held = {pkg["name"] for pkg, _ in _local_crates(root)}
+    content = (workspace/".cargo"/"config.toml").read_text()
+    kept = {url: {n: s for n, s in entries.items() if n not in held} for url, entries in tomllib.loads(content).get("patch", {}).items()}
+    config = root/".cargo"/"config.toml"
+    config.parent.mkdir(exist_ok=True)
+    config.write_text(_strip_patch_tables(content).rstrip() + "\n\n" + _patch_text(kept) + "\n")
+    _sync_cargo_patches(root)
+
+
+def integrate(
+    repos:dict, # Each repo's name and its live checkout; the snapshot goes in `root/<name>`
+    root, # Directory holding the snapshot worktrees and their Cargo config
+    workspace="~/aai-ws", # Workspace whose Cargo config the root copies
+)->dict:
+    "Snapshot each of `repos`, check the snapshots out under `root`, and write `root`'s Cargo config. Returns each repo's snapshot commit."
+    root = Path(root).expanduser().resolve()
+    commits = {}
+    for name, repo in repos.items():
+        repo = Path(repo).expanduser()
+        commits[name] = snapshot(repo)
+        _checkout(repo, root/name, commits[name])
+    _config(root, Path(workspace).expanduser().resolve())
+    return commits
+
+
+def by_owner(
+    diagnostics:dict, # Each repo's name and its `fastws.cargo.Diagnostic`s, with paths relative to the repo, such as one check's errors
+    owners:dict, # Each owner's path globs, matched against `<repo>/<path>`
+)->dict:
+    "Group `diagnostics` by owner, then by file, with each path prefixed by its repo's name. A path that no owner's globs match groups under `None`."
+    groups = {}
+    for repo, ds in diagnostics.items():
+        for d in ds:
+            d = d._replace(path=posixpath.normpath(f"{repo}/{d.path}"))
+            owner = next((o for o, globs in owners.items() if any(fnmatch.fnmatch(d.path, g) for g in globs)), None)
+            groups.setdefault(owner, {}).setdefault(d.path, []).append(d)
+    return groups
+
+
+def owners(
+    path, # Markdown file with a `Files` heading (expands `~`)
+)->dict:
+    "Each owner's path globs from the bullets under `path`'s `Files` heading, one `- Owner: `glob` `glob`` line each, in order, as `by_owner` takes them. The first owner whose globs match a path owns it."
+    files = re.split(r"^#+ Files\s*$", Path(path).expanduser().read_text(), maxsplit=1, flags=re.M)[1]
+    files = re.split(r"^#", files, maxsplit=1, flags=re.M)[0]
+    return {m[1]: re.findall(r"`([^`]+)`", m[2]) for m in re.finditer(r"^- ([^:]+):(.*)$", files, re.M)}
+
+
+def errors(
+    root, # Directory holding the snapshot worktrees (expands `~`)
+    repos, # Names of the worktrees under `root` to check, in order
+)->dict:
+    "Each repo's compiler errors in its own files, from checking its worktree under `root`, as `by_owner` takes them. A check repeats a dependency's errors under a path outside the repo; the dependency's own check reports those."
+    root = Path(root).expanduser()
+    return {r: [d for d in check(root/r).diagnostics if d.level == "error" and not d.path.startswith(("/", ".."))] for r in repos}
+
+
+def code_count(
+    lines, # Rust source lines in order, such as one file's or one side of a diff hunk
+)->int:
+    "How many of `lines` are code: neither blank nor comments (`//` lines and `/* */` blocks)."
+    n, block = 0, False
+    for line in lines:
+        s = line.strip()
+        if block: block = "*/" not in s
+        elif s.startswith("/*"): block = "*/" not in s
+        elif s and not s.startswith("//"): n += 1
+    return n
+
+
+def code_lines(
+    path, # A directory, such as a crate's `src/` (expands `~`)
+)->int:
+    "The code lines of the Rust files under `path`, as `code_count` counts them."
+    return sum(code_count(f.read_text().splitlines()) for f in Path(path).expanduser().rglob("*.rs"))
+
+
+def record(
+    path, # Markdown file to write
+    title:str, # The record's heading, such as "Run 8"
+    commits:dict, # Each repo's snapshot commit, as `integrate` returns them
+    groups:dict, # Messages by owner and file, as `by_owner` returns them
+    edits=(), # Edits made in the snapshot only, one line each
+    tests:dict=None, # Each repo's name and its test `Results`, as `fastws.cargo.run_tests` returns them
+    sizes:dict=None, # Code lines by label, as `code_lines` counts them, such as each repo's `src/` beside its baselines
+    sections:dict=None, # Further sections by heading, each text written as a code block, such as each repo's pytest output
+)->dict:
+    "Write a run's record: the snapshots, each owner's messages by file as `line: message`, any snapshot-only edits, each test binary's outcome with its failures' output, code-line sizes, and any further `sections`. Returns each owner's message count."
+    counts = {o: sum(map(len, files.values())) for o, files in groups.items()}
+    lines = [f"# {title}", "", "Snapshots: " + ", ".join(f"{name} `{c[:9]}`" for name, c in commits.items()), ""]
+    for owner, files in groups.items():
+        lines += [f"## {owner or 'no owner'}: {counts[owner]}", ""]
+        for file, ds in files.items(): lines += [f"- `{file}`", *[f"  - {d.line}: {d.message}" for d in ds]]
+        lines.append("")
+    if edits: lines += ["## Snapshot-only edits", "", *[f"- {e}" for e in edits], ""]
+    if tests: lines += ["## Tests", "", *[f"### {name}\n\n```\n{r.summary(None)}\n```\n" for name, r in tests.items()]]
+    if sizes: lines += ["## Size (code lines, no comments or blanks)", "", *[f"- {k}: {v:,}" for k, v in sizes.items()], ""]
+    for heading, text in (sections or {}).items(): lines += [f"## {heading}", "", f"```\n{text}\n```", ""]
+    Path(path).expanduser().write_text("\n".join(lines))
+    return counts

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws/store.py

@@ -1,12 +1,13 @@
 r"""Builds that `cargo develop` installs, each kept once, read-only, and named by the hash of its inputs and files. A project's installed build is a pointer to one of them, which installing replaces in one step, so a process or a timing that resolves a build never sees it change.
 
 A build is a directory under `STORE/<project>/`, named by its id, that holds `build.json` and a hard link to a blob for each file: `python/…` for the Python package with its extension, `bin/…` for binaries and `data/…` for wheel data. Each blob, under `STORE/blobs/`, is named by its contents' sha256, so files that don't change between builds take no more space. Installed paths are symbolic links through `STORE/<project>/installed`, a link to the installed build's directory. Windows installs copies."""
-import hashlib, json, os, re, shutil, time, tomllib
+import hashlib, json, os, re, shutil, time
 if os.name != 'nt': import fcntl
 from contextlib import contextmanager
 from datetime import datetime
 from pathlib import Path
 from tempfile import TemporaryDirectory
+from ._toml import tomllib
 
 STORE = Path(os.environ.get('XDG_CACHE_HOME') or '~/.cache').expanduser()/'fastws'/'builds' # Each project's builds, and the blobs that their files link to
 KEEP, KEEP_DAYS = 20, 2 # `collect` keeps each project's installed build, its last `KEEP` builds and its builds of the last `KEEP_DAYS` days
@@ -32,7 +33,10 @@ def access(exclusive=False):
         yield
 
 def _digest(path):
-    with open(path, 'rb') as f: return hashlib.file_digest(f, 'sha256').hexdigest()
+    digest = hashlib.sha256()
+    with open(path, 'rb') as f:
+        while chunk := f.read(1024*1024): digest.update(chunk)
+    return digest.hexdigest()
 
 def _blob(path):
     "The blob that holds `path`'s contents, read-only and executable where `path` is, added to the store where it's new."

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.33
+Version: 0.0.34
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License-Expression: Apache-2.0
@@ -205,9 +205,10 @@ Build a mixed Rust/Python project's extension and opted-in binaries, keep them w
 ```bash
 cargo develop
 cargo develop --path /path/to/project
+cargo develop --profile quick
 ```
 
-Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. It always builds with Cargo's `test` profile, matching bare `cargo test`. The project must produce a `cdylib` and set `python-source`. To use release settings without command-line flags, add `[profile.test]` with `inherits = "release"` to `Cargo.toml`. Keep incremental compilation enabled for the local crate in that profile.
+Run from the project root, or give its directory with `--path`. The command reads `python-source`, `module-name`, `manifest-path` and features from `[tool.maturin]`. `[tool.fastws] dev-profile = "quick"` selects a project's default for both `cargo develop` and `fastws.cargo.run_tests`. Without that setting they use `test`. An explicit `--profile NAME` overrides the project default. The project must produce a `cdylib` and set `python-source`. Define the selected profile in `Cargo.toml` and keep incremental compilation enabled for the local crate.
 
 Make the initial editable installation through `ws-sync` or maturin. Repeat that installation when package metadata or entry points change. Running Python processes keep the extension they loaded.
 
@@ -253,11 +254,11 @@ cargo wait --path /path/to/workspace
 
 Use it when another process's unfinished edits stop the workspace compiling, such as a second agent working in the same tree. Each check runs `cargo check --all-targets`, whose rustc calls hold the timing lock shared, as every build's do. After a failed check, `cargo wait` prints Cargo's first error line. It checks again once a source file of a local package changes and then stays unchanged for 2 seconds. Local packages are the workspace's members and its path dependencies, such as a sibling crate that `ws-sync` patches in. When a check passes, it prints `compiles` and exits. After `--patience` seconds, 1800 by default, it prints the last check's errors and exits with status 1.
 
-### Rust tests from Python
+### Rust builds and tests from Python
 
-`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Its `timeout` defaults to 30 seconds shared across the test binaries, excluding compilation. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
+`fastws.cargo.run_tests` builds native tests once with Cargo, without a time limit, then runs the executables that Cargo reports directly from their package directories. Each binary gets its own `timeout`, 30 seconds by default, excluding compilation. A binary still running when its `timeout` ends is killed and counts as a failed binary, with its suite's `timed_out` set. Source edits between building and running do not trigger another build: the results describe the compiled build, not necessarily the latest working tree. It supports directly runnable native tests, not cross-target runners.
 
-Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
+The default profile is `[tool.fastws].dev-profile`, or `test` when unset. Explicit `--profile NAME`, `--profile=NAME`, `--release` and `-r` override it. Pass Cargo build/selection options positionally, and test filters and harness options in `test_args`. A failed binary stops the run unless `--no-fail-fast` is given. Results contain compiler diagnostics and each binary's outcomes and captured failures.
 
 ```python
 from fastws.cargo import run_tests
@@ -268,6 +269,34 @@ focused = run_tests('.', '--test', 'core', test_args=('test_name', '--nocapture'
 
 Doctests are deliberately not run; keep executable examples in ordinary Rust tests. `run_tests` accepts `env` overrides and `timeout=None` to disable its watchdog. It takes no timing lock around the run: rustc's wrapper owns its shared holds, and tests that time own their exclusive holds.
 
+`fastws.cargo.check` runs any Cargo subcommand, `check --all-targets` by default, and returns the same `Results` without test suites. A failing run is a result, not an exception. Each `Diagnostic` in its `diagnostics` holds the message's path, line, column, level and text, and its error code or lint as `code`, such as `E0308` or `clippy::too_many_lines`. The summary lists errors first. `env` sets environment variables for the run, as for `run_tests`. A subcommand that rejects `--message-format`, such as `cargo llvm-lines`, runs with `diagnostics=False`: its result holds the exit status and output, with no diagnostics.
+
+```python
+from fastws.cargo import check
+
+lints = check('.', 'clippy', '--all-targets', '--', '-W', 'clippy::too_many_lines', env={'CLIPPY_CONF_DIR': 'review'})
+errors = [d for d in lints.diagnostics if d.level == 'error']
+sizes = check('.', 'llvm-lines', '--lib', diagnostics=False).text
+```
+
+### Integration snapshots
+
+`fastws.integrate` builds several repos together while people or agents keep editing them. `integrate` snapshots each repo's working tree as a commit (untracked files included, ignored files not), checks each commit out as a detached worktree under one root, and writes the root's Cargo config. `HEAD` and the index of each live repo don't change. The config is the workspace's, with the patches for crates under the root pointing at the root's worktrees, so a build there reads the snapshots and every other local crate from the workspace. Keep the root outside the workspace, where `ws-sync` and uv don't treat its worktrees as members.
+
+`errors` checks each snapshot in turn and keeps the errors in that repo's own files. A repo's check also reports its dependencies' errors, under paths outside the repo, and each dependency's own check already covers those. `owners` reads each owner's path globs from the bullets under a Markdown file's `Files` heading, one ``- Owner: `glob` `glob` `` line each, in order. `by_owner` groups each repo's compiler messages by owner, then by file, and the first owner whose globs match a path owns it. It prefixes each path with its repo's name, so a path into a sibling repo, such as `../rustymath/src/x.rs` from basedpl's check, groups as `rustymath/src/x.rs`. `run_tests` with `--no-fail-fast` runs every test binary, each within its own `timeout`, so a slow or crashing binary doesn't stop the others. `code_count` counts the Rust code lines in a sequence of lines, leaving out blanks and comments. `code_lines` applies it to each Rust file under a directory. `exhash`'s `Journal.stat(count=code_count)` counts the code lines that edits add and remove. `record` writes a run's messages, each repo's test `Results` with each failure's output, and any sizes, as a Markdown record, and returns each owner's message count.
+
+```python
+from fastws.cargo import run_tests
+from fastws.integrate import by_owner, code_lines, errors, integrate, owners, record
+
+root = '~/git/integration'
+commits = integrate({'basedpl': '~/aai-ws/basedpl', 'rustymath': '~/aai-ws/rustymath'}, root)
+errs = errors(root, ['rustymath', 'basedpl'])
+groups = by_owner(errs, owners('~/aai-ws/basedpl/meta/owners.md'))
+tests = {r: run_tests(f'{root}/{r}', '--no-fail-fast') for r in errs if not errs[r]}
+sizes = {r: code_lines(f'{root}/{r}/src') for r in errs}
+counts = record(f'{root}/run8.md', 'Run 8', commits, groups, tests=tests, sizes=sizes)
+```
 ### `ws-sync`
 
 Syncing proceeds in this order:
@@ -293,6 +322,8 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 Sync writes `.cargo/rustc-lock` and makes it Cargo's `rustc-wrapper`. The script runs each rustc call holding the timing lock shared (see `cargo develop` above). Inside the lock it runs the wrapper that the config named before sync first wrote the script, where another tool set one, and otherwise `sccache` when it's installed, so unchanged compilation units are reused across the workspace's otherwise-independent Cargo target directories. Later syncs keep that inner wrapper. The script costs about 9 ms a rustc call, mostly the Python interpreter's start. If `sccache` isn't installed, sync warns with the install command (`cargo install sccache --locked`) and continues; rerun `ws-sync` after installing to enable it. On Windows, where the lock does nothing, the config keeps another tool's wrapper, or else names `sccache`.
 
+For an isolated build workspace, `fastws.core.cargo_setup(root)` regenerates these Cargo patches and the same compiler wrapper without pulling, syncing Python or building.
+
 Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
 
 Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/fastws_cli.egg-info/SOURCES.txt

@@ -2,9 +2,11 @@ LICENSE
 README.md
 pyproject.toml
 fastws/__init__.py
+fastws/_toml.py
 fastws/build_backend.py
 fastws/cargo.py
 fastws/core.py
+fastws/integrate.py
 fastws/news.py
 fastws/procs.py
 fastws/releases.py
@@ -18,6 +20,7 @@ fastws_cli.egg-info/requires.txt
 fastws_cli.egg-info/top_level.txt
 tests/test_build.py
 tests/test_build_backend.py
+tests/test_integrate.py
 tests/test_news.py
 tests/test_procs.py
 tests/test_releases.py

{fastws_cli-0.0.33 → fastws_cli-0.0.34}/tests/test_build.py

@@ -1,5 +1,5 @@
 import subprocess, pytest, fastws.core as core
-from fastws.cargo import build, run_tests, diagnostics
+from fastws.cargo import build, check, run_tests, Suite
 
 
 def _mk_proj(root, name, extra=''):
@@ -41,7 +41,9 @@ def test_build_diagnostics(tmp_path, capfd, body, diagnostic):
     else: assert [m['target']['name'] for m in build(tmp_path)] == ['app']
     stderr = capfd.readouterr().err
     if diagnostic: assert f'[dep] dep/src/lib.rs:1:' in stderr and f': {diagnostic}:' in stderr
-    else: assert not diagnostics(stderr)
+    else: assert 'lib.rs:' not in stderr
+    checked = check(tmp_path)
+    assert checked.ok == (diagnostic != 'error') and [(d.path, d.level) for d in checked.diagnostics] == ([('dep/src/lib.rs', diagnostic)] if diagnostic else [])
 
 
 def test_run_tests(tmp_path):
@@ -64,12 +66,31 @@ pub fn two() -> i32 { 2 }
     (tmp_path/'.cargo/config.toml').write_text('[term]\nquiet = true\n')
     r = run_tests(tmp_path, '--no-fail-fast')
     assert not r.ok
-    assert [(d.path, d.line, d.level) for d in r.diagnostics] == [('src/lib.rs', 1, 'warning')]
+    assert [(d.path, d.line, d.level, d.code) for d in r.diagnostics] == [('src/lib.rs', 1, 'warning', 'unused_variables')]
     suites = {s.name: s for s in r.suites}
     unit, more = suites['unittests src/lib.rs'], suites['tests/more.rs']
     assert len(suites) == 2
     assert unit.tests == {'t::passes': 'ok', 't::fails': 'FAILED', 't::skipped': 'ignored'} and unit.failed
     assert 'broken' in unit.failures['t::fails'] and list(unit.failures) == ['t::fails']
     assert more.tests == {'other': 'ok'} and not more.failed
-    with pytest.raises(TimeoutError, match='started slow test'):
-        run_tests(tmp_path, '--lib', test_args=('skipped', '--ignored', '--nocapture'), timeout=.5)
+    slow = run_tests(tmp_path, '--lib', test_args=('skipped', '--ignored', '--nocapture'), timeout=.5)
+    s, = slow.suites
+    assert not slow.ok and s.failed and s.timed_out and 'started slow test' in s.output
+    assert str(s) == 'unittests src/lib.rs: ran out of time'
+
+    (tmp_path/'pyproject.toml').write_text(r'tool.fastws.dev-profile = "missing"')
+    r = run_tests(tmp_path, '--test', 'more')
+    assert not r.ok and not r.suites and 'missing' in r.text
+    for args in [('--profile', 'test'), ('--profile=test',), ('--release',), ('-r',)]:
+        assert run_tests(tmp_path, '--test', 'more', *args).ok
+
+
+def test_captured_test_output():
+    for failed in (False, True):
+        lines = ['test passes ... ok', 'successes:', '---- passes stdout ----', 'printed', 'successes:', 'passes']
+        if failed:
+            lines[1:1] = ['test fails ... FAILED']
+            lines += ['failures:', '---- fails stdout ----', 'broken', 'failures:', 'fails']
+        lines.append(f'test result: {"FAILED" if failed else "ok"}. 1 passed; {int(failed)} failed; 0 ignored')
+        s = Suite.read('unit', lines, int(failed))
+        assert s.failures == ({'fails': 'broken'} if failed else {}) and 'printed' in s.output

fastws_cli-0.0.34/tests/test_integrate.py

@@ -0,0 +1,46 @@
+import subprocess
+from fastws._toml import tomllib
+from fastws.integrate import _config, code_lines, owners, snapshot
+
+
+def git(repo, *args): return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True).stdout
+
+
+def test_snapshot_takes_the_working_tree_and_leaves_head_and_index(tmp_path):
+    git(tmp_path, "init", "-q")
+    (tmp_path/".gitignore").write_text("ignored\n")
+    (tmp_path/"kept").write_text("old")
+    git(tmp_path, "add", "-A"); git(tmp_path, "-c", "user.name=t", "-c", "user.email=t@t", "commit", "-qm", "base")
+    head = git(tmp_path, "rev-parse", "HEAD")
+    (tmp_path/"kept").write_text("new"); (tmp_path/"untracked").write_text("u"); (tmp_path/"ignored").write_text("i")
+    commit = snapshot(tmp_path)
+    assert git(tmp_path, "show", f"{commit}:kept") == "new"
+    assert git(tmp_path, "show", f"{commit}:untracked") == "u"
+    assert "ignored" not in git(tmp_path, "ls-tree", "--name-only", commit).split()
+    assert git(tmp_path, "rev-parse", "HEAD") == head and git(tmp_path, "diff", "--cached", "--name-only") == ""
+
+
+def test_config_points_only_the_roots_crates_at_the_root(tmp_path):
+    workspace, root = tmp_path/"ws", tmp_path/"root"
+    (workspace/".cargo").mkdir(parents=True)
+    (workspace/".cargo"/"config.toml").write_text(
+        f'[build]\nrustc-wrapper = "/lock"\n\n[patch.crates-io]\na = {{ path = "{workspace}/a" }}\nb = {{ path = "{workspace}/b" }}\n')
+    (root/"a").mkdir(parents=True)
+    (root/"a"/"Cargo.toml").write_text('[package]\nname = "a"\nversion = "0.1.0"\n')
+    _config(root, workspace)
+    config = tomllib.loads((root/".cargo"/"config.toml").read_text())
+    assert config["build"]["rustc-wrapper"] == "/lock"
+    assert config["patch"]["crates-io"] == {"a": {"path": str(root/"a")}, "b": {"path": f"{workspace}/b"}}
+
+
+def test_code_lines_skips_blanks_and_comments(tmp_path):
+    (tmp_path/"a").mkdir()
+    (tmp_path/"a"/"m.rs").write_text("//! Module doc\n\nfn f() {}\n/* a\n   block */\nlet s = \"// not a comment\";\n  /// doc\n}\n")
+    (tmp_path/"notes.md").write_text("fn f() {}\n")
+    assert code_lines(tmp_path) == 3
+
+
+def test_owners_reads_the_files_section_in_order(tmp_path):
+    (tmp_path/"owners.md").write_text(
+        "# Owners\n\n- Before: `x/*`\n\n## Files\n\nThe first match owns a path.\n- Structure: `a/src/p.rs` `b/src/s.rs`\n- Numeric: `b/*`\n\n## Order\n\n- After: `c/*`\n")
+    assert list(owners(tmp_path/"owners.md").items()) == [("Structure", ["a/src/p.rs", "b/src/s.rs"]), ("Numeric", ["b/*"])]