fastws-cli 0.0.14__tar.gz → 0.0.16__tar.gz

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.14
+Version: 0.0.16
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License: Apache-2.0
@@ -82,17 +82,31 @@ AnswerDotAI/fastcore
 jph00/private ~/private
 ```
 
+Select Python extras by attaching brackets to the repo name, before any checkout location:
+
+```text
+AnswerDotAI/fastws[dev]
+owner/repo[dev,test] ~/somewhere
+```
+
+These select `[project.optional-dependencies]`, not `[dependency-groups]`. Sync uses the package name from `pyproject.toml`, so `AnswerDotAI/fastws[dev]` installs `fastws-cli[dev]` editably. For a checkout containing several discovered Python packages, the selection applies to each. uv handles the extras and their dependencies.
+
+Extras from duplicate entries in `repos.txt` and `repos-local.txt` are combined. For example, a personal `AnswerDotAI/fastws[dev]` entry adds development extras to a shared `AnswerDotAI/fastws` entry. The lists are authoritative for those packages' extras: each sync updates existing workspace dependencies, including removing extras no longer selected. Put personal extra selections in `repos-local.txt`, not the generated `pyproject.toml`. Existing version constraints, markers, sources, and unrelated dependencies are preserved.
+
 ### Shared baseline and personal additions
 
 Keep `repos.txt` tracked in the workspace Git repo. Put personal additions in a gitignored `repos-local.txt`, using the same format. Every command reads both lists. `ws-add` and automatic discovery append only to the local list. Edit the shared list deliberately when changing the team's baseline. Duplicate repo names are matched case-insensitively. Conflicting explicit locations, or two different repos targeting the same directory, cause errors.
 
 `--repos-file myrepos.txt` uses `myrepos-local.txt` alongside it. Either list may be absent. Removing a repo from the baseline does not delete its checkout. The next sync discovers an existing root checkout as a personal addition. External checkouts also remain on disk; add them to the local list to keep managing them.
 
-The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it without replacing personal settings. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
+The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it while preserving personal settings except for the repo extras managed above. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
 
 ```gitignore
 /repos-local.txt
 /pyproject.toml
+/package.json
+/package-lock.json
+/node_modules/
 /pyrightconfig.json
 /uv.lock
 /.cargo/
@@ -194,7 +208,7 @@ By default, `ws-sync` uses the active venv's parent as the workspace root. It cr
 
 Project scanning respects `tool.uv.workspace.members` and `exclude`. If a member directory lacks `pyproject.toml`, such as a fresh empty clone, sync warns and skips the `uv sync` step.
 
-The workspace `exclude` list is maintained automatically. An unlisted top-level directory that is not a valid Python project is excluded until it has a `pyproject.toml`. A `repos.txt` checkout is automatically excluded only when it is a Cargo-only Rust crate. A listed checkout with neither `pyproject.toml` nor `Cargo.toml` is treated as a pending member and triggers the warning above.
+Fastws maintains `tool.uv.workspace.exclude`. It excludes unlisted top-level directories without a valid Python project. It also excludes listed Rust or JavaScript projects without `pyproject.toml`. For a listed checkout with none of `pyproject.toml`, `Cargo.toml`, or `package.json`, sync warns and skips installation.
 
 Existing globs, entries for missing directories, and entries for checkouts that are still not Python projects are retained. Use `exclude = [...]` under `[tool.fastws]` to specify exclusions the scan cannot infer, such as keeping a valid project out of the workspace. Adding members preserves hand-written `[tool.uv.sources]` entries, including path and Git sources.
 
@@ -204,6 +218,20 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved.
 
+Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
+
+Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.
+
+An optional tracked `package.json.shared` is merged into the root `package.json` on every sync. Object-valued fields merge one level deep, shared values win conflicts, and arrays replace rather than concatenate. Unrelated local settings remain; removing a shared key does not delete its local value. Workspace discovery still manages `workspaces`. Use this file for shared policy such as version-pinned npm `allowScripts` approvals.
+
+After `uv sync`, fastws runs `<tool> install` at the workspace root. It then runs `<tool> run build` in each JavaScript package containing both `Cargo.toml` and a `build` script. Cargo handles incremental compilation. Other build steps still run.
+
+For npm, sync hides routine install summaries and build-command banners. Install warnings, audit findings, unreviewed install scripts, and failure diagnostics remain visible. Build scripts retain their own output; configure their tools to suppress routine progress. Other package managers retain their normal output.
+
+When Rust-backed JavaScript builds are present, fastws checks before dependency installation that `rustc` and `cargo` on `PATH` are rustup's proxies. Install rustup from https://rustup.rs and select a stable toolchain with `rustup default stable` if not already configured. If another Rust installation shadows rustup, put its bin directory first on `PATH` after Homebrew's shell setup: `export PATH="$HOME/.cargo/bin:$PATH"` (use `$CARGO_HOME/bin` for a custom Cargo home), then open a new shell. Homebrew Rust can remain installed for Homebrew's own use. Fastws reports missing or shadowed rustup without changing your shell or installing a different toolchain. Packages using wasm-pack can then install their WASM target and matching binding tools automatically as part of their build.
+
+The package manager defaults to `npm`. Set `js` under `[tool.fastws]` in the workspace `pyproject.toml` to choose another executable, such as `js = "bun"`. Fastws requires a list for `workspaces` in `package.json`, following the npm format. It stops if it cannot find the selected executable.
+
 At most once per day, sync upgrades dependencies with `uv sync -U` and a parallel `cargo update` in every member with a `Cargo.toml`. It prints the changes and records the run in a stamp file inside the workspace's `.git` directory. Pass `--upgrade` to force an upgrade regardless of the last run time.
 
 Before every uv sync, `ws-sync` writes `.git/fastws-cargo-key` for each member crate. The key hashes `Cargo.lock` contents and the workspace Cargo patch configuration. For Git dependencies redirected to local paths by `[patch."<url>"]`, it also hashes each patched crate's `Cargo.toml`, `build.rs`, and `src` tree, recursively. The file is rewritten only when that content changes, so projects can use `{ file = ".git/fastws-cargo-key" }` in `tool.uv.cache-keys` without rebuilding after a timestamp-only `Cargo.lock` write.

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/README.md

@@ -57,17 +57,31 @@ AnswerDotAI/fastcore
 jph00/private ~/private
 ```
 
+Select Python extras by attaching brackets to the repo name, before any checkout location:
+
+```text
+AnswerDotAI/fastws[dev]
+owner/repo[dev,test] ~/somewhere
+```
+
+These select `[project.optional-dependencies]`, not `[dependency-groups]`. Sync uses the package name from `pyproject.toml`, so `AnswerDotAI/fastws[dev]` installs `fastws-cli[dev]` editably. For a checkout containing several discovered Python packages, the selection applies to each. uv handles the extras and their dependencies.
+
+Extras from duplicate entries in `repos.txt` and `repos-local.txt` are combined. For example, a personal `AnswerDotAI/fastws[dev]` entry adds development extras to a shared `AnswerDotAI/fastws` entry. The lists are authoritative for those packages' extras: each sync updates existing workspace dependencies, including removing extras no longer selected. Put personal extra selections in `repos-local.txt`, not the generated `pyproject.toml`. Existing version constraints, markers, sources, and unrelated dependencies are preserved.
+
 ### Shared baseline and personal additions
 
 Keep `repos.txt` tracked in the workspace Git repo. Put personal additions in a gitignored `repos-local.txt`, using the same format. Every command reads both lists. `ws-add` and automatic discovery append only to the local list. Edit the shared list deliberately when changing the team's baseline. Duplicate repo names are matched case-insensitively. Conflicting explicit locations, or two different repos targeting the same directory, cause errors.
 
 `--repos-file myrepos.txt` uses `myrepos-local.txt` alongside it. Either list may be absent. Removing a repo from the baseline does not delete its checkout. The next sync discovers an existing root checkout as a personal addition. External checkouts also remain on disk; add them to the local list to keep managing them.
 
-The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it without replacing personal settings. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
+The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it while preserving personal settings except for the repo extras managed above. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
 
 ```gitignore
 /repos-local.txt
 /pyproject.toml
+/package.json
+/package-lock.json
+/node_modules/
 /pyrightconfig.json
 /uv.lock
 /.cargo/
@@ -169,7 +183,7 @@ By default, `ws-sync` uses the active venv's parent as the workspace root. It cr
 
 Project scanning respects `tool.uv.workspace.members` and `exclude`. If a member directory lacks `pyproject.toml`, such as a fresh empty clone, sync warns and skips the `uv sync` step.
 
-The workspace `exclude` list is maintained automatically. An unlisted top-level directory that is not a valid Python project is excluded until it has a `pyproject.toml`. A `repos.txt` checkout is automatically excluded only when it is a Cargo-only Rust crate. A listed checkout with neither `pyproject.toml` nor `Cargo.toml` is treated as a pending member and triggers the warning above.
+Fastws maintains `tool.uv.workspace.exclude`. It excludes unlisted top-level directories without a valid Python project. It also excludes listed Rust or JavaScript projects without `pyproject.toml`. For a listed checkout with none of `pyproject.toml`, `Cargo.toml`, or `package.json`, sync warns and skips installation.
 
 Existing globs, entries for missing directories, and entries for checkouts that are still not Python projects are retained. Use `exclude = [...]` under `[tool.fastws]` to specify exclusions the scan cannot infer, such as keeping a valid project out of the workspace. Adding members preserves hand-written `[tool.uv.sources]` entries, including path and Git sources.
 
@@ -179,6 +193,20 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved.
 
+Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
+
+Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.
+
+An optional tracked `package.json.shared` is merged into the root `package.json` on every sync. Object-valued fields merge one level deep, shared values win conflicts, and arrays replace rather than concatenate. Unrelated local settings remain; removing a shared key does not delete its local value. Workspace discovery still manages `workspaces`. Use this file for shared policy such as version-pinned npm `allowScripts` approvals.
+
+After `uv sync`, fastws runs `<tool> install` at the workspace root. It then runs `<tool> run build` in each JavaScript package containing both `Cargo.toml` and a `build` script. Cargo handles incremental compilation. Other build steps still run.
+
+For npm, sync hides routine install summaries and build-command banners. Install warnings, audit findings, unreviewed install scripts, and failure diagnostics remain visible. Build scripts retain their own output; configure their tools to suppress routine progress. Other package managers retain their normal output.
+
+When Rust-backed JavaScript builds are present, fastws checks before dependency installation that `rustc` and `cargo` on `PATH` are rustup's proxies. Install rustup from https://rustup.rs and select a stable toolchain with `rustup default stable` if not already configured. If another Rust installation shadows rustup, put its bin directory first on `PATH` after Homebrew's shell setup: `export PATH="$HOME/.cargo/bin:$PATH"` (use `$CARGO_HOME/bin` for a custom Cargo home), then open a new shell. Homebrew Rust can remain installed for Homebrew's own use. Fastws reports missing or shadowed rustup without changing your shell or installing a different toolchain. Packages using wasm-pack can then install their WASM target and matching binding tools automatically as part of their build.
+
+The package manager defaults to `npm`. Set `js` under `[tool.fastws]` in the workspace `pyproject.toml` to choose another executable, such as `js = "bun"`. Fastws requires a list for `workspaces` in `package.json`, following the npm format. It stops if it cannot find the selected executable.
+
 At most once per day, sync upgrades dependencies with `uv sync -U` and a parallel `cargo update` in every member with a `Cargo.toml`. It prints the changes and records the run in a stamp file inside the workspace's `.git` directory. Pass `--upgrade` to force an upgrade regardless of the last run time.
 
 Before every uv sync, `ws-sync` writes `.git/fastws-cargo-key` for each member crate. The key hashes `Cargo.lock` contents and the workspace Cargo patch configuration. For Git dependencies redirected to local paths by `[patch."<url>"]`, it also hashes each patched crate's `Cargo.toml`, `build.rs`, and `src` tree, recursively. The file is rewritten only when that content changes, so projects can use `{ file = ".git/fastws-cargo-key" }` in `tool.uv.cache-keys` without rebuilding after a timestamp-only `Cargo.lock` write.

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/fastws/__init__.py

@@ -1,4 +1,6 @@
-__version__ = "0.0.14"
+__version__ = "0.0.16"
+
+
 
 
 

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/fastws/core.py

@@ -2,7 +2,7 @@
 
 __all__ = ["ws_setup", "ws_clone", "ws_pull", "ws_status", "ws_branches", "ws_build", "ws_sync", "ws_add", "ws_remove"]
 
-import ast, fnmatch, hashlib, json, os, re, shlex, shutil, subprocess, sys, time
+import ast, fnmatch, glob, hashlib, json, os, re, shlex, shutil, subprocess, sys, time
 from pathlib import Path
 from concurrent.futures import ThreadPoolExecutor
 
@@ -10,6 +10,8 @@ from fastcore.parallel import parallel_async_gen
 from fastcore.script import call_parse
 from fastgit import Git
 from ghapi.core import dep_key, dep_closure
+from packaging.requirements import Requirement
+from packaging.utils import canonicalize_name
 
 try: import tomllib
 except ModuleNotFoundError: import tomli as tomllib
@@ -19,10 +21,12 @@ def _repo_lines(repos_file) -> list[str]:
     if not p.exists(): raise SystemExit(f"File not found: {repos_file}")
     return [line.strip() for line in p.read_text().splitlines() if line.strip() and not line.startswith("#")]
 
-def _parse_repo_line(line: str) -> tuple[str, str|None]:
-    "Split a repos.txt line into its repo spec and optional checkout location"
-    parts = line.split(None, 1)
-    return parts[0], (parts[1].strip() or None) if len(parts) > 1 else None
+def _parse_repo_line(line: str) -> tuple[str, str|None, set[str]]:
+    "Split owner/repo[extra,...] and an optional checkout location."
+    m = re.fullmatch(r'([^\s\[\]]+)(?:\[([^\[\]]+)\])?(?:\s+(.+))?', line.strip())
+    if not m: raise SystemExit(f'Invalid repo entry: {line}')
+    repo, extras, loc = m.groups()
+    return repo, loc, {canonicalize_name(e.strip()) for e in extras.split(',')} if extras else set()
 
 def _load_repos(repos_file: str = "repos.txt") -> list[str]:
     return [_parse_repo_line(l)[0] for l in _repo_lines(repos_file)] if Path(repos_file).exists() else []
@@ -31,21 +35,23 @@ def _local_repos_path(repos_file) -> Path:
     p = Path(repos_file)
     return p.with_name(f'{p.stem}-local{p.suffix}')
 
-def _load_repo_entries(repos_file, root: Path) -> list[tuple[str, Path]]:
-    "Shared and local entries, deduplicated by repo; conflicting checkout locations are errors."
+def _load_repo_entries(repos_file, root: Path) -> list[tuple[str, Path, set[str]]]:
+    "Shared and local entries, with extras unioned by repo; conflicting checkout locations are errors."
     res, locations = {}, {}
     for p in (Path(repos_file), _local_repos_path(repos_file)):
         if not p.exists(): continue
         for line in _repo_lines(p):
-            repo, loc = _parse_repo_line(line)
+            repo, loc, extras = _parse_repo_line(line)
             key = _repo_key(repo)
-            if key in res and loc is None: continue
+            if key in res:
+                res[key][2].update(extras)
+                if loc is None: continue
             d = _resolve_path(root, Path(loc).expanduser()) if loc else root/_repo_dir(repo)
             resolved = d.resolve()
             if key in res and res[key][1].resolve() != resolved: raise SystemExit(f'Conflicting locations for {repo} in {p}')
             if resolved in locations and locations[resolved] != key:
                 raise SystemExit(f'Checkout location {d} is shared by {repo} and {locations[resolved]}')
-            res.setdefault(key, (repo, d))
+            res.setdefault(key, (repo, d, extras))
             locations[resolved] = key
     return list(res.values())
 
@@ -57,7 +63,7 @@ def _resolve_path(root: Path, path: str) -> Path:
 
 def _repo_key(repo: str) -> str: return repo.strip().rstrip("/").removesuffix(".git").casefold()
 
-def _pkg_key(name: str) -> str: return name.casefold()
+def _pkg_key(name: str) -> str: return canonicalize_name(name)
 
 def _fmt_toml_val(v) -> str:
     if isinstance(v, bool): return "true" if v else "false"
@@ -98,16 +104,25 @@ def _ws_cfg(root: Path):
     exclude = ws.get("exclude") or []
     return members, exclude
 
+def _fastws_cfg(root: Path) -> dict:
+    "The `[tool.fastws]` table from the workspace root pyproject.toml (empty when absent)."
+    pyproj = root/"pyproject.toml"
+    if not pyproj.exists(): return {}
+    return tomllib.loads(pyproj.read_text(encoding="utf-8")).get("tool", {}).get("fastws", {})
+
 def _matches_ws(name: str, pattern: str) -> bool:
     pattern = pattern.strip()
     return any(fnmatch.fnmatch(candidate, normalized) for candidate in (name, f"./{name}") for normalized in (pattern, pattern.removeprefix("./")))
 
-def _is_ws_dir(d: Path, members, exclude) -> bool:
-    return d.is_dir() and not d.name.startswith(".") and any(_matches_ws(d.name, o) for o in members) and not any(_matches_ws(d.name, o) for o in exclude)
+def _project_dirs(root: Path, manifest="", members=("*",), exclude=()) -> list[Path]:
+    "Expand member patterns into unique directories, optionally requiring a manifest; exclusions are relative to root."
+    dirs = dict.fromkeys(d for pat in members for d in sorted(root.glob(pat)))
+    return [d for d in dirs if d.is_dir() and (not manifest or (d/manifest).is_file())
+        and not any(_matches_ws(os.path.relpath(d, root), e) for e in exclude)]
 
 def _ws_dirs(root: Path) -> list[Path]:
     members, exclude = _ws_cfg(root)
-    return [d for d in sorted(root.iterdir()) if _is_ws_dir(d, members, exclude)]
+    return [d for d in _project_dirs(root, exclude=exclude) if not d.name.startswith(".") and any(_matches_ws(d.name, o) for o in members)]
 
 def _root_git_dirs(root: Path) -> list[Path]:
     "Every git checkout directly under `root`, whatever the uv workspace config; `_`-prefixed dirs are private"
@@ -152,10 +167,10 @@ def _remove_from_pyproject(pyproject_path: Path, names: list[str]) -> list[str]:
     sources = dict(data.get("tool", {}).get("uv", {}).get("sources", {}))
     deps = list(data.get("project", {}).get("dependencies", []))
     targets = {_pkg_key(n) for n in names}
-    removed = sorted({_pkg_key(s) for s in sources if _pkg_key(s) in targets} | {dep_key(d) for d in deps if dep_key(d) in targets})
+    removed = sorted({_pkg_key(s) for s in sources if _pkg_key(s) in targets} | {_pkg_key(dep_key(d)) for d in deps if _pkg_key(dep_key(d)) in targets})
     if not removed: return []
     sources = {k:v for k,v in sources.items() if _pkg_key(k) not in targets}
-    deps = [d for d in deps if dep_key(d) not in targets]
+    deps = [d for d in deps if _pkg_key(dep_key(d)) not in targets]
     content = _replace_table(content, "tool.uv.sources", "\n".join(f"{k} = {_fmt_source(v)}" for k,v in sources.items()))
     content = _replace_project_dependencies(content, deps)
     pyproject_path.write_text(content)
@@ -178,16 +193,24 @@ def _external_projects(root: Path, dirs: list[Path]) -> list[tuple[str,str]]:
     res = []
     for d in dirs:
         if not d.is_dir(): continue
-        cands = [d] if (d/"pyproject.toml").exists() else sorted(p for p in d.iterdir() if p.is_dir() and not p.name.startswith((".","_")) and (p/"pyproject.toml").exists())
+        cands = [d] if (d/"pyproject.toml").exists() else [p for p in _project_dirs(d, "pyproject.toml") if not p.name.startswith((".", "_"))]
         for c in cands:
             if name := _read_pyproject_name(c/"pyproject.toml"): res.append((name, os.path.relpath(c, root)))
     return res
 
+def _repo_extras(root: Path, entries) -> dict[str, set[str]]:
+    "Package extras selected by the repo lists, including empty selections for removal on sync."
+    return {_pkg_key(name): extras for _,d,extras in entries for name,_ in _external_projects(root, [d])}
+
 def _valid_project_dir(d: Path) -> bool: return (d/"pyproject.toml").exists() and bool(_read_pyproject_name(d/"pyproject.toml"))
 
-def _cargo_only(d: Path) -> bool:
-    "A Rust crate with no Python layer: not a pending scaffold, just not a uv workspace member"
-    return (d/"Cargo.toml").exists() and not (d/"pyproject.toml").exists()
+def _non_python_project(d: Path) -> bool:
+    "A Rust or JS project without a Python layer is not a pending Python scaffold."
+    return not (d/"pyproject.toml").exists() and any((d/f).exists() for f in ("Cargo.toml", "package.json"))
+
+def _pending_dirs(root: Path) -> list[str]:
+    "uv workspace dirs that are not Python projects yet; sync stops rather than let uv fail on them"
+    return [d.name for d in _ws_dirs(root) if not (d/"pyproject.toml").exists()]
 
 def _sync_ws_excludes(pyproject_path: Path, root: Path, tracked: set[str]) -> tuple[list[str], list[str]]:
     """Regenerate `tool.uv.workspace.exclude` and return (added, removed).
@@ -195,7 +218,7 @@ def _sync_ws_excludes(pyproject_path: Path, root: Path, tracked: set[str]) -> tu
     Kept as-is: `[tool.fastws].exclude` entries (intent), globs, missing dirs, and tracked dirs
     (repos.txt checkouts) that are still not valid Python projects. Auto-managed: entries for other
     existing dirs are regenerated each sync, excluding dirs without a valid pyproject (tracked dirs
-    only when they are Cargo-only crates, since a tracked dir with neither file is a pending member
+    only when they are Cargo-only crates or npm-only packages, since a tracked dir with none of those files is a pending member
     awaiting scaffolding) and un-excluding dirs that gained one; deliberately excluding a real
     project takes a `[tool.fastws]` entry."""
     if not pyproject_path.exists(): return [], []
@@ -207,10 +230,8 @@ def _sync_ws_excludes(pyproject_path: Path, root: Path, tracked: set[str]) -> tu
     intent = [e for e in data.get("tool", {}).get("fastws", {}).get("exclude", []) if isinstance(e, str)]
     kept = [e for e in cur if e in intent or any(c in e for c in "*?[") or not (root/e).is_dir() or (e in tracked and not _valid_project_dir(root/e))]
     kept += [e for e in intent if e not in kept]
-    auto = [d.name for d in sorted(root.iterdir())  # chkstyle: ignore-node
-        if d.is_dir() and not d.name.startswith(".") and any(_matches_ws(d.name, m) for m in members)
-        and not any(_matches_ws(d.name, e) for e in kept)
-        and (d.name not in tracked or _cargo_only(d)) and not _valid_project_dir(d)]
+    auto = [d.name for d in _project_dirs(root, exclude=kept) if not d.name.startswith(".") and any(_matches_ws(d.name, m) for m in members)
+        and (d.name not in tracked or _non_python_project(d)) and not _valid_project_dir(d)]
     survivors = set(kept) | set(auto)
     new = [e for e in cur if e in survivors] + [e for e in kept + auto if e not in cur]
     if new == cur: return [], []
@@ -264,7 +285,7 @@ def _replace_project_dependencies(content: str, deps: list[str]) -> str:
     if not (span := _table_span(content, "project")): raise SystemExit("Missing [project] table in pyproject.toml")
     start,end = span
     section = content[start:end]
-    dep_block = "dependencies = [\n" + "".join(f'    "{dep}",\n' for dep in deps) + "]"
+    dep_block = "dependencies = [\n" + "".join(f'    {json.dumps(dep)},\n' for dep in deps) + "]"
     if m := re.search(r"(?m)^dependencies\s*=\s*\[", section):
         arr_start = m.end()-1
         arr_end = _find_array_end(section, arr_start)
@@ -287,29 +308,36 @@ members = ["./*"]
 ''')
     return True
 
-def _sync_ws_pyproject(pyproject_path: Path, template_path: Path, projects: list[str], externals: list[tuple] = None) -> list[str]:
+def _sync_ws_pyproject(pyproject_path: Path, template_path: Path, projects: list[str], externals: list[tuple] = None,
+    extras: dict[str, set[str]] = None) -> list[str]:
     if not pyproject_path.exists():
         if template_path.exists(): shutil.copyfile(template_path, pyproject_path)
         else: _init_ws_pyproject(pyproject_path)
-    content = pyproject_path.read_text()
+    original = content = pyproject_path.read_text()
     data = tomllib.loads(content)
     sources = dict(data.get("tool", {}).get("uv", {}).get("sources", {}))
     source_keys = {_pkg_key(proj) for proj in sources}
     missing = [proj for proj in projects if _pkg_key(proj) not in source_keys]
     ext_missing = [(n,p) for n,p in (externals or []) if _pkg_key(n) not in source_keys]
-    if not missing and not ext_missing: return []
     for proj in missing: sources[proj] = {"workspace": True}
     for n,p in ext_missing: sources[n] = {"path": p, "editable": True}
     deps = list(data.get("project", {}).get("dependencies", []))
-    dep_keys = {dep_key(dep) for dep in deps}
-    for proj in missing + [n for n,_ in ext_missing]:
+    dep_keys = {_pkg_key(dep_key(dep)) for dep in deps}
+    for proj in projects + [n for n,_ in (externals or [])]:
         if _pkg_key(proj) in dep_keys: continue
         deps.append(proj)
         dep_keys.add(_pkg_key(proj))
-    source_lines = "\n".join(f"{proj} = {_fmt_source(src)}" for proj,src in sources.items())
-    content = _replace_table(content, "tool.uv.sources", source_lines)
-    content = _replace_project_dependencies(content, deps)
-    pyproject_path.write_text(content)
+    for i,dep in enumerate(deps):
+        if (key := _pkg_key(dep_key(dep))) not in (extras or {}): continue
+        req = Requirement(dep)
+        if {canonicalize_name(e) for e in req.extras} == extras[key]: continue
+        req.extras = extras[key]
+        deps[i] = str(req)
+    if missing or ext_missing:
+        source_lines = "\n".join(f"{proj} = {_fmt_source(src)}" for proj,src in sources.items())
+        content = _replace_table(content, "tool.uv.sources", source_lines)
+    if deps != data.get("project", {}).get("dependencies", []): content = _replace_project_dependencies(content, deps)
+    if content != original: pyproject_path.write_text(content)
     return missing + [n for n,_ in ext_missing]
 
 def _editable_mapping(path: Path) -> dict[str,str]:
@@ -353,10 +381,10 @@ async def _clone_one(repo: str, d: Path) -> str|None:
     except subprocess.CalledProcessError as e: return f"✗ {d.name}: {e.stderr.strip()}"
 
 async def _clone_repos(entries, workers):
-    async def clone(e): return await _clone_one(*e)
+    async def clone(e): return await _clone_one(e[0], e[1])
     async for _, res in parallel_async_gen(clone, entries, n_workers=workers):
         if res: print(res)
-    if missing := [str(d) for _,d in entries if not (d/'.git').exists()]:
+    if missing := [str(d) for _,d,_ in entries if not (d/'.git').exists()]:
         raise SystemExit(f'Could not clone repos (missing Git checkouts): {", ".join(missing)}')
 
 async def _pull_workspace(root: Path):
@@ -436,7 +464,7 @@ async def ws_pull(
     workers: int = 64,  # Number of parallel workers
 ):
     "Pull updates for repos whose GitHub origin has moved (all repos when that can't be checked)."
-    dirs = [d for _,d in _load_repo_entries(repos_file, Path("."))]
+    dirs = [d for _,d,_ in _load_repo_entries(repos_file, Path("."))]
     try: dirs = await _changed_dirs(dirs)
     except Exception: pass
     await _pull(dirs, workers)
@@ -447,7 +475,7 @@ def ws_status(
     branches: bool = False,  # Show unpushed commit details
 ):
     "Show uncommitted changes and optionally unpushed commit details across repos."
-    for repo,d in _load_repo_entries(repos_file, Path(".")):
+    for repo,d,_ in _load_repo_entries(repos_file, Path(".")):
         if not d.exists(): continue
         g = Git(d)
         if not g.exists: continue
@@ -468,7 +496,7 @@ def ws_branches(
     expected: str = "main",  # Expected branch name
 ):
     "Check if all repos are on the expected branch."
-    for repo,d in _load_repo_entries(repos_file, Path(".")):
+    for repo,d,_ in _load_repo_entries(repos_file, Path(".")):
         if not Path(d).exists():
             print(f"⚠️  {d}: directory not found")
             continue
@@ -503,7 +531,7 @@ def _build_projects(root: Path, repos_file: str, project: str = None) -> list[tu
     "(name, dir) for every project ws-sync installs: workspace members plus external checkouts"
     repos_path = _resolve_path(root, repos_file)
     entries = _load_repo_entries(repos_path, root)
-    ext_dirs = [d for _,d in entries if d.resolve().parent != root.resolve()]
+    ext_dirs = [d for _,d,_ in entries if d.resolve().parent != root.resolve()]
     res = [(name, d) for d in _ws_dirs(root) if (d/"pyproject.toml").exists() and (name := _read_pyproject_name(d/"pyproject.toml"))]
     res += [(n, _resolve_path(root, p)) for n,p in _external_projects(root, ext_dirs)]
     if project is None: return res
@@ -569,7 +597,7 @@ _CARGO_KEY = Path(".git/fastws-cargo-key")
 
 def _crate_dirs(root: Path) -> list[Path]:
     "Root dirs containing a Cargo.toml: the crate view of the workspace, independent of uv membership"
-    return [d for d in sorted(root.iterdir()) if d.is_dir() and not d.name.startswith((".", "_")) and (d/"Cargo.toml").exists()]
+    return [d for d in _project_dirs(root, "Cargo.toml") if not d.name.startswith((".", "_"))]
 
 def _cargo_patches(root: Path):
     "Local Cargo patches keyed by normalized Git URL and package name, plus their config file."
@@ -589,12 +617,10 @@ def _crate_pkgs(d: Path):
     try: data = tomllib.loads((d/"Cargo.toml").read_text())
     except tomllib.TOMLDecodeError: return
     if name := data.get("package", {}).get("name"): yield name, d
-    for pat in data.get("workspace", {}).get("members", []):
-        for m in sorted(d.glob(pat)):
-            if not (m/"Cargo.toml").exists(): continue
-            try: sub = tomllib.loads((m/"Cargo.toml").read_text())
-            except tomllib.TOMLDecodeError: continue
-            if name := sub.get("package", {}).get("name"): yield name, m
+    for m in _project_dirs(d, "Cargo.toml", data.get("workspace", {}).get("members", [])):
+        try: sub = tomllib.loads((m/"Cargo.toml").read_text())
+        except tomllib.TOMLDecodeError: continue
+        if name := sub.get("package", {}).get("name"): yield name, m
 
 def _local_crates(root: Path) -> dict[str, Path]:
     "Package name -> dir for every crate under `root`, nested cargo workspace members included"
@@ -623,6 +649,10 @@ def _strip_patch_tables(content: str) -> str:
         content = content[:m.start()] + content[end:]
     return content
 
+def _entry_changes(current, new):
+    "Added and removed keys, in the order supplied."
+    return [k for k in new if k not in current], [k for k in current if k not in new]
+
 def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     """Regenerate `[patch]` entries in the workspace `.cargo/config.toml` and return (added, removed).
 
@@ -659,7 +689,8 @@ def _sync_cargo_patches(root: Path) -> tuple[list[str], list[str]]:
     config.write_text(new)
     before = {n for entries in old.values() for n in entries}
     after = {n for entries in tables.values() for n in entries}
-    return sorted(after - before), sorted(before - after)
+    added, removed = _entry_changes(before, after)
+    return sorted(added), sorted(removed)
 
 def _sync_cargo_wrapper(root: Path) -> bool:
     "Add sccache to the generated Cargo config when installed, without overriding another wrapper"
@@ -744,6 +775,78 @@ def _cargo_update(root: Path, workers: int = 16):
             elif lines := [l for l in out.splitlines() if l.lstrip().startswith(("Updating ", "Adding ", "Removing ")) and "crates.io index" not in l]:
                 print(f"{name}:\n" + "\n".join(lines))
 
+def _npm_dirs(root: Path) -> list[Path]:
+    "Find root JavaScript packages and their declared workspace members."
+    exclude = _fastws_cfg(root).get("exclude", [])
+    def ok(d): return not d.name.startswith((".", "_")) and d.name != "node_modules"
+    res = []
+    for d in _project_dirs(root, "package.json", exclude=exclude):
+        if not ok(d): continue
+        data = json.loads((d/"package.json").read_text())
+        members = [glob.escape(d.name)] + [f"{glob.escape(d.name)}/{p}" for p in data.get("workspaces", [])]
+        res += _project_dirs(root, "package.json", members, exclude)
+    return list(dict.fromkeys(d for d in res if ok(d)))
+
+def _sync_ws_package_json(root: Path, members: list[Path]) -> tuple[list[str], list[str]]:
+    "Merge shared settings and update workspaces, preserving external paths and globs; return (added, removed)."
+    path = root/"package.json"
+    data = json.loads(path.read_text()) if path.exists() else {"private": True}
+    before = json.dumps(data)
+    shared = root/"package.json.shared"
+    if shared.exists():
+        for k,v in json.loads(shared.read_text()).items():
+            data[k] = data[k] | v if isinstance(v, dict) and isinstance(data.get(k), dict) else v
+    cur = data.get("workspaces", [])
+    if not isinstance(cur, list): raise SystemExit(f"{path}: fastws requires `workspaces` to be a list of package paths.")
+    kept = [e for e in cur if any(c in e for c in "*?[") or not (root/e).resolve().is_relative_to(root.resolve())]
+    new = kept + [e for e in (os.path.relpath(d, root) for d in members) if e not in kept]
+    if new != cur: data["workspaces"] = new
+    if json.dumps(data) == before: return [], []
+    path.write_text(json.dumps(data, indent=2) + "\n")
+    return _entry_changes(cur, new)
+
+def _native_js(members: list[Path]) -> list[Path]:
+    "JavaScript packages with a Rust manifest and a build script."
+    return [d for d in members if (d/"Cargo.toml").exists()
+        and json.loads((d/"package.json").read_text()).get("scripts", {}).get("build")]
+
+def _check_rustup():
+    "Require rustup's compiler proxies for automatic WASM target installation."
+    if not (rustup := shutil.which("rustup")):
+        raise SystemExit("Rust-backed JavaScript builds require rustup. Install it from https://rustup.rs, "
+            "select a stable toolchain with `rustup default stable`, then reopen your shell and rerun ws-sync.")
+    for tool in "rustc","cargo":
+        path = shutil.which(tool)
+        if path and os.path.samefile(path, rustup): continue
+        raise SystemExit(f"Rust-backed JavaScript builds require rustup-managed Rust, but {tool} resolves to {path or 'nothing'}. "
+            "Put rustup's bin directory first on PATH after Homebrew or other toolchain setup in your shell profile "
+            '(normally: export PATH="$HOME/.cargo/bin:$PATH"; use $CARGO_HOME/bin when customized). '
+            "Homebrew Rust can remain installed. Open a new shell and rerun ws-sync.")
+
+def _npm_install(root: Path):
+    "Hide npm's install summary, but retain warnings, audit findings, and failure diagnostics."
+    try: res = subprocess.run(["npm", "install", "--json"], check=True, cwd=root, stdout=subprocess.PIPE, text=True)
+    except subprocess.CalledProcessError as e:
+        if e.stdout: print(e.stdout, file=sys.stderr, end="")
+        raise
+    try: data = json.loads(res.stdout)
+    except json.JSONDecodeError:
+        print(res.stdout, file=sys.stderr, end="")
+        raise
+    if data.get("audit", {}).get("vulnerabilities", {}).get("total") or data.get("unreviewedScripts"):
+        print(f"npm install findings ({root}):\n{res.stdout}", file=sys.stderr, end="")
+
+def _sync_js(root: Path, members: list[Path]) -> list[Path]:
+    "Install the JS workspace, run every native member's build script, and return those members. Cargo handles incremental compilation."
+    tool = _fastws_cfg(root).get("js", "npm")
+    if not shutil.which(tool): raise SystemExit(f"{tool} is not installed: install it, or set [tool.fastws].js to a package manager that is")
+    if tool == "npm": _npm_install(root)
+    else: subprocess.run([tool, "install"], check=True, cwd=root)
+    built = _native_js(members)
+    cmd = [tool, "run", "--silent", "build"] if tool == "npm" else [tool, "run", "build"]
+    for d in built: subprocess.run(cmd, check=True, cwd=d)
+    return built
+
 @call_parse
 async def ws_sync(
     workspace: str = "",  # Workspace root; defaults to active venv parent when available
@@ -765,18 +868,19 @@ async def ws_sync(
 
     if missing_repos := _update_repos_file(repos_path, repos): print(f"Added local repos: {', '.join(missing_repos)}")
     entries = _load_repo_entries(repos_path, root)
-    ext_dirs = [d for _,d in entries if d.resolve().parent != root.resolve()]
-    dirs = [d for _,d in entries]
+    ext_dirs = [d for _,d,_ in entries if d.resolve().parent != root.resolve()]
+    dirs = [d for _,d,_ in entries]
     try: dirs = await _changed_dirs(dirs)
     except Exception: pass
     await _pull(dirs, workers=workers)
 
     if not pyproject_path.exists(): _sync_ws_pyproject(pyproject_path, template_path, [])
-    added_ex, removed_ex = _sync_ws_excludes(pyproject_path, root, {d.name for _,d in entries if d.resolve().parent == root.resolve()})
+    added_ex, removed_ex = _sync_ws_excludes(pyproject_path, root, {d.name for _,d,_ in entries if d.resolve().parent == root.resolve()})
     if added_ex: print(f"Auto-excluded from the workspace: {', '.join(added_ex)}")
     if removed_ex: print(f"No longer excluded from the workspace: {', '.join(removed_ex)}")
 
-    missing_projects = _sync_ws_pyproject(pyproject_path, template_path, _ws_projects(root), _external_projects(root, ext_dirs))
+    missing_projects = _sync_ws_pyproject(pyproject_path, template_path, _ws_projects(root), _external_projects(root, ext_dirs),
+        _repo_extras(root, entries))
     if missing_projects: print(f"Added workspace projects: {', '.join(missing_projects)}")
 
     wrapper_added = _sync_cargo_wrapper(root)
@@ -785,7 +889,13 @@ async def ws_sync(
     if removed_p: print(f"Cargo patches removed: {', '.join(removed_p)}")
     if wrapper_added: print("Cargo builds now use sccache")
 
-    if bad := [d.name for d in _ws_dirs(root) if not (d/"pyproject.toml").exists()]:
+    js_members = _npm_dirs(root)
+    if _native_js(js_members): _check_rustup()
+    added_j, removed_j = _sync_ws_package_json(root, js_members)
+    if added_j: print(f"JS workspace packages added: {', '.join(added_j)}")
+    if removed_j: print(f"JS workspace packages removed: {', '.join(removed_j)}")
+
+    if bad := _pending_dirs(root):
         print(f"⚠️  Skipping uv sync, not Python projects yet (scaffold with e.g. nbdev-new or ship-new, or remove): {', '.join(bad)}")
         return
     up = upgrade or _should_upgrade(root)
@@ -793,6 +903,7 @@ async def ws_sync(
     _sync_cargo_keys(root)
     subprocess.run(["uv", "sync", "-U"] if up else ["uv", "sync"], check=True, cwd=root)
     if up: _upgrade_stamp(root).touch()
+    if js_members: _sync_js(root, js_members)
 
 @call_parse
 async def ws_add(
@@ -848,7 +959,7 @@ def _resolve_removal_target(root: Path, repo: str, repos_path: Path) -> str:
     "Canonical owner/repo for `repo`, matching an existing folder name when `repo` isn't a valid spec."
     if _is_repo_spec(repo): return _normalize_repo(repo)
     if not (root/repo).is_dir(): return _normalize_repo(repo)  # invalid spec and no such folder: raise
-    for r,d in _load_repo_entries(repos_path, root):
+    for r,d,_ in _load_repo_entries(repos_path, root):
         if _repo_dir(r).casefold() == repo.casefold(): return r
     if (root/repo/".git").exists():
         url = Git(root/repo).remote("get-url", "origin", mute_errors=True)
@@ -887,7 +998,7 @@ def ws_remove(
     if _repo_key(repo) in {_repo_key(r) for r in _load_repos(repos_path)}:
         raise SystemExit(f'{repo} belongs to the shared baseline: edit {repos_path} deliberately to remove it')
     d = _resolve_repo_dir(root, repo)
-    for r,location in _load_repo_entries(repos_path, root):
+    for r,location,_ in _load_repo_entries(repos_path, root):
         if _repo_key(r) == _repo_key(repo) and location.resolve() != d:
             raise SystemExit(f'Refusing to remove {repo}: custom checkout location {location}; manage it explicitly')
     repos_path = _local_repos_path(repos_path)

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/fastws/releases.py

@@ -10,21 +10,12 @@ from fastcore.parallel import parallel_async
 from ghapi.core import APIError, GhApi, dep_closure, local_dep_graph
 from packaging.version import InvalidVersion, Version
 
-from .core import _load_repo_entries, _resolve_path, _ws_root
-
-try: import tomllib
-except ModuleNotFoundError: import tomli as tomllib
+from .core import _fastws_cfg, _load_repo_entries, _resolve_path, _ws_root
 
 # Start-of-message regexes for commits that need no release (curation noise, CI config, docs regen)
 DEFAULT_SKIP = ["bump$", "Bump version to ", "meta", "auto", "README", "regen readme", "nbdev regen",
     "update .gitignore", r"\.?gitignore$", "workflow", "chkstyle skip", "tests$", "md$", "docs?$", "clean$", "CI$", "ignore$", "allowed_metadata_keys$"]
 
-def _fastws_cfg(root: Path) -> dict:
-    "The `[tool.fastws]` table from the workspace root pyproject.toml (empty when absent)."
-    pyproj = root/"pyproject.toml"
-    if not pyproj.exists(): return {}
-    return tomllib.loads(pyproj.read_text(encoding="utf-8")).get("tool", {}).get("fastws", {})
-
 def _skip_pats(skip=None, root: Path|None = None) -> list[re.Pattern]:
     "Compiled skip patterns: defaults + `[tool.fastws].release_skip` from the workspace pyproject + `skip`."
     pats = list(DEFAULT_SKIP)
@@ -68,14 +59,14 @@ class ReleaseReport(list):
 def _cwd_project(root: Path, repos_file: str = "repos.txt") -> str|None:
     "Dir name of the workspace member checkout containing cwd (None at the root or outside every member)."
     cwd = Path.cwd().resolve()
-    for _, d in _load_repo_entries(_resolve_path(root, repos_file), root):
+    for _, d, _ in _load_repo_entries(_resolve_path(root, repos_file), root):
         if cwd.is_relative_to(d.resolve()): return d.name
     return None
 
 async def check_releases(project: str = None, skip=None, workspace: str = "", repos_file: str = "repos.txt", nodeps: bool = False) -> ReleaseReport:
     "Sweep every workspace repo for unreleased commits; a `project` (given, or detected from cwd inside a member checkout) limits the sweep to its transitive dependency closure, or to itself alone with `nodeps`. `[tool.fastws].release_exclude` names repos to leave out (apps that deploy rather than release)."
     root = _ws_root(workspace)
-    repos = [r for r,d in _load_repo_entries(_resolve_path(root, repos_file), root)]
+    repos = [r for r,_,_ in _load_repo_entries(_resolve_path(root, repos_file), root)]
     excl = {e.casefold() for e in _fastws_cfg(root).get("release_exclude", [])}
     repos = [r for r in repos if r.split("/")[-1].casefold() not in excl]
     if project is None: project = _cwd_project(root, repos_file)

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/fastws_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fastws-cli
-Version: 0.0.14
+Version: 0.0.16
 Summary: Fast workspace tools for multi-repo management
 Author: Jeremy Howard
 License: Apache-2.0
@@ -82,17 +82,31 @@ AnswerDotAI/fastcore
 jph00/private ~/private
 ```
 
+Select Python extras by attaching brackets to the repo name, before any checkout location:
+
+```text
+AnswerDotAI/fastws[dev]
+owner/repo[dev,test] ~/somewhere
+```
+
+These select `[project.optional-dependencies]`, not `[dependency-groups]`. Sync uses the package name from `pyproject.toml`, so `AnswerDotAI/fastws[dev]` installs `fastws-cli[dev]` editably. For a checkout containing several discovered Python packages, the selection applies to each. uv handles the extras and their dependencies.
+
+Extras from duplicate entries in `repos.txt` and `repos-local.txt` are combined. For example, a personal `AnswerDotAI/fastws[dev]` entry adds development extras to a shared `AnswerDotAI/fastws` entry. The lists are authoritative for those packages' extras: each sync updates existing workspace dependencies, including removing extras no longer selected. Put personal extra selections in `repos-local.txt`, not the generated `pyproject.toml`. Existing version constraints, markers, sources, and unrelated dependencies are preserved.
+
 ### Shared baseline and personal additions
 
 Keep `repos.txt` tracked in the workspace Git repo. Put personal additions in a gitignored `repos-local.txt`, using the same format. Every command reads both lists. `ws-add` and automatic discovery append only to the local list. Edit the shared list deliberately when changing the team's baseline. Duplicate repo names are matched case-insensitively. Conflicting explicit locations, or two different repos targeting the same directory, cause errors.
 
 `--repos-file myrepos.txt` uses `myrepos-local.txt` alongside it. Either list may be absent. Removing a repo from the baseline does not delete its checkout. The next sync discovers an existing root checkout as a personal addition. External checkouts also remain on disk; add them to the local list to keep managing them.
 
-The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it without replacing personal settings. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
+The workspace `pyproject.toml` is local generated state. `ws-sync` creates it from the tracked `pyproject.tmpl` when available, or generates a minimal one, then maintains it while preserving personal settings except for the repo extras managed above. The template supplies starting defaults only: later template edits do not overwrite existing configuration. Ignore these files in the workspace repo (not in its member repos):
 
 ```gitignore
 /repos-local.txt
 /pyproject.toml
+/package.json
+/package-lock.json
+/node_modules/
 /pyrightconfig.json
 /uv.lock
 /.cargo/
@@ -194,7 +208,7 @@ By default, `ws-sync` uses the active venv's parent as the workspace root. It cr
 
 Project scanning respects `tool.uv.workspace.members` and `exclude`. If a member directory lacks `pyproject.toml`, such as a fresh empty clone, sync warns and skips the `uv sync` step.
 
-The workspace `exclude` list is maintained automatically. An unlisted top-level directory that is not a valid Python project is excluded until it has a `pyproject.toml`. A `repos.txt` checkout is automatically excluded only when it is a Cargo-only Rust crate. A listed checkout with neither `pyproject.toml` nor `Cargo.toml` is treated as a pending member and triggers the warning above.
+Fastws maintains `tool.uv.workspace.exclude`. It excludes unlisted top-level directories without a valid Python project. It also excludes listed Rust or JavaScript projects without `pyproject.toml`. For a listed checkout with none of `pyproject.toml`, `Cargo.toml`, or `package.json`, sync warns and skips installation.
 
 Existing globs, entries for missing directories, and entries for checkouts that are still not Python projects are retained. Use `exclude = [...]` under `[tool.fastws]` to specify exclusions the scan cannot infer, such as keeping a valid project out of the workspace. Adding members preserves hand-written `[tool.uv.sources]` entries, including path and Git sources.
 
@@ -204,6 +218,20 @@ Do not commit a `Cargo.lock` generated under these patches. Its source-less loca
 
 When `sccache` is installed, sync also configures it as Cargo's `rustc-wrapper`, allowing unchanged compilation units to be reused across the workspace's otherwise-independent Cargo target directories. An existing wrapper is always preserved.
 
+Fastws discovers JavaScript packages directly under the workspace root. It reads each package's `workspaces` list for declared nested packages. These lists accept paths and globs. Fastws ignores undeclared nested packages. Lockfiles do not affect discovery. `[tool.fastws].exclude` matches paths relative to the workspace root. Fastws skips `node_modules` and directories whose names start with `.` or `_`.
+
+Each sync updates `workspaces` in the root `package.json`. It creates the file when needed. It preserves external paths, globs, and unrelated settings. It reports added and removed packages.
+
+An optional tracked `package.json.shared` is merged into the root `package.json` on every sync. Object-valued fields merge one level deep, shared values win conflicts, and arrays replace rather than concatenate. Unrelated local settings remain; removing a shared key does not delete its local value. Workspace discovery still manages `workspaces`. Use this file for shared policy such as version-pinned npm `allowScripts` approvals.
+
+After `uv sync`, fastws runs `<tool> install` at the workspace root. It then runs `<tool> run build` in each JavaScript package containing both `Cargo.toml` and a `build` script. Cargo handles incremental compilation. Other build steps still run.
+
+For npm, sync hides routine install summaries and build-command banners. Install warnings, audit findings, unreviewed install scripts, and failure diagnostics remain visible. Build scripts retain their own output; configure their tools to suppress routine progress. Other package managers retain their normal output.
+
+When Rust-backed JavaScript builds are present, fastws checks before dependency installation that `rustc` and `cargo` on `PATH` are rustup's proxies. Install rustup from https://rustup.rs and select a stable toolchain with `rustup default stable` if not already configured. If another Rust installation shadows rustup, put its bin directory first on `PATH` after Homebrew's shell setup: `export PATH="$HOME/.cargo/bin:$PATH"` (use `$CARGO_HOME/bin` for a custom Cargo home), then open a new shell. Homebrew Rust can remain installed for Homebrew's own use. Fastws reports missing or shadowed rustup without changing your shell or installing a different toolchain. Packages using wasm-pack can then install their WASM target and matching binding tools automatically as part of their build.
+
+The package manager defaults to `npm`. Set `js` under `[tool.fastws]` in the workspace `pyproject.toml` to choose another executable, such as `js = "bun"`. Fastws requires a list for `workspaces` in `package.json`, following the npm format. It stops if it cannot find the selected executable.
+
 At most once per day, sync upgrades dependencies with `uv sync -U` and a parallel `cargo update` in every member with a `Cargo.toml`. It prints the changes and records the run in a stamp file inside the workspace's `.git` directory. Pass `--upgrade` to force an upgrade regardless of the last run time.
 
 Before every uv sync, `ws-sync` writes `.git/fastws-cargo-key` for each member crate. The key hashes `Cargo.lock` contents and the workspace Cargo patch configuration. For Git dependencies redirected to local paths by `[patch."<url>"]`, it also hashes each patched crate's `Cargo.toml`, `build.rs`, and `src` tree, recursively. The file is rewritten only when that content changes, so projects can use `{ file = ".git/fastws-cargo-key" }` in `tool.uv.cache-keys` without rebuilding after a timestamp-only `Cargo.lock` write.

{fastws_cli-0.0.14 → fastws_cli-0.0.16}/tests/test_sync.py

@@ -1,4 +1,4 @@
-import os, pytest
+import json, os, pytest
 from types import SimpleNamespace
 from fastgit import Git
 import fastws.core as core
@@ -40,40 +40,46 @@ def fake_uv(monkeypatch):
 
 def test_repos_file_roundtrip(tmp_path):
     repos_path = tmp_path/'repos.txt'
-    repos_path.write_text('# header\nAnswerDotAI/fastws\njph00/private ~/private\nfastai/fastai sub/dir\n')
+    repos_path.write_text('# header\nAnswerDotAI/fastws[dev]\njph00/private[docs] ~/private\nfastai/fastai sub/dir\n')
 
     assert core._load_repos(repos_path) == ['AnswerDotAI/fastws', 'jph00/private', 'fastai/fastai']
     assert core._load_repo_entries(repos_path, tmp_path) == [
-        ('AnswerDotAI/fastws', tmp_path/'fastws'),
-        ('jph00/private', core.Path('~/private').expanduser()),
-        ('fastai/fastai', tmp_path/'sub'/'dir')]
+        ('AnswerDotAI/fastws', tmp_path/'fastws', {'dev'}),
+        ('jph00/private', core.Path('~/private').expanduser(), {'docs'}),
+        ('fastai/fastai', tmp_path/'sub'/'dir', set())]
 
     # discovery and ws-add write only extras, never change the shared baseline
     original = repos_path.read_text()
     local = tmp_path/'repos-local.txt'
-    added = core._update_repos_file(repos_path, ['answerdotai/fastws', 'jph00/private', 'fastai/new'])
-    assert added == ['fastai/new']
-    assert local.read_text() == 'fastai/new\n'
+    added = core._update_repos_file(repos_path, ['answerdotai/fastws', 'jph00/private', 'fastai/new[dev]'])
+    assert added == ['fastai/new[dev]']
+    assert local.read_text() == 'fastai/new[dev]\n'
     assert repos_path.read_text() == original
-    assert core._load_repo_entries(repos_path, tmp_path)[-1] == ('fastai/new', tmp_path/'new')
+    assert core._load_repo_entries(repos_path, tmp_path)[-1] == ('fastai/new', tmp_path/'new', {'dev'})
 
     # remove matches case-insensitively, preserves other lines (including comments), and reports absence
     assert core._remove_from_repos_file(local, 'FASTAI/new') is True
     assert core._remove_from_repos_file(repos_path, 'AnswerDotAI/gone') is False
-    assert repos_path.read_text() == '# header\nAnswerDotAI/fastws\njph00/private ~/private\nfastai/fastai sub/dir\n'
+    assert repos_path.read_text() == original
 
 
 def test_repo_lists_deduplicate_and_reject_conflicting_locations(tmp_path):
     base, local = tmp_path/'repos.txt', tmp_path/'repos-local.txt'
-    base.write_text('org/core\n')
-    local.write_text('ORG/core\norg/extra ../external\n')
-    assert core._load_repo_entries(base, tmp_path) == [('org/core', tmp_path/'core'), ('org/extra', tmp_path/'../external')]
+    base.write_text('org/core[dev]\norg/extra[docs] ../external\n')
+    local.write_text('ORG/core[TEST, dev]\norg/extra[dev]\n')
+    assert core._load_repo_entries(base, tmp_path) == [
+        ('org/core', tmp_path/'core', {'dev', 'test'}), ('org/extra', tmp_path/'../external', {'docs', 'dev'})]
     local.write_text('org/core ../elsewhere\n')
     with pytest.raises(SystemExit, match='location'): core._load_repo_entries(base, tmp_path)
     local.write_text('different/core\n')
     with pytest.raises(SystemExit, match='location'): core._load_repo_entries(base, tmp_path)
     base.unlink()
-    assert core._load_repo_entries(base, tmp_path) == [('different/core', tmp_path/'core')]
+    assert core._load_repo_entries(base, tmp_path) == [('different/core', tmp_path/'core', set())]
+
+
+@pytest.mark.parametrize('entry', ['org/repo[dev', 'org/repo[dev]]', 'org/repo[]'])
+def test_invalid_repo_extras(entry):
+    with pytest.raises(SystemExit, match='Invalid repo entry'): core._parse_repo_line(entry)
 
 
 async def test_sync_updates_baseline_before_cloning_and_keeps_removed_repos(tmp_path, monkeypatch, fake_uv):
@@ -87,21 +93,22 @@ async def test_sync_updates_baseline_before_cloning_and_keeps_removed_repos(tmp_
     for name in ('old', 'new', 'extra'):
         d = tmp_path/name
         d.mkdir()
-        (d/'pyproject.toml').write_text(f'[project]\nname = "{name}"\n')
+        (d/'pyproject.toml').write_text(f'[project]\nname = "{name}-pkg"\n[project.optional-dependencies]\ndev = []\ntest = []\n')
         mk_repo(d, origin=origins/f'{name}.git')
     seed.mkdir()
-    (seed/'repos.txt').write_text('org/old\n')
+    (seed/'repos.txt').write_text('org/old[dev]\n')
     (seed/'.gitignore').write_text('*/\npyproject.toml\nrepos-local.txt\n')
     (seed/'pyproject.tmpl').write_text('[project]\nname = "workspace"\ndependencies = []\n[tool.uv.workspace]\nmembers = ["./*"]\n')
     g = mk_repo(seed, origin=origins/'workspace.git')
     Git(tmp_path, raise_exc=True).clone(str(origins/'workspace.git'), str(root))
     Git(root, raise_exc=True).clone('git@github.com:org/extra.git', str(root/'extra'))
+    (root/'repos-local.txt').write_text('org/old[test]\n')
 
     await core.ws_sync(str(root))
     assert (root/'old'/'pyproject.toml').exists()
-    assert (root/'repos.txt').read_text() == 'org/old\n'
-    assert (root/'repos-local.txt').read_text() == 'org/extra\n'
-    assert set(core.tomllib.loads((root/'pyproject.toml').read_text())['project']['dependencies']) == {'old', 'extra'}
+    assert (root/'repos.txt').read_text() == 'org/old[dev]\n'
+    assert (root/'repos-local.txt').read_text() == 'org/old[test]\norg/extra\n'
+    assert set(core.tomllib.loads((root/'pyproject.toml').read_text())['project']['dependencies']) == {'old-pkg[dev,test]', 'extra-pkg'}
 
     (seed/'repos.txt').write_text('org/new\n')
     g.commit('-a', m='new shared baseline')
@@ -111,7 +118,10 @@ async def test_sync_updates_baseline_before_cloning_and_keeps_removed_repos(tmp_
     assert (root/'old'/'pyproject.toml').exists()
     assert (root/'repos.txt').read_text() == 'org/new\n'
     assert set(core._load_repos(root/'repos-local.txt')) == {'org/extra', 'org/old'}
-    assert set(core.tomllib.loads((root/'pyproject.toml').read_text())['project']['dependencies']) == {'old', 'new', 'extra'}
+    assert set(core.tomllib.loads((root/'pyproject.toml').read_text())['project']['dependencies']) == {'old-pkg[test]', 'new-pkg', 'extra-pkg'}
+    (root/'repos-local.txt').write_text('org/extra\n')
+    await core.ws_sync(str(root))
+    assert set(core.tomllib.loads((root/'pyproject.toml').read_text())['project']['dependencies']) == {'old-pkg', 'new-pkg', 'extra-pkg'}
 
     (seed/'repos.txt').write_text('org/missing\n')
     g.commit('-a', m='unavailable member')
@@ -162,6 +172,23 @@ def test_ws_pyproject_preserves_existing_entries(tmp_path):
     assert 'alpha = { workspace = true }' in content
 
 
+def test_sync_extras_preserves_constraints_markers_and_unrelated_settings(tmp_path):
+    pyproject = tmp_path/'pyproject.toml'
+    pyproject.write_text('[project]\nname = "ws"\ndependencies = [\'My_Pkg[old]>=1; python_version >= "3.10"\', "other[keep]"]\n'
+        '[tool.uv.sources]\nMy_Pkg = { path = "../somewhere", editable = true }\n')
+    for extras in ({'dev', 'test'}, set()):
+        core._sync_ws_pyproject(pyproject, tmp_path/'pyproject.tmpl', ['my-pkg'], extras={'my-pkg': extras})
+        data = core.tomllib.loads(pyproject.read_text())
+        dep, other = data['project']['dependencies']
+        req = core.Requirement(dep)
+        assert req.extras == extras and str(req.specifier) == '>=1' and str(req.marker) == 'python_version >= "3.10"'
+        assert other == 'other[keep]' and len(data['project']['dependencies']) == 2
+        assert data['tool']['uv']['sources'] == {'My_Pkg': {'path': '../somewhere', 'editable': True}}
+        before = pyproject.stat().st_mtime_ns
+        core._sync_ws_pyproject(pyproject, tmp_path/'pyproject.tmpl', ['my-pkg'], extras={'my-pkg': extras})
+        assert pyproject.stat().st_mtime_ns == before
+
+
 def test_ws_excludes_generates_from_intent_and_auto(tmp_path):
     pyproject = tmp_path/'pyproject.toml'
     pyproject.write_text(
@@ -205,6 +232,22 @@ def test_external_projects_discovers_root_and_subdir_packages(tmp_path):
         (multi/name/'pyproject.toml').write_text(f'[project]\nname = "{pkg}"\n')
 
     assert core._external_projects(root, [single, multi, tmp_path/'missing']) == [('singlepkg', '../single'), ('tool1', '../multi/tool1')]
+    (root/'repos.txt').write_text(f'org/single[dev] {single}\norg/multi[docs] {multi}\n')
+    extras = core._repo_extras(root, core._load_repo_entries(root/'repos.txt', root))
+    assert extras == {'singlepkg': {'dev'}, 'tool1': {'docs'}}
+    core._sync_ws_pyproject(root/'pyproject.toml', root/'pyproject.tmpl', [], core._external_projects(root, [single, multi]), extras)
+    data = core.tomllib.loads((root/'pyproject.toml').read_text())
+    assert data['project']['dependencies'] == ['singlepkg[dev]', 'tool1[docs]']
+    assert data['tool']['uv']['sources']['tool1'] == {'path': '../multi/tool1', 'editable': True}
+
+
+@pytest.mark.parametrize('manifest', ['pyproject.toml', 'Cargo.toml', 'package.json'])
+def test_project_dirs_expands_patterns_and_filters_manifests(tmp_path, manifest):
+    for name in ('a', 'b', 'empty'): (tmp_path/'pkgs'/name).mkdir(parents=True)
+    for name in ('a', 'b'): (tmp_path/'pkgs'/name/manifest).write_text('')
+    members, exclude = ['pkgs/*', 'pkgs/a'], ['./pkgs/b']
+    assert core._project_dirs(tmp_path, manifest, members, exclude) == [tmp_path/'pkgs'/'a']
+    assert core._project_dirs(tmp_path, members=members, exclude=exclude) == [tmp_path/'pkgs'/'a', tmp_path/'pkgs'/'empty']
 
 
 def test_ws_projects_skip_excluded_dirs_and_template_names(tmp_path):
@@ -454,3 +497,132 @@ def test_sync_cargo_wrapper(tmp_path, monkeypatch):
     config.write_text('[build]\nrustc-wrapper = "other-cache"\n')
     assert not core._sync_cargo_wrapper(tmp_path)
     assert core.tomllib.loads(config.read_text())['build']['rustc-wrapper'] == 'other-cache'
+
+
+def test_npm_dirs_discovers_root_and_declared_packages(tmp_path):
+    for name in ('app', 'crate', 'crate/wasm', 'crate/frontend', 'crate/docs', '_scratch', 'node_modules', 'crate/node_modules'):
+        d = tmp_path/name
+        d.mkdir(parents=True, exist_ok=True)
+        (d/'package.json').write_text('{}')
+    (tmp_path/'app'/'package-lock.json').write_text('{}')
+    (tmp_path/'crate'/'package.json').write_text('{"private": true, "workspaces": ["wasm", "front*", "node_modules", "wasm", "missing"]}')
+    (tmp_path/'crate'/'frontend'/'bun.lock').write_text('')  # lockfiles do not override explicit membership
+
+    assert core._npm_dirs(tmp_path) == [tmp_path/'app', tmp_path/'crate', tmp_path/'crate'/'wasm', tmp_path/'crate'/'frontend']
+
+
+def test_npm_dirs_honours_fastws_exclude(tmp_path):
+    (tmp_path/'pyproject.toml').write_text('[tool.fastws]\nexclude = ["app", "tool*", "py/examples"]\n')
+    for name in ('app', 'lib', 'tools'):
+        (tmp_path/name).mkdir()
+        (tmp_path/name/'package.json').write_text('{}')
+    for name in ('examples', 'wasm'):
+        (tmp_path/'py'/name).mkdir(parents=True)
+        (tmp_path/'py'/name/'package.json').write_text('{}')  # examples is excluded by its root-relative path
+    (tmp_path/'py'/'package.json').write_text('{"private": true, "workspaces": ["*"]}')
+
+    assert core._npm_dirs(tmp_path) == [tmp_path/'lib', tmp_path/'py', tmp_path/'py'/'wasm']
+
+
+def test_sync_ws_package_json_generates_and_preserves(tmp_path):
+    pkg = tmp_path/'package.json'
+    pkg.write_text('{\n  "name": "ws",\n  "private": true,\n  "workspaces": ["../outside", "gone", "tools/*"]\n}\n')
+    members = [tmp_path/'app', tmp_path/'crate'/'wasm']
+
+    added, removed = core._sync_ws_package_json(tmp_path, members)
+    data = json.loads(pkg.read_text())
+    # kept: entries outside the root and globs; managed: existing dirs regenerated from discovery
+    assert data['workspaces'] == ['../outside', 'tools/*', 'app', 'crate/wasm']
+    assert data['name'] == 'ws' and data['private'] is True  # other keys untouched
+    assert added == ['app', 'crate/wasm'] and removed == ['gone']
+
+    content = pkg.read_text()
+    assert core._sync_ws_package_json(tmp_path, members) == ([], [])
+    assert pkg.read_text() == content
+
+    # a missing root package.json is created
+    bare = tmp_path/'ws2'
+    bare.mkdir()
+    assert core._sync_ws_package_json(bare, [bare/'a']) == (['a'], [])
+    assert json.loads((bare/'package.json').read_text()) == {'private': True, 'workspaces': ['a']}
+
+
+@pytest.mark.parametrize('workspaces', [{'packages': ['packages/*']}, {}])
+def test_sync_ws_package_json_rejects_object_form(tmp_path, workspaces):
+    pkg = tmp_path/'package.json'
+    content = json.dumps({'private': True, 'workspaces': workspaces}) + '\n'
+    pkg.write_text(content)
+
+    with pytest.raises(SystemExit, match='workspaces.*list') as exc: core._sync_ws_package_json(tmp_path, [tmp_path/'app'])
+    assert str(pkg) in str(exc.value)
+    assert pkg.read_text() == content
+
+
+def test_sync_ws_package_json_merges_shared_settings(tmp_path):
+    pkg, shared = tmp_path/'package.json', tmp_path/'package.json.shared'
+    pkg.write_text(json.dumps({'name': 'local', 'workspaces': ['app'], 'keywords': ['local'],
+        'allowScripts': {'wasm-pack@0.15.0': False, 'other': False}}))
+    shared.write_text(json.dumps({'keywords': ['shared'], 'allowScripts': {'wasm-pack@0.15.0': True}}))
+    assert core._sync_ws_package_json(tmp_path, [tmp_path/'app']) == ([], [])
+    assert json.loads(pkg.read_text()) == {'name': 'local', 'workspaces': ['app'], 'keywords': ['shared'],
+        'allowScripts': {'wasm-pack@0.15.0': True, 'other': False}}
+    content, mtime = pkg.read_text(), pkg.stat().st_mtime_ns
+    core._sync_ws_package_json(tmp_path, [tmp_path/'app'])
+    assert pkg.stat().st_mtime_ns == mtime
+    shared.write_text('{}')
+    core._sync_ws_package_json(tmp_path, [tmp_path/'app'])
+    assert pkg.read_text() == content
+
+
+def test_ws_excludes_treat_npm_only_dirs_like_cargo_only(tmp_path):
+    pyproject = tmp_path/'pyproject.toml'
+    pyproject.write_text('[project]\nname = "uvws"\n\n[tool.uv.workspace]\nmembers = ["./*"]\nexclude = []\n')
+    for name in ('app', 'pending'): (tmp_path/name).mkdir()
+    (tmp_path/'app'/'package.json').write_text('{}')
+
+    # a tracked npm-only checkout is a valid JS project: excluded from uv, never pending; an empty tracked dir still awaits scaffolding
+    assert core._sync_ws_excludes(pyproject, tmp_path, {'app', 'pending'}) == (['app'], [])
+    assert core._pending_dirs(tmp_path) == ['pending']
+
+
+def test_native_js_requires_rust_and_build_script(tmp_path):
+    app, crate, wasm = [tmp_path/n for n in ('app', 'crate', 'wasm')]
+    for d in (app, crate, wasm): d.mkdir()
+    (app/'package.json').write_text('{"scripts": {"build": "vite build"}}')
+    for d in (crate, wasm): (d/'Cargo.toml').write_text('')
+    (crate/'package.json').write_text('{}')
+    (wasm/'package.json').write_text('{"scripts": {"build": "cargo build"}}')
+    assert core._native_js([app, crate, wasm]) == [wasm]
+
+
+def test_check_rustup_missing(monkeypatch):
+    monkeypatch.setattr(core.shutil, 'which', lambda tool: None)
+    with pytest.raises(SystemExit, match='Install it from https://rustup.rs'): core._check_rustup()
+
+
+@pytest.mark.parametrize('shadowed', [None, 'rustc', 'cargo'])
+def test_check_rustup_proxies(tmp_path, monkeypatch, shadowed):
+    rustup = tmp_path/'rustup'
+    rustup.touch()
+    (tmp_path/'rustc').symlink_to(rustup)
+    (tmp_path/'cargo').hardlink_to(rustup)
+    other = tmp_path/'brew-rust'
+    other.touch()
+    monkeypatch.setattr(core.shutil, 'which', lambda tool: str(other if tool == shadowed else tmp_path/tool))
+    if shadowed:
+        with pytest.raises(SystemExit, match=f'{shadowed} resolves to') as err: core._check_rustup()
+        assert 'Homebrew Rust can remain installed' in str(err.value)
+        assert 'PATH' in str(err.value)
+    else: core._check_rustup()
+
+
+async def test_sync_checks_rustup_before_installs(tmp_path, monkeypatch, fake_uv):
+    (tmp_path/'repos.txt').write_text('')
+    (tmp_path/'pyproject.toml').write_text('[project]\nname = "ws"\n[tool.uv.workspace]\nmembers = ["*"]\n')
+    wasm = tmp_path/'wasm'
+    wasm.mkdir()
+    (wasm/'Cargo.toml').write_text('[package]\nname = "wasm"\n')
+    (wasm/'package.json').write_text('{"name": "wasm", "scripts": {"build": "wasm-pack build"}}')
+    monkeypatch.setattr(core.shutil, 'which', lambda tool: None)
+    with pytest.raises(SystemExit, match='require rustup'): await core.ws_sync(str(tmp_path))
+    assert not fake_uv