macfleet 2.2.0rc1__tar.gz → 2.2.1__tar.gz

{macfleet-2.2.0rc1 → macfleet-2.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: macfleet
-Version: 2.2.0rc1
+Version: 2.2.1
 Summary: Pool Apple Silicon Macs for distributed compute and ML training
 Author: MacFleet Contributors
 License: MIT
@@ -46,29 +46,34 @@ Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
 Requires-Dist: ruff>=0.3.0; extra == "dev"
 Requires-Dist: mypy>=1.8.0; extra == "dev"
 Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
+Requires-Dist: hypothesis>=6.0.0; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: mkdocs-material>=9.5.0; extra == "docs"
 Requires-Dist: mkdocs>=1.5.0; extra == "docs"
 
+<div align="center">
+
 # MacFleet
 
-**Distributed ML training on Apple Silicon.** Pool your Macs into a
-cluster in 5 seconds, run PyTorch or MLX across them, keep zero cloud
-spend.
+### Distributed training for Apple Silicon fleets
 
-```
-  macfleet join              macfleet join            macfleet join
- ┌──────────────┐          ┌──────────────┐          ┌──────────────┐
- │  MacBook Pro │◄────────►│  MacBook Air │◄────────►│  Mac Studio  │
- │  M4 Pro      │  WiFi /  │  M4          │  WiFi /  │  M4 Ultra    │
- │  16 GPU cores│  ETH /   │  10 GPU cores│  ETH /   │  60 GPU cores│
- │  48 GB RAM   │  TB4     │  16 GB RAM   │  TB4     │  192 GB RAM  │
- └──────────────┘          └──────────────┘          └──────────────┘
-         ▲                          ▲                          ▲
-         └──────────────────────────┴──────────────────────────┘
-                        Ring AllReduce (gradient sync)
+Run PyTorch or MLX across multiple Macs with secure peer discovery,
+TLS/HMAC authentication, and framework-agnostic gradient synchronization.
+
+```bash
+pip install macfleet
+macfleet join --bootstrap
 ```
 
+**Local hardware. Real data parallelism. No cloud bill.**
+
+</div>
+
+MacFleet turns a room full of Apple Silicon machines into one training
+pool. Each Mac keeps a full model replica, processes its shard of the
+batch, and synchronizes gradients over a NumPy-only communication layer
+that never imports torch or MLX.
+
 ## Why MacFleet
 
 Apple Silicon is everywhere. Every researcher, student, and founder
@@ -117,40 +122,48 @@ On Mac #1:
 
 ```bash
 macfleet join --bootstrap
-# prints a QR code + pairing URL, also copies URL to pasteboard
+# first run auto-generates a fleet token and prints a short-lived
+# one-time pairing command. The permanent token is not printed.
 ```
 
-On Mac #2 (same Apple ID → Handoff pasteboard sync):
+On Mac #2:
 
 ```bash
-macfleet pair && macfleet join
+macfleet pair --host <Mac-1-IP>:<enrollment-port> --code <one-time-code>
+macfleet join
 ```
 
-Or: scan the QR from Mac #1 with your iPhone camera. Tap the link.
-Done.
+The enrollment code expires after 5 minutes and is single-use by default.
 
-**4. Set `enable_pool_distributed=True` in your script** — training now
-spans both Macs.
+**4. Set `enable_pool_distributed=True` and run the same script on both
+Macs** — training now spans both: the pool forms a gradient mesh, rank 0
+broadcasts initial weights, and every step's gradients are averaged
+across the fleet. The result dict's `params_sha256` matches on both
+Macs when the fleet stayed in sync; `degraded`, `unsynced_steps`, and
+`validation_fallback_steps` tell you if any step fell back locally.
 
 ## Features
 
 - **Dual engine** — PyTorch (MPS) and Apple MLX, same pool infrastructure
 - **Zero config** — mDNS discovery, no coordinator setup, no config files
 - **Safe task dispatch** — `@macfleet.task` registry + msgpack args
-  (no cloudpickle on the wire)
+  (no cloudpickle on the wire; local pickle fallback is explicit opt-in)
 - **Adaptive compression** — auto-selects TopK + FP16 based on link
-  speed (1x–200x reduction)
+  speed (locally; sparse-on-wire arrives in v2.3, see TODOS.md
+  Issue 3)
 - **Heterogeneous scheduling** — faster Macs get bigger batches,
   adjusts for thermal throttling
-- **Secure by default** — auto-generated fleet tokens, HMAC mutual
-  auth, mandatory TLS, per-IP rate limiting
+- **Secure by default** — auto-generated fleet tokens (scrypt-derived
+  keys), client-first HMAC mutual auth (servers reveal nothing to
+  unauthenticated peers), mandatory TLS with channel-bound handshakes
+  (MITM-relay resistant), per-IP rate limiting
 - **Framework-agnostic core** — communication layer uses only numpy,
   never imports torch or mlx
 
 ## Security
 
 Security is on by default. The first `macfleet join` auto-generates a
-fleet token at `~/.macfleet/token` (mode 0600). See the
+fleet token at `~/.macfleet/fleet-token` (mode 0600). See the
 [security reference](docs/reference/security.md) for the full threat
 model.
 
@@ -166,12 +179,18 @@ Short version:
   1s to stop slowloris)
 - **No cloudpickle over the wire** — `@macfleet.task` routes
   registered callables by name, not by pickled closures
+- **One-time pairing** — `macfleet join --bootstrap` exposes only a
+  short-lived enrollment code, not the permanent fleet token
+- **Local audit trail** — auth failures, enrollment, token rotation,
+  legacy pickle use, and degraded training events are written to
+  `~/.macfleet/audit.jsonl` with credential fields redacted
 
 ## CLI
 
 ```
 macfleet join         Join the pool (auto-discovers peers)
-macfleet pair         Read a pairing URL from pasteboard / stdin
+macfleet pair         Pair with a one-time enrollment code
+macfleet rotate-token Rotate the local fleet token
 macfleet status       Show pool members and network info
 macfleet info         Show local hardware profile
 macfleet train        Run training (demo or custom script)
@@ -186,11 +205,16 @@ MacFleet uses **data parallelism**: every Mac holds a full copy of the
 model, trains on a weighted portion of the data, and averages
 gradients via Ring AllReduce after each step.
 
-| Network       | Compression     | 100 MB gradients become |
-|---------------|-----------------|-------------------------|
-| Thunderbolt 4 | None            | 100 MB                  |
-| Ethernet      | TopK 10% + FP16 | ~5 MB                   |
-| WiFi          | TopK 1% + FP16  | ~500 KB                 |
+The compression layer (TopK + FP16) is applied locally before the
+allreduce; v2.2 transmits dense gradients on the wire (sparse
+allreduce is on the v2.3 roadmap as Issue 3). The bandwidth savings
+table below describes the **target** ratios once sparse-on-wire ships:
+
+| Network       | Compression     | 100 MB gradients (v2.3 target) |
+|---------------|-----------------|--------------------------------|
+| Thunderbolt 4 | None            | 100 MB                         |
+| Ethernet      | TopK 10% + FP16 | ~5 MB                          |
+| WiFi          | TopK 1% + FP16  | ~500 KB                        |
 
 ## Requirements
 
@@ -217,7 +241,7 @@ read the Markdown source in `docs/`:
 git clone https://github.com/vikranthreddimasu/MacFleet.git
 cd MacFleet
 pip install -e ".[dev,all]"
-make test       # 425+ tests
+make test       # 447 tests
 make lint       # ruff + mypy
 ```
 

{macfleet-2.2.0rc1 → macfleet-2.2.1}/README.md

@@ -1,22 +1,26 @@
+<div align="center">
+
 # MacFleet
 
-**Distributed ML training on Apple Silicon.** Pool your Macs into a
-cluster in 5 seconds, run PyTorch or MLX across them, keep zero cloud
-spend.
+### Distributed training for Apple Silicon fleets
 
-```
-  macfleet join              macfleet join            macfleet join
- ┌──────────────┐          ┌──────────────┐          ┌──────────────┐
- │  MacBook Pro │◄────────►│  MacBook Air │◄────────►│  Mac Studio  │
- │  M4 Pro      │  WiFi /  │  M4          │  WiFi /  │  M4 Ultra    │
- │  16 GPU cores│  ETH /   │  10 GPU cores│  ETH /   │  60 GPU cores│
- │  48 GB RAM   │  TB4     │  16 GB RAM   │  TB4     │  192 GB RAM  │
- └──────────────┘          └──────────────┘          └──────────────┘
-         ▲                          ▲                          ▲
-         └──────────────────────────┴──────────────────────────┘
-                        Ring AllReduce (gradient sync)
+Run PyTorch or MLX across multiple Macs with secure peer discovery,
+TLS/HMAC authentication, and framework-agnostic gradient synchronization.
+
+```bash
+pip install macfleet
+macfleet join --bootstrap
 ```
 
+**Local hardware. Real data parallelism. No cloud bill.**
+
+</div>
+
+MacFleet turns a room full of Apple Silicon machines into one training
+pool. Each Mac keeps a full model replica, processes its shard of the
+batch, and synchronizes gradients over a NumPy-only communication layer
+that never imports torch or MLX.
+
 ## Why MacFleet
 
 Apple Silicon is everywhere. Every researcher, student, and founder
@@ -65,40 +69,48 @@ On Mac #1:
 
 ```bash
 macfleet join --bootstrap
-# prints a QR code + pairing URL, also copies URL to pasteboard
+# first run auto-generates a fleet token and prints a short-lived
+# one-time pairing command. The permanent token is not printed.
 ```
 
-On Mac #2 (same Apple ID → Handoff pasteboard sync):
+On Mac #2:
 
 ```bash
-macfleet pair && macfleet join
+macfleet pair --host <Mac-1-IP>:<enrollment-port> --code <one-time-code>
+macfleet join
 ```
 
-Or: scan the QR from Mac #1 with your iPhone camera. Tap the link.
-Done.
+The enrollment code expires after 5 minutes and is single-use by default.
 
-**4. Set `enable_pool_distributed=True` in your script** — training now
-spans both Macs.
+**4. Set `enable_pool_distributed=True` and run the same script on both
+Macs** — training now spans both: the pool forms a gradient mesh, rank 0
+broadcasts initial weights, and every step's gradients are averaged
+across the fleet. The result dict's `params_sha256` matches on both
+Macs when the fleet stayed in sync; `degraded`, `unsynced_steps`, and
+`validation_fallback_steps` tell you if any step fell back locally.
 
 ## Features
 
 - **Dual engine** — PyTorch (MPS) and Apple MLX, same pool infrastructure
 - **Zero config** — mDNS discovery, no coordinator setup, no config files
 - **Safe task dispatch** — `@macfleet.task` registry + msgpack args
-  (no cloudpickle on the wire)
+  (no cloudpickle on the wire; local pickle fallback is explicit opt-in)
 - **Adaptive compression** — auto-selects TopK + FP16 based on link
-  speed (1x–200x reduction)
+  speed (locally; sparse-on-wire arrives in v2.3, see TODOS.md
+  Issue 3)
 - **Heterogeneous scheduling** — faster Macs get bigger batches,
   adjusts for thermal throttling
-- **Secure by default** — auto-generated fleet tokens, HMAC mutual
-  auth, mandatory TLS, per-IP rate limiting
+- **Secure by default** — auto-generated fleet tokens (scrypt-derived
+  keys), client-first HMAC mutual auth (servers reveal nothing to
+  unauthenticated peers), mandatory TLS with channel-bound handshakes
+  (MITM-relay resistant), per-IP rate limiting
 - **Framework-agnostic core** — communication layer uses only numpy,
   never imports torch or mlx
 
 ## Security
 
 Security is on by default. The first `macfleet join` auto-generates a
-fleet token at `~/.macfleet/token` (mode 0600). See the
+fleet token at `~/.macfleet/fleet-token` (mode 0600). See the
 [security reference](docs/reference/security.md) for the full threat
 model.
 
@@ -114,12 +126,18 @@ Short version:
   1s to stop slowloris)
 - **No cloudpickle over the wire** — `@macfleet.task` routes
   registered callables by name, not by pickled closures
+- **One-time pairing** — `macfleet join --bootstrap` exposes only a
+  short-lived enrollment code, not the permanent fleet token
+- **Local audit trail** — auth failures, enrollment, token rotation,
+  legacy pickle use, and degraded training events are written to
+  `~/.macfleet/audit.jsonl` with credential fields redacted
 
 ## CLI
 
 ```
 macfleet join         Join the pool (auto-discovers peers)
-macfleet pair         Read a pairing URL from pasteboard / stdin
+macfleet pair         Pair with a one-time enrollment code
+macfleet rotate-token Rotate the local fleet token
 macfleet status       Show pool members and network info
 macfleet info         Show local hardware profile
 macfleet train        Run training (demo or custom script)
@@ -134,11 +152,16 @@ MacFleet uses **data parallelism**: every Mac holds a full copy of the
 model, trains on a weighted portion of the data, and averages
 gradients via Ring AllReduce after each step.
 
-| Network       | Compression     | 100 MB gradients become |
-|---------------|-----------------|-------------------------|
-| Thunderbolt 4 | None            | 100 MB                  |
-| Ethernet      | TopK 10% + FP16 | ~5 MB                   |
-| WiFi          | TopK 1% + FP16  | ~500 KB                 |
+The compression layer (TopK + FP16) is applied locally before the
+allreduce; v2.2 transmits dense gradients on the wire (sparse
+allreduce is on the v2.3 roadmap as Issue 3). The bandwidth savings
+table below describes the **target** ratios once sparse-on-wire ships:
+
+| Network       | Compression     | 100 MB gradients (v2.3 target) |
+|---------------|-----------------|--------------------------------|
+| Thunderbolt 4 | None            | 100 MB                         |
+| Ethernet      | TopK 10% + FP16 | ~5 MB                          |
+| WiFi          | TopK 1% + FP16  | ~500 KB                        |
 
 ## Requirements
 
@@ -165,7 +188,7 @@ read the Markdown source in `docs/`:
 git clone https://github.com/vikranthreddimasu/MacFleet.git
 cd MacFleet
 pip install -e ".[dev,all]"
-make test       # 425+ tests
+make test       # 447 tests
 make lint       # ruff + mypy
 ```
 

{macfleet-2.2.0rc1 → macfleet-2.2.1}/macfleet/__init__.py

@@ -6,11 +6,25 @@ Zero-config discovery. Framework-agnostic engines. Adaptive networking.
 """
 
 import logging
+from typing import TYPE_CHECKING
 
-__version__ = "2.2.0rc1"
+__version__ = "2.2.1"
 
 logging.getLogger(__name__).addHandler(logging.NullHandler())
 
+# Type checkers see real symbols here; runtime uses __getattr__ below to
+# keep heavy framework imports off the cold path.
+if TYPE_CHECKING:
+    from macfleet.compute.authz import TaskAuthorizationError, TaskAuthorizationPolicy
+    from macfleet.compute.models import RemoteTaskError, TaskFuture
+    from macfleet.compute.registry import task
+    from macfleet.engines.mlx_engine import MLXEngine
+    from macfleet.engines.torch_engine import TorchEngine
+    from macfleet.sdk.decorators import distributed
+    from macfleet.sdk.pool import Pool
+    from macfleet.sdk.train import train
+    from macfleet.training.data_parallel import DataParallel
+
 
 def __getattr__(name: str):
     """Lazy imports for heavy modules (avoid importing torch/mlx at module load)."""
@@ -38,6 +52,12 @@ def __getattr__(name: str):
     if name == "RemoteTaskError":
         from macfleet.compute.models import RemoteTaskError
         return RemoteTaskError
+    if name == "TaskAuthorizationError":
+        from macfleet.compute.authz import TaskAuthorizationError
+        return TaskAuthorizationError
+    if name == "TaskAuthorizationPolicy":
+        from macfleet.compute.authz import TaskAuthorizationPolicy
+        return TaskAuthorizationPolicy
     # v2.2 PR 7: @macfleet.task decorator
     if name == "task":
         from macfleet.compute.registry import task
@@ -55,5 +75,7 @@ __all__ = [
     "MLXEngine",
     "TaskFuture",
     "RemoteTaskError",
+    "TaskAuthorizationError",
+    "TaskAuthorizationPolicy",
     "task",
 ]