torchnative 0.0.1a0__py3-none-any.whl

torchnative/__init__.py

@@ -0,0 +1,4 @@
+"""torchnative — on-device test-time learning and federated learning.
+
+See docs/DESIGN.md for the design and its reasoning.
+"""

torchnative/adapt/__init__.py

@@ -0,0 +1,8 @@
+"""Test-time learning methods.
+
+TTL contains TTA contains TTT; the nesting is not flattened into sibling
+modules. Each method declares its own differentiation requirement rather than
+living in a directory named for one, because normalization calibration sits on
+both sides of that line -- recomputing statistics needs no backward pass,
+updating affine parameters by a loss does. See DESIGN.md §3.
+"""

torchnative/api/__init__.py

@@ -0,0 +1,24 @@
+"""Deployment, lifetime policy and device orchestration.
+
+The `torch` import is deferred rather than done at module scope. This
+distribution *provides* `torch` -- the vendored tree plus our `_C` -- rather
+than depending on someone else's, so at import time there may not be one yet:
+the wheels that carry the tree are not wired up, and until they are, importing
+`torchnative` should not require a PyTorch that this package is itself meant to
+supply. See DESIGN.md §2 and the README's Status section.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from torch import nn
+
+
+class TorchNativeAPI(object):
+    def __init__(self, *args, **kwargs):
+        pass
+
+    def deploy(self, model: "nn.Module"):
+        pass

torchnative/delta/__init__.py

@@ -0,0 +1,11 @@
+"""Weight deltas over base weights.
+
+Every adaptation method produces one of these; they differ only in lifetime
+and destination. The boundary is ``model(x)`` -- state a model manages inside
+its own forward (fast weights, caches) is the model's, not ours.
+
+Lifetime names are deliberately unnamed for now. They are driven by system
+events -- backgrounding, user switch, sync window -- not by the domain
+boundaries a benchmark hands you, and they get chosen once the first
+integration shows the real usage. See DESIGN.md §3.
+"""

torchnative/kernels/__init__.py

@@ -0,0 +1,7 @@
+"""Bundle-backed resolver for the Hugging Face ``kernels`` contract.
+
+The contract is adopted as-is; only resolution is inverted. On mobile, kernel
+variants are selected and compiled at build time and read from the app bundle,
+because iOS will not execute downloaded native code. Desktop keeps resolving
+against the Hub. See DESIGN.md §8.
+"""

torchnative/nn/federated/__init__.py

@@ -0,0 +1,7 @@
+"""Federated learning.
+
+A layer above the single-device adaptation methods, not a sibling of them: the
+local step is the same mechanism, with aggregation, transport and privacy on
+top. Its dependencies stay behind the ``federated`` extra so that using
+adaptation alone does not pull in the aggregation stack. See DESIGN.md §3.
+"""

torchnative-0.0.1a0.dist-info/METADATA

@@ -0,0 +1,293 @@
+Metadata-Version: 2.4
+Name: torchnative
+Version: 0.0.1a0
+Summary: Run the real PyTorch ecosystem on device -- not a reimplementation of it
+Author: thisisthepy
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/thisisthepy/torchnative
+Project-URL: Documentation, https://github.com/thisisthepy/torchnative/tree/develop/docs
+Project-URL: Source, https://github.com/thisisthepy/torchnative
+Keywords: pytorch,on-device,edge,federated-learning,test-time-adaptation
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Rust
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Operating System :: Android
+Classifier: Operating System :: iOS
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: POSIX :: Linux
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: federated
+Provides-Extra: test
+Requires-Dist: torch<2.14,>=2.13; extra == "test"
+Dynamic: license-file
+
+<div align="center">
+
+# torchnative
+
+**Run the real PyTorch ecosystem on device — not a reimplementation of it.**
+
+[![PyPI](https://img.shields.io/pypi/v/torchnative?color=blue)](https://pypi.org/project/torchnative/)
+[![Python](https://img.shields.io/badge/python-3.13%2B-blue)](https://www.python.org/)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![Platforms](https://img.shields.io/badge/platforms-macOS%20%7C%20Linux%20%7C%20Android%20%7C%20iOS-lightgrey)](#install)
+[![Status](https://img.shields.io/badge/status-pre--alpha-orange)](#status)
+
+</div>
+
+---
+
+`torchnative` replaces PyTorch's compiled core — `torch._C` — with a native extension, so the
+genuine `torch` and `transformers` packages run on a phone the way they run on a workstation.
+
+Models are not ported, converted, or re-expressed. They are imported.
+
+```python
+from transformers import AutoModelForCausalLM     # the real one
+model = AutoModelForCausalLM.from_pretrained("...")
+model.generate(...)                                # on the device
+```
+
+> [!WARNING]
+> **Pre-alpha.** The operator layer matches upstream PyTorch numerically and 15 of 20 tested
+> architectures reach zero missing operators — but `import transformers` does not work yet, no
+> checkpoint has ever been loaded, and no device has run the built artefact.
+> See [Status](#status) before depending on this.
+
+---
+
+## Why not a reimplementation
+
+Every other route to on-device inference re-expresses the model somewhere else.
+
+| | approach | cost |
+|---|---|---|
+| llama.cpp | architectures rewritten in C++ | each new architecture is a porting task |
+| ExecuTorch · CoreML | ahead-of-time compiled graph | export step, and what runs is not what you wrote |
+| MLC | lowered to its own runtime | same |
+| **torchnative** | **the real Python package** | **the substrate is hard; architectures are free** |
+
+The reason nobody runs the real thing is that `torch._C` cannot be built for mobile. PyTorch's own
+build sets `INTERN_BUILD_MOBILE` for any Android or iOS toolchain, and that path forces
+`BUILD_PYTHON` off — so the mobile build is structurally incapable of producing the Python
+extension module the Python package needs.
+
+`torchnative` supplies that module instead. Everything above it is upstream source, unmodified.
+
+---
+
+## What it does
+
+### 1 · LLM inference
+
+Run `transformers` models directly. No conversion step, no per-architecture port — if
+`transformers` supports it and the operators are covered, it runs.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B")
+tok   = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")
+
+out = model.generate(**tok("On-device inference is", return_tensors="pt"),
+                     max_new_tokens=32, do_sample=True)
+```
+
+**Today:** the operator layer under this is complete for 15 of 20 architectures, and Llama and
+GPT-2 match upstream **token for token and logit for logit** in both greedy and sampling mode.
+The `from_pretrained` path itself is still blocked — see [Status](#status).
+
+### 2 · Federated learning
+
+Devices train locally and share updates, not data. Federated averaging *is* collective
+communication, so this is built on `torch.distributed` rather than beside it — broadcast the
+model, gather the updates, weighted all-reduce.
+
+```python
+from torchnative.nn import federated
+
+engine = federated.Engine(model, rounds=..., aggregator=federated.FedAvg())
+engine.participate()          # local epochs, then contribute a delta
+```
+
+**Today:** planned. `torch.distributed` is being implemented from `world_size = 1` upward, which
+is a truthful description of a single device rather than a stub.
+
+### 3 · Test-time adaptation & training
+
+A model that ships to a device meets data the training set never had. TTA, TTT and the wider
+test-time learning family let it adapt in place — and every method reduces to the same thing: a
+weight delta over base weights, differing only in lifetime and destination.
+
+```python
+from torchnative import adapt
+
+model = adapt.wrap(model, method=adapt.Tent())   # or TTT, memory-based, entropy-based
+model.online()                                   # adapt as it serves
+```
+
+**Today:** planned; the delta abstraction is specified in [`docs/DESIGN.md`](docs/DESIGN.md) §3.
+Lifetime is driven by system events — backgrounding, user switch, sync window — rather than by
+the domain boundaries a benchmark hands you.
+
+---
+
+## How it works
+
+```
+your code · transformers · torch/*.py        upstream Python, unmodified
+──────────────────────────────────────────
+torch._C                                     ← replaced
+  ├── _aten_dispatch                         the single door every operator passes
+  ├── Python spellings                       torch.mm, x.softmax(), F.linear, ...
+  └── kernels                                Rust, backed by candle
+──────────────────────────────────────────
+CPU today · Metal, Vulkan, NPU planned
+```
+
+**One door.** Every operator reaches its kernel through `_aten_dispatch`, and nothing bypasses
+it. That makes the surface measurable — an unimplemented operator names itself rather than
+failing downstream — and it gives graph capture, which NPU backends will need, exactly one place
+to attach.
+
+**Demand-driven.** Nothing is implemented because it might be needed. The shim refuses by name,
+the refusal names the next thing to build, and that list comes from running real models.
+
+**Stable ABI.** Built against CPython's limited API (`abi3-py313`), so one binary per platform
+loads on 3.13, 3.14 and later without a rebuild.
+
+---
+
+## Status
+
+<table>
+<tr><th align="left">Working</th><th align="left"></th></tr>
+<tr><td>ATen operators</td><td><b>91</b>, each compared against upstream</td></tr>
+<tr><td>Golden comparison cases</td><td><b>2095 / 2095</b> — values, shapes, dtypes</td></tr>
+<tr><td>Python spellings</td><td><b>204</b> verified against upstream signatures</td></tr>
+<tr><td>Architectures complete</td><td><b>15 of 20</b> measured</td></tr>
+<tr><td>Build targets</td><td>macOS · Linux · Android arm64 · iOS arm64</td></tr>
+</table>
+
+Complete: Llama · GPT-2 · Qwen2 · Mistral · Gemma · GPT-NeoX · OPT · MPT · StarCoder2 ·
+Persimmon · Cohere · StableLM · OLMo · Phi · BERT
+
+`uniform_` and `normal_` are **bit-identical** to upstream, and `multinomial` consumes the same
+generator stream — a seeded run reproduces exactly.
+
+**Not working yet**
+
+- `import transformers` fails — `torch.distributed` is unimplemented and an unguarded import
+  inside `torch._dynamo` reaches `dist.Store`. Every result above was measured against models
+  transcribed by hand.
+- No checkpoint has ever been loaded. All weights so far are randomly initialised.
+- Mobile is link-verified only — the artefacts build and link, but no device has loaded one.
+- CPU only. No GPU or NPU backend.
+
+Tracked with the measurements behind them in [`docs/DESIGN.md`](docs/DESIGN.md) §11.1.
+
+---
+
+## Verification
+
+Correctness here means *agreeing with upstream PyTorch*, so the strategy is comparison rather
+than assertion.
+
+| | |
+|---|---|
+| **Golden comparison** | Every operator runs on both upstream torch and this shim, compared on value, shape and dtype. It has caught a `float16` GEMM accumulating in `float16` where torch accumulates in `float32`, `cumsum` routed through the wrong kernel, and integer overflow where torch refuses. |
+| **The harness tests itself** | `--self-test` injects a fault shaped like a plausible misimplementation at each comparator and fails if the comparator accepts it — 11 comparators × 11 fault modes, with any comparator never exercised reported as failure. It found that the previous fault injection reached exactly one case out of 1781. |
+| **Tokens are not enough** | A wrong `gelu` approximation produced *identical tokens* while logits differed by 5.9e-04. End-to-end tests compare logits too, with a tolerance measured to sit between normal float32 noise and that failure. |
+
+```sh
+sh rust/torch_c/pytests/run.sh                  # smoke tests + harness self-test
+python tools/golden/compare.py                  # golden comparison against upstream
+python rust/torch_c/pytests/verify_schemas.py   # signature tables vs upstream
+```
+
+---
+
+## Roadmap
+
+The next milestone is the device abstraction, because everything waits on it — a distributed rank
+needs a device to point at, and every accelerator attaches there.
+
+```
+torchnative.nn.federated    rounds · client selection · aggregation · dropout
+  └ torch.distributed       ProcessGroup · collectives (transport)
+      └ backends            ours, via register_backend
+          └ devices         CPU · Metal · Vulkan · NPU
+```
+
+| | |
+|---|---|
+| **Device abstraction** | `torch.device`, per-device dispatch. Everything else waits on it. |
+| **Metal** | candle already has the backend; disabled here for build isolation, not absent. |
+| **`torch.distributed`** | From `world_size = 1` upward. Unblocks `transformers` as a side effect. |
+| **Android GPU** | No candle backend and no `vulkan` slot in the `kernels` contract — genuinely new work. |
+| **NPU** | ANE, NNAPI and QNN take a whole graph ahead of time, so this needs a capture layer rather than another device. The single door is where it attaches. |
+
+---
+
+## Install
+
+> Not published yet. The alpha will carry platform wheels for macOS, Linux, Android and iOS —
+> the extension is native, so `py3-none-any` is not the shape this ships in.
+
+```sh
+pip install torchnative
+```
+
+### Building from source
+
+Requires a Rust toolchain and CPython 3.13+.
+
+```sh
+bash vendor/vendor_torch.sh     # assemble the vendored torch tree
+bash vendor/install_shim.sh     # build the extension and install it
+```
+
+Cross-compilation is documented in [`docs/RUST_CROSSBUILD.md`](docs/RUST_CROSSBUILD.md),
+including the PyO3 configuration iOS needs in order not to link `libpython`.
+
+---
+
+## Repository layout
+
+```
+torchnative/     the Python library
+rust/torch_c/    the torch._C replacement (Rust · PyO3 · candle)
+tools/golden/    the upstream comparison harness
+vendor/          scripts that assemble the vendored torch tree (not checked in)
+docs/            design, measurements, and the reasoning behind open decisions
+```
+
+`docs/` is written to be read. It records what was measured, what was assumed, and where an
+earlier conclusion turned out to be wrong — corrections are left visible rather than edited away.
+Start with [`DESIGN.md`](docs/DESIGN.md); [`SURFACE_HONESTY.md`](docs/SURFACE_HONESTY.md) and
+[`HARNESS.md`](docs/HARNESS.md) show the standard the rest aims for.
+
+---
+
+## Related
+
+- [PythonMultiplatform](https://github.com/thisisthepy/PythonMultiplatform) — embeds CPython 3.13
+  into Kotlin Multiplatform; the deployment target for this library
+- [pypackpack](https://github.com/thisisthepy/pypackpack) — the build and bundling tool
+- [Hugging Face `kernels`](https://github.com/huggingface/kernels) — the fused-kernel contract
+  this adopts, with resolution moved from runtime download to build time, since downloading
+  executable code is not permitted on every target platform
+
+---
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+PyTorch is vendored under its own BSD-3-Clause license. The vendored tree is assembled at build
+time and is not redistributed in this repository.

torchnative-0.0.1a0.dist-info/RECORD

@@ -0,0 +1,12 @@
+torchnative/__init__.py,sha256=4L3O7dxx_sHBBQqLdxioc__EIyHF9bKEbaFLThhEVWI,130
+torchnative/adapt/__init__.py,sha256=ZzN3EtzvwH9FF1uFVxhR3fYcAQlwZi-CHjPDRqbqV-A,402
+torchnative/api/__init__.py,sha256=-OK1hXvpwRswmH_5Bozere1RihBCYPYUxSpuVZa0wQU,761
+torchnative/delta/__init__.py,sha256=sFED7OU2Impia23LrHoLFGztMJRi4Bxe1eVAe948Caw,529
+torchnative/kernels/__init__.py,sha256=1wTSCnbcnhAHKDudhZB2VbuNhd-yY6GSm3Kpb8oOb5Q,344
+torchnative/nn/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+torchnative/nn/federated/__init__.py,sha256=_eGN5zBgFGTIi4cuoQjCmUVotpEYb4cJ6bGW-cF_GpU,332
+torchnative-0.0.1a0.dist-info/licenses/LICENSE,sha256=nroSRfIaQQ-QqbfGvqhCGb11WebGmV-izo-l2ueLZMc,1068
+torchnative-0.0.1a0.dist-info/METADATA,sha256=pFgLEietzqtrT0jq4xe5EoLv7UFAgpoV5LrjDKgOUbk,12678
+torchnative-0.0.1a0.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+torchnative-0.0.1a0.dist-info/top_level.txt,sha256=AJ0pj2SvfDTRa2cnFHAXhpC72DNurvxHJa7RPR4Guvg,12
+torchnative-0.0.1a0.dist-info/RECORD,,

torchnative-0.0.1a0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

torchnative-0.0.1a0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 thisisthepy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

torchnative-0.0.1a0.dist-info/top_level.txt

@@ -0,0 +1 @@
+torchnative