physis-lm 0.1.2__tar.gz

physis_lm-0.1.2/CHANGELOG.md

@@ -0,0 +1,97 @@
+# Changelog
+
+## 0.1.2 (2026-07-12)
+
+Made with AI assistance. New work: a consequential bug fix that unblocks
+generalization, three roadmap items delivered (Sections 3a/3b/4), an Appendix-D
+numerical path, and an honestly-documented scale ceiling. No 0.1.0/0.1.1
+default behaviour changes except the S2 fix (a strict correctness improvement).
+
+- **Bug S2 fix (ON by default; strict improvement):** `AdaptiveSlotPool`
+  (Section 10.1) now RMSNorms its input before the K/V projections and
+  initializes WQs/WKs at 1/sqrt(C). Previously its pre-softmax scores were
+  std ~1e-7, so softmax was uniform across all M slot queries and the deep
+  bottleneck latent collapsed to M copies of one vector — which blocked any task
+  needing a per-position transformation routed through the bottleneck. After the
+  fix, a 3.7M and a 14.7M model both learn a Caesar-shift task to 100% held-out
+  byte and sequence accuracy (seed-disjoint test, 26^16 prompt space). This is
+  the first end-to-end generalization result in the project. Study S8; note S2.
+- **Section 3b — Hugging Face wrapper (`physis_lm.hf`):** `PhysisLMConfig` +
+  `PhysisLMForByteModeling`, round-tripping through save/from_pretrained and
+  registering with `AutoModel`. Non-autoregressive: `generate()` raises and points
+  at `generate_cpd`. Optional import. 5 tests.
+- **Section 3a — JAX parity backend (`physis_lm.jax_backend`):** primitive layers
+  + a full PhysisBlock as pure functions over torch-copied weights, parity-tested
+  to ~1e-10 in x64 (12 tests). Full end-to-end model NOT ported (note J1).
+- **Appendix D — power-iteration spectral norm** (`spectral_norm_method=
+  "power_iteration"`, `spectral_norm_power_iters`): converges to the exact SVD
+  sigma_max; surrogate gradient makes gradcheck fail by design (note D1). Fused
+  CUDA kernels remain out of scope (no GPU).
+- **Documented scale ceiling (study S8b):** on a 4GB CPU box, models above ~15M
+  die from transient OOM spikes mid-training (PLRB's content-dependent tensor
+  shapes fragment the allocator) at ~1GB steady RSS; `malloc_trim` + bs=4 made
+  14.7M stable. An environment limit, not an architecture result.
+- **Real-text language result (study S9), reported honestly:** on 1.1M bytes of
+  Shakespeare, an 11M model plateaus at ~4.8 bits/byte (character marginals; its
+  samples are a repeated character) within ~150 steps, across three chunk lengths
+  and two batch sizes. Cause is compute starvation (CPU sees well under one epoch),
+  not the architecture; whether it models language at real scale is untested here.
+- **Any-to-Physis transplant: deferred.** Prototyped but not shipped in this
+  version (not ready); design retained in docs/SECTIONS_3_4_DESIGN.md.
+
+## 0.1.1 (2026-07-12)
+
+Made with AI assistance. Fixes for four documented findings; no existing 0.1.0
+behaviour changes except the F5 initialization fix (a strict improvement).
+Each fix was re-reproduced against 0.1.0 before being written and is covered by
+a new test. License changed MIT -> Apache-2.0.
+
+- **F5 fix (ON by default):** norm-preserving attention-pool downsampler (W_V
+  init std 1/sqrt(C) + learnable per-channel gain), so the residual stream no
+  longer collapses below RMSNorm's eps floor at init. `pool_norm_preserving=False`
+  restores the exact 0.1.0 init. CV 8.5's one-hot copy is preserved. Study S3 is
+  now a before/after table.
+- **R2 fix (opt-in prototype):** `plrb_soft_routing` (+ `plrb_soft_routing_temp`)
+  adds a differentiable soft-routing path so W_route receives a real gradient —
+  the route Section 9.7 claims but the literal argmax does not provide. Off by
+  default; recovers the hard path as temp -> 0.
+- **F3 fix (opt-in prototype):** `generate_cpd(streaming=True, rehash_period=k)`
+  now runs instead of raising. Implements periodic PLRB re-hashing (Remark 15.1);
+  bit-identical to naive CPD at `rehash_period=1`. Does NOT reach the paper's
+  O(L_out) cost — the DDHH still recomputes each chunk (finding F3 stands, since
+  coarse levels cannot be cached). Study S7 gains a re-hash-drift continuation.
+- **F4 option (opt-in):** `fixed_depth_padding` pins ell/n0 to a constant graph.
+  0.1.1 investigated a true logit-level padding-length invariance and confirmed
+  it is structurally impossible here (extending the input shifts the output
+  placeholders); documented in study S6 and the README's new "Batching and
+  train/inference consistency" section.
+- **Thinker-mode (opt-in prototype, task Sec. 5):** `thinker_steps` and
+  `thinker_scratchpad_slots` add a parallelism-preserving latent-refinement loop
+  over the deep bottleneck (no autoregressive fallback). Default (thinker_steps=1,
+  no scratchpad) is byte-for-byte the single-pass model. Shipped as a mechanism
+  that is live and trainable, explicitly NOT a demonstrated reasoning gain — the
+  steps must be trained to do useful work (note K1).
+- Test suite: 1477 tests, all passing.
+- License: Apache-2.0 (LICENSE + NOTICE added; pyproject/README/PKG-INFO updated).
+
+## 0.1.0 (2026-07-11)
+
+First release. Made with AI assistance.
+
+- Complete, tested PyTorch reference implementation of the Physis-LM preprint:
+  core geometry/oracles, PhysisBlock, DDHH, PLRB (fixed hashing), bottleneck,
+  LP, output-to-input cross-attention, SConM (+ confidence streaming), losses,
+  the full three-stage training schedule, PTCC, CPD, LSR, PCC.
+- Data layer (txt/jsonl/csv/tsv/html/binary) with precise per-format rules and
+  specific error messages; two-stage padding split so the CV 15.3 requirement
+  is enforced inside the model and cannot be bypassed.
+- `physis-lm` CLI: info / selftest / train / generate / scale-study.
+- 1470 tests including the paper's own Computational Verifications re-executed
+  against this code, an end-to-end files->train->generate CLI test, gradchecks,
+  and hypothesis property sweeps.
+- Scale-study module + shipped report (docs/SCALE_FINDINGS.md), including a new
+  finding (F5): geometric residual-stream collapse through the pooling
+  projections at initialization.
+- Disclosed non-goals for 0.1.0: JAX parity backend, Hugging Face wrapper,
+  Appendix D CUDA kernels, PLRB learnable-hashing variant, streaming DDHH
+  (finding F3 makes the paper's version incorrect as specified).

physis_lm-0.1.2/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Omur Bera Isik
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

physis_lm-0.1.2/MANIFEST.in

@@ -0,0 +1,4 @@
+include README.md LICENSE NOTICE CHANGELOG.md pyproject.toml
+recursive-include docs *.md
+recursive-include tests *.py
+global-exclude __pycache__ *.pyc

physis_lm-0.1.2/NOTICE

@@ -0,0 +1,14 @@
+Physis-LM
+Copyright 2026 Omur Bera Isik
+
+This product includes software developed as a reference implementation of the
+Physis-LM preprint ("Physis-LM: A Parallel Non-Autoregressive Language Model
+with Hierarchical Latent Compression and Toward Guaranteed Output Consistency",
+Omur Bera Isik, 2026).
+
+This codebase was made with AI assistance. No trained checkpoint exists for this
+architecture (the paper itself trains nothing); everything here is a tested,
+CPU-runnable reference implementation, not a pretrained model.
+
+Licensed under the Apache License, Version 2.0. See the LICENSE file for the
+full license text.

physis_lm-0.1.2/PKG-INFO

@@ -0,0 +1,332 @@
+Metadata-Version: 2.4
+Name: physis-lm
+Version: 0.1.2
+Summary: Reference PyTorch implementation of the Physis-LM preprint (byte-native, non-autoregressive, hierarchical-hourglass language model). Made with AI assistance.
+Author: Omur Bera Isik
+License: Apache-2.0
+Keywords: language-model,non-autoregressive,byte-level,hierarchical,pytorch,research
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE
+Requires-Dist: numpy>=1.24
+Requires-Dist: torch>=2.1
+Provides-Extra: test
+Requires-Dist: pytest>=7.0; extra == "test"
+Requires-Dist: hypothesis>=6.0; extra == "test"
+Provides-Extra: hf
+Requires-Dist: transformers>=4.40; extra == "hf"
+Provides-Extra: jax
+Requires-Dist: jax>=0.4; extra == "jax"
+Requires-Dist: jaxlib>=0.4; extra == "jax"
+Dynamic: license-file
+
+# physis-lm
+
+Reference PyTorch implementation of the preprint **"Physis-LM: A Parallel
+Non-Autoregressive Language Model with Hierarchical Latent Compression and
+Toward Guaranteed Output Consistency"** (Omur Bera Isik, 2026) — a byte-native,
+non-autoregressive language model built around a Dynamic-Depth Hierarchical
+Hourglass (DDHH), a Physis Long-Range Bridge (PLRB), Length Parameters (LP), and
+a Soft Consensus Module (SConM).
+
+Made with AI assistance. Version 0.1.2.
+
+## What's new in 0.1.2
+
+- **The model now demonstrably generalizes on a task that requires routing a
+  transformation through the bottleneck — after fixing a bug that made it look
+  like it couldn't.** `AdaptiveSlotPool` (Section 10.1) was the only attention in
+  the model that did not RMSNorm its input before the K/Q projections; combined
+  with the shared small init, its pre-softmax scores were std ~1e-7, softmax was
+  uniform for all M slot queries, and the deep bottleneck latent Z collapsed to M
+  copies of one vector (measured: cross-slot cosine 1.0000). A *copy* task hid
+  this (the output->input cross-attention bypasses Z); a Caesar-*shift* task
+  exposed it — training sat at chance. **Fix (bug S2):** RMSNorm before the K/V
+  projections + a 1/sqrt(C) init on WQs/WKs. After it, a 3.7M and a 14.7M model
+  both learn shift+3 to **100% held-out byte AND sequence accuracy** on a
+  seed-disjoint test set (26^16 prompt space, so this is rule-learning, not
+  memorization). See `docs/SCALE_FINDINGS.md` study S8.
+- **On real text, it does NOT yet produce meaningful language — and the reason is
+  compute, not the architecture.** Trained on 1.1M bytes of Shakespeare
+  (next-chunk prediction, an 11M model on CPU), the model's loss falls to ~4.8
+  bits/byte within ~150 steps and then plateaus: it learns the marginal character
+  distribution (its samples are a single repeated character) and little context
+  structure. This reproduced across three chunk lengths and two batch sizes. A
+  byte model needs many epochs over even this small corpus to learn language, and
+  at ~0.5-2s/step on a 4GB CPU box the model sees well under one epoch; plain
+  cross-entropy (without the designed auxiliary losses) and the parallel-chunk
+  objective compound it. Whether the architecture models language at real scale
+  is untested here — gated by hardware, not answered. Study S9.
+- **Section 3b — Hugging Face wrapper (`physis_lm.hf`).** `PhysisLMConfig` +
+  `PhysisLMForByteModeling` round-trip through `save_pretrained`/`from_pretrained`
+  and register with `AutoModel`. It is non-autoregressive, so `generate()` RAISES
+  and points at `generate_cpd` rather than silently running HF's AR decoder.
+  Optional import; `transformers` is not a hard dependency.
+- **Section 3a — JAX parity backend (`physis_lm.jax_backend`).** The primitive
+  layers and a full `PhysisBlock`, ported as pure functions over torch-copied
+  weights and parity-tested to ~1e-10 in x64 (12 tests). Scope is stated plainly:
+  the full end-to-end hourglass/PLRB/SConM forward is **not** ported (note J1).
+- **Appendix D — power-iteration spectral norm (`spectral_norm_method=
+  "power_iteration"`).** The fused CUDA kernels remain out of scope (no GPU), but
+  the numerical method they would accelerate is implemented and selectable;
+  converges to the exact SVD sigma_max. Its autograd surrogate makes gradcheck
+  fail *by design* — documented, not hidden (note D1).
+- **Honest scale ceiling.** On the 4GB CPU sandbox, models above ~15M are killed
+  by transient out-of-memory spikes mid-training (PLRB's content-dependent tensor
+  shapes fragment the allocator) even at ~1GB steady RSS; `malloc_trim` + bs=4
+  made 14.7M stable, and beyond that needs more RAM or a GPU. This is an
+  environment limit, not an architecture result (study S8b).
+
+## What's new in 0.1.1
+
+Four of the reference's documented findings now have fixes. Each was
+re-reproduced against 0.1.0 before being touched, is covered by a new test, and
+— except F5, which only changes an initialization and is a strict improvement —
+is **off by default**, so every 0.1.0 result still reproduces exactly.
+
+- **F5 fix (ON by default):** the attention-pool downsampler is now
+  norm-preserving (W_V init std 1/sqrt(C) + a learnable gain), so the residual
+  stream no longer collapses below RMSNorm's eps floor at init. Strict
+  improvement; `pool_norm_preserving=False` restores the old init.
+- **R2 fix (opt-in prototype):** `plrb_soft_routing=True` gives PLRB a
+  differentiable soft-routing path so `W_route` actually trains — the gradient
+  route the paper claims but the literal argmax doesn't provide.
+- **F3 fix (opt-in prototype):** `generate_cpd(streaming=True)` now runs, via
+  periodic PLRB re-hashing (Remark 15.1). Exact (bit-identical to naive CPD) at
+  `rehash_period=1`. It does **not** reach the paper's O(L_out) cost — the DDHH
+  still recomputes each chunk, because its coarse levels genuinely can't be
+  cached (finding F3 stands) — and that is stated, not hidden.
+- **F4 option (opt-in):** `fixed_depth_padding=True` pins ell/n0 to a constant
+  graph. A true logit-level padding-length invariance was investigated and shown
+  structurally impossible for this architecture; see "Batching and
+  train/inference consistency" below.
+- **Thinker-mode (opt-in prototype):** `thinker_steps > 1` (and optional
+  `thinker_scratchpad_slots`) iterates the deep bottleneck over the latent as a
+  parallelism-preserving deliberation loop (no autoregressive fallback). The
+  mechanism is live and trainable; it is **not** a demonstrated reasoning gain —
+  the extra steps must be trained to do useful work. See "Thinker-mode" below.
+- License is now **Apache-2.0** (was MIT).
+
+Full detail and honest limits: `docs/IMPLEMENTERS_NOTES.md` (notes F3, F4, F5,
+R2) and `docs/SCALE_FINDINGS.md` (studies S3, S6, S7).
+
+## Read this first (honesty statement)
+
+**Physis-LM is currently in the demo stage; we cannot guarantee that everything
+works. If you train the model and get poor results, please be aware that this
+could be one of the contributing factors.**
+
+The paper's own title page states that **no version of this architecture has
+ever been assembled, trained, or evaluated end-to-end**, and that every quality
+claim in its Section 20 is an untested prediction. This package changes the
+first half of that sentence — the architecture is now assembled, correct at the
+paper's own toy scale, and heavily tested — and none of the second half:
+
+- **There are no pretrained weights.** Anywhere. `physis-lm train` really
+  trains (and toy-scale tests verify learning happens), but training a
+  competitive model needs the paper's 50-500B-byte budgets and GPUs.
+- Everything implemented is tested against the paper's own numbered claims;
+  everything not implemented raises a clear error or is listed below.
+- Three paper claims are **demonstrably false as written** (findings F3, F4,
+  F5) and one is self-contradictory (R2); 0.1.1 adds an honest fix or mitigation
+  for each (see "What's new" above) while the underlying findings are still
+  documented in full, not papered over. See `docs/IMPLEMENTERS_NOTES.md` and
+  `docs/SCALE_FINDINGS.md`.
+
+## Install
+
+```bash
+pip install physis-lm            # from a built wheel/PyPI
+# or, from a source checkout:
+pip install -e ".[test]" && pytest
+```
+
+Dependencies: `numpy`, `torch>=2.1` (CPU is fine — this package is developed
+and tested entirely on one CPU core). Python >= 3.10.
+
+## Quickstart — library
+
+```python
+import torch
+from physis_lm import PhysisCoreConfig, PhysisLM
+
+cfg = PhysisCoreConfig(C=32, M=8, H=4, W=8, Nmax=32, nref=2048,
+                       NB=2, K_sconm=2, droute=8, R=2, bucket_target=8)
+model = PhysisLM(cfg)
+
+ids = torch.tensor([list(b"hello physis, this is a byte-native model")])
+out = model(ids)                       # single parallel forward pass
+print(out.logits.shape)                # (1, Nmax, 256) — all positions at once
+print(out.sigma_score)                 # LP completion score (Section 12.6)
+
+text, info = model.generate_cpd(ids, Lmax=128)   # Chunked Parallel Decoding
+print(bytes(text.tolist()))
+```
+
+Training (the full Section 18.4 three-stage schedule, homoscedastic loss
+weighting, curriculum, checkpointing):
+
+```python
+from physis_lm import data as D
+from physis_lm.torch_backend import train as TR
+
+ds = D.dataset_from_paths(["my_corpus/"], context_len=128,
+                          Nmax=cfg.Nmax, pad_byte=cfg.pad_byte)
+provider = D.batch_provider_from_dataset(ds, batch_size=4, pad_byte=cfg.pad_byte)
+history = TR.train(model, provider, TR.TrainSettings(total_steps=1000,
+                                                     ckpt_dir="ckpt"))
+```
+
+The model pads internally to exactly `M * r**ell` (the CV 15.3 hard
+requirement), so **you never need to understand DDHH depth arithmetic**: any
+input length up to `cfg.max_supported_raw_length() - Nmax` just works, and
+anything else raises an error telling you to increase `nref`.
+
+## Quickstart — CLI
+
+```bash
+physis-lm selftest                     # end-to-end pipeline check (~5 s)
+physis-lm info --size base             # configs + parameter-count oracles
+physis-lm train --data my_files/ --steps 500 --ckpt-dir ckpt
+physis-lm generate --ckpt ckpt/final.pt --prompt "hello" --max-bytes 200
+physis-lm scale-study                  # regenerate docs/SCALE_FINDINGS.md
+```
+
+`train` accepts `.txt`, `.jsonl` (`--text-field`), `.csv`/`.tsv`, `.html`
+(`--html-mode strip|raw`), and raw binary files; malformed inputs fail with
+messages naming the file, line, and problem.
+
+## Status table
+
+Per-module status; each module's docstring carries the same note, and
+`docs/IMPLEMENTERS_NOTES.md` holds the full list of paper
+ambiguities/contradictions and the decision taken for each (tags like "note B7"
+below resolve there).
+
+| Module | Paper | Status | Tested by | Limitations / notes |
+|---|---|---|---|---|
+| `core` geometry, padding, output slice | 7.2, Rem. 7.1, CV 15.3, Rem. 15.4 | implemented | test_core (incl. the literal CV 15.3 sweep, hypothesis sweeps) | ell >= 1 clamp (G1) |
+| `core` complexity + parameter oracles | 7.7, C.1/C.6, Prop 9.1, 17.5, Sec. 1, 12.2, 18.7, App. A | implemented | test_core reproduces every printed table/figure | paper's 130,808 misprint recorded (P1) |
+| `core` LSH oracle + config | 9.3, Prop 9.2, CV 9.3, App. A.1 | implemented | test_core (collision/recall sims; config rejections) | Omega = f(seed, round, B) (R1) |
+| RMSNorm / RoPE / SwiGLU / spectral / local attention | 6.2-6.6, 28.1-28.4, CV 8.3, Prop 8.2 | implemented | test_torch_layers (banded==O(n^2) oracle, CV 8.3 both halves, circulant Jacobian, gradchecks) | banded impl is O(nW) memory; no CUDA kernels (App. D out of scope, no GPU) |
+| PhysisBlock, pooling, upsample, bottleneck, LP | 6.5, 7.3-7.4, 10, 12, CV 8.5, CV 12.6 | implemented | test_torch_blocks (CV 8.5 both cases, all three LP propositions vs autograd) | B3, B8-B10 readings; 0.1.2 fixes AdaptiveSlotPool bottleneck-collapse bug S2 (RMSNorm + 1/sqrt(C) init) |
+| PLRB | 8-9, 19.5, C.6 | implemented (fixed hashing; 0.1.1 opt-in soft routing) | test_torch_plrb (segmented == mask oracle, sentinel, zero W_route grad by default, soft-routing gives W_route a real gradient) | default trains everything except W_route as literally specified (R2); 0.1.1 `plrb_soft_routing` fixes this (prototype); learnable-Omega variant a disclosed stub (R3); dense hashing = C.6 trap at large n (study S5) |
+| SConM + streaming readout | 14, CV 14.2, Lemma 14.8, D.2 | implemented | test_torch_sconm (LOO exact single/multi-head, LSE recovery, vertex bound, scopes) | no contraction claim — conditional on unmeasured L_g (S1; studies S1/S2); 0.1.2 adds opt-in power-iteration spectral norm (note D1) |
+| Full model, integration, LSR, CPD | App. B, CV 15.2/15.3, Rem. 15.4, 7.8.2, 15 | implemented | test_torch_model (the paper's own sixteen-join toy-scale integration test, ablations, LSR bit-exactness, CPD incl. streaming exact-at-period-1) | 0.1.1 streaming CPD runs via periodic PLRB re-hash, exact at rehash_period=1, but does not reach O(L_out) — DDHH still recomputes (finding F3); CPD stops gracefully at the context limit |
+| Losses, schedules, training loop, PTCC | 16.3, 18, 19, CV 18.2/18.4/18.10 | implemented | test_training (a real toy run that learns + checkpoint round-trip), test_data_ptcc_pcc | L1, T1-T3 decisions; no mixed precision / Flash Attention / grad checkpointing (throughput devices, no GPU here) |
+| PCC | 24, CV 24.2 | implemented | test_data_ptcc_pcc (losslessness incl. adversarial streams, tier accounting) | ratios are corpus claims — only CV 24.2's qualitative pattern asserted (C1) |
+| Data layer + CLI | task Sec. 5 | implemented | test_data_ptcc_pcc, test_cli (end-to-end train->generate on real files) | formats defined precisely in data.py's docstring |
+| Scale studies | task Sec. 2 | implemented | test_scale_studies | untrained instantiations only; see report header |
+| Thinker-mode (deliberation loop) | task Sec. 5 | prototype (opt-in) | test_torch_model (default == single-pass, live + trainable, parallelism preserved) | mechanism only; genuine reasoning gains require training the steps (note K1) |
+| JAX parity backend (layers + PhysisBlock) | task Sec. 3a | implemented (layer/block level) | test_jax_parity (12 tests, torch-vs-JAX at copied weights, ~1e-10 in x64) | full end-to-end hourglass/PLRB/SConM forward NOT ported — tested foundation only (note J1) |
+| Hugging Face wrapper | task Sec. 3b | implemented | test_hf (config round-trip, save/from_pretrained identical logits, AutoModel, non-AR contract) | optional import; `generate()` raises -> `generate_cpd`; no pretrained weights |
+| Appendix D spectral-norm kernel | App. D | power-iteration path implemented; fused CUDA kernels out of scope | test_torch_sconm (converges to SVD sigma_max; gradient flow) | no GPU for fused kernels; gradcheck fails by design (surrogate gradient, note D1) |
+| Any-to-Physis weight transplant | task Sec. 4 | **deferred to a later release** | — | intentionally not shipped in this version; design in docs/SECTIONS_3_4_DESIGN.md |
+
+Test suite: **1497 tests, all passing** (`pytest -q`), dominated by cheap
+parametrized pure-math sweeps plus hypothesis-generated cases; every
+Computational Verification remark in the paper that can run on CPU is
+re-executed against this codebase's actual code.
+
+## The findings and their 0.1.1 fixes
+
+- **F3** — Streaming DDHH's premise ("levels above j* do not change") is false:
+  spectral mixing is a global circular convolution at every level (quantified in
+  study S7). *0.1.1:* `generate_cpd(streaming=True)` now runs — bit-identical to
+  naive CPD at `rehash_period=1`, and periodic PLRB re-hashing (Remark 15.1) for
+  larger periods — but the DDHH still recomputes each chunk, so it does not reach
+  the paper's O(L_out) streaming cost. The finding stands; the fix is honest
+  about what it does and doesn't buy.
+- **F4** — the literal Section 19.5 claim that padding to a longer batch length
+  leaves behavior identical is structurally impossible (padding changes ell, and
+  extending the input shifts the output placeholders). The achievable invariant
+  — batch-content independence at fixed shape — holds to 1e-10 and is tested
+  (study S6). *0.1.1:* `fixed_depth_padding=True` pins ell/n0 to a constant
+  graph; a true logit-level length invariance was investigated and shown
+  impossible here.
+- **F5** (found by this implementation) — at initialization the residual stream
+  collapsed geometrically through the attention-pool downsamplers (~x0.02-0.09
+  per level), pushing deep levels under RMSNorm's eps floor (study S3). *0.1.1
+  (ON by default):* the norm-preserving downsampler holds every level above the
+  floor across the whole grid measured; S3 is now a before/after table.
+- **R2** — Section 9.7 is self-contradictory: it says the routing argmax "is not
+  differentiated through" yet "gradients flow normally through W_route". The
+  literal argmax gives W_route exactly zero gradient (tested), so the default
+  fixed-hashing path trains everything except W_route. *0.1.1:*
+  `plrb_soft_routing=True` supplies a differentiable soft-routing path so
+  W_route trains (prototype; off by default).
+
+## Batching and train/inference consistency
+
+Because n0 = M * r**ell depends on the padded input length, this architecture's
+output for a fixed sequence is **not** invariant to how much padding you add:
+different padded lengths mean a different number of DDHH levels and a different
+spectral DFT length (finding F4). Two practical consequences:
+
+1. The guarantee you *can* rely on is **batch-content independence at a fixed
+   batch shape**: a sequence's logits do not depend on which other sequences
+   share its batch, as long as the batch's padded length is the same. This holds
+   to floating-point precision and is tested.
+2. To keep training and inference consistent, either (a) use the **same nref and
+   the same batch-shaping discipline** in both — so a given input lands at the
+   same ell in training and at inference — or (b) set
+   `PhysisCoreConfig(fixed_depth_padding=True)`, which pins every input to
+   ell_max / n0 = M*r**ell_max. Option (b) makes the computation graph constant
+   across input lengths (removing the batch-shape-dependent DFT-length drift) at
+   the cost of always running at maximum depth; it does not make two
+   different-length inputs produce identical logits (that is impossible here),
+   but it removes padding-length as a source of train/inference mismatch.
+
+## Thinker-mode (prototype)
+
+`PhysisCoreConfig(thinker_steps=T, thinker_scratchpad_slots=S)` turns on an
+opt-in deliberation loop (task Section 5). Instead of a single deep-bottleneck
+pass over the compressed latent Z, the bottleneck is applied T times, adding a
+learnable per-step code each iteration, with S optional learnable "scratchpad"
+latent slots as working memory (dropped before readout so the output shape is
+unchanged). Every latent slot is refined simultaneously at each step, and the
+loop iterates over deliberation steps, never over output tokens — so the
+architecture's parallelism is preserved and there is **no** autoregressive
+fallback. `thinker_steps=1` with no scratchpad (the default) is byte-for-byte
+the single-pass model.
+
+What this is and isn't: the mechanism is implemented, live (non-zero step codes
+change the output), and trainable (gradients reach the step codes and
+scratchpad) — all tested. It is **not** a demonstrated reasoning improvement.
+The per-step codes are zero at initialization, so an untrained model's first
+step equals the ordinary pass and further steps just re-apply the bottleneck;
+making the extra steps do genuinely new work is a *training* problem (train with
+`thinker_steps > 1`, most plausibly with a ponder/ACT-style objective). This
+release ships the mechanism honestly labeled as a prototype, not a claim that
+iterating an untrained bottleneck reasons better. See `docs/IMPLEMENTERS_NOTES.md`
+note K1.
+
+## Repository layout
+
+```
+physis_lm/
+  core.py            geometry, cost/parameter oracles, LSH math, config (no torch)
+  data.py            byte-level corpus loading + batching (stage-1 padding only)
+  pcc.py             Physis Context Compression (Section 24; no torch)
+  cli.py             the `physis-lm` command
+  scale_studies.py   Section-2 measurements -> markdown report
+  torch_backend/
+    layers.py blocks.py plrb.py sconm.py model.py losses.py train.py ptcc.py
+tests/               1470 tests; conftest.py holds the paper's toy scale
+docs/                PLAN.md (pre-implementation plan), IMPLEMENTERS_NOTES.md,
+                     SCALE_FINDINGS.md
+```
+
+## Citing
+
+If you use this implementation, cite the Physis-LM preprint (Omur Bera Isik,
+2026). This codebase: `physis-lm` 0.1.2, made with AI assistance, Apache-2.0 license.