latentport 0.2.0__tar.gz

latentport-0.2.0/.dockerignore

@@ -0,0 +1,11 @@
+**
+!src/
+!src/**
+!docker/
+!docker/**
+!pyproject.toml
+!README.md
+!LICENSE
+!MANIFEST.in
+**/__pycache__
+**/*.egg-info

latentport-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,26 @@
+# Contributing
+
+Start with a focused issue or a small patch. Include a runnable reproduction for fixes.
+
+```powershell
+python -m pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cpu
+python -m pip install -e ".[gguf,dev]"
+python -m pytest -q
+```
+
+Lightweight packaging checks also run without inference extras:
+`python -m unittest discover -s tests -p test_distribution.py`.
+
+GPU checks are separate and require an idle supported GPU. Use `scripts/smoke_gguf.py`
+for state/MTP correctness and `scripts/check_long_chat.py --policy cached_handoff`
+for the bounded long-context conversation check. Both accept explicit model paths and
+require `--gpu-is-free`. Keep large weights, captured states and build output outside
+this repository. Never submit credentials or private conversation snapshots.
+
+Behavioral contracts matter: no silent target replay; exact compatibility checks;
+immutable published states; independent branches; rollback before publishing a speculative
+state; and honest timing including source and translation work.
+
+Native changes must preserve the pinned ABI or explicitly version it. Follow
+[the native guide](docs/native-abi.md). New platforms and bridges need their own
+qualification; passing CPU mocks does not establish real-model correctness.

latentport-0.2.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

latentport-0.2.0/MANIFEST.in

@@ -0,0 +1,12 @@
+include LICENSE README.md
+recursive-include native *.h *.cpp *.cu *.py CMakeLists.txt
+recursive-include native LICENSE*
+recursive-include scripts *.py *.ps1
+recursive-include docs *.md
+recursive-include docs *.json
+recursive-include tests *.py
+recursive-include examples *.py
+recursive-include docs/assets *.gif
+include CONTRIBUTING.md
+recursive-include docker Dockerfile *.txt
+include .dockerignore

latentport-0.2.0/PKG-INFO

@@ -0,0 +1,247 @@
+Metadata-Version: 2.4
+Name: latentport
+Version: 0.2.0
+Summary: Programmable full-state transfer between hybrid language models
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://latentport.com
+Project-URL: Repository, https://github.com/LatentPort/latentport
+Project-URL: Documentation, https://github.com/LatentPort/latentport/tree/main/docs
+Project-URL: Models, https://huggingface.co/latentport
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: gguf
+Requires-Dist: huggingface_hub<2,>=1.33; extra == "gguf"
+Requires-Dist: safetensors<1,>=0.8; extra == "gguf"
+Requires-Dist: numpy<3,>=2.3; extra == "gguf"
+Requires-Dist: torch<3,>=2.11; extra == "gguf"
+Requires-Dist: transformers==5.17.0; extra == "gguf"
+Provides-Extra: bf16
+Requires-Dist: latentport[gguf]; extra == "bf16"
+Requires-Dist: accelerate<2,>=1.15; extra == "bf16"
+Requires-Dist: fla-core==0.6.0; extra == "bf16"
+Requires-Dist: causal-conv1d==1.7.0; extra == "bf16"
+Provides-Extra: reference
+Requires-Dist: latentport[gguf]; extra == "reference"
+Requires-Dist: accelerate<2,>=1.15; extra == "reference"
+Requires-Dist: bitsandbytes<0.51,>=0.50.2; extra == "reference"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: build>=1; extra == "dev"
+Requires-Dist: twine>=6; extra == "dev"
+Provides-Extra: compact
+Requires-Dist: latentport[bf16]; extra == "compact"
+Dynamic: license-file
+
+# LatentPort
+
+**LatentPort makes LLM internal states programmable.**
+
+We're building a library of learned state transformations that enable models and
+agents to transfer, compact, optimise and combine their internal states during
+inference.
+
+This allows developers to improve model performance, reduce inference costs and
+build longer-running agents without repeatedly processing the same context.
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/LatentPort/latentport/main/docs/assets/latentport-ascii.gif" alt="LatentPort: programmable model state" width="640">
+</p>
+
+[Website](https://latentport.com) | [Get started](#get-started) | [Models](https://huggingface.co/latentport) | [Cookbook](https://github.com/LatentPort/latentport/blob/main/docs/cookbook.md) | [API](https://github.com/LatentPort/latentport/blob/main/docs/api.md)
+
+![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue)
+[![Runtime: 0.1](https://img.shields.io/badge/runtime-0.1-blue)](https://github.com/LatentPort/latentport/releases/tag/lpcompact-v0.1)
+
+## A library of state operations
+
+A model's internal state carries what it has already processed. LatentPort exposes
+that state as an object developers can transform, persist, reuse and branch.
+Learned operators work directly on state tensors, allowing inference to continue
+from the transformed state.
+
+| Operation | Purpose | Available today |
+| --- | --- | --- |
+| **Transfer** | Move working context into another model's state space. | Learned Qwen3.5-4B to Qwen3.8-27B bridge via `bridge.translate(state)`. |
+| **Compact** | Reduce attention memory while retaining the model's recurrent and convolutional state. | LPCompact 0.1 for Qwen3.8-27B via `state.compact()`. |
+| **Optimise** | Improve a state for the task or inference objective ahead. | Part of the roadmap. |
+| **Combine** | Bring information from multiple states into a shared working state. | Part of the roadmap. |
+
+The shared runtime also provides capture, append, save, load and fork operations.
+Snapshots are immutable, so branches can share a starting context while advancing
+independently. Each learned transformation is trained for its supported model
+architecture or model pair.
+
+## Get started
+
+Install the lightweight Python package with Python 3.11 or later:
+
+```shell
+python -m pip install latentport
+latentport doctor
+```
+
+The base package includes the API, diagnostics and demo interface. Inference
+dependencies are optional: see [installation](https://github.com/LatentPort/latentport/blob/main/docs/installation.md) for
+`latentport[gguf]`, `latentport[bf16]` and CUDA PyTorch setup.
+
+### Try the browser demo
+
+On a qualified NVIDIA GPU host with Docker GPU access configured:
+
+```shell
+docker run --rm --gpus all -p 127.0.0.1:8768:8768 -v latentport-cache:/cache ghcr.io/latentport/latentport:0.2.0-gguf-cu128-sm120
+```
+
+Open **http://localhost:8768**. The demo prepares the models, transfers context,
+forks specialist sessions, and checks a generated repair with executable tests.
+The initial transfer profile requires an SM120 GPU with at least 32 GB VRAM and
+approximately 22.10 GB of model downloads; subsequent runs reuse the volume.
+Windows Docker GPU execution requires the WSL 2 backend.
+
+**Release status:** the Python package is available on [PyPI](https://pypi.org/project/latentport/).
+The GHCR images are pending GPU qualification and publication; until then, use
+the [local image build](https://github.com/LatentPort/latentport/blob/main/docs/installation.md).
+
+Choose the setup for your operation:
+
+- **Compaction:** [BF16 and GGUF setup](https://github.com/LatentPort/latentport/blob/main/docs/compaction.md), including the BF16 kernel dependencies and native GGUF runtime.
+- **Model transfer:** [Windows GGUF setup](https://github.com/LatentPort/latentport/blob/main/docs/installation.md) and [conversation recipes](https://github.com/LatentPort/latentport/blob/main/docs/cookbook.md).
+
+Set `LATENTPORT_HOME` to a directory with enough space for downloaded models and
+caches. The runtime downloads transformation weights when needed; base models are
+separate artifacts. Prebuilt GGUF runtimes are available for Windows x64 / CUDA 13
+and Linux x64 / CUDA 12.8 on Blackwell SM120.
+
+## Compact a running model's state
+
+[LPCompact-Qwen3.8-27B](https://huggingface.co/latentport/LPCompact-Qwen3.8-27B)
+provides a learned selector through a single operation:
+
+```python
+state = state.compact()
+```
+
+Given an agent's accumulated `history`, continue from its compacted state:
+
+```python
+import latentport as lp
+
+with lp.load_model(backend="bf16") as model:
+    state = model.prefill(history)
+    state = state.compact()
+    answer = model.session(state).ask("Continue the task.")
+    print(answer.text)
+```
+
+The default policy retains one eighth of attention KV positions on histories with
+at least 65,536 retained positions. A 4x setting is available with
+`state.compact(ratio=4)`. Recurrent state, convolution state and the original
+logical positions are preserved. Compaction uses no text summary or historical
+token replay.
+
+One selector checkpoint serves Qwen3.8-27B BF16 and GGUF weight quantizations.
+For GGUF, use `backend="gguf"` and supply `model_path` with your chosen base-model
+file. See the [compaction guide](https://github.com/LatentPort/latentport/blob/main/docs/compaction.md) for context sizing, policy
+details and the `state.compaction` receipt.
+
+## Transfer context between models
+
+The current transfer bridge lets Qwen3.5-4B read a context and Qwen3.8-27B continue
+from the translated state. It transfers attention K/V, recurrent memory,
+convolution history and the terminal hidden state.
+
+```python
+import latentport as lp
+
+with lp.load(backend="gguf") as port:
+    port.warm()
+    context = port.source.capture([("user", "Repository context: ...")])
+    transferred = port.bridge.translate(context)
+    answer = port.target.session(transferred).ask("Explain the repository.")
+    print(answer.text)
+```
+
+For a complete conversation, `port.composite.session()` handles the initial
+handoff and retains the target's state across turns. The default `cached_handoff`
+policy translates once, then processes new input on the target. An explicit
+`strict` policy keeps the source model responsible for new user prompts.
+
+The [cookbook](https://github.com/LatentPort/latentport/blob/main/docs/cookbook.md) covers both policies, answer continuation,
+streaming and MTP. Existing local GGUF files can be supplied through `source_path`
+and `target_path`.
+
+## Reuse, branch and resume
+
+The same state abstraction supports persistence and independent continuations:
+
+```python
+review_state, planning_state = state.fork(2)
+state.save("saved-state")
+restored = lp.State.load("saved-state")
+```
+
+Prepare a repository once, let different agents work from the same snapshot, or
+save a long-running session and resume it later. Restoring a state reuses the
+captured context. Appending new tool results processes the new input.
+
+Each snapshot records its exact model identity and runtime compatibility. Forks
+share immutable storage; execution uses the selected backend's scheduling.
+
+## Measured results
+
+| Transformation | Measurement | Validation |
+| --- | --- | --- |
+| LPCompact, 128K history at 8x | Attention KV tensors reduced from **8 GiB to 1 GiB**. | BF16, Q4_K_M and Q8_0 tested at 64K and 128K on RTX PRO 6000; continued generation, save/load, forks and repeated compaction passed. |
+| 4B to 27B transfer, approximately 16K prompts | **2.19x paired time-to-first-token speedup**; native median 24.663 s, composite median 11.087 s. | E120 resident NF4/BF16 research runtime on RTX 5090; paired 95% interval 2.15-2.23x. |
+
+These measurements describe their stated workloads and configurations. Compaction
+reduces attention-state storage; model weights and recurrent state retain their
+existing representation. The transfer timing includes prefill, translation,
+installation and the first emitted token, excluding cold model loading.
+
+See [compaction validation](https://github.com/LatentPort/latentport/blob/main/docs/compaction-validation.json),
+[transfer results](https://github.com/LatentPort/latentport/blob/main/docs/composite-results.json) and
+[benchmark methodology](https://github.com/LatentPort/latentport/blob/main/docs/benchmarks.md) for the full records.
+
+## A state operating layer for inference
+
+The long-term direction is a state operating layer for AI inference:
+
+- **Model-specific translators** connect the internal state spaces of compatible models.
+- **Reusable operators** transfer, compact, optimise and combine state through a shared interface.
+- **A policy engine** selects transformations automatically according to the application's capability, latency and memory requirements.
+
+The current release provides explicit transfer and compaction operations, plus
+the state lifecycle primitives. Learned optimisation, state combination and
+automatic policy selection are the next layers of that vision.
+
+## One runtime, separate transformation models
+
+The shared Python runtime lives in this GitHub repository. Each trained state
+transformation has its own Hugging Face repository, with model compatibility,
+weights, versions and a model card.
+
+- [LPCompact-Qwen3.8-27B](https://huggingface.co/latentport/LPCompact-Qwen3.8-27B): the compaction selector shared across BF16 and GGUF.
+- [4B to 27B transfer bridge](https://huggingface.co/latentport/latentport-4-27b-research-preview): the current directional translator; [GGUF artifacts](https://huggingface.co/latentport/latentport-4-27b-research-preview-GGUF) are also available.
+
+Transformation repositories contain their own learned artifacts. Base-model
+weights are obtained separately.
+
+## Demos and documentation
+
+- [Memory transplant demo](https://github.com/LatentPort/latentport/blob/main/examples/memory_transplant/README.md): live state transfer, specialist branches and a repair checked by executable tests.
+- [State demos](https://github.com/LatentPort/latentport/blob/main/docs/state-demos.md): cached conversations, branching, save/resume and fresh context updates.
+- [Time-to-first-token demo](https://github.com/LatentPort/latentport/blob/main/docs/ttft-demo.md): compare native 27B and composite inference on a fresh 16K prompt.
+- [Compaction guide](https://github.com/LatentPort/latentport/blob/main/docs/compaction.md), [cookbook](https://github.com/LatentPort/latentport/blob/main/docs/cookbook.md) and [API contracts](https://github.com/LatentPort/latentport/blob/main/docs/api.md).
+- [Native integration](https://github.com/LatentPort/latentport/blob/main/docs/native-abi.md) and [contributing](CONTRIBUTING.md).
+
+Report reproducible issues at [GitHub Issues](https://github.com/LatentPort/latentport/issues).
+Include the output of `python -m latentport doctor`, model identities and a minimal
+example, with private prompts and credentials removed.
+
+## License
+
+The LatentPort runtime and transformation weights are Apache-2.0. Qwen base models
+retain their upstream licenses, and llama.cpp retains its MIT license. See
+[LICENSE](LICENSE) and the notices supplied with the model and native downloads.