tensorcode 0.1.0a1__tar.gz → 0.4.0a3__tar.gz

tensorcode-0.4.0a3/.gitignore

@@ -0,0 +1,18 @@
+# Local environments, credentials and editor preferences
+.venv/
+.env
+.env.*
+.vscode/
+.idea/
+
+# Python/build/test artifacts
+__pycache__/
+*.py[cod]
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+build/
+dist/

tensorcode-0.4.0a3/CHANGELOG.md

@@ -0,0 +1,76 @@
+# Changelog
+
+All notable changes to the `tensorcode` Python package. Versions follow
+[PEP 440](https://peps.python.org/pep-0440/); alpha releases may break APIs.
+See [updating development code](docs/migration.md) for import changes.
+
+## 0.4.0a3 (unreleased)
+
+### Added
+
+- Text operations (`Classify`, `Decide`, `Score`, `Retrieve`) accept
+  `decoding='likelihood'`, which scores every authored alternative in one
+  encoder pass and returns a full distribution instead of generated JSON.
+- `tensorcode.ops.text.ask` / `aask` ask several named structured questions
+  about the same messages; a shared external model that implements
+  `complete_questions` answers them in one request.
+- `tensorcode.integrations.JevModel`, an explicit HTTP adapter for typed
+  classification, decision and rubric-score requests.
+- `tensorcode.training` calibration utilities: `TemperatureCalibration`,
+  `evaluate_calibration` and `fit_threshold`, fitted only on held-out scores.
+- `readout='output_encoding'` for owned text and image encoders: an owned
+  trainable token appended after the context.
+- Investigator `verification_scope='joint'` checks a hypothesis against the
+  combined evidence and keeps each per-source check.
+- `Investigator.new_cognitive_session(...)` / `load_cognitive_session(...)`,
+  `Planner.new_executor(...)` and `tools.actions.action_loop(...)` factories.
+
+### Changed
+
+- Public boundaries are smaller. The session, memory, cognition, execution and
+  tracing machinery moved under `tensorcode._internal`. Use root
+  `tensorcode.trace`, `Trace`, `InputRef` and `OutputRef`;
+  `training.Trainer.from_tool(...)` / `Trainer.from_ops(...)`; and
+  `training.load_experience(...)`. `tensorcode.runtime` and
+  `tensorcode.tracing` are removed without compatibility aliases.
+- Public operations are constructed from JSON configuration. They own their
+  weights and persist with `save_pretrained` / `from_pretrained`. Supplied
+  implementations use the explicit `from_module` (vector) and `from_model`
+  (text) factories.
+- The Chatbot workspace uses a bounded (`relative_rms_bounded`) memory update.
+  Artifacts must match the current architecture exactly.
+- Public records, operations, tools and trainer properties now have
+  docstrings. Every `ops.vec` submodule declares `__all__`, so internal
+  helpers no longer leak through star imports.
+- Package metadata now includes classifiers, keywords and project, docs and
+  changelog URLs. The sdist includes only the documented top-level paths.
+- Tests of the repository's internal `.development` experiments skip, rather
+  than fail, when run from the unpacked sdist, which does not ship that tree.
+
+### Removed
+
+- `DecisionPipeline`, `runtime.*` session/memory/storage classes, the free
+  checkpoint functions and `training.ToolTrainer`. See the
+  [migration table](docs/migration.md).
+
+## 0.4.0a2
+
+- Vector APIs are organized by operation. Concrete encoders live in
+  `tensorcode.ops.vec.encode` and decoders in `tensorcode.ops.vec.decode`.
+  Backend modules are private.
+- Alpha model artifacts were recreated with the current public operation
+  identities.
+
+## 0.4.0a1
+
+- Added owned pretrained vector operations: transformer `TextEncoder` and
+  `ImageEncoder`, a seq2seq `TextDecoder` and a latent-diffusion
+  `ImageDecoder` (`tensorcode[diffusion]`).
+
+## 0.3.0
+
+- Added the owned pretrained tool lifecycle (`save_pretrained`,
+  `from_pretrained`, Hugging Face Hub loading) and the learned evidence
+  workspace shared by Chatbot, Investigator, Planner, Decision and Scene.
+  Built distributions are attached to the
+  [GitHub release](https://github.com/TensaCo/tensacode-py/releases/tag/v0.3.0).

tensorcode-0.4.0a3/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.5
+Name: tensorcode
+Version: 0.4.0a3
+Summary: Composable cognitive operations and traceable, trainable Python programs
+Project-URL: Homepage, https://tensorcode.dev
+Project-URL: Documentation, https://tensorcode.dev/docs/
+Project-URL: Source, https://github.com/TensaCo/tensacode-py
+Project-URL: Issues, https://github.com/TensaCo/tensacode-py/issues
+Project-URL: Changelog, https://github.com/TensaCo/tensacode-py/blob/main/CHANGELOG.md
+Author-email: Jacob Valdez <jacob.valdez@tensaco.ai>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agents,cognitive-architecture,llm,machine-learning,pytorch,tracing,training,transformers
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pillow>=10; extra == 'dev'
+Requires-Dist: pyarrow>=18; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Provides-Extra: diffusion
+Requires-Dist: diffusers<1,>=0.40; extra == 'diffusion'
+Requires-Dist: huggingface-hub>=0.34; extra == 'diffusion'
+Requires-Dist: pillow>=10; extra == 'diffusion'
+Requires-Dist: safetensors>=0.4; extra == 'diffusion'
+Requires-Dist: tokenizers>=0.21; extra == 'diffusion'
+Requires-Dist: torch>=2.4; extra == 'diffusion'
+Requires-Dist: torchvision>=0.19; extra == 'diffusion'
+Requires-Dist: transformers<6,>=5.0; extra == 'diffusion'
+Provides-Extra: local
+Requires-Dist: pillow>=10; extra == 'local'
+Requires-Dist: torch>=2.4; extra == 'local'
+Requires-Dist: torchvision>=0.19; extra == 'local'
+Requires-Dist: transformers<6,>=5.0; extra == 'local'
+Provides-Extra: pretrained
+Requires-Dist: huggingface-hub>=0.34; extra == 'pretrained'
+Requires-Dist: pillow>=10; extra == 'pretrained'
+Requires-Dist: safetensors>=0.4; extra == 'pretrained'
+Requires-Dist: tokenizers>=0.21; extra == 'pretrained'
+Requires-Dist: torch>=2.4; extra == 'pretrained'
+Requires-Dist: torchvision>=0.19; extra == 'pretrained'
+Requires-Dist: transformers<6,>=5.0; extra == 'pretrained'
+Provides-Extra: tools
+Requires-Dist: huggingface-hub>=0.34; extra == 'tools'
+Requires-Dist: pillow>=10; extra == 'tools'
+Requires-Dist: safetensors>=0.4; extra == 'tools'
+Requires-Dist: tokenizers>=0.21; extra == 'tools'
+Requires-Dist: torch>=2.4; extra == 'tools'
+Requires-Dist: torchvision>=0.19; extra == 'tools'
+Requires-Dist: transformers<6,>=5.0; extra == 'tools'
+Provides-Extra: vec
+Requires-Dist: numpy>=1.26; extra == 'vec'
+Requires-Dist: safetensors>=0.4; extra == 'vec'
+Requires-Dist: torch>=2.4; extra == 'vec'
+Description-Content-Type: text/markdown
+
+# TensorCode
+
+**Website:** [tensorcode.dev](https://tensorcode.dev) ·
+**Docs:** [tensorcode.dev/docs](https://tensorcode.dev/docs/) ·
+**Source:** [GitHub](https://github.com/TensaCo/tensacode-py) ·
+[Changelog](https://github.com/TensaCo/tensacode-py/blob/main/CHANGELOG.md)
+
+TensorCode builds trainable Python programs from callable operations and small
+tools that own their models. You compose encoders, scorers and decoders
+(`tensorcode.ops`), or use a complete tool such as `Investigator`, `Planner` or
+`Chatbot` (`tensorcode.tools`). You collect reviewed feedback with explicit
+provenance, train with PyTorch, and save everything as a data-only artifact that
+reloads in a fresh process or from the Hugging Face Hub. Tracing records which
+operation produced which value, so supervised local tensor paths can be replayed
+and trained. It does not make arbitrary Python or remote model calls
+differentiable.
+
+> **Status: 0.4.0 alpha.** APIs may change between alphas. The core package has
+> no dependencies and importing it does not import PyTorch or touch the network.
+> Measured behavior and its limits are in [validation](docs/validation.md).
+> Consistent benefits of the learned cognitive workspace are not yet established.
+
+## Install
+
+Python 3.11+. PyPI currently hosts only an older 0.1 alpha, so install 0.4 from
+GitHub:
+
+```bash
+python -m pip install "tensorcode[tools] @ git+https://github.com/TensaCo/tensacode-py"
+```
+
+From a checkout, use `python -m pip install -e '.[tools]'`. Choose the extras
+for the interfaces you use:
+
+| Extra | Adds |
+|---|---|
+| `tools` | Owned models, training and Hugging Face loading (PyTorch, Transformers) |
+| `vec` | Vector operations only (PyTorch, NumPy, safetensors) |
+| `local` | Adapter for a local multimodal Transformers model |
+| `diffusion` | `tools` plus diffusers for image decoders |
+| `pretrained` | Alias of `tools` |
+| `dev` | pytest, build, Pillow and PyArrow; the full test suite also needs `diffusion` |
+
+## 30-second example
+
+This trains an `Investigator` to rank two supplied hypotheses from log evidence.
+It saves the model and reloads it. It runs offline on CPU in a few seconds.
+
+```python
+import torch
+from tensorcode import training
+from tensorcode.tools.investigator import Investigator
+
+torch.manual_seed(0)
+model = Investigator({"vocabulary": ["database", "network", "connection", "refused", "packet", "loss"],
+                      "dimensions": 16, "slots": 2, "steps": 1})
+trainer = training.Trainer.from_tool(model, optimizer=torch.optim.AdamW(model.parameters(), lr=0.01))
+
+hypotheses = [{"id": "database", "text": "database connection refused"},
+              {"id": "network", "text": "network packet loss"}]
+
+def case(log_line):
+    return {"question": "which component failed",
+            "evidence": [{"source_id": "log:1", "text": log_line}],
+            "hypotheses": hypotheses}
+
+# Reviewed feedback, with explicit provenance, becomes training experience.
+experiences = [trainer.capture(case("connection refused"), "database", source="review:1"),
+               trainer.capture(case("packet loss"), "network", source="review:2")]
+losses = trainer.fit(experiences, epochs=30)
+
+model.save_pretrained("./investigator")
+restored = Investigator.from_pretrained("./investigator")
+print(restored(case("packet loss"))["selected_id"])  # network
+```
+
+Two authored cases show the lifecycle. They do not show that the model can
+investigate anything. The result also includes every candidate's score and the
+source-linked evidence. Probabilities are uncalibrated. The
+[quickstart](docs/quickstart.md) extends this to persisted experience files,
+resumable training checkpoints and loading in a fresh process.
+
+## What is inside
+
+- **`tensorcode.ops.vec`, `ops.text`, `ops.graph`**: operations with one calling
+  convention, `op(value, *, context=None)`. Operations are built from JSON
+  configuration, and learned vector operations own their weights. Text
+  operations wrap an owned seq2seq model or an explicit external provider.
+  Graph operations are reserved symbolic interfaces that raise
+  `NotImplementedError`.
+- **`tensorcode.tools`**: `Chatbot`, `Investigator`, `Decision`, `Planner` and
+  `Scene`, complete trainable models with `save_pretrained` / `from_pretrained`.
+- **`tensorcode.trace()` and `tensorcode.training`**: dependency capture,
+  explicit supervision, `Trainer.from_tool` / `Trainer.from_ops`, portable
+  experience and complete checkpoints.
+- **`tensorcode.integrations`**: explicit adapters for OpenAI-compatible
+  endpoints, local Transformers models and Jev.
+
+Generated hypotheses are not evidence, and generated plans are not executable
+code. Evidence, policies and actions stay explicit in your code.
+
+## Guides
+
+- [Quickstart](docs/quickstart.md): a runnable offline training lifecycle.
+- [Developer documentation](docs/README.md): operation and model contracts.
+- [Pretrained checkpoints](docs/pretrained.md): hosted tools and their measured
+  scope. Saved artifacts must match the current architecture exactly: the hosted
+  Chatbot and cognitive Investigator checkpoints need source commit `6607a8b`;
+  only the Hotpot Investigator and Decision have refreshed revisions verified on
+  current source.
+- [Evidence and cognition](docs/cognition.md): hypotheses, revisions, memory and outcome feedback.
+- [Training](docs/training.md): tracing, replay and resumable checkpoints.
+- [Examples](examples/README.md): learning agents and practical applications.
+- [Validation](docs/validation.md): measured behavior and limitations.
+- [Updating development code](docs/migration.md): import changes since earlier alphas.
+
+## Development
+
+```bash
+python -m pip install -e '.[tools,diffusion,dev]'
+python -m pytest -q
+python -m build
+```
+
+Library code lives in `src/tensorcode`. See the [test guide](tests/README.md) for
+how the suite is organized. This alpha API replaces the former provider-owned
+tools, and there is no legacy compatibility layer. [MIT license](LICENSE).

tensorcode-0.4.0a3/README.md

@@ -0,0 +1,127 @@
+# TensorCode
+
+**Website:** [tensorcode.dev](https://tensorcode.dev) ·
+**Docs:** [tensorcode.dev/docs](https://tensorcode.dev/docs/) ·
+**Source:** [GitHub](https://github.com/TensaCo/tensacode-py) ·
+[Changelog](https://github.com/TensaCo/tensacode-py/blob/main/CHANGELOG.md)
+
+TensorCode builds trainable Python programs from callable operations and small
+tools that own their models. You compose encoders, scorers and decoders
+(`tensorcode.ops`), or use a complete tool such as `Investigator`, `Planner` or
+`Chatbot` (`tensorcode.tools`). You collect reviewed feedback with explicit
+provenance, train with PyTorch, and save everything as a data-only artifact that
+reloads in a fresh process or from the Hugging Face Hub. Tracing records which
+operation produced which value, so supervised local tensor paths can be replayed
+and trained. It does not make arbitrary Python or remote model calls
+differentiable.
+
+> **Status: 0.4.0 alpha.** APIs may change between alphas. The core package has
+> no dependencies and importing it does not import PyTorch or touch the network.
+> Measured behavior and its limits are in [validation](docs/validation.md).
+> Consistent benefits of the learned cognitive workspace are not yet established.
+
+## Install
+
+Python 3.11+. PyPI currently hosts only an older 0.1 alpha, so install 0.4 from
+GitHub:
+
+```bash
+python -m pip install "tensorcode[tools] @ git+https://github.com/TensaCo/tensacode-py"
+```
+
+From a checkout, use `python -m pip install -e '.[tools]'`. Choose the extras
+for the interfaces you use:
+
+| Extra | Adds |
+|---|---|
+| `tools` | Owned models, training and Hugging Face loading (PyTorch, Transformers) |
+| `vec` | Vector operations only (PyTorch, NumPy, safetensors) |
+| `local` | Adapter for a local multimodal Transformers model |
+| `diffusion` | `tools` plus diffusers for image decoders |
+| `pretrained` | Alias of `tools` |
+| `dev` | pytest, build, Pillow and PyArrow; the full test suite also needs `diffusion` |
+
+## 30-second example
+
+This trains an `Investigator` to rank two supplied hypotheses from log evidence.
+It saves the model and reloads it. It runs offline on CPU in a few seconds.
+
+```python
+import torch
+from tensorcode import training
+from tensorcode.tools.investigator import Investigator
+
+torch.manual_seed(0)
+model = Investigator({"vocabulary": ["database", "network", "connection", "refused", "packet", "loss"],
+                      "dimensions": 16, "slots": 2, "steps": 1})
+trainer = training.Trainer.from_tool(model, optimizer=torch.optim.AdamW(model.parameters(), lr=0.01))
+
+hypotheses = [{"id": "database", "text": "database connection refused"},
+              {"id": "network", "text": "network packet loss"}]
+
+def case(log_line):
+    return {"question": "which component failed",
+            "evidence": [{"source_id": "log:1", "text": log_line}],
+            "hypotheses": hypotheses}
+
+# Reviewed feedback, with explicit provenance, becomes training experience.
+experiences = [trainer.capture(case("connection refused"), "database", source="review:1"),
+               trainer.capture(case("packet loss"), "network", source="review:2")]
+losses = trainer.fit(experiences, epochs=30)
+
+model.save_pretrained("./investigator")
+restored = Investigator.from_pretrained("./investigator")
+print(restored(case("packet loss"))["selected_id"])  # network
+```
+
+Two authored cases show the lifecycle. They do not show that the model can
+investigate anything. The result also includes every candidate's score and the
+source-linked evidence. Probabilities are uncalibrated. The
+[quickstart](docs/quickstart.md) extends this to persisted experience files,
+resumable training checkpoints and loading in a fresh process.
+
+## What is inside
+
+- **`tensorcode.ops.vec`, `ops.text`, `ops.graph`**: operations with one calling
+  convention, `op(value, *, context=None)`. Operations are built from JSON
+  configuration, and learned vector operations own their weights. Text
+  operations wrap an owned seq2seq model or an explicit external provider.
+  Graph operations are reserved symbolic interfaces that raise
+  `NotImplementedError`.
+- **`tensorcode.tools`**: `Chatbot`, `Investigator`, `Decision`, `Planner` and
+  `Scene`, complete trainable models with `save_pretrained` / `from_pretrained`.
+- **`tensorcode.trace()` and `tensorcode.training`**: dependency capture,
+  explicit supervision, `Trainer.from_tool` / `Trainer.from_ops`, portable
+  experience and complete checkpoints.
+- **`tensorcode.integrations`**: explicit adapters for OpenAI-compatible
+  endpoints, local Transformers models and Jev.
+
+Generated hypotheses are not evidence, and generated plans are not executable
+code. Evidence, policies and actions stay explicit in your code.
+
+## Guides
+
+- [Quickstart](docs/quickstart.md): a runnable offline training lifecycle.
+- [Developer documentation](docs/README.md): operation and model contracts.
+- [Pretrained checkpoints](docs/pretrained.md): hosted tools and their measured
+  scope. Saved artifacts must match the current architecture exactly: the hosted
+  Chatbot and cognitive Investigator checkpoints need source commit `6607a8b`;
+  only the Hotpot Investigator and Decision have refreshed revisions verified on
+  current source.
+- [Evidence and cognition](docs/cognition.md): hypotheses, revisions, memory and outcome feedback.
+- [Training](docs/training.md): tracing, replay and resumable checkpoints.
+- [Examples](examples/README.md): learning agents and practical applications.
+- [Validation](docs/validation.md): measured behavior and limitations.
+- [Updating development code](docs/migration.md): import changes since earlier alphas.
+
+## Development
+
+```bash
+python -m pip install -e '.[tools,diffusion,dev]'
+python -m pytest -q
+python -m build
+```
+
+Library code lives in `src/tensorcode`. See the [test guide](tests/README.md) for
+how the suite is organized. This alpha API replaces the former provider-owned
+tools, and there is no legacy compatibility layer. [MIT license](LICENSE).

tensorcode-0.4.0a3/docs/README.md

@@ -0,0 +1,66 @@
+# TensorCode documentation
+
+TensorCode separates callable operations, owned trainable models, application
+runtime state and durable training experience. Start with the
+[quickstart](quickstart.md) for a complete offline model lifecycle.
+
+| Guide | Contents |
+|---|---|
+| [Pretrained checkpoints](pretrained.md) | Hosted tool catalog, pinned loading, scope and publication |
+| [Quickstart](quickstart.md) | Install, initialize, collect feedback, train, save and reload |
+| [Operations](operations.md) | Vector/message contracts, encoders, decoders and symbolic graph interfaces |
+| [Pretrained vector models](latent-models.md) | Transformer encoding, text/diffusion decoding, training, canonical imports and owned configuration |
+| [Tools](tools.md) | Owned models, shared workspace, pretrained artifacts and sessions |
+| [Cognition](cognition.md) | Sourced evidence, generated proposals, verification, revision and action feedback |
+| [Training](training.md) | Tool objectives, tracing, replay and resumable checkpoints |
+| [Examples](../examples/README.md) | Learning agents and applications with explicit inputs |
+| [Validation](validation.md) | Measured behavior, pretrained provenance and remaining gaps |
+| [Evaluation records](results/README.md) | Machine-readable measurements |
+| [Troubleshooting](troubleshooting.md) | Loading, gradients, replay and session errors |
+| [Updating development code](migration.md) | New public imports, tool factories and saved-data continuity |
+
+## Public boundaries
+
+`tensorcode.ops.{vec,text,graph}` contains callable operations. Developers assign
+names for cognitive roles; the same transform can participate in interpretation,
+revision or response formulation. Representation contracts matter: equal tensor
+shapes do not make independently trained encoders interchangeable. Concrete vector
+encoders live in `tensorcode.ops.vec.encode`, and decoders in
+`tensorcode.ops.vec.decode`; root exports are conveniences. Backend modules are
+private. See the [operation API](operations.md) and
+[owned operation contracts](latent-models.md#public-paths-and-owned-configuration).
+
+Learned operation constructors accept JSON configuration and own their model
+parameters. Explicit `from_foundation` factories initialize supported pretrained
+architectures; `save_pretrained`/`from_pretrained` persist owned artifacts.
+Advanced vector `from_module` and text `from_model` factories integrate supplied
+implementations with explicit reconstruction limits. Weightless operations accept
+optional configuration; graph calls still raise `NotImplementedError`.
+
+`tensorcode.tools` contains owned PyTorch models: Chatbot, Investigator, Planner,
+Decision and Scene. Scene combines image patches and text through the shared
+workspace to rank supplied descriptions or, with an owned language foundation,
+produce explicitly unverified image interpretations. Neither mode constructs
+symbolic scene graphs. Constructors initialize all
+parameters; `from_pretrained` loads complete model artifacts from a local directory
+or the Hugging Face Hub. An external foundation model can bootstrap training, but
+its inherited competence does not establish that a new workspace has learned.
+
+Tools construct their sessions and execution helpers. `tools.cognition` exposes
+evidence and interpretation records; `tools.actions` exposes action callback
+records and an explicit bounded-loop factory. Planner exposes structured plan
+contracts and `new_executor(...)`. Evidence, policies and action implementations
+remain explicit, while memory storage and state transitions stay internal.
+Generated hypotheses are not source evidence; generated plans are not executable
+action code. The general core supplies no domain ontology or implicit authority.
+
+Root `tensorcode.trace()` and `tensorcode.training` capture operation dependencies and
+train supported local tensor paths independently of an agent harness. Tracing does
+not make arbitrary Python, remote model calls or discrete choices differentiable.
+Graph operations currently declare symbolic interfaces without implementations.
+
+Choose `training.Trainer.from_tool(model)` for a declared model objective, or
+`training.Trainer.from_ops(operations, losses=...)` for a composed program. Load
+saved experience through `training.load_experience(...)`; save optimization state
+through the trainer's checkpoint methods. Root `Trace`, `InputRef` and `OutputRef`
+provide public trace types without exposing the implementation modules.