tensorless 0.1.0__tar.gz → 0.3.0__tar.gz

tensorless-0.3.0/PKG-INFO

@@ -0,0 +1,77 @@
+Metadata-Version: 2.4
+Name: tensorless
+Version: 0.3.0
+Summary: ML with maximum automation and minimum setup.
+Author: Tensorless Contributors
+License: MIT
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Dynamic: license-file
+
+# Tensorless
+
+Tensorless trains small PyTorch models with sensible defaults. It supports
+text generation, text classification, tabular classification, and regression.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+## Train on your data
+
+```python
+import tensorless as tl
+
+model = tl.train("./corpus.txt", task="text-generation")
+print(model.generate("The", max_new_tokens=40))
+```
+
+Text files are trained as next-token language models. BPE is the default
+tokenizer; use `tokenizer="char"` for a character-level model. Tensorless
+derives model size, batch size, epochs, validation, device, and BPE vocabulary
+size from the data, while every setting can be overridden.
+
+Long text is tokenized lazily and fed through PyTorch in fixed-size batches.
+The automatic batch size uses a token budget; reduce `batch_size` if your
+available memory is limited.
+
+## English starter pretraining
+
+```python
+import tensorless as tl
+
+model = tl.pretrain(out="english.tl", epochs=20, max_seq_len=128)
+print(model.generate("A complete sentence", max_new_tokens=30))
+```
+
+This offline starter corpus contains English prose and grammar examples. It is
+for demos and smoke tests, not a replacement for a large language dataset. For
+real pretraining, pass your own `.txt` corpus to `tl.train()` and increase the
+training settings as your hardware allows.
+
+## Other tasks
+
+```python
+tl.train("reviews/", task="text-classification")
+tl.train("housing.csv", task="regression")
+```
+
+Tabular preprocessing automatically handles numeric values, ISO dates, and
+high-cardinality categories. Missing and rare values are handled using the
+fitted training data, and the same preprocessing is stored in the `.tl` file.
+
+Models are saved as `.tl` files and can be loaded later:
+
+```python
+model = tl.load("model.tl")
+print(model.info())
+```
+
+See the [documentation](docs/quickstart.md) for data formats, configuration,
+checkpointing, and the command-line interface.

tensorless-0.3.0/README.md

@@ -0,0 +1,63 @@
+# Tensorless
+
+Tensorless trains small PyTorch models with sensible defaults. It supports
+text generation, text classification, tabular classification, and regression.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+## Train on your data
+
+```python
+import tensorless as tl
+
+model = tl.train("./corpus.txt", task="text-generation")
+print(model.generate("The", max_new_tokens=40))
+```
+
+Text files are trained as next-token language models. BPE is the default
+tokenizer; use `tokenizer="char"` for a character-level model. Tensorless
+derives model size, batch size, epochs, validation, device, and BPE vocabulary
+size from the data, while every setting can be overridden.
+
+Long text is tokenized lazily and fed through PyTorch in fixed-size batches.
+The automatic batch size uses a token budget; reduce `batch_size` if your
+available memory is limited.
+
+## English starter pretraining
+
+```python
+import tensorless as tl
+
+model = tl.pretrain(out="english.tl", epochs=20, max_seq_len=128)
+print(model.generate("A complete sentence", max_new_tokens=30))
+```
+
+This offline starter corpus contains English prose and grammar examples. It is
+for demos and smoke tests, not a replacement for a large language dataset. For
+real pretraining, pass your own `.txt` corpus to `tl.train()` and increase the
+training settings as your hardware allows.
+
+## Other tasks
+
+```python
+tl.train("reviews/", task="text-classification")
+tl.train("housing.csv", task="regression")
+```
+
+Tabular preprocessing automatically handles numeric values, ISO dates, and
+high-cardinality categories. Missing and rare values are handled using the
+fitted training data, and the same preprocessing is stored in the `.tl` file.
+
+Models are saved as `.tl` files and can be loaded later:
+
+```python
+model = tl.load("model.tl")
+print(model.info())
+```
+
+See the [documentation](docs/quickstart.md) for data formats, configuration,
+checkpointing, and the command-line interface.

{tensorless-0.1.0 → tensorless-0.3.0}/docs/architecture.md

@@ -22,7 +22,8 @@ tensorless/
 │   ├── detector.py               Dataset -> task string
 │   └── config.py                   Dataset + TrainConfig -> ResolvedConfig
 ├── tokenization/
-│   └── char_tokenizer.py          CharTokenizer: vocab build/encode/decode/save/load
+│   ├── char_tokenizer.py          CharTokenizer: vocab build/encode/decode/save/load
+│   └── bpe_tokenizer.py           BPETokenizer: corpus-trained subword encoding
 ├── models/
 │   ├── transformer.py              TinyTransformer (GPT-style decoder)
 │   ├── mlp.py                       TabularMLP

{tensorless-0.1.0 → tensorless-0.3.0}/docs/automatic_mode.md

@@ -31,14 +31,23 @@ Once the task is known, `tensorless/auto/config.py` picks:
 
 - **model type**: `transformer` for text tasks, `mlp` for tabular tasks
 - **size** (`d_model`, `layers`, `heads`): scaled to dataset size, from
-  a tiny 2-layer/64-dim model for a few hundred examples up to an
+  a tiny 2-layer/64-dim model for a few hundred effective text examples up to an
   8-layer/384-dim model for 50,000+ examples
 
+For text corpora, effective examples include corpus character count, so a
+single large `.txt` file is not treated like one training example. BPE
+vocabulary size is also bounded from corpus character diversity rather than
+always using a fixed oversized vocabulary.
+
 This is a heuristic, not a search — the goal is "a model that trains
 quickly and doesn't wildly overfit or underfit for typical dataset
 sizes," not the best possible architecture. Override any of it:
 `tl.train("./data", d_model=512, layers=6)`.
 
+For a packaged English grammar starter corpus, use
+`tl.pretrain(out="english.tl")`. It is intended for demos and smoke tests;
+larger local corpora should be passed to `tl.train()`.
+
 ## 3. Hyperparameter selection
 
 Batch size, epoch count, learning rate, warmup steps, and the

{tensorless-0.1.0 → tensorless-0.3.0}/docs/checkpointing.md

@@ -40,6 +40,10 @@ Writes are atomic: Tensorless writes to a temporary file in the same
 directory and renames it into place, so a crash mid-write never leaves a
 corrupt checkpoint that would block resumption.
 
+When loading `.tl` files, Tensorless fills compatible fields introduced by
+older versions with safe defaults. Files created by a newer unsupported format
+version are rejected with an upgrade message instead of being partially read.
+
 ## How resumption works
 
 When `tl.train()` finds a checkpoint whose `dataset_fingerprint` matches

{tensorless-0.1.0 → tensorless-0.3.0}/docs/configuration.md

@@ -26,6 +26,8 @@ defaults.
 | `ff_mult` | auto | Feed-forward expansion multiplier (transformer only) |
 | `dropout` | `0.1` | Dropout probability |
 | `max_seq_len` | `256` (text) / `1` (tabular) | Max sequence length in tokens (text tasks) |
+| `tokenizer` | `"bpe"` | Text tokenizer: `"bpe"` or `"char"` |
+| `bpe_vocab_size` | `1000` | Maximum vocabulary size when `tokenizer="bpe"` |
 
 ## Optimization
 
@@ -45,7 +47,7 @@ defaults.
 | Field | Default | Description |
 |---|---|---|
 | `val_split` | `0.1` (if ≥50 examples, else `0`) | Fraction of data held out for validation |
-| `patience` | `5` | Epochs without improvement before stopping early |
+| `patience` | `3` | Epochs without improvement before stopping early |
 | `min_delta` | `1e-4` | Minimum improvement to count as "improved" |
 
 ## Hardware

{tensorless-0.1.0 → tensorless-0.3.0}/docs/installation.md

@@ -7,26 +7,6 @@
 - Optional: a CUDA-capable GPU, or a TPU with `torch_xla` installed, for
   faster training. Tensorless works fine on CPU-only machines too.
 
-## Install from source
-
-Tensorless isn't published to PyPI yet. Install it directly from a
-checkout of this repository:
-
-```bash
-git clone https://github.com/tensorless/tensorless.git
-cd tensorless
-pip install -e .
-```
-
-The `-e` (editable) flag means changes to the source are picked up
-immediately without reinstalling — useful if you're also contributing.
-
-If you plan to run the test suite, install the dev extras too:
-
-```bash
-pip install -e ".[dev]"
-```
-
 ## Verify your installation
 
 ```bash

{tensorless-0.1.0 → tensorless-0.3.0}/docs/limitations.md

@@ -6,12 +6,10 @@ current limits:
 
 ## Models
 
-- The default text tokenizer is **character-level**, not subword/BPE.
-  This means no vocabulary file dependency and it works on any language
-  out of the box, but it's less token-efficient than production
-  tokenizers (GPT-style BPE, SentencePiece), so very long documents take
-  more sequence positions to represent, and generation quality per
-  parameter is lower than a comparably-sized BPE-tokenized model.
+- The default text tokenizer is the dependency-free **BPE** tokenizer, which
+  learns merges from the training corpus. It is more token-efficient than the
+  optional `tokenizer="char"` alternative, but still simpler than production
+  tokenizers such as GPT-style BPE or SentencePiece.
 - Models are trained **from scratch** every time — there's no
   fine-tuning of pretrained checkpoints. This keeps things dependency-
   free and fast to set up, but means text-generation quality on small

{tensorless-0.1.0 → tensorless-0.3.0}/docs/roadmap.md

@@ -5,18 +5,7 @@ a commitment or timeline.
 
 ## Near-term
 
-- **Subword/BPE tokenization** as an alternative to the default
-  character-level tokenizer, for better efficiency on larger text
-  corpora
-- **`.tl` format migration** — forward-compatible loading of files
-  written by older Tensorless versions
-- **Better tabular preprocessing** — handling of date/datetime columns,
-  high-cardinality categorical columns, and more robust outlier handling
-  for regression targets
 - **Progress bars** for training (currently plain print-based logging)
-- **`tl.train(..., resume=False)`** enforcement — currently `resume` is
-  accepted in `TrainConfig` but the automatic resume decision doesn't
-  yet fully respect an explicit `False` override in every code path
 - **Multi-GPU / distributed training** for larger datasets
 
 ## Medium-term
@@ -38,8 +27,6 @@ a commitment or timeline.
 
 - **Alternate backends** (JAX, a lightweight NumPy-only backend) behind
   the same `tl.train()`/`tl.load()` API
-- **Streaming/out-of-core training** for datasets too large to fit in
-  memory
 - **Export to other formats** (ONNX, TorchScript) from a `.tl` file for
   deployment outside Python
 

{tensorless-0.1.0 → tensorless-0.3.0}/docs/training.md

@@ -38,6 +38,11 @@ A directory can mix multiple files of the *same* format (e.g. several
 plain text files with structured (JSON/CSV) files in the same directory
 raises a `DataError` asking you to separate them.
 
+For tabular data, numeric columns are robustly scaled and missing values use
+the training median. ISO-8601 date and datetime columns are converted to
+numeric timestamps. Categorical columns are frequency-ranked and capped at
+1,000 learned values; rare or unseen values use the `<unk>` category.
+
 Tensorless never modifies, moves, or deletes files in your dataset
 directory. It only ever reads from `path`; all output goes to the `out`
 file and `checkpoint_dir`.
@@ -68,7 +73,9 @@ By default, Tensorless holds out `val_split` of the data (10% for
 datasets with 50+ examples, 0% for smaller ones where a held-out split
 wouldn't be meaningful) and tracks validation loss after each epoch. If
 validation loss doesn't improve by at least `min_delta` for `patience`
-consecutive epochs, training stops early.
+consecutive epochs, training stops early. The automatic default is 3
+consecutive epochs, and the completed model is still written to the `.tl`
+output file.
 
 ## Checkpointing during training
 
@@ -81,8 +88,8 @@ for resumption.
 
 | Task | Model | What's learned |
 |---|---|---|
-| `text-generation` | Small GPT-style decoder transformer, char-level tokenizer | Next-character prediction over your text |
-| `text-classification` | Same transformer backbone, classification head on the final token | Text → one of your labeled classes |
+| `text-generation` | Small GPT-style decoder transformer, BPE tokenizer by default | Next-token prediction over your text |
+| `text-classification` | Same transformer backbone, BPE tokenizer by default, classification head on the final token | Text → one of your labeled classes |
 | `classification` | MLP with per-column categorical embeddings | Row of features → one of your labeled classes |
 | `regression` | Same MLP, single continuous output | Row of features → a number |
 

{tensorless-0.1.0 → tensorless-0.3.0}/examples/text_generation_example.py

@@ -32,12 +32,12 @@ def main():
     model = tl.train(
         CORPUS_PATH,
         out=os.path.join(DATA_DIR, "text_gen_example.tl"),
-        epochs=5,
-        d_model=64,
-        layers=2,
-        heads=4,
-        batch_size=16,
-        max_seq_len=64,
+        # epochs=5,
+        # d_model=64,
+        # layers=2,
+        # heads=4,
+        # batch_size=16,
+        # max_seq_len=64,
     )
 
     print("\nGenerated text:")

{tensorless-0.1.0 → tensorless-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "tensorless"
-version = "0.1.0"
+version = "0.3.0"
 description = "ML with maximum automation and minimum setup."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -22,3 +22,6 @@ tensorless = "tensorless.cli.main:main"
 
 [tool.setuptools.packages.find]
 include = ["tensorless*"]
+
+[tool.setuptools.package-data]
+tensorless = ["data/*.txt"]

{tensorless-0.1.0 → tensorless-0.3.0}/tensorless/__init__.py

@@ -12,7 +12,7 @@ ML with maximum automation and minimum setup.
 See https://github.com/tensorless/tensorless for full documentation.
 """
 
-from .api import train, run, load, inspect
+from .api import train, pretrain, run, load, inspect
 from .config import TrainConfig
 from .errors import (
     TensorlessError,
@@ -26,6 +26,7 @@ from ._version import __version__
 
 __all__ = [
     "train",
+    "pretrain",
     "run",
     "load",
     "inspect",

{tensorless-0.1.0 → tensorless-0.3.0}/tensorless/api.py

@@ -12,6 +12,7 @@
 from __future__ import annotations
 
 import dataclasses
+import importlib.resources
 import os
 from typing import Any, Optional
 
@@ -92,7 +93,7 @@ def train(path: str, **kwargs: Any) -> LoadedModel:
                 return LoadedModel(existing)
 
         # 2. Is there an interrupted / matching checkpoint to resume from?
-        if checkpoint_mgr.exists():
+        if checkpoint_mgr.exists() and user_cfg.resume is not False:
             ckpt = checkpoint_mgr.load()
             if ckpt.get("dataset_fingerprint") == fingerprint:
                 if ckpt.get("training_complete"):
@@ -119,6 +120,10 @@ def train(path: str, **kwargs: Any) -> LoadedModel:
                         "checkpoint -- retraining from scratch."
                     )
                 checkpoint_mgr.clear()
+        elif checkpoint_mgr.exists():
+            if user_cfg.verbose:
+                print("[tensorless] resume=False -- ignoring the existing checkpoint.")
+            checkpoint_mgr.clear()
     else:
         checkpoint_mgr.clear()
 
@@ -162,6 +167,17 @@ def train(path: str, **kwargs: Any) -> LoadedModel:
     return LoadedModel(payload)
 
 
+def pretrain(
+    out: str = "english_pretrained.tl", language: str = "english", **kwargs: Any
+) -> LoadedModel:
+    """Pretrain a small language model on the built-in starter corpus."""
+    if language.lower() != "english":
+        raise ValueError("The built-in pretraining corpus currently supports only 'english'.")
+    corpus = importlib.resources.files("tensorless.data").joinpath("english_grammar.txt")
+    options = {"task": "text-generation", "out": out, **kwargs}
+    return train(str(corpus), **options)
+
+
 def _finalize_from_checkpoint(ckpt: dict, out: str, verbose: bool) -> LoadedModel:
     payload = {
         "tl_format_version": _tl_format_version,

{tensorless-0.1.0 → tensorless-0.3.0}/tensorless/auto/config.py

@@ -43,15 +43,17 @@ def _auto_model_size(n_examples: int, kind: str):
             return 128, 4, 1, 2
 
 
-def _auto_batch_size(n_examples: int) -> int:
+def _auto_batch_size(n_examples: int, max_seq_len: int) -> int:
+    token_budget = 8192
+    sequence_batch = max(1, token_budget // max_seq_len)
     if n_examples < 200:
-        return 8
+        return min(8, sequence_batch)
     elif n_examples < 2000:
-        return 16
+        return min(16, sequence_batch)
     elif n_examples < 20000:
-        return 32
+        return min(32, sequence_batch)
     else:
-        return 64
+        return min(64, sequence_batch)
 
 
 def _auto_epochs(n_examples: int) -> int:
@@ -65,12 +67,29 @@ def _auto_epochs(n_examples: int) -> int:
         return 5
 
 
+def _effective_text_size(ds: Dataset) -> int:
+    """Estimate useful training examples for raw corpora."""
+    if ds.kind in ("text", "text_labeled"):
+        return max(len(ds), sum(len(text) for text in ds.texts) // 200)
+    return len(ds)
+
+
+def _auto_vocab_size(ds: Dataset) -> int:
+    if ds.kind not in ("text", "text_labeled"):
+        return 1000
+    unique_chars = len(set("".join(ds.texts)))
+    return min(4096, max(64, unique_chars * 8))
+
+
 def resolve_config(ds: Dataset, user: TrainConfig) -> ResolvedConfig:
-    n = len(ds)
+    n = _effective_text_size(ds)
     task = user.task or detect_task(ds)
     model_type = user.model_type or (
         "transformer" if task in ("text-generation", "text-classification") else "mlp"
     )
+    tokenizer = user.tokenizer or "bpe"
+    if tokenizer not in ("char", "bpe"):
+        raise ValueError("tokenizer must be 'char' or 'bpe'")
 
     d_model, layers, heads, ff_mult = _auto_model_size(n, ds.kind)
 
@@ -79,6 +98,7 @@ def resolve_config(ds: Dataset, user: TrainConfig) -> ResolvedConfig:
     out = user.out or "model.tl"
     checkpoint_dir = user.checkpoint_dir or (out + ".ckpt")
 
+    max_seq_len = user.max_seq_len or (256 if ds.kind in ("text", "text_labeled") else 1)
     resolved = ResolvedConfig(
         out=out,
         force=bool(user.force),
@@ -91,17 +111,19 @@ def resolve_config(ds: Dataset, user: TrainConfig) -> ResolvedConfig:
         heads=user.heads or heads,
         ff_mult=user.ff_mult or ff_mult,
         dropout=user.dropout if user.dropout is not None else 0.1,
-        max_seq_len=user.max_seq_len or (256 if ds.kind in ("text", "text_labeled") else 1),
+        max_seq_len=max_seq_len,
+        tokenizer=tokenizer,
+        bpe_vocab_size=user.bpe_vocab_size if user.bpe_vocab_size is not None else _auto_vocab_size(ds),
         optimizer=user.optimizer or "adamw",
         learning_rate=user.learning_rate or (3e-4 if model_type == "transformer" else 1e-3),
         weight_decay=user.weight_decay if user.weight_decay is not None else 0.01,
-        batch_size=user.batch_size or _auto_batch_size(n),
+        batch_size=user.batch_size if user.batch_size is not None else _auto_batch_size(n, max_seq_len),
         epochs=user.epochs or _auto_epochs(n),
         max_steps=user.max_steps,
         grad_clip=user.grad_clip if user.grad_clip is not None else 1.0,
         warmup_steps=user.warmup_steps if user.warmup_steps is not None else min(100, max(1, n // 10)),
         val_split=user.val_split if user.val_split is not None else (0.1 if n >= 50 else 0.0),
-        patience=user.patience or 5,
+        patience=user.patience if user.patience is not None else 3,
         min_delta=user.min_delta if user.min_delta is not None else 1e-4,
         device=device,
         precision=precision,

{tensorless-0.1.0 → tensorless-0.3.0}/tensorless/config.py

@@ -33,6 +33,8 @@ class TrainConfig:
     ff_mult: Optional[int] = None
     dropout: Optional[float] = None
     max_seq_len: Optional[int] = None
+    tokenizer: Optional[str] = None           # "char" or "bpe"
+    bpe_vocab_size: Optional[int] = None
 
     # --- optimization ---
     optimizer: Optional[str] = None             # "adamw", "adam", "sgd"
@@ -89,6 +91,8 @@ class ResolvedConfig:
     ff_mult: int
     dropout: float
     max_seq_len: int
+    tokenizer: str
+    bpe_vocab_size: int
 
     optimizer: str
     learning_rate: float