tensorless 0.1.0__tar.gz

tensorless-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tensorless Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tensorless-0.1.0/MANIFEST.in

@@ -0,0 +1,5 @@
+include README.md
+include LICENSE
+include pyproject.toml
+recursive-include docs *.md
+recursive-include examples *.py

tensorless-0.1.0/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.4
+Name: tensorless
+Version: 0.1.0
+Summary: ML with maximum automation and minimum setup.
+Author: Tensorless Contributors
+License: MIT
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Dynamic: license-file
+
+# Tensorless
+
+**ML with maximum automation and minimum setup.**
+
+```python
+import tensorless as tl
+
+tl.train("./data")
+```
+
+That's it. Tensorless inspects your dataset, figures out what kind of
+task you're trying to solve, builds and configures a model, trains it,
+validates it, checkpoints it, and saves a single portable `model.tl`
+file you can move anywhere.
+
+```python
+model = tl.run("model.tl")   # interactive chat, if it's a text model
+# or
+model = tl.load("model.tl")
+model.predict(...)
+```
+
+Simple by default. Powerful when you need it:
+
+```python
+tl.train(
+    "./data",
+    d_model=512,
+    layers=6,
+    learning_rate=3e-4,
+    batch_size=32,
+)
+```
+
+## Why Tensorless
+
+Most ML frameworks assume you already know what model you want, how big
+it should be, which optimizer and learning rate to use, and how to wire
+up checkpointing and resumption yourself. Tensorless flips that: it
+makes a reasonable, working choice for all of that automatically, and
+lets you override exactly the parts you care about.
+
+It also remembers what it already did. Run `tl.train("./data")` twice on
+the same dataset and it won't retrain — it'll just hand you back the
+model it already trained. Change the data, and it retrains. Get
+interrupted partway through a long run, and the next call resumes right
+where it left off. This is the **Smart Auto Check**, and it's the core
+idea the whole framework is built around.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+See [docs/installation.md](docs/installation.md) for details and
+requirements.
+
+## Documentation
+
+| Doc | What's in it |
+|---|---|
+| [Installation](docs/installation.md) | Requirements, install steps, verifying your setup |
+| [Quick Start](docs/quickstart.md) | The fastest path to a trained model |
+| [Beginner Tutorial](docs/tutorial.md) | A guided, from-scratch walkthrough |
+| [Automatic Mode](docs/automatic_mode.md) | How auto-detection and auto-configuration work, and the Smart Auto Check |
+| [Training](docs/training.md) | `tl.train()` in depth, all supported tasks and data formats |
+| [Inference](docs/inference.md) | `tl.run()`, `tl.load()`, and the prediction API |
+| [Checkpointing & Resume](docs/checkpointing.md) | How checkpoints work and how resumption is decided |
+| [The `.tl` Format](docs/tl_format.md) | What's inside a `.tl` file and why it's portable |
+| [Configuration](docs/configuration.md) | Every override you can pass, and what it does |
+| [CLI](docs/cli.md) | `tensorless train / run / inspect / info` |
+| [API Reference](docs/api_reference.md) | Full function/class signatures |
+| [Examples](docs/examples.md) | Worked examples for each supported task |
+| [Troubleshooting](docs/troubleshooting.md) | Common errors and what to do about them |
+| [Architecture](docs/architecture.md) | How the codebase is organized, for contributors |
+| [Contributing](docs/contributing.md) | How to add models, backends, or data formats |
+| [Roadmap](docs/roadmap.md) | What's planned |
+| [Limitations](docs/limitations.md) | What Tensorless deliberately doesn't do (yet) |
+
+## Supported today
+
+- **Text generation** (language modeling) from `.txt`/`.md` files or JSON/JSONL with a `text` field
+- **Text classification** from a directory of class subfolders (`positive/`, `negative/`, ...) or labeled JSON/JSONL
+- **Tabular classification and regression** from CSV/TSV/JSON/JSONL with a target column
+
+## Project status
+
+Tensorless is an early-stage, actively developed framework. The core
+loop — inspect, auto-configure, train, checkpoint, save, reload, infer —
+is real and tested end-to-end (see [tests/](tests/)). See
+[docs/limitations.md](docs/limitations.md) for what's intentionally out
+of scope right now, and [docs/roadmap.md](docs/roadmap.md) for what's next.
+
+## License
+
+MIT

tensorless-0.1.0/README.md

@@ -0,0 +1,97 @@
+# Tensorless
+
+**ML with maximum automation and minimum setup.**
+
+```python
+import tensorless as tl
+
+tl.train("./data")
+```
+
+That's it. Tensorless inspects your dataset, figures out what kind of
+task you're trying to solve, builds and configures a model, trains it,
+validates it, checkpoints it, and saves a single portable `model.tl`
+file you can move anywhere.
+
+```python
+model = tl.run("model.tl")   # interactive chat, if it's a text model
+# or
+model = tl.load("model.tl")
+model.predict(...)
+```
+
+Simple by default. Powerful when you need it:
+
+```python
+tl.train(
+    "./data",
+    d_model=512,
+    layers=6,
+    learning_rate=3e-4,
+    batch_size=32,
+)
+```
+
+## Why Tensorless
+
+Most ML frameworks assume you already know what model you want, how big
+it should be, which optimizer and learning rate to use, and how to wire
+up checkpointing and resumption yourself. Tensorless flips that: it
+makes a reasonable, working choice for all of that automatically, and
+lets you override exactly the parts you care about.
+
+It also remembers what it already did. Run `tl.train("./data")` twice on
+the same dataset and it won't retrain — it'll just hand you back the
+model it already trained. Change the data, and it retrains. Get
+interrupted partway through a long run, and the next call resumes right
+where it left off. This is the **Smart Auto Check**, and it's the core
+idea the whole framework is built around.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+See [docs/installation.md](docs/installation.md) for details and
+requirements.
+
+## Documentation
+
+| Doc | What's in it |
+|---|---|
+| [Installation](docs/installation.md) | Requirements, install steps, verifying your setup |
+| [Quick Start](docs/quickstart.md) | The fastest path to a trained model |
+| [Beginner Tutorial](docs/tutorial.md) | A guided, from-scratch walkthrough |
+| [Automatic Mode](docs/automatic_mode.md) | How auto-detection and auto-configuration work, and the Smart Auto Check |
+| [Training](docs/training.md) | `tl.train()` in depth, all supported tasks and data formats |
+| [Inference](docs/inference.md) | `tl.run()`, `tl.load()`, and the prediction API |
+| [Checkpointing & Resume](docs/checkpointing.md) | How checkpoints work and how resumption is decided |
+| [The `.tl` Format](docs/tl_format.md) | What's inside a `.tl` file and why it's portable |
+| [Configuration](docs/configuration.md) | Every override you can pass, and what it does |
+| [CLI](docs/cli.md) | `tensorless train / run / inspect / info` |
+| [API Reference](docs/api_reference.md) | Full function/class signatures |
+| [Examples](docs/examples.md) | Worked examples for each supported task |
+| [Troubleshooting](docs/troubleshooting.md) | Common errors and what to do about them |
+| [Architecture](docs/architecture.md) | How the codebase is organized, for contributors |
+| [Contributing](docs/contributing.md) | How to add models, backends, or data formats |
+| [Roadmap](docs/roadmap.md) | What's planned |
+| [Limitations](docs/limitations.md) | What Tensorless deliberately doesn't do (yet) |
+
+## Supported today
+
+- **Text generation** (language modeling) from `.txt`/`.md` files or JSON/JSONL with a `text` field
+- **Text classification** from a directory of class subfolders (`positive/`, `negative/`, ...) or labeled JSON/JSONL
+- **Tabular classification and regression** from CSV/TSV/JSON/JSONL with a target column
+
+## Project status
+
+Tensorless is an early-stage, actively developed framework. The core
+loop — inspect, auto-configure, train, checkpoint, save, reload, infer —
+is real and tested end-to-end (see [tests/](tests/)). See
+[docs/limitations.md](docs/limitations.md) for what's intentionally out
+of scope right now, and [docs/roadmap.md](docs/roadmap.md) for what's next.
+
+## License
+
+MIT

tensorless-0.1.0/docs/api_reference.md

@@ -0,0 +1,103 @@
+# API Reference
+
+## `tensorless.train(path, **kwargs) -> LoadedModel`
+
+Train a model on the dataset at `path`. See
+[configuration.md](configuration.md) for every valid keyword argument.
+Implements the Smart Auto Check (see [automatic_mode.md](automatic_mode.md)).
+
+Raises:
+- `tensorless.DataError` — dataset missing, empty, malformed, or unsupported format
+- `tensorless.ConfigError` — invalid/unknown config argument, or dataset changed with `ask_on_data_change=True`
+- `tensorless.CheckpointError` — an existing checkpoint is corrupt or unreadable
+- `tensorless.SerializationError` — an existing `.tl` file is corrupt
+
+## `tensorless.inspect(path) -> InspectionReport`
+
+Load and analyze a dataset without training. Prints a human-readable
+report and returns a structured object.
+
+```python
+@dataclass
+class InspectionReport:
+    path: str
+    fingerprint: str
+    kind: str              # "text" | "text_labeled" | "tabular"
+    task: str               # "text-generation" | "text-classification" | "classification" | "regression"
+    n_examples: int
+    n_files: int
+    columns: List[str]
+    sample: Any
+    warnings: List[str]
+    recommendations: List[str]
+    stats: Dict[str, Any]
+```
+
+## `tensorless.load(path, device=None) -> LoadedModel`
+
+Load a trained `.tl` file for inference. `device` overrides the device
+recorded at training time (e.g. load a GPU-trained model on a CPU-only
+machine with `device="cpu"`).
+
+Raises `tensorless.SerializationError` if the file is missing, corrupt,
+or from an unsupported future format version.
+
+## `tensorless.run(path, prompt=None) -> Any`
+
+Convenience wrapper around `load()` for quick command-line-style usage.
+See [inference.md](inference.md#tlrun--the-cli-friendly-shortcut).
+
+## `class tensorless.runtime.LoadedModel`
+
+Returned by both `train()` and `load()`.
+
+| Method | Applies to | Description |
+|---|---|---|
+| `.generate(prompt="", max_new_tokens=200, temperature=0.8, top_k=40)` | `text-generation` | Generate a text continuation |
+| `.chat()` | `text-generation` | Interactive terminal chat loop |
+| `.predict(x)` | all tasks | Unified prediction API; `x` is a string (text tasks) or dict/list-of-dicts (tabular tasks) |
+| `.info()` | all | Dict summary: task, model type, versions, config, metrics, param count |
+
+Attributes: `.task`, `.model_type`, `.config`, `.meta`, `.metrics`,
+`.dataset_fingerprint`, `.model` (the underlying `torch.nn.Module`),
+`.tokenizer` (`CharTokenizer` or `None`), `.preprocessor`
+(`TabularPreprocessor` or `None`).
+
+## `class tensorless.TrainConfig`
+
+The dataclass of every trainable override; see
+[configuration.md](configuration.md) for field-by-field defaults.
+
+## Errors — `tensorless.errors`
+
+All Tensorless exceptions inherit from `TensorlessError`:
+
+```python
+try:
+    tl.train("./data")
+except tl.TensorlessError as e:
+    ...
+```
+
+| Class | Raised when |
+|---|---|
+| `DataError` | Dataset can't be read, is empty, or is malformed |
+| `ConfigError` | Invalid/unknown configuration, or a Smart-Auto-Check conflict |
+| `ModelError` | Unsupported task/model combination, or a runtime prediction-API misuse |
+| `CheckpointError` | Checkpoint missing, corrupt, or incompatible |
+| `SerializationError` | `.tl` file can't be written or read |
+
+## Lower-level modules
+
+These aren't part of the stable public API but are documented here for
+contributors — see [architecture.md](architecture.md) for how they fit
+together:
+
+- `tensorless.data.loader.load_dataset(path) -> Dataset`
+- `tensorless.data.fingerprint.fingerprint_path(path) -> str`
+- `tensorless.auto.detector.detect_task(ds) -> str`
+- `tensorless.auto.config.resolve_config(ds, TrainConfig) -> ResolvedConfig`
+- `tensorless.models.registry.build_model(task, model_type, cfg, meta) -> nn.Module`
+- `tensorless.training.trainer.run_training(...) -> dict`
+- `tensorless.checkpoint.manager.CheckpointManager`
+- `tensorless.serialization.tl_format.save_tl(path, payload)` / `load_tl(path) -> dict`

tensorless-0.1.0/docs/architecture.md

@@ -0,0 +1,125 @@
+# Architecture
+
+This page is for people extending or contributing to Tensorless, not
+end users.
+
+## Package layout
+
+```
+tensorless/
+├── __init__.py           public API surface: train, run, load, inspect
+├── api.py                 orchestration for train()/run(): Smart Auto Check + wiring
+├── config.py               TrainConfig (user-facing) / ResolvedConfig (fully resolved)
+├── errors.py                exception hierarchy
+├── runtime.py               LoadedModel: rebuilds a model from a .tl payload for inference
+├── _version.py               package + .tl format version numbers
+├── data/
+│   ├── loader.py              path -> Dataset (txt/json/jsonl/csv/dirs)
+│   ├── fingerprint.py          content-based dataset hashing
+│   ├── inspector.py             tl.inspect() report generation
+│   └── tabular.py                 TabularPreprocessor: numeric/categorical encoding
+├── auto/
+│   ├── detector.py               Dataset -> task string
+│   └── config.py                   Dataset + TrainConfig -> ResolvedConfig
+├── tokenization/
+│   └── char_tokenizer.py          CharTokenizer: vocab build/encode/decode/save/load
+├── models/
+│   ├── transformer.py              TinyTransformer (GPT-style decoder)
+│   ├── mlp.py                       TabularMLP
+│   └── registry.py                   build_model(task, model_type, cfg, meta)
+├── training/
+│   ├── data_prep.py                  Dataset -> DataLoaders per task
+│   ├── trainer.py                     the training loop (run_training)
+│   └── early_stopping.py               EarlyStopping
+├── checkpoint/
+│   └── manager.py                      CheckpointManager: atomic save/load/clear
+├── serialization/
+│   └── tl_format.py                     save_tl/load_tl: the .tl file format
+├── devices/
+│   └── device.py                         hardware auto-detection + torch.device resolution
+└── cli/
+    └── main.py                            argparse-based CLI
+```
+
+## Data flow for `tl.train("./data")`
+
+```
+path
+  │
+  ▼
+data.loader.load_dataset(path)          -> Dataset (kind, texts/records, columns)
+  │
+  ▼
+auto.detector.detect_task(ds)           -> "text-generation" | "text-classification"
+  │                                          | "classification" | "regression"
+  ▼
+auto.config.resolve_config(ds, TrainConfig) -> ResolvedConfig (every field concrete)
+  │
+  ▼
+training.data_prep.prepare_*(ds, cfg)   -> PreparedData (train/val DataLoaders,
+  │                                          meta, tokenizer/preprocessor)
+  ▼
+models.registry.build_model(task, model_type, cfg, meta) -> nn.Module
+  │
+  ▼
+training.trainer.run_training(...)      -> trains, checkpoints periodically,
+  │                                          returns final weights + metrics
+  ▼
+serialization.tl_format.save_tl(out, payload) -> model.tl
+  │
+  ▼
+runtime.LoadedModel(payload)            -> returned to the caller
+```
+
+`api.train()` wraps this whole flow with the Smart Auto Check (see
+[automatic_mode.md](automatic_mode.md)): before any of the above runs,
+it checks for an existing complete `.tl` file or a resumable checkpoint
+matching the dataset's fingerprint.
+
+## Design principles
+
+- **Every automatic decision is explainable.** Auto-detection and
+  auto-configuration are simple, inspectable heuristics (see
+  `auto/detector.py`, `auto/config.py`), not opaque search or ML-driven
+  meta-learning. Anyone reading the code can predict what Tensorless
+  will choose for a given dataset.
+- **Never silently touch user data.** `data/loader.py` only reads;
+  nothing in the training path writes to or deletes files under the
+  dataset path.
+- **Fail with actionable errors.** All user-facing errors are
+  `TensorlessError` subclasses with messages that say what's wrong and
+  usually what to do about it (see `errors.py` and
+  [troubleshooting.md](troubleshooting.md)).
+- **Checkpoints are self-describing.** A checkpoint carries its own
+  config, meta, and dataset fingerprint, so resuming never depends on
+  external state being reconstructed correctly by the caller.
+- **A `.tl` file is the unit of portability.** Nothing about inference
+  should require the original dataset, training script, or checkpoint
+  directory to still exist.
+
+## Extending Tensorless
+
+### Adding a new model type
+
+1. Implement your `nn.Module` in `models/your_model.py`.
+2. Register it in `models/registry.py`'s `build_model()`.
+3. Add a branch in `training/trainer.py`'s `_compute_loss()` for how to
+   compute loss for your task/model_type combination.
+4. If it needs new data prep, add a `prepare_*()` function in
+   `training/data_prep.py`.
+5. If it changes what needs to go in the `.tl` file, extend the `meta`
+   dict produced by data prep — `build_model()` receives it and can pull
+   out whatever fields it needs.
+
+### Adding a new data format
+
+Add a branch to `data/loader.py`'s `load_dataset()` / `_load_directory()`
+for the new extension, producing a `Dataset` with the appropriate `kind`.
+
+### Adding a new backend/device
+
+Extend `devices/device.py`'s `_*_available()` checks and
+`auto_select_device()` / `get_torch_device()`.
+
+See [contributing.md](contributing.md) for the contribution process
+itself (tests, PRs, etc).

tensorless-0.1.0/docs/automatic_mode.md

@@ -0,0 +1,100 @@
+# Automatic Mode
+
+Tensorless's whole premise is that `tl.train("./data")` should just
+work. This page explains exactly what "automatic" means at each stage,
+so the behavior is predictable rather than magic.
+
+## 1. Task detection
+
+`tensorless/auto/detector.py` looks at the *shape* of your loaded
+dataset (see [training.md](training.md#supported-data-formats) for how
+data is loaded) and picks one of four tasks:
+
+| Dataset shape | Detected task |
+|---|---|
+| Plain text (one or more `.txt`/`.md` files, or JSON/JSONL records with a `text` field and no label) | `text-generation` |
+| Text with labels (class subfolders, or JSON/JSONL with `text` + `label`) | `text-classification` |
+| Tabular data whose target column is numeric with many distinct values | `regression` |
+| Tabular data whose target column is categorical, or numeric with few distinct integer values | `classification` |
+
+The target column for tabular data is chosen by looking for a column
+named `label`, `target`, `class`, `category`, `y`, or `output` (case
+insensitive); if none of those exist, the **last column** in the file is
+used, which is a common convention in tabular datasets.
+
+You can always override detection explicitly: `tl.train("./data",
+task="regression")`.
+
+## 2. Architecture selection
+
+Once the task is known, `tensorless/auto/config.py` picks:
+
+- **model type**: `transformer` for text tasks, `mlp` for tabular tasks
+- **size** (`d_model`, `layers`, `heads`): scaled to dataset size, from
+  a tiny 2-layer/64-dim model for a few hundred examples up to an
+  8-layer/384-dim model for 50,000+ examples
+
+This is a heuristic, not a search — the goal is "a model that trains
+quickly and doesn't wildly overfit or underfit for typical dataset
+sizes," not the best possible architecture. Override any of it:
+`tl.train("./data", d_model=512, layers=6)`.
+
+## 3. Hyperparameter selection
+
+Batch size, epoch count, learning rate, warmup steps, and the
+validation split are all similarly scaled to dataset size. See
+[configuration.md](configuration.md) for the exact defaults and how to
+override each one.
+
+## 4. Hardware selection
+
+`tensorless/devices/device.py` picks, in order: TPU (if `torch_xla` is
+installed and usable) → CUDA GPU (if available) → Apple MPS (if
+available) → CPU. Precision is chosen alongside it: `bf16` on TPU,
+`bf16` or `fp16` on GPU depending on hardware support, and `fp32`
+everywhere else for stability.
+
+This selection isn't blind — Tensorless actually checks each backend is
+usable (not just importable) before choosing it, and if a device you
+explicitly request isn't available, it downgrades gracefully rather
+than erroring.
+
+## 5. The Smart Auto Check
+
+This is the part that makes repeated `tl.train("./data")` calls safe and
+cheap. Every dataset gets a **fingerprint**: a hash of every file's
+content, size, and relative path under the given directory (see
+`tensorless/data/fingerprint.py`). This fingerprint is content-based, not
+timestamp-based, so touching a file without changing it, or copying a
+dataset to a new machine, doesn't trigger a false "changed" signal.
+
+When you call `tl.train(path, out="model.tl")`, Tensorless checks, in
+order:
+
+1. **Does `model.tl` already exist, with a fingerprint matching the
+   current dataset, and `training_complete=True`?**
+   → Return it immediately. No training happens.
+
+2. **Is there a checkpoint at `model.tl.ckpt/` whose fingerprint matches
+   the current dataset?**
+   - If that checkpoint's training wasn't finished
+     (`training_complete=False`) → **resume** from it.
+   - If it was actually finished but the final `.tl` file is missing
+     (e.g. the process died right after the last checkpoint write, before
+     the `.tl` file could be written) → package that checkpoint into
+     `model.tl` directly, no retraining needed.
+
+3. **Is there a checkpoint whose fingerprint does *not* match?** → The
+   dataset changed since that checkpoint was created.
+   - By default, Tensorless retrains from scratch automatically (and
+     prints a message explaining why).
+   - Pass `ask_on_data_change=True` to instead raise a `ConfigError`
+     and let you decide, rather than silently retraining.
+
+4. **None of the above?** → Train from scratch.
+
+`force=True` skips all of this and always retrains from scratch,
+clearing any existing checkpoint first.
+
+See [checkpointing.md](checkpointing.md) for what's actually inside a
+checkpoint and exactly how resumption reconstructs training state.

tensorless-0.1.0/docs/checkpointing.md

@@ -0,0 +1,76 @@
+# Checkpointing & Resume
+
+## Where checkpoints live
+
+Every training run writes to a checkpoint directory, by default
+`<out>.ckpt/` (e.g. `model.tl.ckpt/checkpoint.pt`). You never need to
+create or manage this directory yourself.
+
+## What's in a checkpoint
+
+`tensorless/checkpoint/manager.py` writes a single file,
+`checkpoint.pt`, containing:
+
+- `model_state_dict` — model weights
+- `optimizer_state_dict` — optimizer momentum/variance buffers
+- `scheduler_state_dict` — learning rate schedule position
+- `epoch`, `global_step` — where training left off
+- `early_stopping_best`, `early_stopping_bad_checks` — early stopping state
+- `config` — the fully resolved training configuration used
+- `meta` — task-specific sizing info (vocab size, number of classes, etc.)
+- `tokenizer_state` / `preprocessor_state` — whichever applies to the task
+- `dataset_fingerprint` — the fingerprint of the dataset this checkpoint
+  was trained on (see [automatic_mode.md](automatic_mode.md#5-the-smart-auto-check))
+- `training_complete` — whether this checkpoint represents a finished run
+
+This is everything needed to either resume training or reconstruct the
+final `.tl` file — nothing about resumption depends on the original
+dataset still being on disk in the same location, only on it being
+fingerprint-identical to what was originally used.
+
+## When checkpoints are written
+
+- Every `checkpoint_every` steps (default: 50) during training, with
+  `training_complete=False`
+- At the end of every epoch, with `training_complete` set to whether
+  that was the last epoch
+- On early stopping, with `training_complete=True`
+
+Writes are atomic: Tensorless writes to a temporary file in the same
+directory and renames it into place, so a crash mid-write never leaves a
+corrupt checkpoint that would block resumption.
+
+## How resumption works
+
+When `tl.train()` finds a checkpoint whose `dataset_fingerprint` matches
+the current dataset and `training_complete=False`, it:
+
+1. Loads the tokenizer/preprocessor state from the checkpoint (so the
+   vocabulary or column encoding is identical to the original run)
+2. Rebuilds the exact same model architecture from the checkpoint's
+   saved `config`
+3. Loads model, optimizer, and scheduler state
+4. Continues the training loop from the saved `epoch` / `global_step`
+
+**Important:** any config overrides you pass to the resuming
+`tl.train()` call (e.g. a different `d_model`) are ignored in favor of
+the checkpoint's original config. This is intentional — a resumed run
+must use the same architecture as the interrupted run, or the saved
+weights simply won't fit. If you want different hyperparameters,
+either delete the checkpoint directory first or pass `force=True`.
+
+## Manually clearing a checkpoint
+
+```python
+from tensorless.checkpoint.manager import CheckpointManager
+CheckpointManager("model.tl.ckpt").clear()
+```
+
+or simply delete the directory:
+
+```bash
+rm -rf model.tl.ckpt
+```
+
+`tl.train(..., force=True)` does this for you automatically before
+retraining.