altamt 1.0__tar.gz

altamt-1.0/LICENSE

@@ -0,0 +1,15 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

altamt-1.0/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: altamt
+Version: 1.0
+Summary: ALTAMT: Advanced Lightweight Translation AI Model Transformer - bidirectional Kinyarwanda <-> English machine translation optimized for CPU inference.
+Author: ALTAMT Contributors
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/altamt-ai/altamt
+Project-URL: Documentation, https://github.com/altamt-ai/altamt#readme
+Project-URL: Issues, https://github.com/altamt-ai/altamt/issues
+Keywords: machine-translation,kinyarwanda,nlp,transformer,low-resource,onnx,quantization
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.1
+Requires-Dist: sentencepiece>=0.1.99
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2.0
+Requires-Dist: pyarrow>=14.0
+Requires-Dist: sacrebleu>=2.4
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: tqdm>=4.66
+Provides-Extra: onnx
+Requires-Dist: onnx>=1.15; extra == "onnx"
+Requires-Dist: onnxruntime>=1.17; extra == "onnx"
+Provides-Extra: benchmark
+Requires-Dist: transformers>=4.40; extra == "benchmark"
+Requires-Dist: evaluate>=0.4; extra == "benchmark"
+Requires-Dist: psutil>=5.9; extra == "benchmark"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: mypy>=1.8; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Provides-Extra: all
+Requires-Dist: altamt[benchmark,dev,onnx]; extra == "all"
+Dynamic: license-file
+
+# altamt
+
+**Bidirectional Kinyarwanda ⇄ English machine translation — fast on your CPU.**
+
+`altamt` (Advanced Lightweight Translation AI Model Transformer) is a compact (~130–195M parameter) modern Transformer (RMSNorm · SwiGLU · Grouped-Query Attention · RoPE) that translates in **both directions with one model** and **auto-detects the input language**. No GPU required.
+
+## Installation
+
+```bash
+pip install altamt
+# optional extras
+pip install "altamt[onnx]"        # ONNX Runtime export & inference
+pip install "altamt[benchmark]"   # compare against NLLB / Opus-MT baselines
+```
+
+## Quickstart (Python)
+
+```python
+from altamt import Translator
+
+translator = Translator(model_path="path/to/checkpoint")
+
+# Language auto-detection: just pass text.
+result = translator.translate("Mwaramutse nshuti zanjye")
+print(result)
+# {'translated_text': 'Good morning my friends',
+#  'detected_src': 'rw', 'tgt': 'en', 'latency_ms': 42.1}
+
+# Or pin the direction explicitly:
+translator.translate("How are you today?", src_lang="en", tgt_lang="rw")
+
+# Batch translation — directions can even be mixed in one batch:
+translator.translate_batch(["Mwaramutse", "Good morning"])
+```
+
+### Make it faster: INT8 quantization
+
+```python
+translator = Translator("path/to/checkpoint", quantize=True, num_threads=8)
+translator.warmup()
+```
+
+INT8 dynamic quantization typically gives ~2× lower latency and ~4× smaller weight matrices on x86 CPUs, with a negligible quality drop.
+
+## Quickstart (CLI)
+
+```bash
+altamt translate "Mwaramutse nshuti zanjye" --model path/to/checkpoint
+# Good morning my friends
+# [rw -> en | 42.1 ms]
+
+altamt translate "Hello" --model path/to/checkpoint --tgt-lang rw --int8 --json
+
+altamt benchmark  --model path/to/checkpoint --test-file test.parquet --int8
+altamt export-onnx --model path/to/checkpoint --output-dir ./onnx_model
+altamt train      --config config.yaml
+```
+
+## Why altamt?
+
+- **One model, both directions** — a target-language tag (`<2en>` / `<2rw>`) steers the decoder, so RW→EN and EN→RW share all parameters and vocabulary.
+- **Automatic language detection** — a built-in, microsecond-fast character n-gram detector picks the direction when you don't.
+- **Built for CPU** — Grouped-Query Attention + full KV-cached decoding + INT8 quantization target <100 ms per sentence on a modern 8-core CPU (beam=1, short sentences).
+- **Multi-format data** — training and benchmarking read `.json`, `.jsonl` and `.parquet` interchangeably.
+- **Honest benchmarking** — `altamt benchmark` reports SacreBLEU, chrF++, latency, throughput and memory, and can run NLLB/Opus-MT baselines through the same harness, emitting Markdown and LaTeX tables.
+
+## Indicative CPU performance
+
+Numbers depend on your hardware, sentence length, beam size and thread count; measure on your machine with `altamt benchmark`. On a modern 8-core x86 CPU (beam=1, ~20-token sentences, `num_threads=8`), the INT8 `base` model targets **<100 ms/sentence**, with fp32 roughly 2× slower and ~780 MB peak RAM.
+
+## Documentation
+
+Full architecture details, data format specs, training and paper-publishing guides live in the [GitHub README](https://github.com/altamt-ai/altamt#readme).
+
+## License
+
+Apache 2.0

altamt-1.0/README.md

@@ -0,0 +1,361 @@
+# ALTAMT — Advanced Lightweight Translation AI Model Transformer
+
+**Multilingual machine translation centered on Kinyarwanda ⇄ English, engineered for sub-100ms CPU inference.**
+
+One shared model can now serve any set of language pairs (e.g. `rw-en`, `en-rw`, `en-fr`, `fr-en`), auto-detect the source language at inference, and transparently **pivot** through a bridge language (default English) for pairs it was never trained on (`rw→fr` via `rw→en→fr`). Training scales from a laptop to multi-node clusters via **DDP** and **DeepSpeed**.
+
+This is the developer README (architecture, data, training, benchmarking, publishing). For the user-facing quickstart, see [`README_pypi.md`](README_pypi.md).
+
+---
+
+## 1. Architecture
+
+ALTAMT is a modern encoder–decoder Transformer. Every architectural decision below exists either to raise quality per parameter (low-resource setting) or to cut CPU decoding latency.
+
+### 1.1 Components
+
+| Component | Choice | Why |
+|---|---|---|
+| Normalization | **RMSNorm**, pre-norm | Cheaper than LayerNorm (no mean/bias), more stable deep stacks |
+| FFN | **SwiGLU** | Consistently better quality/param than ReLU/GELU FFNs |
+| Attention | **Grouped-Query Attention** (12 Q heads / 4 KV heads in `base`) | 3× smaller KV projections and KV cache → the decode loop is memory-bandwidth-bound on CPU, so this directly cuts latency |
+| Positions | **RoPE** in encoder & decoder *self*-attention | No learned position table, clean length extrapolation. Cross-attention is position-free (standard) |
+| Embeddings | **Three-way weight tying** (encoder emb = decoder emb = LM head) | Saves ~24M params at 32k vocab; regularizes the low-resource setting |
+| Kernel | `torch.scaled_dot_product_attention` + **FlashAttention** | Causality is declared via `is_causal=True` (training) or omitted entirely (single-token cached steps, unpadded batches) instead of materializing mask tensors — mask-free calls are what makes the flash kernel eligible; masked encoder/cross attention uses the fused memory-efficient kernel. Combined with GQA via `enable_gqa=True`. A test suite pins all paths (full pass, cached steps, chunked prefill, padded/unpadded) to identical numerics |
+| Decoding | **Full KV cache** (growing self-attn cache + one-shot cross-attn cache) | O(T) decoding instead of O(T²) forwards — the single biggest CPU win |
+| Compression | **INT8 dynamic quantization** + **ONNX export** | ~2× speedup / ~4× smaller weights on x86; portable runtime |
+
+### 1.2 One model, many directions
+
+Instead of one model per pair, ALTAMT trains a **single shared model** tagged on **both** sides (NLLB style):
+
+```
+encoder input : <2rw> SP(src_text) </s>     # source-language tag
+decoder input : <2en> SP(tgt_text)          # forced-BOS target-language tag
+labels        : SP(tgt_text) </s>
+```
+
+The **target** tag is the decoder's forced first token: generation simply *starts* from the right tag, so a **single batch can mix arbitrary directions** (each row carries its own start tag) — which `Translator.translate_batch` exploits, including for pivot-routed rows.
+
+The **source** tag matters once you have more than two languages. Without it the encoder must infer the input language from content alone, and that is precisely where same-script pairs lose accuracy: English and French share the alphabet, punctuation, digits, proper nouns and thousands of cognates, so a short or entity-heavy sentence is genuinely ambiguous. Both tags come from the same `<2xx>` piece inventory — position disambiguates them — so no tokenizer change is needed. The setting lives in `data.add_src_lang_tag` and is **recorded in the checkpoint's `vocab_config.json`**, so `Translator` configures itself to match how the model was trained; a silent mismatch here degrades quality with no error, which is why it is never left to the caller to remember.
+
+Language tags are **dynamic**: one `<2xx>` piece per language in `data.languages` is reserved as a SentencePiece user-defined symbol at contiguous ids starting at 4, and both tokenizer and model self-discover the language set from the vocabulary at load time — nothing about `rw`/`en` is hard-coded.
+
+All languages share one 32k SentencePiece BPE vocabulary (`byte_fallback=True`, `character_coverage=1.0`), so cognates, named entities and numbers share subwords, and no input can ever fail to encode. The pair inventory a checkpoint was actually trained on is recorded in its bundled `vocab_config.json` (`trained_pairs`) and drives pivot routing at inference — with direct `rw↔fr` bitext in the corpus, that pair is trained directly and no pivot (and no compounding of pivot errors) is involved.
+
+### 1.3 Presets & parameter budget
+
+| Preset | d_model | layers (enc/dec) | heads (Q/KV) | d_ff | Params |
+|---|---|---|---|---|---|
+| `small` (`configs/small.yaml`) | 640 | 10 / 10 | 10 / 2 | 2048 | ~130M |
+| `base` (`configs/base.yaml`) | 768 | **16 / 8** | 12 / 4 | 3072 | ~245M |
+| `tiny` (tests/CI only) | 64 | 2 / 2 | 4 / 2 | 128 | ~0.2M |
+
+`base` uses a **deep encoder and a shallow decoder** rather than a symmetric stack. For translation this is close to free quality: encoder depth is where translation accuracy comes from, while autoregressive decoding cost scales with *decoder* depth × output length. Moving layers from the decoder to the encoder therefore raises quality **and** keeps decoding fast — 16/8 at `d_ff=3072` decodes at roughly the same speed as the old symmetric 12/12 at `d_ff=2048` while carrying substantially more encoder capacity.
+
+`small` is deliberately a good **distillation student**: a strong recipe is to distill from NLLB-200-1.3B/3.3B teacher translations of monolingual Kinyarwanda text (sequence-level KD), then fine-tune on clean bitext.
+
+### 1.4 Language identification
+
+`altamt.models.lid.LanguageDetector` is a **dependency-free** character-trigram Naive Bayes classifier with a function-word prior. It ships with built-in seed profiles for `rw`, `en` and `fr` (works out of the box, microsecond latency) and can be retrained on your corpora:
+
+```python
+from altamt.models import LanguageDetector
+det = LanguageDetector.train({"rw": ["mono.rw.txt"], "en": ["mono.en.txt"]})
+det.save("lid_profile.json")
+```
+
+It powers **source-language auto-detection at inference**: when `src_lang` is omitted, the `Translator` restricts the detector to the model's supported languages (minus the requested target) via `detect(text, allowed=...)`, and prints a stderr notice such as `[altamt] auto-detected source language: 'rw'`. It is also used (optionally) as a **bitext quality filter** in preprocessing (`clean_corpus(lid_filter=True)`), dropping pairs whose source text clearly isn't in the declared source language.
+
+---
+
+## 2. Repository layout
+
+```
+altamt/
+├── pyproject.toml            # packaging (setuptools, src layout)
+├── configs/                  # ready-to-edit training configs
+├── src/altamt/
+│   ├── cli.py                # altamt {train,translate,translate-doc,benchmark,export-onnx,quantize,train-tokenizer,doctor}
+│   ├── config.py             # typed dataclasses + YAML I/O + presets
+│   ├── data/                 # dataset.py (json/jsonl/parquet), preprocessor.py, tokenizer.py
+│   ├── models/               # modules.py (RMSNorm/SwiGLU/RoPE/GQA), transformer.py, lid.py
+│   ├── training/             # trainer.py (AMP, cosine LR, checkpoints), utils.py
+│   ├── inference/            # engine.py (Translator), quantize.py (INT8 + ONNX)
+│   └── benchmarking/         # evaluator.py (BLEU/chrF++/latency/RAM), report_generator.py
+└── tests/                    # pytest suite (tiny model, trains a 300-piece tokenizer)
+```
+
+## 3. Developer setup
+
+```bash
+git clone https://github.com/altamt-ai/altamt && cd altamt
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[all]"        # core + onnx + benchmark + dev
+pytest                          # full suite runs on CPU in <1 min
+```
+
+## 4. Data format specification
+
+All loaders accept **`.json`**, **`.jsonl`** and **`.parquet`** interchangeably (mixable within one run). Required columns/keys:
+
+| key | type | values |
+|---|---|---|
+| `src_lang` | str | any code in `data.languages` (e.g. `"rw"`, `"en"`, `"fr"`) |
+| `tgt_lang` | str | any code in `data.languages` |
+| `src_text` | str | source sentence |
+| `tgt_text` | str | reference translation |
+
+The language inventory is declared once in the config (`data.languages: [rw, en, fr]`); rows in other languages are rejected at load time, and mixing many pairs in one corpus (or across several `train_files`) is fully supported — each row is its own direction.
+
+**Every pair is trained both ways.** By default (`data.ensure_bidirectional: true`) any direction whose reverse is missing gets flipped rows added automatically at load time — a corpus with `en->fr` rows but no `fr->en` rows trains `fr->en` too (you'll see a `[altamt] added missing reverse direction(s) fr-en ...` notice). Pairs that already exist in both directions are never duplicated, unlike `add_reverse_direction: true`, which flips *every* row (use that when your corpus deliberately lists each pair only once). The guarantee applies to the validation set as well, so eval metrics and sample blocks cover both directions of every pair.
+
+**JSONL** (recommended for large corpora):
+```json
+{"src_lang": "rw", "tgt_lang": "en", "src_text": "Mwaramutse", "tgt_text": "Good morning"}
+```
+
+**JSON** — either a top-level array of such records, or `{"data": [...]}`.
+
+**Parquet** — same four columns; written e.g. via `df.to_parquet("train.parquet")`.
+
+If your corpus lists each pair only once, set `data.add_reverse_direction: true` to train both directions from it.
+
+### Recommended public sources
+Digital Umuganda / Mbaza datasets, JW300 (rw–en), Flores-200 dev/devtest (evaluation), TICO-19, plus back-translated monolingual Kinyarwanda news/wiki text. Always hold Flores-200 devtest out for reporting.
+
+### Cleaning
+```python
+from altamt.data import load_parallel_files, clean_corpus
+df = load_parallel_files(["raw.parquet", "extra.jsonl"])
+df, stats = clean_corpus(df, max_len_ratio=3.0, lid_filter=True)
+print(stats.summary())
+df.to_parquet("data/train.parquet")
+```
+
+## 5. Training locally
+
+```bash
+# 1. Train the shared 32k tokenizer straight from dataset files.
+#    --output may be a DIRECTORY: all artifacts (altamt.model, altamt.vocab,
+#    vocab_config.json) are written directly into it.
+altamt train-tokenizer --files data/train.parquet --output tokenizer/ \
+    --languages rw en fr
+
+#    ...or drive it entirely from the config's tokenizer: section
+#    (corpus = data.train_files, output = data.tokenizer_path,
+#     languages = data.languages):
+altamt train-tokenizer --config configs/base.yaml
+
+# 2. Edit configs/base.yaml (paths, batch size), then:
+altamt train --config configs/base.yaml
+```
+
+**Terminal overrides.** Every configuration value can be overridden on the command line — CLI values take precedence over the YAML file, are type-checked against the config schema, re-validated exactly like YAML values, and echoed at startup (`[altamt] config override: training.lr: 0.0005 -> 0.0001`). Two equivalent syntaxes, freely mixable:
+
+```bash
+altamt train --config configs/base.yaml \
+    --set training.lr=1e-4 -s training.batch_size=32 -s data.languages=rw,en,fr
+
+altamt train --config configs/base.yaml \
+    --training.lr 1e-4 --training.batch_size 32 --data.languages rw,en,fr
+```
+
+Bare field names work when unique across sections (`-s lr=1e-4`); ambiguous ones (`vocab_size` exists in both `model` and `tokenizer`) must be qualified. Lists accept CSV or YAML (`rw,en,fr` / `"[rw, en, fr]"`), booleans accept `true/false/yes/no/on/off/1/0`, and `null` clears optional values (`-s training.finetune_from=null`). Typos get did-you-mean suggestions. `altamt train-tokenizer` supports the same mechanism (`--tokenizer.vocab_size 16000`), with or without `--config`.
+
+The most common data/output settings also have **dedicated flags** — usable for fresh training, `--resume-from` runs, and `--finetune-from` runs alike, and taking precedence over `--set`:
+
+```bash
+altamt train --config configs/base.yaml \
+    --train-files data/a.parquet data/b.jsonl \
+    --valid-files data/val.jsonl \
+    --tokenizer-path tokenizer/altamt.model \
+    --languages rw en fr \
+    --output-dir runs/exp2
+```
+
+File lists and languages accept space- or comma-separated values; `--valid-files none` disables validation for the run. Like every override, dedicated flags are validated against the config schema, echoed at startup, and baked into the `config.yaml` written to the run directory and its checkpoints.
+
+**Adding languages to an existing tokenizer.** `--extend-from old_tokenizer/` retrains on the new corpus while preserving the original language-tag order and appending new `<2xx>` tags, so existing tag ids stay stable; `build_embedding_remap(old, new)` then maps shared pieces so pretrained embeddings can be transplanted.
+
+**Fine-tuning.** Set `training.finetune_from: checkpoints/altamt-base/best` (or pass `--finetune-from`) to initialize from a pretrained checkpoint: weights are loaded, a fresh optimizer/scheduler is built, `global_step` restarts at 0, and regularization knobs (`dropout`, `attention_dropout`, `label_smoothing`) are taken from *your* config while the architecture comes from the checkpoint. If your tokenizer differs from the checkpoint's (e.g. you extended it with French), embeddings are automatically remapped and resized — new pieces get fresh init, shared pieces keep their trained rows. This is exactly how you add a language pair to an existing bilingual model.
+
+**Resuming interrupted training.** Any checkpoint directory saved by the trainer is fully resumable — just point `--resume-from` (or `training.resume_from`) at it:
+
+```bash
+# resume from a checkpoint directory (best/ or last/, or a shared copy of one)
+altamt train --config configs/base.yaml --resume-from runs/base/best
+
+# resume from a rotating step checkpoint, or the run dir (auto-picks the newest)
+altamt train --config configs/base.yaml --resume-from runs/base/step-24000.pt
+altamt train --config configs/base.yaml --resume-from runs/base
+
+# DeepSpeed runs resume from their ZeRO-sharded engine checkpoints
+altamt train --config configs/base.yaml --resume-from runs/base/ds_checkpoints
+altamt train --config configs/base.yaml --resume-from runs/base/ds_checkpoints@step-24000
+```
+
+Resume restores everything mid-run: model weights, optimizer and LR-scheduler state, the AMP grad scaler, `global_step`, the best validation loss, and the accumulated **total training time** (logs and `metrics.jsonl` keep counting from where the previous run stopped; the startup notice shows the prior training time). Checkpoint directories from older versions of this code lack `training_state.pt` (weights only); resuming from one produces an error naming the correct step checkpoint. Use `--finetune-from` instead of `--resume-from` when you want to start a *fresh* run (new schedule, step 0) initialized from a checkpoint's weights.
+
+**Distributed training.** `training.distributed: auto` (default) picks up `torchrun`/`deepspeed` launch environments automatically; `none`, `ddp` and `deepspeed` force a mode. Gradients sync via DDP or DeepSpeed ZeRO (`deepspeed_stage: 2` default, stage 3 supported, or point `deepspeed_config` at your own JSON); data is sharded with a `DistributedSampler`, validation losses are all-reduced, and logging/samples/checkpoint writes happen on rank 0 (ZeRO-3 weight consolidation is handled collectively).
+
+```bash
+# single node, 4 GPUs
+torchrun --nproc-per-node 4 -m altamt.cli train --config configs/base.yaml
+
+# 2 nodes × 8 GPUs
+torchrun --nnodes 2 --nproc-per-node 8 --rdzv-backend c10d \
+         --rdzv-endpoint $MASTER:29500 -m altamt.cli train --config configs/base.yaml
+
+# DeepSpeed ZeRO
+deepspeed --num_gpus 8 -m altamt.cli train --config configs/base.yaml
+```
+
+**Multi-GPU efficiency.** Gradients are all-reduced **once per optimizer step** (non-final accumulation micro-batches run under `DDP.no_sync()`), buffer broadcasts are disabled (all buffers are deterministic caches), gradients live directly in the allreduce buckets (`gradient_as_bucket_view`), and the NCCL communicator is bound to each rank's GPU explicitly (`device_id` in `init_process_group` — this also removes the *"Guessing device ID based on global rank"* warning). The **quality eval is sharded across ranks**: every rank decodes a disjoint slice of the selected validation sentences and the shards are merged with one `all_gather`, so all GPUs work on the eval *and* no rank ever idles at a barrier while another decodes alone — the latter is what triggers NCCL's 10-minute watchdog (*"Watchdog caught collective operation timeout"*) and kills the whole run. As a backstop the collective timeout is configurable (`training.collective_timeout_min`, default 120). The startup header proves the rank→GPU mapping, e.g. `device cuda:0 + cuda:1 (2× NVIDIA H200)`. Every progress log line reports **`data-wait %`** — the fraction of time the GPU sat idle waiting for batches; if it exceeds ~30% you'll get a one-time hint to raise `data.num_workers` (workers are persistent and prefetch 4 batches each). Two heartbeats print at the start of every run: the first-step time (includes pipeline warmup, so its projection is pessimistic) and, at step 10, the **steady-state s/step with a warmup-free ETA** — trust the second one. If per-step time stays high with low data-wait on a multi-GPU cloud box, the interconnect is the limit: check `nvidia-smi topo -m` (PCIe-only pairs without P2P pay heavily per allreduce; larger `grad_accum_steps` amortizes it further).
+
+**If the startup header says `device cpu` on a GPU machine, stop the run.** It means `torch.cuda.is_available()` returned False — training a full-size model on CPU takes minutes per optimizer step and looks frozen. altamt now handles this end to end with the **environment doctor**:
+
+```bash
+altamt doctor            # diagnose: wheel CUDA vs driver CUDA, GPUs, status, exact fix
+altamt doctor --fix      # reinstall the torch wheel matching your driver (asks first; -y to skip)
+altamt train ... --auto-fix-torch    # single-process runs: fix + relaunch automatically
+```
+
+The doctor distinguishes the cases precisely: `driver_too_old` (e.g. `torch 2.13.0+cu130` on a CUDA-12.8 driver — the classic silent-CPU trap), `cpu_build` (CPU-only torch on a GPU machine), and `mismatch` (compatible wheel but CUDA still blocked — container without `--gpus`, `CUDA_VISIBLE_DEVICES`, etc.; diagnosed but never blindly reinstalled). It computes the newest published wheel your driver can run (cu126/cu128/cu130/...) and prints the exact `pip` commands. `--fix` verifies success in a fresh interpreter afterwards. Safety rails: the fix never runs from inside a multi-rank `torchrun`/`deepspeed` launch (N ranks would race pip — run `altamt doctor --fix` once standalone instead), and `--auto-fix-torch` re-execs the training command after reinstalling so the new build is actually used.
+
+**CPU machines and CPU inference stay first-class and quiet.** A box with no GPUs gets status `ok_cpu` — no warnings, no fix suggestions; `Translator` defaults to `device="cpu"` silently (that's the optimized path). Requesting `device="cuda"` at inference when CUDA is unavailable raises a clear error containing the fix; `device="auto"` falls back to CPU, adding a single stderr note only when GPUs are physically present but unusable (suppressed with `verbose=False`). Set `training.device: cuda` (or pass `--training.device cuda`) to make silent CPU fallback a hard error during training.
+
+Tokenizer hyper-parameters (`vocab_size`, `model_type`, `character_coverage`, `byte_fallback`, `hard_vocab_limit`, ...) live in the `tokenizer:` section of the YAML config (`altamt.config.TokenizerConfig`); CLI flags like `--vocab-size` override it. If SentencePiece reports *"Vocabulary size too high (32000). Please set it to a value <= N"*, your corpus is too small for that many pieces — lower `tokenizer.vocab_size` to ≤ N, add more data, or pass `--no-hard-vocab-limit` (config: `hard_vocab_limit: false`) to auto-shrink to the largest achievable vocabulary. `altamt train` then reconciles `model.vocab_size` with the trained tokenizer automatically.
+
+**Batching: token budget, not sentence count.** The corpus spans 3–350 words per sentence, and a padded batch is as wide as its longest member — so a fixed *sentence* count spends most of every tensor on padding. On this corpus, 64-sentence random batches are only **~24 % real tokens**. `training.max_tokens` switches to length-grouped, token-budget batching (`TokenBucketBatchSampler`: shuffle → chunk → sort by length within a chunk → fill to the token budget → shuffle batch order), which reaches **~97 % real tokens**. At the same peak tensor size that is **~4× less padded compute per epoch** — the largest quality-per-hour lever in this trainer. `batch_size` then acts only as an upper bound on sentences per batch, and step counts are equalized across ranks so DDP can never deadlock on a ragged tail.
+
+**Data is tokenized once.** Both sides of the corpus are encoded up front into flat `int32` arrays and cached on disk (`.altamt_cache/`, keyed on file mtimes + tokenizer + options), so `__getitem__` is a pure array slice. SentencePiece leaves the hot loop entirely (`data-wait %` goes to ~0 with `num_workers: 2`), exact token lengths are known before training, and over-long pairs can be **dropped** rather than truncated — `length_policy: drop`. This matters: truncating a target teaches the model to stop mid-sentence, which later shows up as unfinished translations.
+
+The trainer gives you bf16/fp16 autocast, gradient accumulation (effective batch = `max_tokens × grad_accum_steps × world_size`), linear-warmup + cosine LR, label smoothing 0.1, grad clipping, JSONL metric logs (`metrics.jsonl`), rotating step checkpoints (`step-N.pt`), and **fully self-contained, shareable** `best/` and `last/` checkpoint directories bundling: `model.pt` (weights) + `training_state.pt` (optimizer, LR scheduler, grad scaler, step counter, best val loss, cumulative training time) + `config.json` + `config.yaml` + the SentencePiece tokenizer + `vocab_config.json` (languages, tag ids, trained pairs) + `training_summary.json` (provenance). One directory is everything anyone needs to **serve** (`Translator(path)`), **resume** (`--resume-from path`), or **fine-tune** (`--finetune-from path`) — delete `training_state.pt` if you want a smaller inference-only copy. (DeepSpeed runs shard optimizer state across ranks, so they resume from `ds_checkpoints` instead.)
+
+Terminal logging shows **total training time** (cumulative across resumed runs), progress %, loss/ppl/lr/grad-norm/tokens-per-sec, `data-wait %`, and a live ETA on every log line. Time is tracked as a first-class metric alongside ETA: each progress line leads with the labeled time trained so far and ends with the projected total wall time, e.g.
+
+```
+[1d 02:13 trained] step 20,000/100,000 20.0% | loss 2.0000 | ppl 7.4 | lr 5.00e-04 | grad 0.50 | 24,000 tok/s | data-wait 5% | eta 04:26:40 | total ≈ 1d 06:40
+```
+
+so "we've been training for a day, ~4.5 hours to go, ~1d 6h total" is readable at a glance (and stays correct across `--resume-from`, since elapsed time is cumulative). The same figures land in `metrics.jsonl` — `elapsed`/`elapsed_sec`, `eta`/`eta_sec`, `projected_total`/`projected_total_sec`, and `time_progress_pct` (share of the projected total already completed) — and total training time is also persisted in every step checkpoint (so it keeps accumulating after `--resume-from`) and in a `training_summary.json` inside `best/` and `last/`. Each validation pass reports `val_loss`/`val_ppl` and prints a per-direction quality table covering **every trained language pair** — the `eval_gen_samples` generation budget is split evenly across directions, so no pair is ever missing from the metrics:
+
+```
+[02:14:09] eval @ step 24,000  val_loss 2.3141 | val_ppl 10.1 | eval took 42s | total training time 02:14:09 | ✓ new best
+    direction      BLEU   chrF++  sentences
+    en -> fr      21.10    44.02         85
+    en -> rw      25.56    48.92         85
+    rw -> en      28.04    46.29         86
+```
+
+BLEU is corpus-level **SacreBLEU** and chrF++ is `CHRF(word_order=2)`; both are 0–100, higher is better (a legend prints under the first table). The same numbers are stored in `metrics.jsonl` as a readable nested record: `{"quality": {"rw-en": {"bleu": 28.04, "chrf++": 46.29, "sentences": 86}, ...}}`.
+
+**Reading the eval numbers honestly.** Three settings decide whether the reported BLEU reflects the model or the measurement:
+
+* `eval_gen_samples` (default **-1 = the entire validation set**, the most faithful SacreBLEU; the eval pause grows with the set size). Set e.g. `1024` for a fast fixed random sample split evenly across directions — a few dozen sentences per direction makes corpus BLEU swing ±3–5 points between evals, so don't go below ~256 per direction when comparing checkpoints.
+* `gen_max_new_tokens` / `eval_length_ratio`. The budget is `min(gen_max_new_tokens, src_tokens × eval_length_ratio + 16)` — computed per batch from the actual source length. A small *fixed* budget silently truncates long hypotheses, and BLEU's brevity penalty then reports a score far below the model's real quality. This is the single most common reason in-training BLEU looks disappointing.
+* `eval_beam_size` (default 1). Greedy keeps the in-training curve cheap and comparable across steps; **beam 4 typically scores 1–2 BLEU higher**. Use `altamt benchmark` for the number you quote.
+
+The scored sentences are a **seeded random sample** per direction (`eval_seed`), not the first N rows — the head of a validation file is usually one domain or one source corpus. References are the raw target strings, not tokenizer round-trips. The trainer also prints fixed sample translations (source / hypothesis / reference — the same sentences every time, so quality progress is directly comparable). Sample blocks cover **every training direction** by default (`training.sample_directions: all`), grouped with one labeled sub-block per direction, so all trained languages (rw/en/fr/...) appear in the logs; `num_samples` sentences are shown per direction. Restrict it with a single pair (`sample_directions: rw-en`), a list (`[rw-en, en-fr]`), or `mixed` for one arbitrary block. Sample sentences come from the validation set when it has the direction and **fall back to the training set otherwise** (with an explicit note), so directions like `fr->rw`/`rw->fr` still appear even when the validation file lacks direct rows for them. Sampling defaults to every eval (`sample_every: 0`); set an explicit step count to change the cadence. `suppress_warnings: true` hides Python/torch warnings and `clear_terminal: true` clears the screen when `altamt train` starts.
+
+**Suggested recipe (base, 2× A100/H100):** `max_tokens: 8192` per GPU × `grad_accum_steps: 2` × 2 ranks ≈ 32k tokens/step, lr 7e-4, 4k warmup, 120k steps, then optional 10–20k fine-tuning steps on the cleanest subset at lr 1e-4. On a single GPU, raise `grad_accum_steps` to 4 to keep the same effective batch. If you hit OOM, halve `max_tokens` and double `grad_accum_steps` — the effective batch (and therefore the LR) is unchanged.
+
+## 6. Inference & compression
+
+```python
+from altamt import Translator
+t = Translator("checkpoints/altamt-base/best", quantize=True, num_threads="auto")
+t.warmup()
+
+# tgt_lang is REQUIRED; src_lang is optional and auto-detected when omitted
+# (a stderr notice reports what was detected).
+r = t.translate("Mwaramutse nshuti zanjye", tgt_lang="en")
+print(r["translated_text"], r["detected_src"], r["route"])   # ... rw rw->en
+
+# Untrained pairs route through the pivot language (default "en") automatically:
+r = t.translate("Mwaramutse", tgt_lang="fr")     # route: rw->en->fr, pivot: en
+# Batches may mix directions (per-sentence tgt_lang) and mix direct/pivoted rows:
+t.translate_batch(["Mwaramutse", "Good morning"], tgt_lang=["en", "rw"])
+```
+
+```bash
+altamt translate "Mwaramutse" --model checkpoints/altamt-base/best \
+    --tgt-lang en          # --src-lang optional (auto-detected); --json for full records
+```
+
+Routing uses the checkpoint's recorded `trained_pairs`: direct pairs decode in one hop; otherwise the engine pivots (`pivot_lang="en"` by default, configurable) and reports `route` (`"rw->en->fr"`), `pivot` and the intermediate `pivot_text` in the result. If no route exists it raises a `ValueError` listing the trained pairs. Passing `src_lang == tgt_lang` is an error, and the target must always be given explicitly.
+
+### 6.1 Documents (multi-page input)
+
+**Long input is handled automatically.** A sentence-level model given a five-page document as one sequence is far outside its training distribution: positions past `max_src_len` were never seen during training, and the generation budget truncates the output — exactly why a single paragraph used to translate well while a long document came back garbled. `translate()` now detects input beyond the model's trained sentence length and transparently routes it through the document pipeline (segment → batch-translate → reassemble); the caller just sees the translated result. `translate_document()` remains the explicit entry point with full control (`unwrap`, `max_segment_tokens`, `progress`).
+
+```python
+doc = open("rapport.txt", encoding="utf-8").read()      # 5–20 pages is fine
+r = t.translate_document(doc, tgt_lang="en", progress=True)
+print(r["translated_text"])       # layout preserved
+print(r["segments"], r["chars_in"], r["chars_out"])
+
+t.translate_file("rapport.txt", tgt_lang="en")          # -> rapport.en.txt
+```
+
+```bash
+altamt translate-doc rapport.txt --model checkpoints/altamt-base/best \
+    --tgt-lang en -o rapport.en.txt --beam-size 4
+cat report.md | altamt translate-doc --model ... --tgt-lang rw > report.rw.md
+```
+
+What it does (`altamt.data.segmenter`):
+
+1. **Blocks** — splits on blank lines, keeping the exact separators so paragraph spacing survives byte-for-byte.
+2. **Lines** — hard-wrapped prose (the usual shape of text extracted from PDF/DOCX, with a newline at every rendered line) is *unwrapped* into one logical unit, while headings, bullets, numbered items and table rows stay separate. `--unwrap never` keeps every source line as its own unit; `--unwrap always` joins every multi-line paragraph.
+3. **Markers** — a leading `- `, `1. `, `## `, `(a) ` is stripped before translation and re-attached after, so the model translates prose instead of reproducing scaffolding. Titles and initials (`M. Dupont`, `J. R. R. Tolkien`) are deliberately *not* treated as list markers.
+4. **Sentences** — abbreviation-, initial- and decimal-aware splitting tuned for en/fr/rw (`fig. 3`, `3.5`, `art. 5`, `Dr. Smith` do not split).
+5. **Packing** — sentences are packed into segments up to `--segment-tokens` (default: 80 % of the checkpoint's training `max_src_len`), so several short sentences keep their mutual context while nothing ever exceeds the trained length. A single over-long sentence is split at clause boundaries first, and only then at a word-aligned hard cut.
+6. **Pass-through** — rules, bare URLs and numeric-only rows are copied verbatim rather than hallucinated over.
+7. **Batching** — every segment of the whole document is translated in **length-sorted, token-budgeted batches**, so a 20-page document is one efficient batched pass, not thousands of one-sentence forward passes.
+
+Reassembly restores the original paragraphs, line breaks, headings and markers exactly; with an identity translation the round-trip is byte-identical.
+
+### 6.2 Device efficiency
+
+`num_threads="auto"` (default) pins PyTorch to the machine's *physical* cores and disables the inter-op pool. This matters on SMT machines: the PyTorch default of one thread per logical core over-subscribes the cores and makes short-sentence decoding measurably slower. Pass an integer to override, or `None` to leave the process defaults alone.
+
+Generation batches are formed against a token budget (`batch_tokens`, defaulting to 4 096 on CPU and 32 768 on CUDA) and divided by the beam width, since beam search multiplies the effective batch. On GPU, generation runs under bf16/fp16 autocast automatically. Grouped-query attention uses PyTorch's native `enable_gqa=True` path when available (≥ 2.5), which avoids materializing a full-width key/value tensor on every decode step — the memory traffic GQA exists to avoid. Beam search does **not** permute the cross-attention KV cache: all beams of a sentence attend to the same encoder states, so the permutation is a no-op, and skipping it removes a per-layer, per-step copy that dominated beam search on long inputs.
+
+### 6.3 Compression & export
+
+```bash
+altamt quantize    --model checkpoints/altamt-base/best --output-dir int8_model
+altamt export-onnx --model checkpoints/altamt-base/best --output-dir onnx_model
+```
+
+Notes: the **PyTorch INT8 path keeps the KV cache** and is the recommended low-latency route; the ONNX export produces cache-free `encoder.onnx`/`decoder.onnx` graphs (maximum portability — use `altamt.inference.OnnxTranslator`). Latency targets assume single sentences of ≤ ~40 tokens on a modern 8-core x86 CPU with `num_threads="auto"`.
+
+## 7. Benchmarking
+
+```bash
+altamt benchmark \
+  --model checkpoints/altamt-base/best \
+  --test-file flores200.devtest.parquet \
+  --int8 \
+  --baseline facebook/nllb-200-distilled-600M \
+  --baseline Helsinki-NLP/opus-mt-rw-en \
+  --output-dir benchmark_results --formats md latex
+```
+
+Per direction and per system this reports **SacreBLEU** (13a), **chrF++** (`CHRF(word_order=2)` — the more reliable metric for morphologically rich Kinyarwanda), mean **latency (ms/sentence)**, **output tokens/sec**, **peak RSS memory** and **parameter count**, and writes `report.md` plus a booktabs `report.tex` you can `\input{}` directly into a paper. Baselines run through the *same* harness (`pip install altamt[benchmark]`), so numbers are directly comparable.
+
+## 8. Publishing
+
+**PyPI:**
+```bash
+python -m build
+twine upload dist/*        # README_pypi.md is the PyPI landing page
+```
+
+**Paper guidelines:** report chrF++ alongside BLEU (always with SacreBLEU signatures), evaluate on Flores-200 devtest for comparability, report latency with hardware + thread count + beam size, include both fp32 and INT8 rows, and release the tokenizer + config with the checkpoint for full reproducibility. The generated LaTeX table follows ACL formatting conventions.
+
+## 9. License
+
+Apache 2.0.

altamt-1.0/README_pypi.md

@@ -0,0 +1,77 @@
+# altamt
+
+**Bidirectional Kinyarwanda ⇄ English machine translation — fast on your CPU.**
+
+`altamt` (Advanced Lightweight Translation AI Model Transformer) is a compact (~130–195M parameter) modern Transformer (RMSNorm · SwiGLU · Grouped-Query Attention · RoPE) that translates in **both directions with one model** and **auto-detects the input language**. No GPU required.
+
+## Installation
+
+```bash
+pip install altamt
+# optional extras
+pip install "altamt[onnx]"        # ONNX Runtime export & inference
+pip install "altamt[benchmark]"   # compare against NLLB / Opus-MT baselines
+```
+
+## Quickstart (Python)
+
+```python
+from altamt import Translator
+
+translator = Translator(model_path="path/to/checkpoint")
+
+# Language auto-detection: just pass text.
+result = translator.translate("Mwaramutse nshuti zanjye")
+print(result)
+# {'translated_text': 'Good morning my friends',
+#  'detected_src': 'rw', 'tgt': 'en', 'latency_ms': 42.1}
+
+# Or pin the direction explicitly:
+translator.translate("How are you today?", src_lang="en", tgt_lang="rw")
+
+# Batch translation — directions can even be mixed in one batch:
+translator.translate_batch(["Mwaramutse", "Good morning"])
+```
+
+### Make it faster: INT8 quantization
+
+```python
+translator = Translator("path/to/checkpoint", quantize=True, num_threads=8)
+translator.warmup()
+```
+
+INT8 dynamic quantization typically gives ~2× lower latency and ~4× smaller weight matrices on x86 CPUs, with a negligible quality drop.
+
+## Quickstart (CLI)
+
+```bash
+altamt translate "Mwaramutse nshuti zanjye" --model path/to/checkpoint
+# Good morning my friends
+# [rw -> en | 42.1 ms]
+
+altamt translate "Hello" --model path/to/checkpoint --tgt-lang rw --int8 --json
+
+altamt benchmark  --model path/to/checkpoint --test-file test.parquet --int8
+altamt export-onnx --model path/to/checkpoint --output-dir ./onnx_model
+altamt train      --config config.yaml
+```
+
+## Why altamt?
+
+- **One model, both directions** — a target-language tag (`<2en>` / `<2rw>`) steers the decoder, so RW→EN and EN→RW share all parameters and vocabulary.
+- **Automatic language detection** — a built-in, microsecond-fast character n-gram detector picks the direction when you don't.
+- **Built for CPU** — Grouped-Query Attention + full KV-cached decoding + INT8 quantization target <100 ms per sentence on a modern 8-core CPU (beam=1, short sentences).
+- **Multi-format data** — training and benchmarking read `.json`, `.jsonl` and `.parquet` interchangeably.
+- **Honest benchmarking** — `altamt benchmark` reports SacreBLEU, chrF++, latency, throughput and memory, and can run NLLB/Opus-MT baselines through the same harness, emitting Markdown and LaTeX tables.
+
+## Indicative CPU performance
+
+Numbers depend on your hardware, sentence length, beam size and thread count; measure on your machine with `altamt benchmark`. On a modern 8-core x86 CPU (beam=1, ~20-token sentences, `num_threads=8`), the INT8 `base` model targets **<100 ms/sentence**, with fp32 roughly 2× slower and ~780 MB peak RAM.
+
+## Documentation
+
+Full architecture details, data format specs, training and paper-publishing guides live in the [GitHub README](https://github.com/altamt-ai/altamt#readme).
+
+## License
+
+Apache 2.0