@tuned-tensor/cli 0.20.0 → 0.22.0

package/README.md

@@ -94,8 +94,12 @@ npm link
 ## Behavior spec workflow
 
 `tunedtensor.json` is the source of truth for behavior and training settings.
-Use `/spec`, `/spec diff`, `/spec validate`, and `/spec history` in the shell to
-review it without a model call. Ask the agent for changes, inspect its proposed
+In the shell, `/spec` shows a readable overview grouped into behavior,
+examples, training, evaluation and runtime sections; `/spec <section>` (for
+example `/spec examples`) shows one section in full and `/spec show` prints the
+highlighted JSON. `/spec diff`, `/spec validate`, and `/spec history` complete
+the review, all without a model call. Outside the shell, `tt spec view` and
+`tt spec <section>` print the same readable view. Ask the agent for changes, inspect its proposed
 diff, then press `y` when tt asks (or `/approve` later) to save a validated edit
 with revision history. Pipeline
 runs use the saved spec. See [the spec workflow](docs/spec-workflow.md).
@@ -226,8 +230,15 @@ that request real execution fail before the child command is dispatched. Real
 training remains available through the explicit direct `tt pipeline run` command.
 A changed workspace or spec requires a new review.
 
-Useful shell controls include `/help`, `/status`, `/context`, `/model`,
-`/login`, `/cd`, `/clear`, and `/exit`.
+Useful shell controls include `/help`, `/status`, `/context`, `/system`,
+`/model`, `/login`, `/cd`, `/clear`, and `/exit`.
+When the shell opens, the TT logo mark (drawn in the terminal) greets you with the current spec, a
+live machine panel (GPU utilization, VRAM and temperature from `nvidia-smi`,
+CPU, RAM, and free space in the model cache) and whether this machine can
+fine-tune the spec's base model, plus a suggested next step. `/system` refreshes
+that panel and lists every certified base model's verdict. Fit verdicts come
+from the same certified memory profiles as `tt hardware`; a fresh
+`tt hardware` snapshot takes precedence over the quick live check.
 The shell banner and `/context` show the inference selection (`agent
 provider/model`) separately from the workflow model. The managed alias identifies
 inference through the server-selected model. Run `/model` with no arguments to
@@ -260,18 +271,40 @@ optional `gpu` configuration moves its GPU processes to your AWS instance.
 The portable document contract lives in `@tuned-tensor/pipeline-contract`. The
 CLI adds execution planning (step selection) on top of it.
 
-Two local engines share that document version:
+Three local engines share that document version:
 
 - **Adapter** (default): LoRA SFT on a certified Hugging Face checkpoint.
   Runs through `tt pipeline run`. Uses `train` / `evaluate` / `compare` with
   `evaluate.with.evaluator: "behavior"`.
+  Certified models (`Qwen/Qwen3.5-2B`, Nemotron 3.5 Lightning, Muse Glimmer)
+  have a reviewed revision, architecture, and memory profile. Any other
+  Hugging Face text model can be trained as **uncertified** by pinning its
+  commit; it loads as a causal LM with safetensors weights, no remote code,
+  and generic LoRA defaults (override them in `hyperparameters`):
+
+  ```json
+  {
+    "base_model": "mistralai/Mistral-7B-Instruct-v0.3",
+    "hyperparameters": { "base_model_revision": "<40-char commit SHA>" }
+  }
+  ```
+
+  Pick an instruct/chat variant (the trainer needs a chat template) and check
+  hardware fit with `tt doctor`, since TT has no memory profile for it.
 - **Foundation**: from-scratch tokenizer + GPT. Also runs through
   `tt pipeline run`. Uses
   `tokenize` / `pretrain` / `finetune` / optional `rl` plus `bpb`, `chat`, and
   `inference` evaluators. Requires a foundation `tunedtensor.json`
   (`engine: "foundation"`, at least two examples, no `base_model`).
-
-Both engines are spec-driven. `system_prompt`, `guidelines`, and `constraints`
+- **Decision**: fine-tunes a small typed decision model
+  (`convaiinnovations/laya`, 421M) that answers one choice, yes/no or score
+  question per input with a probability per label, instead of generating text.
+  Uses the adapter vocabulary (`train` / `evaluate` / `compare`) and runs on
+  CPU, Apple MPS or CUDA. Requires `engine: "decision"` and a `decision`
+  question; example outputs are labels. See
+  [Decision models](docs/local-runtime/decision.md).
+
+All engines are spec-driven. `system_prompt`, `guidelines`, and `constraints`
 are compiled into one canonical system instruction, and `examples` supply the
 demonstrated behavior. Adapter training, foundation tokenizer/pretraining,
 chat SFT, optional RL, and evaluation all receive that same compiled
@@ -427,6 +460,12 @@ tt pipeline run --spec tunedtensor.json --dry-run
 tt pipeline run --spec tunedtensor.json
 ```
 
+`tt models prefetch` shows a progress bar with transfer rate and ETA. Large
+downloads survive flaky networks: TT pins the requested revision to one commit,
+retries interrupted transfers with backoff (`TT_HF_DOWNLOAD_MAX_ATTEMPTS`,
+default 10), and keeps completed files in the Hugging Face cache, so running the
+command again resumes instead of starting over.
+
 These commands read runtime and evaluation settings from the core spec. The preview needs
 no GPU or AWS connection. Results return to the local run and model stores:
 
@@ -435,6 +474,83 @@ tt runs report <run-id>
 tt models verify local-<run-id>
 ```
 
+### Train on a Hugging Face dataset
+
+Point `dataset_prebuilt` at files in a Hugging Face dataset repo instead of
+local paths. It works like `base_model`: name the repo, pin a commit, and TT
+downloads it into the same Hugging Face cache.
+
+```json
+{
+  "dataset_prebuilt": {
+    "huggingface": {
+      "repo": "SetFit/tweet_sentiment_extraction",
+      "revision": "0d5359cfbb2470332f2b82e62269cf755e0ac5c3",
+      "columns": { "input": "text", "output": "label_text" }
+    },
+    "training": "train.jsonl",
+    "test": "test.jsonl"
+  }
+}
+```
+
+```bash
+tt datasets prefetch tunedtensor.json   # download with progress; rerun to resume
+tt datasets verify tunedtensor.json     # offline checksum check of the cached files
+tt pipeline run --spec tunedtensor.json # downloads automatically if still missing
+```
+
+- Split paths are files inside the repo and must be JSONL. Parquet and CSV are
+  not supported yet.
+- Without `columns`, files must already be TT chat JSONL whose system message
+  matches the spec. With `columns`, each record's `input` and `output` fields
+  become chat rows that use the spec's instruction. Records missing either
+  field are skipped. Evaluation splits keep the first of any duplicate input,
+  and training drops inputs that also appear in an evaluation split. The run
+  report lists every skipped or dropped count.
+- Without `revision`, TT uses the cached `main` snapshot and records the commit
+  it resolved. `tt validate` and `tt datasets prefetch` flag the spec as
+  unpinned. Pin the commit for reproducible runs.
+- Runs never contact the Hub after the download. Each run checks every file
+  against its content-addressed blob and records the repo, commit, file
+  SHA-256s and conversion counts in `run_metadata.dataset_source`.
+- Gated datasets use `HF_TOKEN`, as model downloads do.
+
+### Run records and audit
+
+`tt pipeline run` prints each stage live with the time since the run started.
+`--verbose` adds training and evaluation process output, and `--quiet` hides
+both. Full logs stay in the run directory either way.
+
+Every run report has a `provenance` block recording what produced it:
+
+- TT, Node and platform versions, and the digest of the locked Python runtime;
+- where training ran (local, or the AWS instance and region);
+- the spec file and snapshot digests;
+- the base model id, revision and content fingerprint;
+- every dataset file's SHA-256, the compiled training data digest, and the
+  evaluation split and seed;
+- the hyperparameters and the training exit code.
+
+The run's event log is hash-chained: each event records the previous event's
+hash. When a run completes, TT records the report's SHA-256.
+
+```bash
+tt runs audit <run-id>
+```
+
+`tt runs audit` checks the event chain and re-hashes the report, then prints
+the stage timeline with durations and the provenance. Its verdict is
+`verified`, `incomplete` (older TT versions or unfinished runs) or `tampered`.
+`tampered` exits with code 2 and names the edited event or file; removing the
+completion event or emptying the log also counts as tampering.
+
+`tt publish` runs the same audit and uploads the summary with the report. It
+refuses tampered evidence. It also stops on unverifiable evidence, such as runs
+from older TT versions or logs with unreadable lines, unless you pass
+`--allow-unverified`. The chain makes edits evident; it is not a signature, so
+anyone with write access to the store can rewrite the whole log.
+
 Serving still runs on the machine executing `tt serve` and requires a local
 Linux/CUDA GPU, even when training used AWS: