@tuned-tensor/cli 0.16.0 → 0.18.0

package/docs/spec-workflow.md

@@ -0,0 +1,131 @@
+# Work from the behavior spec
+
+Explore the [interactive architecture diagram](spec-architecture.html) for source
+excerpts, code structure and command examples. Open the HTML file in a browser.
+
+`tunedtensor.json` is the local recipe for the model's behavior, examples and
+training settings. Ask the agent to improve that recipe, review the proposed
+diff, approve the edit, then preview or execute a pipeline against the saved spec.
+The agent reads the actual file before proposing changes; it does not use the
+conversation transcript as a replacement for the current spec.
+
+## Review without a model call
+
+Inside the TT shell, in your project directory:
+
+```text
+/spec
+/spec validate
+/spec diff
+/spec history
+```
+
+`/spec` shows the complete current file, validation results and SHA-256.
+`/spec diff` shows pending spec edits in the current conversation; otherwise it
+shows the latest approved edit, or external changes since that revision.
+`/spec history` lists up to 50 recent revision records. These commands require
+no credentials and work before the agent is configured.
+
+Outside the shell, use `tt spec show`, `tt spec validate`, `tt spec diff`, and
+`tt spec history`. Add a workspace-relative path, for example
+`tt spec show feedback/tunedtensor.json`. For continued work on a nested
+project, `/cd feedback` makes its `tunedtensor.json` the current recipe.
+`--json` returns structured review results. `tt spec validate` exits nonzero
+for an invalid spec.
+
+## Edit, review, approve
+
+```text
+› Keep my training settings, but require a single lowercase sentiment label.
+  [agent reads the spec and proposes a diff]
+› /spec diff
+› /approve
+› /spec
+› /spec history
+```
+
+`get_local_spec` returns the document, validation diagnostics and content hash.
+`prepare_update_local_spec` requires that hash and a patch of supported fields.
+Arrays replace their whole field; `hyperparameters` and `foundation` merge by
+key. `runtime` and `evaluation` merge recursively; null removes optional settings.
+`pipeline` replaces the entire recipe (null restores the derived default). Unmentioned fields remain unchanged. Identity and engine cannot change
+through this edit tool. The full resulting document must pass the runtime's
+strict schema, readiness checks, nonblank content checks and distinct-example
+checks. Small example sets produce a warning; validation is not a claim of model
+quality or GPU readiness.
+
+Proposing an edit does not write anything. `/approve` revalidates the workspace,
+parent directory, current file bytes and proposed result, acquires the spec's
+writer lock and atomically replaces the file. `/reject` leaves it unchanged.
+If the file changes after review, prepare a fresh edit. A saved edit invalidates
+any previously prepared pipeline preview for the old spec.
+
+Secure agent edits currently require Linux filesystem handles. Reads work on
+other supported CLI platforms; edit the JSON with your editor there. Spec reads
+and edits are bounded to 200 KB. Large model-facing proposals may hit the
+agent's existing smaller review budget; split the request or use your editor.
+
+Revision records live beside the spec under `.tuned-tensor/spec-history/` and
+contain before/after snapshots and hashes. They can contain your training
+examples. They use private permissions. History records changes made through
+this edit workflow; it is not a file watcher or a complete history of external
+editor changes. A `prepared` record without an `applied` record is an incomplete
+journal entry: inspect the current file and its hash before attempting another
+edit. A crashed writer can leave `.tunedtensor-spec.lock`; after confirming no
+TT writer is still running, remove that lock manually. The lock coordinates TT
+writers; avoid concurrent external editor saves during approval.
+
+## Execute the saved recipe
+
+```bash
+tt spec validate
+tt pipeline run --spec tunedtensor.json --dry-run
+tt pipeline run --spec tunedtensor.json
+```
+
+The pipeline uses the optional `pipeline` section or derives from the current spec. An adjacent
+`tunedtensor.pipeline.json` is no longer loaded implicitly. To select an advanced
+recipe explicitly, pass `--file`; a missing explicit recipe or `--spec` file fails. Foundation
+training parameters in an explicit recipe must agree with the spec, including
+in previews and the runtime's own execution guard. Change the spec and regenerate
+the recipe when they disagree. Explicit recipes may still select supported
+stages; `--only` and `--skip` remain available.
+
+CLI planning, validation, execution and agent pipeline proposals share
+`pipelineForRunInput` in `src/pipeline.ts`. It uses the same
+`validateBehaviorSpec` rules as spec review and editing, defined beside the
+runtime parser in `src/local-runtime/local-project.ts`. The edit module only
+owns workspace access, patches and revision history.
+
+The command reads one spec snapshot for planning, hardware warnings and execution. Plan/dry-run
+JSON and final result JSON identify its path and SHA-256. The spec's compiled
+instructions and examples feed training and task evaluation; the independently
+configured general-regression suite keeps its own prompt. Provider choice and
+GPU placement is configured in `runtime.gpu` in the spec. Model-provider choice
+remains in agent configuration. Evaluation policy belongs in `evaluation`;
+artifact/store locations and model paths belong in `runtime`. All relative paths
+resolve beside the spec. New projects default to `.tuned-tensor/artifacts` and
+`.tuned-tensor/store` beside it (unless `TT_LOCAL_HOME` overrides the store).
+Foundation supports `runtime.artifactRoot` and `runtime.gpu`; its evaluation
+and training settings remain under `foundation`.
+
+Only `tunedtensor.json` is required. `tt pipeline init --spec tunedtensor.json`
+embeds an advanced recipe when needed; omit this step to keep automatic derivation.
+An explicit `--file` cannot override a conflicting embedded recipe.
+
+For existing projects, run `tt pipeline migrate --spec tunedtensor.json` to
+consolidate adjacent pipeline and runner files. It validates the result first,
+keeps exclusive `.bak` copies, and preserves resolved paths and the old store
+location. Conflicting sources, enabled legacy `dryRun`, or unsupported foundation
+runner settings require reconciliation first. Existing `.bak` files are never
+overwritten. Avoid concurrent edits during migration. Legacy runner files remain
+readable with a warning, but cannot coexist with inline runtime/evaluation settings.
+A legacy adjacent pipeline is used only with explicit `--file` or after migration.
+
+Runs write generated `resolved-workflow.json` snapshots for diagnosis and
+reproducibility; these are outputs, not additional user-authored configuration.
+
+
+Agent `/approve` for a pipeline still runs a preview only. Real training uses
+the direct command above. The spec is the behavior recipe; approval of a spec
+edit is not approval to launch training.

package/examples/single-spec/adapter/tunedtensor.json

@@ -0,0 +1,49 @@
+{
+  "id": "22222222-2222-4222-8222-222222222222",
+  "name": "Local Smoke Assistant",
+  "description": "Tiny local runner smoke test.",
+  "system_prompt": "Return exactly one sentiment label.",
+  "guidelines": [
+    "Reply with positive or negative."
+  ],
+  "constraints": [
+    "Do not include extra commentary."
+  ],
+  "base_model": "Qwen/Qwen3.5-2B",
+  "examples": [
+    {
+      "input": "Classify sentiment: I love this product.",
+      "output": "positive"
+    },
+    {
+      "input": "Classify sentiment: This was disappointing.",
+      "output": "negative"
+    },
+    {
+      "input": "Classify sentiment: The update works perfectly.",
+      "output": "positive"
+    },
+    {
+      "input": "Classify sentiment: Support never replied.",
+      "output": "negative"
+    }
+  ],
+  "hyperparameters": {
+    "n_epochs": 1,
+    "learning_rate": 0.0002,
+    "batch_size": 1,
+    "gradient_accumulation_steps": 1,
+    "lora_rank": 4,
+    "lora_alpha": 8,
+    "max_seq_length": 256,
+    "max_eval_examples": 2
+  },
+  "evaluation": {
+    "inference": {
+      "maxNewTokens": 32
+    },
+    "scoring": {
+      "mode": "exact_match"
+    }
+  }
+}

package/examples/single-spec/foundation/tunedtensor.json

@@ -0,0 +1,31 @@
+{
+  "engine": "foundation",
+  "name": "Foundation smoke chat",
+  "description": "Tiny from-scratch tokenizer, pretrain, chat SFT, and eval.",
+  "system_prompt": "You are a helpful assistant.",
+  "guidelines": [
+    "Answer the user request directly."
+  ],
+  "constraints": [],
+  "examples": [
+    {
+      "input": "Say hello in one sentence.",
+      "output": "Hello \u2014 how can I help you today?"
+    },
+    {
+      "input": "What is 2 + 2?",
+      "output": "2 + 2 = 4."
+    }
+  ],
+  "foundation": {
+    "vocab_size": 256,
+    "max_chars": 20000,
+    "depth": 2,
+    "pretrain_steps": 2,
+    "finetune_steps": 2,
+    "rl_steps": 0,
+    "batch_size": 2,
+    "sequence_length": 64,
+    "nproc_per_node": 1
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tuned-tensor/cli",
-  "version": "0.16.0",
+  "version": "0.18.0",
   "description": "Local TT agent and CUDA fine-tuning CLI for Tuned Tensor workflows",
   "repository": {
     "type": "git",
@@ -34,8 +34,11 @@
     "training/foundation/uv.lock",
     "training/foundation/src/*.py",
     "docs/local-runtime",
+    "docs/spec-workflow.md",
+    "docs/spec-architecture.html",
     "examples/local-runtime/*.json",
-    "examples/local-runtime/*.jsonl"
+    "examples/local-runtime/*.jsonl",
+    "examples/single-spec/**/*.json"
   ],
   "engines": {
     "node": ">=22.19.0"
@@ -66,6 +69,6 @@
     "tsup": "^8.5.1",
     "tsx": "^4.23.12",
     "typescript": "^5.9.3",
-    "vitest": "^4.1.8"
+    "vitest": "^4.1.11"
   }
 }

package/training/adapter/src/serve.py

@@ -93,7 +93,13 @@ def build_vllm_args(model_source: str, adapter_path: str | None, temp_dir: Path)
         # tool calls/results verbatim while merging leading system context.
         prefix = "{%- set tt = namespace(system=" + json.dumps(prompt) + ", history=[]) -%}"
         prefix += "{%- for message in messages -%}{%- if message.role == 'system' -%}"
+        # vLLM may normalize text into OpenAI content parts for the model's
+        # native template. Merge those parts without stringifying the list.
+        prefix += "{%- if message.content is string -%}"
         prefix += "{%- set tt.system = tt.system + '\\n\\n' + message.content -%}"
+        prefix += "{%- else -%}{%- for part in message.content -%}{%- if part.type == 'text' -%}"
+        prefix += "{%- set tt.system = tt.system + '\\n\\n' + part.text -%}"
+        prefix += "{%- endif -%}{%- endfor -%}{%- endif -%}"
         prefix += "{%- else -%}{%- set tt.history = tt.history + [message] -%}{%- endif -%}{%- endfor -%}"
         prefix += "{%- set messages = [{'role': 'system', 'content': tt.system}] + tt.history -%}"
         path = temp_dir / "chat-template.jinja"