@tuned-tensor/cli 0.17.0 → 0.18.0

package/docs/local-runtime/aws-gpu.md

@@ -1,6 +1,6 @@
 # Use your AWS GPU with the local pipeline
 
-TT runs one pipeline on your laptop. Add `gpu` to `local-runner.json` to send
+TT runs one pipeline on your laptop. Add `runtime.gpu` to `tunedtensor.json` to send
 adapter training and CUDA evaluation, or foundation pretraining, fine-tuning,
 RL and evaluation, to an existing EC2 instance in your account. Foundation
 tokenization, dataset preparation, adapter scoring/comparisons, run history,
@@ -52,21 +52,23 @@ only needs `ec2:DescribeInstances` for lookup.
 
 ## 3. Configure the connection
 
-Save this as `local-runner.json` beside `tunedtensor.json`,
-replacing the example values with your own. You can also copy
-[`examples/local-runtime/aws-runner.json`](../../examples/local-runtime/aws-runner.json).
-All runner paths refer to the laptop; TT translates the paths needed remotely.
+Add this `runtime` section to your existing `tunedtensor.json`, replacing the
+example values with your own. All paths refer to the laptop; TT translates
+paths needed remotely. Existing sidecars can be consolidated with
+`tt pipeline migrate`, which retains `.bak` copies.
 
 ```json
 {
-  "gpu": {
-    "provider": "aws",
-    "profile": "research",
-    "region": "eu-west-1",
-    "instanceId": "i-0123456789abcdef0",
-    "user": "ubuntu",
-    "identityFile": "~/.ssh/research-gpu.pem",
-    "maxSeconds": 86400
+  "runtime": {
+    "gpu": {
+      "provider": "aws",
+      "profile": "research",
+      "region": "eu-west-1",
+      "instanceId": "i-0123456789abcdef0",
+      "user": "ubuntu",
+      "identityFile": "~/.ssh/research-gpu.pem",
+      "maxSeconds": 86400
+    }
   }
 }
 ```
@@ -94,7 +96,7 @@ Verify the instance's SSH host key for the selected IP address and connect
 once yourself before using TT. TT requires a known host key and noninteractive
 SSH authentication. Omit
 `identityFile` to use your SSH agent or SSH configuration. Relative identity
-file paths resolve beside the runner config; `~` expands locally. Set
+file paths resolve beside the core spec; `~` expands locally. Set
 `privateIp: true` when connected to the instance's VPC through a VPN or other
 private network. The default uses its public IPv4 address. Security groups and
 network routes must allow the laptop to reach that address over SSH.
@@ -137,28 +139,27 @@ Run these commands on your laptop in the project directory. Adapter workflow:
 ```bash
 # Fetch the base model locally first.
 tt validate tunedtensor.json
-tt models prefetch tunedtensor.json --config local-runner.json
-tt doctor tunedtensor.json --config local-runner.json
-tt pipeline run --spec tunedtensor.json --config local-runner.json --dry-run
-tt pipeline run --spec tunedtensor.json --config local-runner.json
+tt models prefetch tunedtensor.json
+tt doctor tunedtensor.json
+tt pipeline run --spec tunedtensor.json --dry-run
+tt pipeline run --spec tunedtensor.json
 ```
 
 For a foundation spec, no base-model prefetch is needed. Start a new run with:
 
 ```bash
 tt validate tunedtensor.json
-tt doctor tunedtensor.json --config local-runner.json
-tt pipeline run --spec tunedtensor.json --config local-runner.json \
+tt doctor tunedtensor.json
+tt pipeline run --spec tunedtensor.json \
   --dry-run
-tt pipeline run --spec tunedtensor.json --config local-runner.json
+tt pipeline run --spec tunedtensor.json
 ```
 
 Choose the command block for your spec's engine. `tt doctor` connects to the
 instance and prepares/probes its locked Python/CUDA runtime, which may take
 time on first use. It does not start training or reserve GPU capacity.
 `tt hardware` reports the laptop's hardware; it does not inspect the AWS
-instance. `tt pipeline run` discovers `local-runner.json` beside the selected
-spec when `--config` is omitted.
+instance. `tt pipeline run` reads `runtime.gpu` from the selected spec.
 
 Keep pipeline step targets set to `local`: the target describes the
 orchestrator, and `gpu` selects where its GPU processes execute. CPU adapter
@@ -172,10 +173,10 @@ Wait for a successful completion message. Adapter runs return a run ID and a
 local model ID; use the actual IDs from the output:
 
 ```bash
-tt runs list --config local-runner.json
-tt runs report RUN_ID --config local-runner.json
-tt models list --config local-runner.json
-tt models verify MODEL_ID --config local-runner.json
+tt runs list
+tt runs report RUN_ID
+tt models list
+tt models verify MODEL_ID
 ```
 
 For foundation runs, inspect `report.json` in the run directory printed by the

package/docs/local-runtime/serving.md

@@ -38,7 +38,7 @@ not mutate the training runtime.
 ```bash
 tt models prefetch tunedtensor.json
 
-tt serve local-<run-id> --config local-runner.json \
+tt serve local-<run-id> \
   --context-length 8192 --max-tokens 512
 ```
 
@@ -81,7 +81,7 @@ Export a new isolated configuration rather than replacing global settings:
 
 ```bash
 export PI_CODING_AGENT_DIR="$(mktemp -d)"
-tt serve local-<run-id> --config local-runner.json \
+tt serve local-<run-id> \
   --context-length 8192 --max-tokens 512 \
   --print-client-config pi > "$PI_CODING_AGENT_DIR/models.json"
 ```

package/docs/local-runtime/spark.md

@@ -78,28 +78,23 @@ Edit both generated examples in `tunedtensor.json`. For a meaningful run,
 replace them with a larger, representative dataset and a separate validation
 split.
 
-The generated `local-runner.json` uses CUDA and project-local artifacts. A
-durable Spark configuration can set:
+Initialization creates only `tunedtensor.json`; CUDA and project-local storage
+are defaults. For durable Spark paths, add this optional section to the spec:
 
 ```json
 {
-  "artifactRoot": "/home/eve/tuned-tensor-runs/artifacts",
-  "storeRoot": "/home/eve/tuned-tensor-runs/store",
-  "paths": {
-    "modelCache": "/home/eve/.cache/huggingface"
-  },
-  "evaluation": {
-    "inference": {
-      "device": "cuda"
-    },
-    "scoring": {
-      "mode": "exact_match"
-    },
-    "timeoutMs": 1800000
+  "runtime": {
+    "artifactRoot": "/home/eve/tuned-tensor-runs/artifacts",
+    "storeRoot": "/home/eve/tuned-tensor-runs/store",
+    "paths": { "modelCache": "/home/eve/.cache/huggingface" }
   }
 }
 ```
 
+Existing projects can use `tt pipeline migrate` to fold sidecar settings into
+the spec while keeping backups.
+
+
 Every Python stage uses the locked runtime included in the npm package; a
 source checkout and a custom runner path are neither required nor supported.
 
@@ -110,7 +105,7 @@ tt doctor tunedtensor.json
 tt validate tunedtensor.json
 tt models prefetch tunedtensor.json
 tt models verify-base tunedtensor.json
-tt pipeline run --spec tunedtensor.json --config local-runner.json
+tt pipeline run --spec tunedtensor.json
 ```
 
 `doctor` resolves the same bundled project and paths the run will use, imports
@@ -136,7 +131,7 @@ tt serve local-<run-id> --spec tunedtensor.json --port 8000
 ```
 
 `tt serve active` fails unless an adapter is activated. Activation requires a
-`generalRegression` suite in `local-runner.json`. Serve a specific adapter
+`generalRegression` suite in `tunedtensor.json`. Serve a specific adapter
 with `tt serve local-<run-id>` until then.
 
 In another shell:

package/docs/spec-architecture.html

@@ -85,9 +85,9 @@
     <div class="tt-level">
       <div class="tt-unit"><button class="btn" type="button" data-component="validation" aria-pressed="false">Shared validation</button><small class="text-small">schema → readiness checks</small></div>
       <div class="tt-unit"><button class="btn" type="button" data-component="plan" aria-pressed="false">Derive the pipeline</button><small class="text-small">pipelineForRunInput → plan</small></div>
-      <div class="tt-unit"><button class="btn" type="button" data-component="config" aria-pressed="false">Runner configuration</button><small class="text-small">GPU placement · paths · scoring</small></div>
+      <div class="tt-unit"><button class="btn" type="button" data-component="config" aria-pressed="false">Spec runtime and evaluation</button><small class="text-small">GPU placement · paths · scoring</small></div>
     </div>
-    <div class="tt-arrow text-small">↓ same spec and plan · preview or explicit run · runner config supplies execution policy</div>
+    <div class="tt-arrow text-small">↓ same spec and plan · preview or explicit run · core spec supplies execution policy</div>
     <div class="tt-boundary text-small">03 / EXECUTE & EVALUATE</div>
     <div class="tt-level tt-pair">
       <div class="tt-unit"><button class="btn" type="button" data-component="adapter" aria-pressed="false">Adapter engine</button><small class="text-small">baseline → LoRA → candidate → compare</small></div>
@@ -167,7 +167,7 @@
     "summary": "One saved recipe for instructions, examples, engine and training parameters.",
     "input": "Reviewed user intent: prompt, guidelines, constraints, examples and settings.",
     "output": "One parsed snapshot, identified by a SHA-256, for planning and execution.",
-    "boundary": "The spec defines behavior and training. Runner configuration still defines placement, paths and evaluation policy.",
+    "boundary": "The spec defines behavior and training. Its runtime and evaluation sections define placement, paths and evaluation policy.",
     "flow": [
       "read bytes",
       "parse once",
@@ -185,26 +185,26 @@
     "example": "/spec\n/spec diff\n/spec validate\n/spec history\n\n# Outside the TT shell:\ntt spec show --json"
   },
   "config": {
-    "title": "Runner configuration",
+    "title": "Spec runtime and evaluation",
     "location": "Project · JSON",
     "summary": "Execution placement, filesystem paths and evaluation policy remain explicit runtime choices.",
-    "input": "local-runner.json beside the spec, or --config.",
+    "input": "runtime and evaluation sections in tunedtensor.json; legacy --config remains compatible.",
     "output": "GPU dispatch, artifact locations, inference/scoring configuration and regression policy.",
-    "boundary": "This is not a second behavior recipe. Evaluation settings remain here; moving them into the spec would require a separate schema change.",
+    "boundary": "These optional sections belong to the core spec. Runtime defaults apply when omitted; legacy sidecars can be migrated with backups.",
     "flow": [
       "config",
       "resolve paths",
       "choose process host"
     ],
-    "path": "examples/local-runtime/aws-runner.json",
+    "path": "examples/single-spec/adapter/tunedtensor.json",
     "line": 1,
-    "code": "{\n  \"artifactRoot\": \".tuned-tensor/artifacts\",\n  \"storeRoot\": \".tuned-tensor/store\",\n  \"gpu\": {\n    \"provider\": \"aws\",\n    \"profile\": \"research\",\n    \"region\": \"eu-west-1\",\n    \"instanceId\": \"i-0123456789abcdef0\",\n    \"user\": \"ubuntu\",\n    \"identityFile\": \"~/.ssh/research-gpu.pem\",\n    \"maxSeconds\": 86400\n  }\n}",
+    "code": "{\n  \"id\": \"22222222-2222-4222-8222-222222222222\",\n  \"name\": \"Local Smoke Assistant\",\n  \"description\": \"Tiny local runner smoke test.\",\n  \"system_prompt\": \"Return exactly one sentiment label.\",\n  \"guidelines\": [\n    \"Reply with positive or negative.\"\n  ],\n  \"constraints\": [\n    \"Do not include extra commentary.\"\n  ],\n  \"base_model\": \"Qwen/Qwen3.5-2B\",\n  \"examples\": [\n    {\n      \"input\": \"Classify sentiment: I love this product.\",\n      \"output\": \"positive\"\n    },\n    {\n      \"input\": \"Classify sentiment: This was disappointing.\",\n      \"output\": \"negative\"\n    },\n    {\n      \"input\": \"Classify sentiment: The update works perfectly.\",\n      \"output\": \"positive\"\n    },\n    {\n      \"input\": \"Classify sentiment: Support never replied.\",\n      \"output\": \"negative\"\n    }\n  ],\n  \"hyperparameters\": {\n    \"n_epochs\": 1,\n    \"learning_rate\": 0.0002,\n    \"batch_size\": 1,\n    \"gradient_accumulation_steps\": 1,\n    \"lora_rank\": 4,\n    \"lora_alpha\": 8,\n    \"max_seq_length\": 256,\n    \"max_eval_examples\": 2\n  },\n  \"evaluation\": {\n    \"inference\": {\n      \"maxNewTokens\": 32\n    },\n    \"scoring\": {\n      \"mode\": \"exact_match\"\n    }\n  }\n}\n",
     "files": [
-      "examples/local-runtime/aws-runner.json",
+      "examples/single-spec/adapter/tunedtensor.json",
       "src/local-runtime/contracts.ts",
       "src/commands/pipeline.ts"
     ],
-    "example": "tt doctor tunedtensor.json --config local-runner.json\ntt pipeline run --spec tunedtensor.json --config local-runner.json --dry-run"
+    "example": "tt doctor tunedtensor.json\ntt pipeline run --spec tunedtensor.json --dry-run"
   },
   "agent": {
     "title": "Agent & model provider",
@@ -242,8 +242,8 @@
       "applySpecUpdate"
     ],
     "path": "src/spec-workspace.ts",
-    "line": 133,
-    "code": "export async function prepareSpecUpdate(workspaceRoot: string, specPath: string, expectedSha256: string, changes: unknown) {\n  const current = await inspectLocalSpec(workspaceRoot, specPath);\n  if (current.sha256 !== expectedSha256) throw new Error(\"The behavior spec changed; inspect it and prepare the edit again.\");\n  if (!isObject(current.document)) throw new Error(\"Repair the spec JSON before preparing an edit.\");\n  if (!isObject(changes) || !Object.keys(changes).length || Object.keys(changes).some(key => !EDIT_KEYS.has(key))) {\n    throw new Error(\"Provide supported spec field changes; identity and engine cannot be changed by an edit.\");\n  }\n  const next = { ...current.document, ...changes };\n  for (const key of [\"hyperparameters\", \"foundation\"] as const) {\n    if (isObject(changes[key])) {\n      const previous = isObject(current.document[key]) ? current.document[key] : {};\n      next[key] = { ...previous, ...changes[key] };\n    }\n  }\n  const validation = validateDocument(next, current.path);\n  if (!validation.valid) throw new Error(`Invalid behavior spec:\\n${validation.errors.join(\"\\n\")}`);\n  const source = `${JSON.stringify(next, null, 2)}\\n`;\n  if (Buffer.byteLength(source) > MAX_BYTES) throw new Error(\"Updated spec is too large.\");\n  const diff = specDiff(current.document, next);\n  if (diff === \"No changes.\") throw new Error(\"The proposed edit makes no changes.\");\n  const update: SpecUpdate = {\n    spec_path: current.displayPath,\n    expected_sha256: current.sha256,\n    after_sha256: specHash(source),",
+    "line": 138,
+    "code": "export async function prepareSpecUpdate(workspaceRoot: string, specPath: string, expectedSha256: string, changes: unknown) {\n  const current = await inspectLocalSpec(workspaceRoot, specPath);\n  if (current.sha256 !== expectedSha256) throw new Error(\"The behavior spec changed; inspect it and prepare the edit again.\");\n  if (!isObject(current.document)) throw new Error(\"Repair the spec JSON before preparing an edit.\");\n  if (!isObject(changes) || !Object.keys(changes).length || Object.keys(changes).some(key => !EDIT_KEYS.has(key))) {\n    throw new Error(\"Provide supported spec field changes; identity and engine cannot be changed by an edit.\");\n  }\n  const next = { ...current.document, ...changes };\n  function mergeSettings(before: Record<string, unknown>, patch: Record<string, unknown>): Record<string, unknown> {\n    const result = { ...before };\n    for (const [key, value] of Object.entries(patch)) {\n      if (value === null) delete result[key];\n      else result[key] = isObject(value) && isObject(result[key]) ? mergeSettings(result[key], value) : value;\n    }\n    return result;\n  }\n  for (const key of [\"runtime\", \"evaluation\", \"pipeline\"]) {\n    if (changes[key] === null) delete next[key];\n    else if (key !== \"pipeline\" && isObject(changes[key])) next[key] = mergeSettings(isObject(current.document[key]) ? current.document[key] : {}, changes[key]);\n  }\n  for (const key of [\"hyperparameters\", \"foundation\"] as const) {\n    if (isObject(changes[key])) {\n      const previous = isObject(current.document[key]) ? current.document[key] : {};\n      next[key] = { ...previous, ...changes[key] };\n    }\n  }\n  const validation = validateDocument(next, current.path);\n  if (!validation.valid) throw new Error(`Invalid behavior spec:\\n${validation.errors.join(\"\\n\")}`);",
     "files": [
       "src/spec-workspace.ts",
       "src/agent-approval.ts",
@@ -264,8 +264,8 @@
       "preview / run"
     ],
     "path": "src/pipeline.ts",
-    "line": 196,
-    "code": "export function pipelineForRunInput(input: LocalRunInput, requested?: unknown): NormalizedPipeline {\n  if (input.kind === \"request\") {\n    // Preserve the direct CLI's full run-request compatibility.\n    assertLocalRunInputReady(input.request);\n  } else {\n    const validation = validateBehaviorSpec(input);\n    if (!validation.valid) {\n      throw new Error(`Invalid behavior spec:\\n${validation.errors.join(\"\\n\")}`);\n    }\n  }\n\n  const foundation = input.kind === \"foundation-spec\";\n  const recipe = requested !== undefined ? requested : (foundation\n    ? pipelineFromFoundationHyperparameters(input.spec.name, input.spec.foundation)\n    : canonicalPipeline(\"local\"));\n  const pipeline = parsePipeline(recipe);\n  if (isFoundationPipeline(pipeline) !== foundation) {\n    const recipeEngine = isFoundationPipeline(pipeline) ? \"a foundation\" : \"an adapter\";\n    const specEngine = foundation ? \"a foundation\" : \"an adapter\";\n    throw new Error(`Pipeline is ${recipeEngine} recipe, but the behavior spec is ${specEngine} spec. Use a matching recipe or omit --file.`);\n  }\n  if (foundation) assertFoundationSettingsMatch(input.spec.foundation, pipeline);\n  return pipeline;\n}",
+    "line": 197,
+    "code": "export function pipelineForRunInput(input: LocalRunInput, requested?: unknown): NormalizedPipeline {\n  if (input.kind === \"request\") {\n    // Preserve the direct CLI's full run-request compatibility.\n    assertLocalRunInputReady(input.request);\n  } else {\n    const validation = validateBehaviorSpec(input);\n    if (!validation.valid) {\n      throw new Error(`Invalid behavior spec:\\n${validation.errors.join(\"\\n\")}`);\n    }\n  }\n\n  const foundation = input.kind === \"foundation-spec\";\n  const embedded = input.kind === \"request\" ? undefined : input.spec.pipeline;\n  if (embedded && requested !== undefined && canonicalJson(parsePipeline(embedded)) !== canonicalJson(parsePipeline(requested))) {\n    throw new Error(\"Conflicting pipeline settings: update tunedtensor.json.pipeline before running a different recipe.\");\n  }\n  const recipe = requested !== undefined ? requested : embedded ?? (foundation\n    ? pipelineFromFoundationHyperparameters(input.spec.name, input.spec.foundation)\n    : canonicalPipeline(\"local\"));\n  const pipeline = parsePipeline(recipe);\n  if (isFoundationPipeline(pipeline) !== foundation) {\n    const recipeEngine = isFoundationPipeline(pipeline) ? \"a foundation\" : \"an adapter\";\n    const specEngine = foundation ? \"a foundation\" : \"an adapter\";\n    throw new Error(`Pipeline is ${recipeEngine} recipe, but the behavior spec is ${specEngine} spec. Use a matching recipe or omit --file.`);\n  }\n  if (foundation) assertFoundationSettingsMatch(input.spec.foundation, pipeline);\n  return pipeline;\n}",
     "files": [
       "src/pipeline.ts",
       "src/commands/pipeline.ts",
@@ -314,7 +314,7 @@
     ],
     "path": "src/local-runtime/foundation-runner.ts",
     "line": 271,
-    "code": "export async function runFoundationPipeline(args: {\n  spec: LocalFoundationSpecFile;\n  plan: ExecutionPlan;\n  specPath: string;\n  outputDir?: string;\n  resume?: boolean;\n  spawnStep?: FoundationStepSpawn;\n  gpu?: AwsGpuConfig;\n}): Promise<FoundationPipelineResult> {\n  assertFoundationPlanSupported(args.spec, args.plan);\n  assertFoundationSettingsMatch(args.spec.foundation, args.plan);\n  const outputDir = resolve(\n    args.outputDir ?? join(dirname(resolve(args.specPath)), DEFAULT_FOUNDATION_RUNS_DIR, randomUUID()),\n  );\n  if (args.gpu && args.spec.foundation.checkpoint_backup_dir\n    && localPathsOverlap(outputDir, args.spec.foundation.checkpoint_backup_dir)) {\n    throw new Error(\"AWS checkpoint_backup_dir must be separate from the foundation run directory; overlapping backups can overwrite returned checkpoints.\");\n  }",
+    "code": "export async function runFoundationPipeline(args: {\n  spec: LocalFoundationSpecFile;\n  plan: ExecutionPlan;\n  specPath: string;\n  outputDir?: string;\n  resume?: boolean;\n  spawnStep?: FoundationStepSpawn;\n  gpu?: AwsGpuConfig;\n}): Promise<FoundationPipelineResult> {\n  assertFoundationPlanSupported(args.spec, args.plan);\n  assertFoundationSettingsMatch(args.spec.foundation, args.plan);\n  const outputDir = resolve(\n    args.outputDir ?? (args.spec.runtime?.artifactRoot\n      ? join(resolve(dirname(args.specPath), args.spec.runtime.artifactRoot), \"foundation-runs\", randomUUID())\n      : join(dirname(resolve(args.specPath)), DEFAULT_FOUNDATION_RUNS_DIR, randomUUID())),\n  );\n  if (args.gpu && args.spec.foundation.checkpoint_backup_dir\n    && localPathsOverlap(outputDir, args.spec.foundation.checkpoint_backup_dir)) {\n    throw new Error(\"AWS checkpoint_backup_dir must be separate from the foundation run directory; overlapping backups can overwrite returned checkpoints.\");\n  }\n  let outputExists = false;\n  try {\n    const outputInfo = await lstat(outputDir);\n    if (!args.resume) {\n      throw new Error(`Foundation output directory already exists; pass --resume ${outputDir} to recover it.`);\n    }\n    if (outputInfo.isSymbolicLink()) {\n      throw new Error(`Foundation resume directory must not be a symbolic link: ${outputDir}`);\n    }\n    if (!outputInfo.isDirectory()) {\n      throw new Error(`Foundation resume path must be a directory: ${outputDir}`);\n    }\n    outputExists = true;\n  } catch (error) {\n    if ((error as NodeJS.ErrnoException).code !== \"ENOENT\") throw error;\n  }\n  if (!outputExists) {\n    await mkdir(outputDir, { recursive: true, mode: 0o700 });\n  }\n  await makePrivateTree(outputDir);\n  const workflowPath = join(outputDir, \"resolved-workflow.json\");\n  const workflow = { spec: args.spec, plan: args.plan };\n  if (outputExists) {\n    const previous = await readFile(workflowPath, \"utf8\").catch((error) => {\n      if (error.code === \"ENOENT\") return undefined;\n      throw error;\n    });\n    if (previous && JSON.stringify(JSON.parse(previous)) !== JSON.stringify(workflow)) {\n      throw new Error(\"Foundation workflow configuration changed; resume requires the original resolved workflow.\");\n    }\n  }\n  await writePrivateTextAtomic(workflowPath, `${JSON.stringify(workflow, null, 2)}\\n`);",
     "files": [
       "src/local-runtime/foundation-runner.ts",
       "src/pipeline.ts",
@@ -345,7 +345,7 @@
       "src/local-runtime/process-runner.ts",
       "docs/local-runtime/aws-gpu.md"
     ],
-    "example": "tt doctor tunedtensor.json --config local-runner.json\ntt pipeline run --spec tunedtensor.json --config local-runner.json --dry-run\n# After checking the preview and GPU readiness:\ntt pipeline run --spec tunedtensor.json --config local-runner.json"
+    "example": "tt doctor tunedtensor.json\ntt pipeline run --spec tunedtensor.json --dry-run\n# After checking the preview and GPU readiness:\ntt pipeline run --spec tunedtensor.json"
   },
   "lora": {
     "title": "LoRA training implementation",
@@ -447,7 +447,7 @@
       "src/paths.ts",
       "src/agent-store.ts"
     ],
-    "example": "~/.tuned-tensor/config.json\n~/.tuned-tensor/agent/\n~/.tuned-tensor/cache/\n<project>/.tuned-tensor/artifacts/\n<project>/.tuned-tensor/store/  # when configured\n<spec-directory>/.tuned-tensor/foundation-runs/\n# Store fallback: ~/.tuned-tensor/store/"
+    "example": "~/.tuned-tensor/config.json\n~/.tuned-tensor/agent/\n~/.tuned-tensor/cache/\n<project>/.tuned-tensor/artifacts/\n<project>/.tuned-tensor/store/  # default for spec projects\n<spec-directory>/.tuned-tensor/foundation-runs/\n# Store fallback: ~/.tuned-tensor/store/"
   },
   "serve": {
     "title": "Serving & API clients",
@@ -472,7 +472,7 @@
       "training/serving/pyproject.toml",
       "docs/local-runtime/serving.md"
     ],
-    "example": "tt serve local-<run-id> --config local-runner.json\n# Or a foundation checkpoint:\ntt serve foundation --checkpoint <model-dir> --tokenizer <tokenizer.json>"
+    "example": "tt serve local-<run-id>\n# Or a foundation checkpoint:\ntt serve foundation --checkpoint <model-dir> --tokenizer <tokenizer.json>"
   },
   "publish": {
     "title": "Publish & account operations",

package/docs/spec-workflow.md

@@ -47,7 +47,8 @@ for an invalid spec.
 `get_local_spec` returns the document, validation diagnostics and content hash.
 `prepare_update_local_spec` requires that hash and a patch of supported fields.
 Arrays replace their whole field; `hyperparameters` and `foundation` merge by
-key. Unmentioned fields remain unchanged. Identity and engine cannot change
+key. `runtime` and `evaluation` merge recursively; null removes optional settings.
+`pipeline` replaces the entire recipe (null restores the derived default). Unmentioned fields remain unchanged. Identity and engine cannot change
 through this edit tool. The full resulting document must pass the runtime's
 strict schema, readiness checks, nonblank content checks and distinct-example
 checks. Small example sets produce a warning; validation is not a claim of model
@@ -82,7 +83,7 @@ tt pipeline run --spec tunedtensor.json --dry-run
 tt pipeline run --spec tunedtensor.json
 ```
 
-The default pipeline derives from the current spec. An adjacent
+The pipeline uses the optional `pipeline` section or derives from the current spec. An adjacent
 `tunedtensor.pipeline.json` is no longer loaded implicitly. To select an advanced
 recipe explicitly, pass `--file`; a missing explicit recipe or `--spec` file fails. Foundation
 training parameters in an explicit recipe must agree with the spec, including
@@ -100,10 +101,30 @@ The command reads one spec snapshot for planning, hardware warnings and executio
 JSON and final result JSON identify its path and SHA-256. The spec's compiled
 instructions and examples feed training and task evaluation; the independently
 configured general-regression suite keeps its own prompt. Provider choice and
-GPU placement remain separate runtime choices in agent configuration and
-`local-runner.json`. Evaluation policy (inference/scoring settings and the
-general-regression suite) also remains in the runner config. Moving that policy
-into the spec would be a separate schema change.
+GPU placement is configured in `runtime.gpu` in the spec. Model-provider choice
+remains in agent configuration. Evaluation policy belongs in `evaluation`;
+artifact/store locations and model paths belong in `runtime`. All relative paths
+resolve beside the spec. New projects default to `.tuned-tensor/artifacts` and
+`.tuned-tensor/store` beside it (unless `TT_LOCAL_HOME` overrides the store).
+Foundation supports `runtime.artifactRoot` and `runtime.gpu`; its evaluation
+and training settings remain under `foundation`.
+
+Only `tunedtensor.json` is required. `tt pipeline init --spec tunedtensor.json`
+embeds an advanced recipe when needed; omit this step to keep automatic derivation.
+An explicit `--file` cannot override a conflicting embedded recipe.
+
+For existing projects, run `tt pipeline migrate --spec tunedtensor.json` to
+consolidate adjacent pipeline and runner files. It validates the result first,
+keeps exclusive `.bak` copies, and preserves resolved paths and the old store
+location. Conflicting sources, enabled legacy `dryRun`, or unsupported foundation
+runner settings require reconciliation first. Existing `.bak` files are never
+overwritten. Avoid concurrent edits during migration. Legacy runner files remain
+readable with a warning, but cannot coexist with inline runtime/evaluation settings.
+A legacy adjacent pipeline is used only with explicit `--file` or after migration.
+
+Runs write generated `resolved-workflow.json` snapshots for diagnosis and
+reproducibility; these are outputs, not additional user-authored configuration.
+
 
 Agent `/approve` for a pipeline still runs a preview only. Real training uses
 the direct command above. The spec is the behavior recipe; approval of a spec

package/examples/single-spec/adapter/tunedtensor.json

@@ -0,0 +1,49 @@
+{
+  "id": "22222222-2222-4222-8222-222222222222",
+  "name": "Local Smoke Assistant",
+  "description": "Tiny local runner smoke test.",
+  "system_prompt": "Return exactly one sentiment label.",
+  "guidelines": [
+    "Reply with positive or negative."
+  ],
+  "constraints": [
+    "Do not include extra commentary."
+  ],
+  "base_model": "Qwen/Qwen3.5-2B",
+  "examples": [
+    {
+      "input": "Classify sentiment: I love this product.",
+      "output": "positive"
+    },
+    {
+      "input": "Classify sentiment: This was disappointing.",
+      "output": "negative"
+    },
+    {
+      "input": "Classify sentiment: The update works perfectly.",
+      "output": "positive"
+    },
+    {
+      "input": "Classify sentiment: Support never replied.",
+      "output": "negative"
+    }
+  ],
+  "hyperparameters": {
+    "n_epochs": 1,
+    "learning_rate": 0.0002,
+    "batch_size": 1,
+    "gradient_accumulation_steps": 1,
+    "lora_rank": 4,
+    "lora_alpha": 8,
+    "max_seq_length": 256,
+    "max_eval_examples": 2
+  },
+  "evaluation": {
+    "inference": {
+      "maxNewTokens": 32
+    },
+    "scoring": {
+      "mode": "exact_match"
+    }
+  }
+}

package/examples/single-spec/foundation/tunedtensor.json

@@ -0,0 +1,31 @@
+{
+  "engine": "foundation",
+  "name": "Foundation smoke chat",
+  "description": "Tiny from-scratch tokenizer, pretrain, chat SFT, and eval.",
+  "system_prompt": "You are a helpful assistant.",
+  "guidelines": [
+    "Answer the user request directly."
+  ],
+  "constraints": [],
+  "examples": [
+    {
+      "input": "Say hello in one sentence.",
+      "output": "Hello \u2014 how can I help you today?"
+    },
+    {
+      "input": "What is 2 + 2?",
+      "output": "2 + 2 = 4."
+    }
+  ],
+  "foundation": {
+    "vocab_size": 256,
+    "max_chars": 20000,
+    "depth": 2,
+    "pretrain_steps": 2,
+    "finetune_steps": 2,
+    "rl_steps": 0,
+    "batch_size": 2,
+    "sequence_length": 64,
+    "nproc_per_node": 1
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tuned-tensor/cli",
-  "version": "0.17.0",
+  "version": "0.18.0",
   "description": "Local TT agent and CUDA fine-tuning CLI for Tuned Tensor workflows",
   "repository": {
     "type": "git",
@@ -37,7 +37,8 @@
     "docs/spec-workflow.md",
     "docs/spec-architecture.html",
     "examples/local-runtime/*.json",
-    "examples/local-runtime/*.jsonl"
+    "examples/local-runtime/*.jsonl",
+    "examples/single-spec/**/*.json"
   ],
   "engines": {
     "node": ">=22.19.0"
@@ -68,6 +69,6 @@
     "tsup": "^8.5.1",
     "tsx": "^4.23.12",
     "typescript": "^5.9.3",
-    "vitest": "^4.1.8"
+    "vitest": "^4.1.11"
   }
 }

package/training/adapter/src/serve.py

@@ -93,7 +93,13 @@ def build_vllm_args(model_source: str, adapter_path: str | None, temp_dir: Path)
         # tool calls/results verbatim while merging leading system context.
         prefix = "{%- set tt = namespace(system=" + json.dumps(prompt) + ", history=[]) -%}"
         prefix += "{%- for message in messages -%}{%- if message.role == 'system' -%}"
+        # vLLM may normalize text into OpenAI content parts for the model's
+        # native template. Merge those parts without stringifying the list.
+        prefix += "{%- if message.content is string -%}"
         prefix += "{%- set tt.system = tt.system + '\\n\\n' + message.content -%}"
+        prefix += "{%- else -%}{%- for part in message.content -%}{%- if part.type == 'text' -%}"
+        prefix += "{%- set tt.system = tt.system + '\\n\\n' + part.text -%}"
+        prefix += "{%- endif -%}{%- endfor -%}{%- endif -%}"
         prefix += "{%- else -%}{%- set tt.history = tt.history + [message] -%}{%- endif -%}{%- endfor -%}"
         prefix += "{%- set messages = [{'role': 'system', 'content': tt.system}] + tt.history -%}"
         path = temp_dir / "chat-template.jinja"