@tuned-tensor/local 0.1.9 → 0.2.1

package/docs/architecture.md

@@ -13,13 +13,16 @@ packaged as a standalone CLI.
   compatible hosted exports.
 - Artifact store: writes datasets, logs, model outputs, and reports beneath a
   configured local artifact root.
-- Local state store: persists specs, runs, progress events, reports, and model
-  records as JSON/JSONL under a configured `storeRoot` for the dashboard and CLI.
-- Training adapter: launches a uv-managed Python process with local input,
-  model cache, and output directories passed through environment variables.
-- Evaluation adapter: runs local Hugging Face/PEFT inference for baseline and
-  candidate evaluation. Evaluation can also use an OpenRouter LLM judge to score
-  locally generated outputs.
+- Local state store: keeps a SQLite metadata index for specs, runs, progress
+  events, and model records while preserving JSON/JSONL files under a
+  configured `storeRoot` for portable artifacts, recovery, and inspection.
+- Training adapter: launches either a uv-managed Python process or an arbitrary
+  command with local input, model cache, and output directories passed through
+  environment variables.
+- Evaluation adapter: runs local Hugging Face/PEFT inference, an arbitrary
+  batch command, or per-example commands for baseline and candidate evaluation.
+  Evaluation can also use an OpenRouter LLM judge to score locally generated
+  outputs.
 - Report writer: emits structured JSON reports that are portable across local
   and hosted environments.
 - Dashboard/API server: serves local run/model/spec metadata from the local
@@ -43,11 +46,12 @@ The first runnable milestone should support:
 - one local run at a time;
 - one GPU device;
 - local filesystem artifacts;
-- local file-backed run/model/spec tracking;
+- SQLite-backed run/model/spec tracking with recoverable JSON files;
 - standalone `tt-local init`, `validate`, `run`, `serve`, and inspection
   commands;
-- uv-based training;
-- native Transformers/PEFT evaluation with optional OpenRouter judge scoring;
+- uv-based or command-based training;
+- native Transformers/PEFT, batch-command, or per-example command evaluation
+  with optional OpenRouter judge scoring;
 - local dashboard and CLI inspection commands;
 - a tiny smoke-run fixture that finishes quickly.
 
@@ -56,12 +60,12 @@ The first runnable milestone should support:
 The public runner should accept a plain JSON run request and a local runner
 configuration. The run request describes the behavior spec, base model,
 examples, and hyperparameters. The local runner configuration describes artifact
-paths, store paths, uv entrypoint settings, model cache paths, OpenRouter judge
-settings, and timeout limits.
+paths, store paths, uv or command entrypoint settings, model cache paths,
+OpenRouter judge settings, and timeout limits.
 
 ## Evaluation Loop
 
-The local loop intentionally compares like-for-like model artifacts:
+The default Transformers loop intentionally compares like-for-like model artifacts:
 
 - baseline loads the original Hugging Face base model from the behavior spec;
 - training writes a Hugging Face/PEFT artifact under the run directory;
@@ -72,25 +76,33 @@ The first native backend is `transformers`. `llama.cpp`/GGUF conversion is not
 part of the default path because conversion would make the baseline/candidate
 comparison less direct.
 
+Custom model families can use `training.backend: "command"` and
+`evaluation.inference.provider: "batch_command"`. Batch evaluators receive the
+same `--input`/`--output` JSON files as the bundled Transformers evaluator, so a
+custom workflow can load any artifact format as long as it writes compatible
+evaluation results.
+
 ## Local State Layout
 
 The local store is intentionally transparent and easy to back up:
 
+- `metadata.sqlite`
 - `specs/<spec-id>/spec.json`
 - `runs/<run-id>/request.json`
 - `runs/<run-id>/state.json`
 - `runs/<run-id>/progress.jsonl`
 - `runs/<run-id>/run-report.json`
 - `models/<model-id>/model.json`
-- `catalog/*.jsonl`
 
-Catalog files are append-only indexes for fast listing. `tt-local store
-rebuild-index` can reconstruct them from the canonical per-object files.
+SQLite is the primary metadata index for CLI and dashboard listings. The
+per-object JSON files remain the recoverable source for artifacts and
+human-readable inspection. `tt-local store rebuild-index` can reconstruct the
+SQLite index from the canonical per-object files.
 
 ## Dashboard API
 
-The local server is a lightweight Node HTTP server, not a separate database.
-Current endpoints include:
+The local server is a lightweight Node HTTP server backed by the local state
+store. Current endpoints include:
 
 - `GET /api/health`
 - `GET /api/runs`, `GET /api/runs/:id`, `GET /api/runs/:id/events`

package/docs/spark.md

@@ -41,7 +41,8 @@ tt-local run examples/smoke-run-request.json --config examples/local-runner.json
 
 ## Real Training
 
-For real training, set `dryRun: false` and configure the uv training entrypoint:
+For real training with the bundled SFT workflow, set `dryRun: false` and
+configure the uv training entrypoint:
 
 ```json
 {
@@ -73,6 +74,33 @@ that same base model plus the fine-tuned artifact from the run directory. Set
 `paths.modelCache` to a persistent Spark-local cache so training and evaluation
 reuse downloads.
 
+Custom workflows such as a from-scratch nanoGPT trainer can use command
+entrypoints instead:
+
+```json
+{
+  "artifactRoot": "/home/eve/tt-local-artifacts",
+  "dryRun": false,
+  "training": {
+    "backend": "command",
+    "command": ["python", "nanogpt/train.py"]
+  },
+  "evaluation": {
+    "inference": {
+      "provider": "batch_command",
+      "command": ["python", "nanogpt/evaluate.py"]
+    },
+    "scoring": {
+      "mode": "exact_match"
+    }
+  }
+}
+```
+
+The training command receives the same environment variables listed above. The
+batch evaluator is called with `--input <path> --output <path>` and should write
+the same results JSON shape as the bundled evaluator.
+
 The included first-pass SFT script can be run by the local runner through uv:
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tuned-tensor/local",
-  "version": "0.1.9",
+  "version": "0.2.1",
   "description": "TT Local: tuning tensors locally on hardware you control.",
   "license": "Apache-2.0",
   "private": false,
@@ -34,11 +34,13 @@
     "node": ">=22"
   },
   "devDependencies": {
+    "@types/better-sqlite3": "^7.6.13",
     "@types/node": "^25.0.0",
     "tsx": "^4.22.4",
     "typescript": "^6.0.0"
   },
   "dependencies": {
+    "better-sqlite3": "^12.11.1",
     "zod": "^4.4.3"
   }
 }