@mcptoolshop/style-dataset-lab 3.0.0 → 3.0.1

package/CHANGELOG.md

@@ -2,23 +2,57 @@
 
 All notable changes to this project will be documented in this file.
 
-## [2.6.0] - 2026-07-01
+## [3.0.1] - 2026-04-21
+
+### Fixed — Dogfood swarm health pass (Stage A: bugs & security)
+
+- **Four Laws enforcement** (`lib/snapshot.js`, `lib/split.js`, `lib/export.js`):
+  - Export now inherits `snapshot.config_fingerprint` instead of recomputing; throws `FINGERPRINT_DRIFT` on mismatch (Law 4).
+  - Deterministic majority-lane per family with lexicographic tie-break, recorded in `audit.family_lane_decisions` (Law 3).
+  - Unassigned-family fallback now throws instead of silently landing in train.
+  - `existsSync` guards across snapshot/split/export/eval/training-manifest/training-package/impl-pack prevent ID collisions from overwriting frozen artifacts (Law 1).
+  - `included.jsonl` entries now carry `rules_checked`, `profile_id`, and `config_fingerprint` for explainable inclusion (Law 2).
+- **Security hardening**: path-traversal rejection on `record.asset_path` and all user-supplied `--source`/`--guide`/`--anchor`/`--ref`/`--outputs`/`--prompt-file`/`--packet-file`/`--domain` flags; URL validation on ComfyUI endpoints; safe filename regex at selection boundaries.
+- **CLI structured errors**: ~25 scripts converted from raw `throw new Error` to `inputError`/`runtimeError` so exit codes are correct (1 = user error, 2 = runtime).
+- **Pipeline correctness**: brief fingerprint uses recursive `canonicalize()` — deterministic across nested key reordering; override precedence fixed (`||` → `??` so legit zero/empty overrides are preserved); negative-prompt dedup switched from substring to token-exact match; adapter output filenames keyed by `record.id` to prevent basename collisions silently dropping records.
+- **Re-ingest**: now actually copies images to `outputs/candidates/` instead of writing records that point at missing files.
+- **Curate ordering**: `curate.js` moves image first, then updates record; bulk-curate trio gained `--dry-run` and standard error handling.
+
+### Added — Dogfood swarm humanization pass (Stage B/C)
+
+- **ComfyUI progress**: `submitAndWait` emits heartbeats every 15 s (queued → generating → completed); bails fast on `execution_error` or unknown status shapes instead of timing out silently.
+- **Atomic manifest writes**: run/batch manifests use temp-file + rename; `checkpointRunManifest` saves progress after every slot so a crash no longer orphans GPU work.
+- **`schema_version` enforcement**: every manifest (`snapshot`, `split`, `export`, `eval-pack`, `eval-run`, `training-manifest`, `training-package`, `implementation-pack`) now stamps `schema_version: '2.2.0'` with warn-on-mismatch loaders.
+- **Export truthfulness**: tracks `expected`/`actual`/`failed` counts with per-record reasons and stderr warnings (no more silent-skip under `catch {}`).
+- **Snapshot resilience**: malformed record JSON lands in `errors[]` instead of aborting the snapshot.
+- **Split audit**: `deviation_from_target` per lane + `overall_deviation_score` in audit output.
+- **Eval scorecards**: `sample_record_ids` up to 5 per failure bucket so operators can investigate.
+- **CLI help**: `sdlab <cmd> --help` now works for 15 top commands via `HELP_TEXT` registry.
+- **Did-you-mean**: hand-rolled Levenshtein suggestion for typo'd commands and flags.
+- **`--project` fallback**: loud 2-line stderr warning when falling back to `star-freight` (silence with `SDLAB_QUIET_FALLBACK=1`).
+- **Signal handling**: `SIGINT` (130), `SIGTERM` (143), `uncaughtException`, `unhandledRejection` handlers at `main()` so Ctrl+C surfaces a clean message.
+- **ETA**: generate and batch-generate print ETA every 5 items with rolling averages.
+- **`result()`/`success()` helpers**: artifact paths always print regardless of `--quiet`.
+- **`sdlab init`**: scaffolded projects now get a `README.md` with quick commands, layout, and TODO sections.
+- **Per-domain example waves**: all 5 domain templates ship `inputs/prompts/example-wave.json` with subjects matching their lane `id_patterns`.
+- **Troubleshooting section**: README covers ComfyUI connection failure, missing weights, doctor errors, `--project` fallback, and bug reporting.
+- **Install as primary CTA**: landing page leads with a copy-to-clipboard install command; `og:image`/`twitter:card` meta added via `SiteLayout`.
 
-### Added
+### Changed
+
+- Moved `HANDOFF.md`, `WAVE_PLAN.md`, `WAVE27A_SESSION_STATE.md` to `docs/internal/` with a README explaining the archive.
+- `lib/deprecation.js`: `--game` deprecation target bumped from "v3.0" (lying, package is 3.0) to "v4.0".
+- Handbook `reference.md` and `architecture.md` rewritten against the v3.0 CLI surface (`--project`, `projects/<name>/`); `--game` demoted to a Legacy section.
+- One-off Star Freight wave-curate scripts excluded from the npm tarball via `files`-field negations (137 → 135 files, 179.3 → 173.9 kB).
+
+### Infrastructure
+
+- **Test suite**: 0 → 98 tests across 12 files (`tests/lib-pipeline/`, `tests/lib-dataset/`, `tests/cli-scripts/`).
+- **CI**: added `pull_request` trigger; `publish.yml` now runs `npm ci` + `npm run verify` + `npm test` + tag-vs-version guard before `npm publish`; `permissions: contents: read` and `persist-credentials: false` across all workflows; npm cache on every `setup-node`.
+- **`package-lock.json`** regenerated from stale v2.2.1 to v3.0.1.
+- **Governance**: added `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `.github/ISSUE_TEMPLATE/{bug_report,feature_request}.md`, `.github/PULL_REQUEST_TEMPLATE.md`.
+- **Dependabot**: groups restricted to `minor`/`patch` so breaking majors get individual PRs.
 
-- **Batch production modes (Phase 4D)** — coordinated multi-slot generation with reviewable sheet output
-  - `sdlab batch generate --mode <id>` — execute a batch production mode (expression sheets, environment boards, etc.)
-  - `sdlab batch show [batch-id]` — list all batches or show batch details
-  - `sdlab batch sheet <batch-id>` — re-render batch sheet from saved manifest
-  - `--dry-run` flag prepares batch directory and briefs without submitting to ComfyUI
-- **Batch modes** (`lib/batch-modes.js`) — load, validate, and list batch mode definitions from `workflows/batch-modes/`
-- **Batch compiler** (`lib/batch-compiler.js`) — expand one batch mode into multiple slot briefs with controlled per-slot deltas
-- **Batch runs** (`lib/batch-runs.js`) — batch directory creation (`batch_YYYY-MM-DD_NNN`), coordinated slot execution, manifest save/load
-- **Batch sheet renderer** (`lib/batch-sheet-render.js`) — HTML review surfaces with CSS grid layout, dark theme, zero dependencies
-- **4 starter batch modes** for Star Freight: expression-sheet, environment-board, silhouette-pack, continuity-variants
-- **JSON schemas** for batch mode definitions and batch manifests (`schemas/`)
-- Doctor validates batch mode definitions (batch_type, subject_mode, slot uniqueness, layout, workflow references)
-- Init scaffolds `workflows/batch-modes/` and `batches/` directories
 ## [3.0.0] - 2026-04-16
 
 ### Added
@@ -55,6 +89,25 @@ All notable changes to this project will be documented in this file.
 - **Handbook** updated: index shows production loop diagram, new [Production Loop](handbook/production-loop/) page with complete end-to-end walkthrough
 - **Package metadata** updated: description, keywords (`generation-workflow`, `batch-production`, `reintegration`), version badge
 - Version bumped to **v3.0.0** — the repo is now a full production workbench, not just a dataset preparation system
+
+## [2.6.0] - 2026-04-16
+
+### Added
+
+- **Batch production modes (Phase 4D)** — coordinated multi-slot generation with reviewable sheet output
+  - `sdlab batch generate --mode <id>` — execute a batch production mode (expression sheets, environment boards, etc.)
+  - `sdlab batch show [batch-id]` — list all batches or show batch details
+  - `sdlab batch sheet <batch-id>` — re-render batch sheet from saved manifest
+  - `--dry-run` flag prepares batch directory and briefs without submitting to ComfyUI
+- **Batch modes** (`lib/batch-modes.js`) — load, validate, and list batch mode definitions from `workflows/batch-modes/`
+- **Batch compiler** (`lib/batch-compiler.js`) — expand one batch mode into multiple slot briefs with controlled per-slot deltas
+- **Batch runs** (`lib/batch-runs.js`) — batch directory creation (`batch_YYYY-MM-DD_NNN`), coordinated slot execution, manifest save/load
+- **Batch sheet renderer** (`lib/batch-sheet-render.js`) — HTML review surfaces with CSS grid layout, dark theme, zero dependencies
+- **4 starter batch modes** for Star Freight: expression-sheet, environment-board, silhouette-pack, continuity-variants
+- **JSON schemas** for batch mode definitions and batch manifests (`schemas/`)
+- Doctor validates batch mode definitions (batch_type, subject_mode, slot uniqueness, layout, workflow references)
+- Init scaffolds `workflows/batch-modes/` and `batches/` directories
+
 ## [2.5.0] - 2026-04-16
 
 ### Added

package/CODE_OF_CONDUCT.md

@@ -0,0 +1,9 @@
+# Code of Conduct
+
+This project follows the [Contributor Covenant v2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
+
+In short: be respectful, assume good faith, and focus discussion on the technical work. Harassment, personal attacks, and discrimination are not tolerated.
+
+## Reporting
+
+Report concerns by emailing `64996768+mcp-tool-shop@users.noreply.github.com`. Reports are handled confidentially. The full Contributor Covenant enforcement ladder applies (correction → warning → temporary ban → permanent ban).

package/CONTRIBUTING.md

@@ -0,0 +1,71 @@
+# Contributing to style-dataset-lab
+
+Thanks for your interest. This repo is the canon-aware dataset pipeline for visual AI projects — it prioritizes determinism, auditability, and a single source of truth per project. Contributions that reinforce those properties are welcome.
+
+## Dev setup
+
+```bash
+git clone https://github.com/mcp-tool-shop-org/style-dataset-lab
+cd style-dataset-lab
+npm install        # zero runtime deps; installs dev-only tooling
+npm test           # runs the full node --test suite
+npm run verify     # sanity smoke: help output + project doctor on star-freight
+```
+
+Requires Node.js 20+. ComfyUI is only needed for changes that touch the generation path (`lib/comfyui.js`, `scripts/generate.js`, `scripts/run-generate.js`). Doctrine, snapshot, split, export, and curation paths can all be exercised without a GPU — see the "Try it without ComfyUI" section of the README.
+
+## Coding style
+
+- **ESM only.** Every file uses `import` / `export`. No CommonJS.
+- **No TypeScript.** Plain JavaScript. Types live in JSDoc when they help.
+- **Built-in test runner.** Tests use `node --test` from Node 20+ — no Jest, Vitest, Mocha.
+- **No runtime dependencies.** `package.json` intentionally has zero `dependencies`. Dev-only tooling goes in `devDependencies`; a new runtime dep requires a very good reason.
+- **Structured errors.** Errors follow the Structured Error Shape — `code`, `message`, `hint`, optional `cause`, optional `retryable`. See `lib/errors.js` (or equivalent) for helpers.
+- **Canon is truth.** Never hand-edit generated artifacts (snapshots, splits, exports, runs, critiques, selections). Edit the upstream config or record and re-run.
+
+## Testing
+
+Every change ships with tests. For bug fixes, add a regression test that would have failed before the fix.
+
+```bash
+# Run the full suite
+npm test
+
+# Run a single file
+node --test tests/path/to/file.test.js
+```
+
+CI runs the same suite on Node 20 and Node 22 (see `.github/workflows/ci.yml`).
+
+## PR checklist
+
+Before opening a PR:
+
+- [ ] `npm test` passes locally
+- [ ] `npm run verify` passes locally
+- [ ] New features have test coverage
+- [ ] `CHANGELOG.md` has an entry under `## [Unreleased]`
+- [ ] If you touched `package.json`, you regenerated `package-lock.json` (`npm install --package-lock-only`)
+- [ ] If you added or changed a config file schema, `schemas/` is updated too
+- [ ] Commit message explains *why*, not just *what*
+
+## Shipcheck
+
+Every release passes `npx @mcptoolshop/shipcheck audit` with all hard gates (A–D) green. Before we publish, we also re-read `SHIP_GATE.md` against the current repo state. If your change affects security posture, error handling, operator docs, or shipping hygiene, update `SHIP_GATE.md` in the same PR.
+
+See the [shipcheck docs](https://github.com/mcp-tool-shop-org/shipcheck) for the full quality-gate definition.
+
+## Reporting bugs
+
+File an issue using the bug-report template at https://github.com/mcp-tool-shop-org/style-dataset-lab/issues/new/choose. Minimum repro info:
+
+- `sdlab --version`
+- `node --version`
+- ComfyUI version (if relevant)
+- Project name and relevant config file excerpt
+- `sdlab project doctor --project <name>` output
+- The failing command and full structured error output
+
+## Code of conduct
+
+Be decent. See [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — we use the Contributor Covenant.

package/README.md

@@ -28,7 +28,8 @@ sdlab generate inputs/prompts/wave1.json --project my-project
 sdlab curate <id> approved "Strong silhouette, correct faction palette"
 
 # Bind approved work to constitution rules
-sdlab bind --project my-project
+# (`sdlab bind` is a shorter alias for `canon-bind`)
+sdlab canon-bind --project my-project
 
 # Freeze a versioned dataset
 sdlab snapshot create --project my-project
@@ -158,6 +159,50 @@ npm install -g @mcptoolshop/style-dataset-lab
 
 Requires Node.js 20+ and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) on localhost:8188 for generation.
 
+### Try it without ComfyUI
+
+You can explore the full non-generation surface — inspection, curation, snapshot, split, export — using the bundled Star Freight project without installing ComfyUI or downloading any SDXL weights.
+
+```bash
+# Scaffold a fresh project (no ComfyUI needed)
+sdlab init test --domain game-art
+
+# Run the canonical health check (no ComfyUI needed)
+sdlab project doctor --project test
+
+# Dry-run a snapshot against the bundled Star Freight corpus
+sdlab snapshot create --dry-run --project star-freight
+```
+
+`sdlab project doctor` validates every project config (constitution, lanes, rubric, terminology) and reports eligibility without touching the GPU. Any command that mutates generated state accepts `--dry-run` to preview the effect first.
+
+If you forget `--project`, the CLI falls back to the first project it finds under `projects/` and prints a warning — pass `--project` explicitly to silence it.
+
+## Troubleshooting
+
+Common failure modes and fixes:
+
+**`ECONNREFUSED 127.0.0.1:8188` on any `sdlab generate` / `sdlab run generate` / `sdlab batch generate`**
+ComfyUI isn't running. Start ComfyUI (`python main.py --listen 127.0.0.1 --port 8188`) and confirm with `curl http://127.0.0.1:8188/system_stats`. To point at a different host/port, set `COMFY_URL=http://host:port`.
+
+**`missing checkpoint` / `LoRA weight not found`**
+Your workflow profile names a model file that isn't in ComfyUI's `models/checkpoints/` or `models/loras/` folder. Open `projects/<project>/workflows/profiles/<profile>.json`, locate the `checkpoint` or `lora` field, and either download the referenced weight or swap it for one you already have. Re-run `sdlab project doctor --project <project>` to confirm the fix.
+
+**`sdlab project doctor` errors**
+Doctor returns structured error codes. Common ones:
+- `E_PROJECT_NOT_FOUND` — the project directory doesn't exist under `projects/`. Check spelling.
+- `E_CONFIG_INVALID` — one of the five JSON config files failed schema validation. The `hint` field names the bad file and field.
+- `E_RECORD_DRIFT` — a record's config fingerprint no longer matches its source. Re-curate or re-bind as the hint suggests.
+
+**`No --project specified, falling back to <name>`**
+A soft warning. Pass `--project <name>` explicitly to select the right project and silence the warning.
+
+**Painterly / VRAM out-of-memory issues**
+See `docs/internal/HANDOFF.md` for the painterly denoise tuning notes. In short: lower the denoise strength, reduce batch size, or switch to a smaller checkpoint in your workflow profile.
+
+**Reporting bugs**
+File an issue at https://github.com/mcp-tool-shop-org/style-dataset-lab/issues with your sdlab version (`sdlab --version`), Node version (`node -v`), the full command, and the structured error output. A bug-report template prefills the fields.
+
 ## Security
 
 Local-only. No telemetry, no analytics, no external requests. Images stay on your GPU and filesystem.

package/SECURITY.md

@@ -2,11 +2,12 @@
 
 ## Supported Versions
 
-| Version | Supported |
-|---------|-----------|
-| 2.x     | Yes       |
-| 1.x     | No        |
-| 0.x     | No        |
+| Version | Supported              |
+|---------|------------------------|
+| 3.x     | Yes                    |
+| 2.x     | Yes (transitional)     |
+| 1.x     | No                     |
+| 0.x     | No                     |
 
 ## Threat Model