@mcptoolshop/style-dataset-lab 2.3.0 → 3.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +178 -0
- package/CODE_OF_CONDUCT.md +9 -0
- package/CONTRIBUTING.md +71 -0
- package/README.md +82 -9
- package/SECURITY.md +6 -5
- package/bin/sdlab.js +359 -11
- package/lib/adapters/comfyui-runner.js +170 -0
- package/lib/adapters/comfyui-workflows.js +163 -0
- package/lib/adapters/diffusers-lora.js +7 -2
- package/lib/adapters/generic-image-caption.js +6 -1
- package/lib/args.js +200 -13
- package/lib/batch-compiler.js +122 -0
- package/lib/batch-modes.js +111 -0
- package/lib/batch-runs.js +237 -0
- package/lib/batch-sheet-render.js +209 -0
- package/lib/brief-compiler.js +625 -0
- package/lib/brief-render.js +205 -0
- package/lib/card.js +11 -4
- package/lib/comfyui.js +100 -6
- package/lib/config.js +68 -3
- package/lib/critique-engine.js +413 -0
- package/lib/critique-render.js +171 -0
- package/lib/deprecation.js +1 -1
- package/lib/eval-pack.js +94 -4
- package/lib/eval-runs.js +42 -3
- package/lib/export.js +144 -34
- package/lib/generated-provenance.js +87 -0
- package/lib/implementation-packs.js +15 -1
- package/lib/log.js +41 -4
- package/lib/paths.js +72 -3
- package/lib/refine-briefs.js +336 -0
- package/lib/reingest-selected.js +184 -0
- package/lib/reingest.js +15 -1
- package/lib/run-summary.js +132 -0
- package/lib/runtime-runs.js +218 -0
- package/lib/selections.js +375 -0
- package/lib/snapshot.js +100 -7
- package/lib/split.js +175 -17
- package/lib/training-manifests.js +17 -3
- package/lib/training-packages.js +28 -14
- package/lib/workflow-profiles.js +161 -0
- package/package.json +13 -4
- package/scripts/batch-generate.js +176 -0
- package/scripts/batch-sheet.js +46 -0
- package/scripts/batch-show.js +86 -0
- package/scripts/brief-compile.js +134 -0
- package/scripts/brief-show.js +53 -0
- package/scripts/canon-bind.js +7 -9
- package/scripts/card.js +22 -25
- package/scripts/compare.js +38 -33
- package/scripts/critique-show.js +50 -0
- package/scripts/critique.js +70 -0
- package/scripts/curate.js +91 -46
- package/scripts/doctor.js +172 -6
- package/scripts/eligibility.js +17 -14
- package/scripts/eval-pack.js +13 -11
- package/scripts/eval-run.js +30 -29
- package/scripts/export.js +27 -31
- package/scripts/generate-controlnet.js +31 -17
- package/scripts/generate-identity.js +37 -20
- package/scripts/generate-ipadapter.js +22 -16
- package/scripts/generate.js +68 -13
- package/scripts/implementation-pack.js +19 -14
- package/scripts/init.js +137 -17
- package/scripts/painterly.js +8 -10
- package/scripts/refine.js +119 -0
- package/scripts/reingest-selected.js +78 -0
- package/scripts/reingest.js +37 -18
- package/scripts/run-generate.js +76 -0
- package/scripts/run-list.js +58 -0
- package/scripts/run-show.js +63 -0
- package/scripts/select.js +119 -0
- package/scripts/selection-show.js +88 -0
- package/scripts/snapshot.js +22 -24
- package/scripts/split.js +26 -31
- package/scripts/training-manifest.js +29 -24
- package/scripts/training-package.js +24 -20
- package/scripts/training-profile.js +13 -8
- package/scripts/workflow-list.js +61 -0
- package/scripts/workflow-show.js +97 -0
- package/templates/domains/architecture/inputs/prompts/example-wave.json +32 -0
- package/templates/domains/architecture/terminology.json +10 -8
- package/templates/domains/architecture/workflows/profiles/environment-moodboard.json +62 -0
- package/templates/domains/architecture/workflows/profiles/interior-variant-pack.json +60 -0
- package/templates/domains/character-design/inputs/prompts/example-wave.json +32 -0
- package/templates/domains/character-design/project.json +4 -2
- package/templates/domains/character-design/workflows/profiles/character-portrait-set.json +61 -0
- package/templates/domains/character-design/workflows/profiles/expression-sheet.json +59 -0
- package/templates/domains/creature-design/inputs/prompts/example-wave.json +32 -0
- package/templates/domains/creature-design/project.json +4 -2
- package/templates/domains/creature-design/terminology.json +1 -0
- package/templates/domains/creature-design/workflows/profiles/creature-portrait-set.json +61 -0
- package/templates/domains/creature-design/workflows/profiles/silhouette-exploration.json +58 -0
- package/templates/domains/game-art/inputs/prompts/example-wave.json +37 -0
- package/templates/domains/game-art/project.json +4 -2
- package/templates/domains/game-art/workflows/profiles/character-portrait-set.json +62 -0
- package/templates/domains/game-art/workflows/profiles/environment-moodboard.json +61 -0
- package/templates/domains/vehicle-mech/inputs/prompts/example-wave.json +37 -0
- package/templates/domains/vehicle-mech/project.json +4 -2
- package/templates/domains/vehicle-mech/workflows/profiles/variant-pack.json +60 -0
- package/templates/domains/vehicle-mech/workflows/profiles/vehicle-silhouette-pack.json +59 -0
- package/scripts/bulk-curate-wave2-5.js +0 -138
- package/scripts/bulk-curate-waves11-18.js +0 -413
- package/scripts/curate-wave25.js +0 -209
- package/scripts/migrate-records.js +0 -141
- package/templates/inputs/prompts/example-wave.json +0 -27
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,184 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
|
|
5
|
+
## [3.0.1] - 2026-04-21
|
|
6
|
+
|
|
7
|
+
### Fixed — Dogfood swarm health pass (Stage A: bugs & security)
|
|
8
|
+
|
|
9
|
+
- **Four Laws enforcement** (`lib/snapshot.js`, `lib/split.js`, `lib/export.js`):
|
|
10
|
+
- Export now inherits `snapshot.config_fingerprint` instead of recomputing; throws `FINGERPRINT_DRIFT` on mismatch (Law 4).
|
|
11
|
+
- Deterministic majority-lane per family with lexicographic tie-break, recorded in `audit.family_lane_decisions` (Law 3).
|
|
12
|
+
- Unassigned-family fallback now throws instead of silently landing in train.
|
|
13
|
+
- `existsSync` guards across snapshot/split/export/eval/training-manifest/training-package/impl-pack prevent ID collisions from overwriting frozen artifacts (Law 1).
|
|
14
|
+
- `included.jsonl` entries now carry `rules_checked`, `profile_id`, and `config_fingerprint` for explainable inclusion (Law 2).
|
|
15
|
+
- **Security hardening**: path-traversal rejection on `record.asset_path` and all user-supplied `--source`/`--guide`/`--anchor`/`--ref`/`--outputs`/`--prompt-file`/`--packet-file`/`--domain` flags; URL validation on ComfyUI endpoints; safe filename regex at selection boundaries.
|
|
16
|
+
- **CLI structured errors**: ~25 scripts converted from raw `throw new Error` to `inputError`/`runtimeError` so exit codes are correct (1 = user error, 2 = runtime).
|
|
17
|
+
- **Pipeline correctness**: brief fingerprint uses recursive `canonicalize()` — deterministic across nested key reordering; override precedence fixed (`||` → `??` so legit zero/empty overrides are preserved); negative-prompt dedup switched from substring to token-exact match; adapter output filenames keyed by `record.id` to prevent basename collisions silently dropping records.
|
|
18
|
+
- **Re-ingest**: now actually copies images to `outputs/candidates/` instead of writing records that point at missing files.
|
|
19
|
+
- **Curate ordering**: `curate.js` moves image first, then updates record; bulk-curate trio gained `--dry-run` and standard error handling.
|
|
20
|
+
|
|
21
|
+
### Added — Dogfood swarm humanization pass (Stage B/C)
|
|
22
|
+
|
|
23
|
+
- **ComfyUI progress**: `submitAndWait` emits heartbeats every 15 s (queued → generating → completed); bails fast on `execution_error` or unknown status shapes instead of timing out silently.
|
|
24
|
+
- **Atomic manifest writes**: run/batch manifests use temp-file + rename; `checkpointRunManifest` saves progress after every slot so a crash no longer orphans GPU work.
|
|
25
|
+
- **`schema_version` enforcement**: every manifest (`snapshot`, `split`, `export`, `eval-pack`, `eval-run`, `training-manifest`, `training-package`, `implementation-pack`) now stamps `schema_version: '2.2.0'` with warn-on-mismatch loaders.
|
|
26
|
+
- **Export truthfulness**: tracks `expected`/`actual`/`failed` counts with per-record reasons and stderr warnings (no more silent-skip under `catch {}`).
|
|
27
|
+
- **Snapshot resilience**: malformed record JSON lands in `errors[]` instead of aborting the snapshot.
|
|
28
|
+
- **Split audit**: `deviation_from_target` per lane + `overall_deviation_score` in audit output.
|
|
29
|
+
- **Eval scorecards**: `sample_record_ids` up to 5 per failure bucket so operators can investigate.
|
|
30
|
+
- **CLI help**: `sdlab <cmd> --help` now works for 15 top commands via `HELP_TEXT` registry.
|
|
31
|
+
- **Did-you-mean**: hand-rolled Levenshtein suggestion for typo'd commands and flags.
|
|
32
|
+
- **`--project` fallback**: loud 2-line stderr warning when falling back to `star-freight` (silence with `SDLAB_QUIET_FALLBACK=1`).
|
|
33
|
+
- **Signal handling**: `SIGINT` (130), `SIGTERM` (143), `uncaughtException`, `unhandledRejection` handlers at `main()` so Ctrl+C surfaces a clean message.
|
|
34
|
+
- **ETA**: generate and batch-generate print ETA every 5 items with rolling averages.
|
|
35
|
+
- **`result()`/`success()` helpers**: artifact paths always print regardless of `--quiet`.
|
|
36
|
+
- **`sdlab init`**: scaffolded projects now get a `README.md` with quick commands, layout, and TODO sections.
|
|
37
|
+
- **Per-domain example waves**: all 5 domain templates ship `inputs/prompts/example-wave.json` with subjects matching their lane `id_patterns`.
|
|
38
|
+
- **Troubleshooting section**: README covers ComfyUI connection failure, missing weights, doctor errors, `--project` fallback, and bug reporting.
|
|
39
|
+
- **Install as primary CTA**: landing page leads with a copy-to-clipboard install command; `og:image`/`twitter:card` meta added via `SiteLayout`.
|
|
40
|
+
|
|
41
|
+
### Changed
|
|
42
|
+
|
|
43
|
+
- Moved `HANDOFF.md`, `WAVE_PLAN.md`, `WAVE27A_SESSION_STATE.md` to `docs/internal/` with a README explaining the archive.
|
|
44
|
+
- `lib/deprecation.js`: `--game` deprecation target bumped from "v3.0" (lying, package is 3.0) to "v4.0".
|
|
45
|
+
- Handbook `reference.md` and `architecture.md` rewritten against the v3.0 CLI surface (`--project`, `projects/<name>/`); `--game` demoted to a Legacy section.
|
|
46
|
+
- One-off Star Freight wave-curate scripts excluded from the npm tarball via `files`-field negations (137 → 135 files, 179.3 → 173.9 kB).
|
|
47
|
+
|
|
48
|
+
### Infrastructure
|
|
49
|
+
|
|
50
|
+
- **Test suite**: 0 → 98 tests across 12 files (`tests/lib-pipeline/`, `tests/lib-dataset/`, `tests/cli-scripts/`).
|
|
51
|
+
- **CI**: added `pull_request` trigger; `publish.yml` now runs `npm ci` + `npm run verify` + `npm test` + tag-vs-version guard before `npm publish`; `permissions: contents: read` and `persist-credentials: false` across all workflows; npm cache on every `setup-node`.
|
|
52
|
+
- **`package-lock.json`** regenerated from stale v2.2.1 to v3.0.1.
|
|
53
|
+
- **Governance**: added `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `.github/ISSUE_TEMPLATE/{bug_report,feature_request}.md`, `.github/PULL_REQUEST_TEMPLATE.md`.
|
|
54
|
+
- **Dependabot**: groups restricted to `minor`/`patch` so breaking majors get individual PRs.
|
|
55
|
+
|
|
56
|
+
## [3.0.0] - 2026-04-16
|
|
57
|
+
|
|
58
|
+
### Added
|
|
59
|
+
|
|
60
|
+
- **Selection + Reintegration (Phase 4E)** — close the production loop from generation back to corpus
|
|
61
|
+
- `sdlab select --run <id> --approve <files>` — select approved outputs from a run
|
|
62
|
+
- `sdlab select --batch <id> --approve <slot:file,...>` — select approved outputs from a batch
|
|
63
|
+
- `sdlab selection show [selection-id]` — list selections or show details
|
|
64
|
+
- `sdlab reingest selected --selection <id>` — re-ingest selected outputs as candidate records
|
|
65
|
+
- `--reason` and `--tags` flags for selection context
|
|
66
|
+
- `--dry-run` and `--json` output support
|
|
67
|
+
- **Selection engine** (`lib/selections.js`) — create selection artifacts from runs or batches
|
|
68
|
+
- Selection ID generation (`selection_YYYY-MM-DD_NNN`)
|
|
69
|
+
- Run and batch selection with output validation
|
|
70
|
+
- Chosen files copied into `selections/<id>/chosen/`
|
|
71
|
+
- Manifest, summary.json, and summary.md written per selection
|
|
72
|
+
- **Generated provenance** (`lib/generated-provenance.js`) — build provenance blocks for re-ingested records
|
|
73
|
+
- Traces back through run → brief → workflow → seed → config fingerprint
|
|
74
|
+
- Normalizes run and batch source context into a single shape
|
|
75
|
+
- **Re-ingest from selections** (`lib/reingest-selected.js`) — create candidate records from selections
|
|
76
|
+
- Records carry `source: generated` provenance with full generation provenance block
|
|
77
|
+
- Images copied to `inbox/generated/`
|
|
78
|
+
- Records tagged: `generated`, `selected`, workflow ID, subject ID
|
|
79
|
+
- No auto-bypass around review — records are staged for normal curation
|
|
80
|
+
- Provenance log written as `provenance.jsonl`
|
|
81
|
+
- **JSON schemas** for selections and generated records (`schemas/`)
|
|
82
|
+
- Doctor validates new project directories (`selections/`, `inbox/generated/`)
|
|
83
|
+
- Init scaffolds `selections/` and `inbox/generated/` directories
|
|
84
|
+
|
|
85
|
+
### Changed
|
|
86
|
+
|
|
87
|
+
- **README** rewritten to describe the full production loop: canon → dataset → training → brief → run → critique → batch → select → re-ingest
|
|
88
|
+
- **Landing page** updated: new headline, description, 4-tab preview (Start, Dataset, Produce, Close the loop), 7 feature cards including production workflow and selection
|
|
89
|
+
- **Handbook** updated: index shows production loop diagram, new [Production Loop](handbook/production-loop/) page with complete end-to-end walkthrough
|
|
90
|
+
- **Package metadata** updated: description, keywords (`generation-workflow`, `batch-production`, `reintegration`), version badge
|
|
91
|
+
- Version bumped to **v3.0.0** — the repo is now a full production workbench, not just a dataset preparation system
|
|
92
|
+
|
|
93
|
+
## [2.6.0] - 2026-04-16
|
|
94
|
+
|
|
95
|
+
### Added
|
|
96
|
+
|
|
97
|
+
- **Batch production modes (Phase 4D)** — coordinated multi-slot generation with reviewable sheet output
|
|
98
|
+
- `sdlab batch generate --mode <id>` — execute a batch production mode (expression sheets, environment boards, etc.)
|
|
99
|
+
- `sdlab batch show [batch-id]` — list all batches or show batch details
|
|
100
|
+
- `sdlab batch sheet <batch-id>` — re-render batch sheet from saved manifest
|
|
101
|
+
- `--dry-run` flag prepares batch directory and briefs without submitting to ComfyUI
|
|
102
|
+
- **Batch modes** (`lib/batch-modes.js`) — load, validate, and list batch mode definitions from `workflows/batch-modes/`
|
|
103
|
+
- **Batch compiler** (`lib/batch-compiler.js`) — expand one batch mode into multiple slot briefs with controlled per-slot deltas
|
|
104
|
+
- **Batch runs** (`lib/batch-runs.js`) — batch directory creation (`batch_YYYY-MM-DD_NNN`), coordinated slot execution, manifest save/load
|
|
105
|
+
- **Batch sheet renderer** (`lib/batch-sheet-render.js`) — HTML review surfaces with CSS grid layout, dark theme, zero dependencies
|
|
106
|
+
- **4 starter batch modes** for Star Freight: expression-sheet, environment-board, silhouette-pack, continuity-variants
|
|
107
|
+
- **JSON schemas** for batch mode definitions and batch manifests (`schemas/`)
|
|
108
|
+
- Doctor validates batch mode definitions (batch_type, subject_mode, slot uniqueness, layout, workflow references)
|
|
109
|
+
- Init scaffolds `workflows/batch-modes/` and `batches/` directories
|
|
110
|
+
|
|
111
|
+
## [2.5.0] - 2026-04-16
|
|
112
|
+
|
|
113
|
+
### Added
|
|
114
|
+
|
|
115
|
+
- **Critique + Refine loop (Phase 4C)** — inspect run outputs and generate focused next-pass briefs
|
|
116
|
+
- `sdlab critique --run <id>` — generate structured critique of a completed run
|
|
117
|
+
- `sdlab critique show --run <id>` — display saved critique (text, JSON, markdown)
|
|
118
|
+
- `sdlab refine --run <id> --pick <file>` — generate refined next-pass brief from critique
|
|
119
|
+
- CLI override flags: `--preserve`, `--push`, `--suppress` for manual delta control
|
|
120
|
+
- **Critique engine** (`lib/critique-engine.js`) — brief-driven candidate analysis
|
|
121
|
+
- 10 critique dimensions activated by workflow canon focus and drift guards
|
|
122
|
+
- Per-candidate structured notes: strengths, drift issues, preserve/correct next pass
|
|
123
|
+
- Run-level recommendation: accept_one, refine_from_one, rerun_broader, discard_run
|
|
124
|
+
- Workflow-specific emphasis by output mode (portrait, expression, moodboard, etc.)
|
|
125
|
+
- **Refine briefs** (`lib/refine-briefs.js`) — delta-layered brief generation
|
|
126
|
+
- Preserve/push/suppress instructions derived from critique + CLI overrides
|
|
127
|
+
- Prompt refinement: original prompt preserved, delta instructions appended
|
|
128
|
+
- Negative prompt extension: suppress items added without replacing existing negatives
|
|
129
|
+
- Refined briefs point back to parent brief + source run + picked candidate
|
|
130
|
+
- **JSON schemas** for critique reports and refined briefs (`schemas/`)
|
|
131
|
+
- Critique and refine artifacts stored in run directory: `runs/<id>/critique.json`, `runs/<id>/refine/`
|
|
132
|
+
|
|
133
|
+
## [2.4.0] - 2026-04-16
|
|
134
|
+
|
|
135
|
+
### Added
|
|
136
|
+
|
|
137
|
+
- **ComfyUI runtime adapter (Phase 4B)** — execute compiled briefs through ComfyUI and capture run artifacts
|
|
138
|
+
- `sdlab run generate --brief <id>` — execute a brief through ComfyUI, capture outputs to runs/
|
|
139
|
+
- `sdlab run show <id>` — show run manifest and results
|
|
140
|
+
- `sdlab run list` — list all runs for a project
|
|
141
|
+
- `--dry-run` flag prepares run directory without submitting to ComfyUI
|
|
142
|
+
- `--seed <n>` override for reproducible generation
|
|
143
|
+
- **Run management** (`lib/runtime-runs.js`) — run ID generation (`run_YYYY-MM-DD_NNN`), seed plans (fixed/increment/random), run directory preparation, manifest save/load
|
|
144
|
+
- **ComfyUI workflow adapter** (`lib/adapters/comfyui-workflows.js`) — builds ComfyUI prompt graphs from compiled briefs, template resolution by output mode
|
|
145
|
+
- **ComfyUI runner** (`lib/adapters/comfyui-runner.js`) — orchestrates seed plan → graph build → submit → poll → download → manifest for each image
|
|
146
|
+
- **Run summaries** (`lib/run-summary.js`) — generates summary.json and summary.md in each run directory
|
|
147
|
+
- **Runtime templates** (`runtime/comfyui/txt2img-standard.json`) — declarative workflow template metadata
|
|
148
|
+
- Doctor validates runtime templates (Section 10)
|
|
149
|
+
- `runs/` added to project scaffold
|
|
150
|
+
|
|
151
|
+
## [2.3.0] - 2026-04-15
|
|
152
|
+
|
|
153
|
+
### Added
|
|
154
|
+
|
|
155
|
+
- **Production workbench spine (Phase 4A)** — workflow profiles and brief compiler
|
|
156
|
+
- `sdlab workflow list` — list workflow profiles for a project
|
|
157
|
+
- `sdlab workflow show <id>` — show workflow profile details
|
|
158
|
+
- `sdlab brief compile` — compile a deterministic generation brief from project truth
|
|
159
|
+
- `sdlab brief show <id>` — show compiled brief (text, JSON, or markdown)
|
|
160
|
+
- **Workflow profiles** — production recipes defining output shape, canon constraints, drift guards, and runtime defaults
|
|
161
|
+
- 6 output modes: portrait_set, expression_sheet, variant_pack, moodboard, silhouette_sheet, turnaround
|
|
162
|
+
- 12 canon focus categories: silhouette, material_language, palette, anatomy, costume_logic, era_logic, scale_logic, lighting, surface_wear, gesture, composition, faction_read
|
|
163
|
+
- Subject mode enforcement: required, optional, forbidden
|
|
164
|
+
- **Brief compiler** — merges 6 layers of project truth into a frozen generation contract:
|
|
165
|
+
- Project canon → lane config → subject identity → training asset hints → workflow profile → CLI overrides
|
|
166
|
+
- Structured prompt assembly (no freeform prompt soup)
|
|
167
|
+
- Deterministic output: same inputs produce identical briefs
|
|
168
|
+
- Config fingerprint (SHA-256) for change detection
|
|
169
|
+
- Outputs JSON + human-readable Markdown
|
|
170
|
+
- **3 starter workflows for Star Freight**: character-portrait-set, expression-sheet, environment-moodboard
|
|
171
|
+
- **10 domain template workflows** across 5 domains (game-art, character-design, creature-design, architecture, vehicle-mech)
|
|
172
|
+
- **New library modules:** `lib/workflow-profiles.js`, `lib/brief-compiler.js`, `lib/brief-render.js`
|
|
173
|
+
- **Doctor validates workflow profiles** — checks schema, lane references, subject mode, runtime defaults
|
|
174
|
+
- **Init scaffolds `workflows/profiles/` and `briefs/`** directories and copies domain workflow templates
|
|
175
|
+
|
|
176
|
+
### Key properties
|
|
177
|
+
|
|
178
|
+
- Workflow profiles are NOT training profiles — they drive generation, not training
|
|
179
|
+
- Briefs are frozen contracts — the creative runtime boundary between planning and execution
|
|
180
|
+
- Subject constraints are hard when subject_mode = required (fail loudly if missing)
|
|
181
|
+
- Drift guards appear as explicit warnings, never hidden in metadata
|
|
182
|
+
|
|
5
183
|
## [2.2.0] - 2026-04-16
|
|
6
184
|
|
|
7
185
|
### Added
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# Code of Conduct
|
|
2
|
+
|
|
3
|
+
This project follows the [Contributor Covenant v2.1](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).
|
|
4
|
+
|
|
5
|
+
In short: be respectful, assume good faith, and focus discussion on the technical work. Harassment, personal attacks, and discrimination are not tolerated.
|
|
6
|
+
|
|
7
|
+
## Reporting
|
|
8
|
+
|
|
9
|
+
Report concerns by emailing `64996768+mcp-tool-shop@users.noreply.github.com`. Reports are handled confidentially. The full Contributor Covenant enforcement ladder applies (correction → warning → temporary ban → permanent ban).
|
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Contributing to style-dataset-lab
|
|
2
|
+
|
|
3
|
+
Thanks for your interest. This repo is the canon-aware dataset pipeline for visual AI projects — it prioritizes determinism, auditability, and a single source of truth per project. Contributions that reinforce those properties are welcome.
|
|
4
|
+
|
|
5
|
+
## Dev setup
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab
|
|
9
|
+
cd style-dataset-lab
|
|
10
|
+
npm install # zero runtime deps; installs dev-only tooling
|
|
11
|
+
npm test # runs the full node --test suite
|
|
12
|
+
npm run verify # sanity smoke: help output + project doctor on star-freight
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Requires Node.js 20+. ComfyUI is only needed for changes that touch the generation path (`lib/comfyui.js`, `scripts/generate.js`, `scripts/run-generate.js`). Doctrine, snapshot, split, export, and curation paths can all be exercised without a GPU — see the "Try it without ComfyUI" section of the README.
|
|
16
|
+
|
|
17
|
+
## Coding style
|
|
18
|
+
|
|
19
|
+
- **ESM only.** Every file uses `import` / `export`. No CommonJS.
|
|
20
|
+
- **No TypeScript.** Plain JavaScript. Types live in JSDoc when they help.
|
|
21
|
+
- **Built-in test runner.** Tests use `node --test` from Node 20+ — no Jest, Vitest, Mocha.
|
|
22
|
+
- **No runtime dependencies.** `package.json` intentionally has zero `dependencies`. Dev-only tooling goes in `devDependencies`; a new runtime dep requires a very good reason.
|
|
23
|
+
- **Structured errors.** Errors follow the Structured Error Shape — `code`, `message`, `hint`, optional `cause`, optional `retryable`. See `lib/errors.js` (or equivalent) for helpers.
|
|
24
|
+
- **Canon is truth.** Never hand-edit generated artifacts (snapshots, splits, exports, runs, critiques, selections). Edit the upstream config or record and re-run.
|
|
25
|
+
|
|
26
|
+
## Testing
|
|
27
|
+
|
|
28
|
+
Every change ships with tests. For bug fixes, add a regression test that would have failed before the fix.
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
# Run the full suite
|
|
32
|
+
npm test
|
|
33
|
+
|
|
34
|
+
# Run a single file
|
|
35
|
+
node --test tests/path/to/file.test.js
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
CI runs the same suite on Node 20 and Node 22 (see `.github/workflows/ci.yml`).
|
|
39
|
+
|
|
40
|
+
## PR checklist
|
|
41
|
+
|
|
42
|
+
Before opening a PR:
|
|
43
|
+
|
|
44
|
+
- [ ] `npm test` passes locally
|
|
45
|
+
- [ ] `npm run verify` passes locally
|
|
46
|
+
- [ ] New features have test coverage
|
|
47
|
+
- [ ] `CHANGELOG.md` has an entry under `## [Unreleased]`
|
|
48
|
+
- [ ] If you touched `package.json`, you regenerated `package-lock.json` (`npm install --package-lock-only`)
|
|
49
|
+
- [ ] If you added or changed a config file schema, `schemas/` is updated too
|
|
50
|
+
- [ ] Commit message explains *why*, not just *what*
|
|
51
|
+
|
|
52
|
+
## Shipcheck
|
|
53
|
+
|
|
54
|
+
Every release passes `npx @mcptoolshop/shipcheck audit` with all hard gates (A–D) green. Before we publish, we also re-read `SHIP_GATE.md` against the current repo state. If your change affects security posture, error handling, operator docs, or shipping hygiene, update `SHIP_GATE.md` in the same PR.
|
|
55
|
+
|
|
56
|
+
See the [shipcheck docs](https://github.com/mcp-tool-shop-org/shipcheck) for the full quality-gate definition.
|
|
57
|
+
|
|
58
|
+
## Reporting bugs
|
|
59
|
+
|
|
60
|
+
File an issue using the bug-report template at https://github.com/mcp-tool-shop-org/style-dataset-lab/issues/new/choose. Minimum repro info:
|
|
61
|
+
|
|
62
|
+
- `sdlab --version`
|
|
63
|
+
- `node --version`
|
|
64
|
+
- ComfyUI version (if relevant)
|
|
65
|
+
- Project name and relevant config file excerpt
|
|
66
|
+
- `sdlab project doctor --project <name>` output
|
|
67
|
+
- The failing command and full structured error output
|
|
68
|
+
|
|
69
|
+
## Code of conduct
|
|
70
|
+
|
|
71
|
+
Be decent. See [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) — we use the Contributor Covenant.
|
package/README.md
CHANGED
|
@@ -11,11 +11,11 @@
|
|
|
11
11
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
12
12
|
</p>
|
|
13
13
|
|
|
14
|
-
Write your visual rules. Generate art. Judge every image against those rules. Ship the results as versioned, auditable training data.
|
|
14
|
+
Write your visual rules. Generate art. Judge every image against those rules. Ship the results as versioned, auditable training data — then put trained models to work in real production workflows and feed the best outputs back into your corpus.
|
|
15
15
|
|
|
16
|
-
Style Dataset Lab connects the thing you wrote down about your art style to the dataset you actually train from. You define a constitution — silhouette rules, palette constraints, material language, whatever matters for your project. The pipeline generates candidates, scores them against those rules, and packages the approved work into reproducible datasets where every record explains why it was included.
|
|
16
|
+
Style Dataset Lab connects the thing you wrote down about your art style to the dataset you actually train from, and then closes the loop all the way through production. You define a constitution — silhouette rules, palette constraints, material language, whatever matters for your project. The pipeline generates candidates, scores them against those rules, and packages the approved work into reproducible datasets where every record explains why it was included.
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
Then the production workbench takes over: compile generation briefs from project truth, run them through ComfyUI, critique the outputs, batch-produce expression sheets and environment boards, select the best results, and re-ingest them as new candidates. The loop closes: produce, select, review, strengthen.
|
|
19
19
|
|
|
20
20
|
## The pipeline
|
|
21
21
|
|
|
@@ -28,25 +28,37 @@ sdlab generate inputs/prompts/wave1.json --project my-project
|
|
|
28
28
|
sdlab curate <id> approved "Strong silhouette, correct faction palette"
|
|
29
29
|
|
|
30
30
|
# Bind approved work to constitution rules
|
|
31
|
-
sdlab bind
|
|
31
|
+
# (`sdlab bind` is a shorter alias for `canon-bind`)
|
|
32
|
+
sdlab canon-bind --project my-project
|
|
32
33
|
|
|
33
34
|
# Freeze a versioned dataset
|
|
34
35
|
sdlab snapshot create --project my-project
|
|
35
36
|
sdlab split build
|
|
36
37
|
sdlab export build
|
|
37
38
|
|
|
38
|
-
# Build a training package
|
|
39
|
+
# Build a training package
|
|
39
40
|
sdlab training-manifest create --profile character-style-lora
|
|
40
41
|
sdlab training-package build
|
|
41
|
-
|
|
42
|
-
|
|
42
|
+
|
|
43
|
+
# Compile a production brief and run it
|
|
44
|
+
sdlab brief compile --workflow character-portrait-set --subject kael_maren
|
|
45
|
+
sdlab run generate --brief brief_2026-04-16_001
|
|
46
|
+
|
|
47
|
+
# Critique, refine, batch-produce
|
|
48
|
+
sdlab critique --run run_2026-04-16_001
|
|
49
|
+
sdlab refine --run run_2026-04-16_001 --pick 001.png
|
|
50
|
+
sdlab batch generate --mode expression-sheet --subject kael_maren
|
|
51
|
+
|
|
52
|
+
# Select the best outputs and bring them back
|
|
53
|
+
sdlab select --run run_2026-04-16_001 --approve 001.png,003.png
|
|
54
|
+
sdlab reingest selected --selection selection_2026-04-16_001
|
|
43
55
|
```
|
|
44
56
|
|
|
45
|
-
That last command is the point.
|
|
57
|
+
That last command is the point. Selected outputs come back through the same review process as everything else. The corpus grows and the rules hold.
|
|
46
58
|
|
|
47
59
|
## What it produces
|
|
48
60
|
|
|
49
|
-
Seven
|
|
61
|
+
Seven dataset artifacts and a full production workbench. Each artifact links to its predecessors so you can trace any training record back to the rule that approved it.
|
|
50
62
|
|
|
51
63
|
| Artifact | What it is |
|
|
52
64
|
|----------|-----------|
|
|
@@ -58,6 +70,17 @@ Seven versioned, checksummed artifacts. Each links to its predecessors so you ca
|
|
|
58
70
|
| **Eval scorecard** | Per-task pass/fail from scoring generated outputs against eval packs. |
|
|
59
71
|
| **Implementation pack** | Prompt examples, known failures, continuity tests, and re-ingest guidance. |
|
|
60
72
|
|
|
73
|
+
The production workbench adds:
|
|
74
|
+
|
|
75
|
+
| Surface | What it does |
|
|
76
|
+
|---------|-------------|
|
|
77
|
+
| **Compiled brief** | Deterministic generation instruction from workflow profile + project truth. |
|
|
78
|
+
| **Run** | Frozen execution artifact: brief + seeds + ComfyUI outputs + manifest. |
|
|
79
|
+
| **Critique** | Structured multi-dimension evaluation of run outputs against canon. |
|
|
80
|
+
| **Batch** | Coordinated multi-slot production (expression sheets, environment boards, silhouette packs). |
|
|
81
|
+
| **Selection** | Creative decision artifact: which outputs were chosen, why, and where they came from. |
|
|
82
|
+
| **Re-ingest** | Selected outputs return as candidate records with full generation provenance. |
|
|
83
|
+
|
|
61
84
|
## Why this exists
|
|
62
85
|
|
|
63
86
|
Training data is the highest-leverage artifact in any visual AI pipeline. But most training data is a folder of images with no history, no judgment trail, and no connection to the style rules it was supposed to follow.
|
|
@@ -94,6 +117,12 @@ projects/my-project/
|
|
|
94
117
|
splits/ Train/val/test partitions
|
|
95
118
|
exports/ Versioned export packages
|
|
96
119
|
training/ Profiles, manifests, packages, eval runs, implementations
|
|
120
|
+
workflows/ Workflow profiles + batch mode definitions
|
|
121
|
+
briefs/ Compiled generation briefs
|
|
122
|
+
runs/ Execution artifacts (brief + outputs + manifest)
|
|
123
|
+
batches/ Coordinated multi-slot productions
|
|
124
|
+
selections/ Chosen outputs with reasons and provenance
|
|
125
|
+
inbox/generated/ Re-ingested images awaiting review
|
|
97
126
|
```
|
|
98
127
|
|
|
99
128
|
## Trust properties
|
|
@@ -130,6 +159,50 @@ npm install -g @mcptoolshop/style-dataset-lab
|
|
|
130
159
|
|
|
131
160
|
Requires Node.js 20+ and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) on localhost:8188 for generation.
|
|
132
161
|
|
|
162
|
+
### Try it without ComfyUI
|
|
163
|
+
|
|
164
|
+
You can explore the full non-generation surface — inspection, curation, snapshot, split, export — using the bundled Star Freight project without installing ComfyUI or downloading any SDXL weights.
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
# Scaffold a fresh project (no ComfyUI needed)
|
|
168
|
+
sdlab init test --domain game-art
|
|
169
|
+
|
|
170
|
+
# Run the canonical health check (no ComfyUI needed)
|
|
171
|
+
sdlab project doctor --project test
|
|
172
|
+
|
|
173
|
+
# Dry-run a snapshot against the bundled Star Freight corpus
|
|
174
|
+
sdlab snapshot create --dry-run --project star-freight
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
`sdlab project doctor` validates every project config (constitution, lanes, rubric, terminology) and reports eligibility without touching the GPU. Any command that mutates generated state accepts `--dry-run` to preview the effect first.
|
|
178
|
+
|
|
179
|
+
If you forget `--project`, the CLI falls back to the first project it finds under `projects/` and prints a warning — pass `--project` explicitly to silence it.
|
|
180
|
+
|
|
181
|
+
## Troubleshooting
|
|
182
|
+
|
|
183
|
+
Common failure modes and fixes:
|
|
184
|
+
|
|
185
|
+
**`ECONNREFUSED 127.0.0.1:8188` on any `sdlab generate` / `sdlab run generate` / `sdlab batch generate`**
|
|
186
|
+
ComfyUI isn't running. Start ComfyUI (`python main.py --listen 127.0.0.1 --port 8188`) and confirm with `curl http://127.0.0.1:8188/system_stats`. To point at a different host/port, set `COMFY_URL=http://host:port`.
|
|
187
|
+
|
|
188
|
+
**`missing checkpoint` / `LoRA weight not found`**
|
|
189
|
+
Your workflow profile names a model file that isn't in ComfyUI's `models/checkpoints/` or `models/loras/` folder. Open `projects/<project>/workflows/profiles/<profile>.json`, locate the `checkpoint` or `lora` field, and either download the referenced weight or swap it for one you already have. Re-run `sdlab project doctor --project <project>` to confirm the fix.
|
|
190
|
+
|
|
191
|
+
**`sdlab project doctor` errors**
|
|
192
|
+
Doctor returns structured error codes. Common ones:
|
|
193
|
+
- `E_PROJECT_NOT_FOUND` — the project directory doesn't exist under `projects/`. Check spelling.
|
|
194
|
+
- `E_CONFIG_INVALID` — one of the five JSON config files failed schema validation. The `hint` field names the bad file and field.
|
|
195
|
+
- `E_RECORD_DRIFT` — a record's config fingerprint no longer matches its source. Re-curate or re-bind as the hint suggests.
|
|
196
|
+
|
|
197
|
+
**`No --project specified, falling back to <name>`**
|
|
198
|
+
A soft warning. Pass `--project <name>` explicitly to select the right project and silence the warning.
|
|
199
|
+
|
|
200
|
+
**Painterly / VRAM out-of-memory issues**
|
|
201
|
+
See `docs/internal/HANDOFF.md` for the painterly denoise tuning notes. In short: lower the denoise strength, reduce batch size, or switch to a smaller checkpoint in your workflow profile.
|
|
202
|
+
|
|
203
|
+
**Reporting bugs**
|
|
204
|
+
File an issue at https://github.com/mcp-tool-shop-org/style-dataset-lab/issues with your sdlab version (`sdlab --version`), Node version (`node -v`), the full command, and the structured error output. A bug-report template prefills the fields.
|
|
205
|
+
|
|
133
206
|
## Security
|
|
134
207
|
|
|
135
208
|
Local-only. No telemetry, no analytics, no external requests. Images stay on your GPU and filesystem.
|
package/SECURITY.md
CHANGED
|
@@ -2,11 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
## Supported Versions
|
|
4
4
|
|
|
5
|
-
| Version | Supported
|
|
6
|
-
|
|
7
|
-
|
|
|
8
|
-
|
|
|
9
|
-
|
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
|---------|------------------------|
|
|
7
|
+
| 3.x | Yes |
|
|
8
|
+
| 2.x | Yes (transitional) |
|
|
9
|
+
| 1.x | No |
|
|
10
|
+
| 0.x | No |
|
|
10
11
|
|
|
11
12
|
## Threat Model
|
|
12
13
|
|