@mcptoolshop/style-dataset-lab 2.1.0 → 2.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.es.md +137 -108
- package/README.fr.md +137 -108
- package/README.hi.md +137 -108
- package/README.it.md +137 -108
- package/README.ja.md +137 -108
- package/README.md +80 -144
- package/README.pt-BR.md +137 -108
- package/README.zh.md +137 -108
- package/bin/sdlab.js +26 -0
- package/lib/adapters/diffusers-lora.js +110 -0
- package/lib/adapters/generic-image-caption.js +94 -0
- package/lib/eval-runs.js +285 -0
- package/lib/implementation-packs.js +316 -0
- package/lib/reingest.js +131 -0
- package/lib/training-manifests.js +172 -0
- package/lib/training-packages.js +241 -0
- package/lib/training-profiles.js +97 -0
- package/package.json +1 -1
- package/scripts/doctor.js +5 -0
- package/scripts/eval-run.js +121 -0
- package/scripts/implementation-pack.js +78 -0
- package/scripts/init.js +5 -0
- package/scripts/reingest.js +86 -0
- package/scripts/training-manifest.js +119 -0
- package/scripts/training-package.js +84 -0
- package/scripts/training-profile.js +63 -0
package/README.md
CHANGED
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="README.ja.md">日本語</a> | <a href="README.zh.md">中文</a> | <a href="README.es.md">Español</a> | <a href="README.fr.md">Français</a> | <a href="README.hi.md">हिन्दी</a> | <a href="README.it.md">Italiano</a> | <a href="README.pt-BR.md">Português (BR)</a>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
1
5
|
<p align="center">
|
|
2
6
|
<img src="https://raw.githubusercontent.com/mcp-tool-shop-org/brand/main/logos/style-dataset-lab/readme.png" width="400">
|
|
3
7
|
</p>
|
|
@@ -7,196 +11,128 @@
|
|
|
7
11
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
8
12
|
</p>
|
|
9
13
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
Turn approved visual work into versioned, review-backed datasets, splits, export packages, and eval packs.
|
|
14
|
+
Write your visual rules. Generate art. Judge every image against those rules. Ship the results as versioned, auditable training data.
|
|
13
15
|
|
|
14
|
-
|
|
16
|
+
Style Dataset Lab connects the thing you wrote down about your art style to the dataset you actually train from. You define a constitution — silhouette rules, palette constraints, material language, whatever matters for your project. The pipeline generates candidates, scores them against those rules, and packages the approved work into reproducible datasets where every record explains why it was included.
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
The pipeline produces four artifacts:
|
|
19
|
-
|
|
20
|
-
| Artifact | What it is |
|
|
21
|
-
|----------|-----------|
|
|
22
|
-
| **Snapshot** | Frozen, fingerprinted selection of eligible records. Every inclusion has an explicit reason trace. |
|
|
23
|
-
| **Split** | Leakage-safe train/val/test partition. Records sharing a subject family always land in the same split. |
|
|
24
|
-
| **Export package** | Self-contained dataset: manifest, metadata, images, splits, dataset card, and checksums. |
|
|
25
|
-
| **Eval pack** | Canon-aware verification tasks: lane coverage, forbidden drift, anchor/gold, subject continuity. |
|
|
18
|
+
The loop closes: train a model, generate new outputs, score them against the same canon, re-ingest what passes. The dataset grows and the rules hold.
|
|
26
19
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
1. **Provenance** -- full generation history (checkpoint, LoRA, seed, sampler, cfg, timing)
|
|
30
|
-
2. **Canon binding** -- which constitution rules this asset passes, fails, or partially meets
|
|
31
|
-
3. **Quality judgment** -- approved/rejected/borderline with per-dimension scores
|
|
32
|
-
|
|
33
|
-
Works for game art, character design, creature design, architecture, vehicle/mech concepts, and any domain where visual production needs to stay on-model.
|
|
34
|
-
|
|
35
|
-
## Quick start
|
|
20
|
+
## The pipeline
|
|
36
21
|
|
|
37
22
|
```bash
|
|
38
|
-
#
|
|
39
|
-
npm install -g @mcptoolshop/style-dataset-lab
|
|
40
|
-
|
|
41
|
-
# Scaffold a new project
|
|
23
|
+
# Write your canon. Scaffold the project.
|
|
42
24
|
sdlab init my-project --domain character-design
|
|
43
25
|
|
|
44
|
-
#
|
|
45
|
-
sdlab
|
|
46
|
-
|
|
26
|
+
# Generate candidates via ComfyUI, then review them
|
|
27
|
+
sdlab generate inputs/prompts/wave1.json --project my-project
|
|
28
|
+
sdlab curate <id> approved "Strong silhouette, correct faction palette"
|
|
47
29
|
|
|
48
|
-
|
|
30
|
+
# Bind approved work to constitution rules
|
|
31
|
+
sdlab bind --project my-project
|
|
49
32
|
|
|
50
|
-
|
|
33
|
+
# Freeze a versioned dataset
|
|
34
|
+
sdlab snapshot create --project my-project
|
|
35
|
+
sdlab split build
|
|
36
|
+
sdlab export build
|
|
51
37
|
|
|
52
|
-
|
|
53
|
-
sdlab
|
|
54
|
-
sdlab
|
|
55
|
-
|
|
56
|
-
sdlab
|
|
57
|
-
sdlab generate:identity <packet> # Named-subject identity images
|
|
58
|
-
sdlab generate:controlnet # ControlNet-guided generation
|
|
59
|
-
sdlab generate:ipadapter # IP-Adapter reference-guided
|
|
60
|
-
|
|
61
|
-
sdlab curate <id> <status> <explanation> # Record review judgment
|
|
62
|
-
sdlab compare <a> <b> <winner> <reason> # Pairwise A-vs-B comparison
|
|
63
|
-
sdlab bind [--project <name>] # Bind records to constitution rules
|
|
64
|
-
sdlab painterly [--project <name>] # Post-processing style pass
|
|
65
|
-
|
|
66
|
-
sdlab snapshot create [--profile <name>] # Create frozen dataset snapshot
|
|
67
|
-
sdlab snapshot list # List all snapshots
|
|
68
|
-
sdlab snapshot diff <a> <b> # Compare two snapshots
|
|
69
|
-
sdlab eligibility audit # Audit record training eligibility
|
|
70
|
-
sdlab split build [--snapshot <id>] # Build train/val/test split
|
|
71
|
-
sdlab split audit <id> # Audit split for leakage + balance
|
|
72
|
-
sdlab card generate # Generate dataset card (md + JSON)
|
|
73
|
-
sdlab export build [--snapshot <id>] # Build versioned export package
|
|
74
|
-
sdlab eval-pack build # Build canon-aware eval pack
|
|
38
|
+
# Build a training package and close the loop
|
|
39
|
+
sdlab training-manifest create --profile character-style-lora
|
|
40
|
+
sdlab training-package build
|
|
41
|
+
sdlab eval-run create && sdlab eval-run score <id> --outputs results.jsonl
|
|
42
|
+
sdlab reingest generated --source ./outputs --manifest <id>
|
|
75
43
|
```
|
|
76
44
|
|
|
77
|
-
|
|
45
|
+
That last command is the point. Generated outputs come back through the same review process as everything else. The loop closes.
|
|
78
46
|
|
|
79
|
-
##
|
|
47
|
+
## What it produces
|
|
80
48
|
|
|
81
|
-
Each
|
|
49
|
+
Seven versioned, checksummed artifacts. Each links to its predecessors so you can trace any training record back to the rule that approved it.
|
|
82
50
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
records/ Per-asset JSON (provenance + judgment + canon)
|
|
93
|
-
inputs/prompts/ Prompt packs (JSON)
|
|
94
|
-
outputs/ Generated images (gitignored)
|
|
95
|
-
comparisons/ A-vs-B preference judgments
|
|
96
|
-
snapshots/ Frozen dataset snapshots
|
|
97
|
-
splits/ Train/val/test partitions
|
|
98
|
-
exports/ Versioned export packages
|
|
99
|
-
eval-packs/ Canon-aware eval instruments
|
|
100
|
-
```
|
|
51
|
+
| Artifact | What it is |
|
|
52
|
+
|----------|-----------|
|
|
53
|
+
| **Snapshot** | Frozen record selection with config fingerprint. Every inclusion has an explicit reason. |
|
|
54
|
+
| **Split** | Train/val/test partition where subject families never cross boundaries. |
|
|
55
|
+
| **Export package** | Self-contained dataset: manifest, metadata, images, splits, dataset card, checksums. |
|
|
56
|
+
| **Eval pack** | Canon-aware test tasks: lane coverage, forbidden drift, anchor/gold, subject continuity. |
|
|
57
|
+
| **Training package** | Trainer-ready layout via adapters (`diffusers-lora`, `generic-image-caption`). Same truth, different format. |
|
|
58
|
+
| **Eval scorecard** | Per-task pass/fail from scoring generated outputs against eval packs. |
|
|
59
|
+
| **Implementation pack** | Prompt examples, known failures, continuity tests, and re-ingest guidance. |
|
|
101
60
|
|
|
102
|
-
##
|
|
61
|
+
## Why this exists
|
|
103
62
|
|
|
104
|
-
|
|
105
|
-
canon → generate → curate → bind → snapshot → split → export → eval
|
|
106
|
-
| | | | | | | |
|
|
107
|
-
rules ComfyUI judgment rules frozen subject package verify
|
|
108
|
-
selection isolation
|
|
109
|
-
```
|
|
63
|
+
Training data is the highest-leverage artifact in any visual AI pipeline. But most training data is a folder of images with no history, no judgment trail, and no connection to the style rules it was supposed to follow.
|
|
110
64
|
|
|
111
|
-
|
|
112
|
-
2. **Generate** -- ComfyUI produces candidates with full provenance
|
|
113
|
-
3. **Curate** -- approve/reject with per-dimension scores and failure modes
|
|
114
|
-
4. **Bind** -- link each asset to constitution rules with pass/fail/partial verdicts
|
|
115
|
-
5. **Snapshot** -- freeze eligible records into a deterministic, fingerprinted selection
|
|
116
|
-
6. **Split** -- partition into train/val/test with subject isolation and lane balance
|
|
117
|
-
7. **Export** -- build a self-contained package with manifest, metadata, images, and checksums
|
|
118
|
-
8. **Eval** -- generate canon-aware test instruments for model verification
|
|
65
|
+
Style Dataset Lab makes the connection explicit. Your constitution defines the rules. Your rubric defines the scoring dimensions. Your curation records the judgment. Your canon binding proves the connection. And your dataset carries all of that forward as structured, queryable, reproducible truth.
|
|
119
66
|
|
|
120
|
-
|
|
67
|
+
The practical result: when your LoRA drifts, you can ask *why*. When your next training round needs better data, you know exactly which records are near-misses and what single rule they failed. When a new team member asks what the project's visual language is, the answer isn't a Figma board — it's a searchable constitution with 1,182 graded examples.
|
|
121
68
|
|
|
122
|
-
##
|
|
69
|
+
## Five domains, real rules
|
|
123
70
|
|
|
124
|
-
Each domain
|
|
71
|
+
Not placeholder templates. Each domain ships with production-grade constitution rules, lane definitions, scoring rubrics, and group vocabulary.
|
|
125
72
|
|
|
126
|
-
| Domain | Lanes |
|
|
127
|
-
|
|
128
|
-
| **game-art** | character, environment, prop, UI, ship, interior, equipment | Silhouette at gameplay scale, faction
|
|
129
|
-
| **character-design** | portrait, full_body, turnaround, expression_sheet, action_pose |
|
|
130
|
-
| **creature-design** | concept, orthographic, detail_study, action, scale_reference, habitat |
|
|
131
|
-
| **architecture** | exterior, interior, streetscape, structural_detail, ruin, landscape |
|
|
132
|
-
| **vehicle-mech** | exterior, cockpit, component, schematic, silhouette_sheet, damage_variant | Mechanical logic,
|
|
73
|
+
| Domain | Lanes | What gets judged |
|
|
74
|
+
|--------|-------|-----------------|
|
|
75
|
+
| **game-art** | character, environment, prop, UI, ship, interior, equipment | Silhouette at gameplay scale, faction read, wear and aging |
|
|
76
|
+
| **character-design** | portrait, full_body, turnaround, expression_sheet, action_pose | Proportions, costume logic, personality, gesture clarity |
|
|
77
|
+
| **creature-design** | concept, orthographic, detail_study, action, scale_reference, habitat | Anatomy, evolutionary logic, silhouette distinction |
|
|
78
|
+
| **architecture** | exterior, interior, streetscape, structural_detail, ruin, landscape | Structure, material consistency, perspective, era coherence |
|
|
79
|
+
| **vehicle-mech** | exterior, cockpit, component, schematic, silhouette_sheet, damage_variant | Mechanical logic, design language, access points, damage narrative |
|
|
133
80
|
|
|
134
|
-
##
|
|
81
|
+
## Project structure
|
|
135
82
|
|
|
136
|
-
|
|
83
|
+
Each project is self-contained. Five JSON config files define the rules; everything else is data.
|
|
137
84
|
|
|
138
85
|
```
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
86
|
+
projects/my-project/
|
|
87
|
+
project.json Identity + generation defaults
|
|
88
|
+
constitution.json Rules with rationale templates
|
|
89
|
+
lanes.json Subject lanes with detection patterns
|
|
90
|
+
rubric.json Scoring dimensions + thresholds
|
|
91
|
+
terminology.json Group vocabulary + detection order
|
|
92
|
+
records/ Per-asset JSON (provenance + judgment + canon)
|
|
93
|
+
snapshots/ Frozen dataset snapshots
|
|
94
|
+
splits/ Train/val/test partitions
|
|
95
|
+
exports/ Versioned export packages
|
|
96
|
+
training/ Profiles, manifests, packages, eval runs, implementations
|
|
147
97
|
```
|
|
148
98
|
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
**Splits** assign records to train/val/test partitions with subject isolation (no subject family appears in multiple splits) and lane-balanced distribution. Seeded PRNG ensures identical results from the same seed.
|
|
99
|
+
## Trust properties
|
|
152
100
|
|
|
153
|
-
|
|
101
|
+
These are not aspirational. They are enforced.
|
|
154
102
|
|
|
155
|
-
|
|
103
|
+
- **Snapshots are immutable.** Config fingerprint (SHA-256) proves nothing changed.
|
|
104
|
+
- **Splits prevent leakage.** Subject families (by identity, lineage, or ID suffix) never cross partition boundaries.
|
|
105
|
+
- **Manifests are frozen contracts.** Export hash + config fingerprint. If anything changes, create a new one.
|
|
106
|
+
- **Adapters cannot mutate truth.** Different layout, same records. No additions, no removals, no reclassification.
|
|
107
|
+
- **Generated outputs re-enter through review.** No bypass. Curate and bind like everything else.
|
|
156
108
|
|
|
157
|
-
|
|
109
|
+
## Star Freight
|
|
158
110
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
Clone the repo for a complete working example: 1,182 records, 28 prompt waves, 5 factions, 7 lanes, 24 constitution rules, and 892 approved assets from a gritty sci-fi RPG.
|
|
111
|
+
The repo includes a complete working example: 1,182 records, 5 factions, 7 lanes, 24 constitution rules, 892 approved assets, 2 training profiles. A gritty sci-fi RPG visual canon, fully curated.
|
|
162
112
|
|
|
163
113
|
```bash
|
|
164
114
|
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab
|
|
165
115
|
cd style-dataset-lab
|
|
166
|
-
|
|
167
|
-
# Validate the project
|
|
168
116
|
sdlab project doctor --project star-freight
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
sdlab snapshot create --project star-freight # 839 eligible records
|
|
172
|
-
sdlab split build --project star-freight # ~80/10/10, zero leakage
|
|
173
|
-
sdlab export build --project star-freight # package with checksums
|
|
174
|
-
sdlab eval-pack build --project star-freight # 78 eval records
|
|
117
|
+
sdlab snapshot create --project star-freight # 839 eligible records
|
|
118
|
+
sdlab split build --project star-freight # zero subject leakage
|
|
175
119
|
```
|
|
176
120
|
|
|
177
|
-
##
|
|
121
|
+
## Downstream formats
|
|
178
122
|
|
|
179
|
-
|
|
123
|
+
`sdlab` owns the dataset. Format conversion is handled by [`repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset): TRL, LLaVA, Qwen2-VL, JSONL, Parquet, and more. `repo-dataset` renders; it never decides inclusion.
|
|
180
124
|
|
|
181
|
-
|
|
182
|
-
# Rename your data directory
|
|
183
|
-
mv games projects
|
|
125
|
+
## Install
|
|
184
126
|
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
sdlab bind --project star-freight # canonical form
|
|
127
|
+
```bash
|
|
128
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
188
129
|
```
|
|
189
130
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
**Local-only.** Talks to ComfyUI on `localhost:8188`. No telemetry, no analytics, no external requests. Images stay on your GPU and filesystem.
|
|
131
|
+
Requires Node.js 20+ and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) on localhost:8188 for generation.
|
|
193
132
|
|
|
194
|
-
##
|
|
133
|
+
## Security
|
|
195
134
|
|
|
196
|
-
-
|
|
197
|
-
- SDXL-compatible checkpoint + style LoRA
|
|
198
|
-
- Node.js 20+
|
|
199
|
-
- [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) for training export
|
|
135
|
+
Local-only. No telemetry, no analytics, no external requests. Images stay on your GPU and filesystem.
|
|
200
136
|
|
|
201
137
|
## License
|
|
202
138
|
|
package/README.pt-BR.md
CHANGED
|
@@ -13,158 +13,187 @@
|
|
|
13
13
|
|
|
14
14
|
# style-dataset-lab
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
Transforme trabalhos visuais aprovados em conjuntos de dados versionados, com revisões, divisões, pacotes de exportação e pacotes de avaliação.
|
|
17
17
|
|
|
18
18
|
## O que é isso
|
|
19
19
|
|
|
20
|
-
|
|
20
|
+
Um **pipeline de produção de conjuntos de dados e de referências visuais**. Defina a aparência do seu projeto. Crie conjuntos de dados de acordo com as regras estabelecidas. Produza pacotes de dados reproduzíveis com divisões seguras contra vazamento de informações. Gere pacotes de avaliação para a verificação futura do modelo.
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
O pipeline produz quatro artefatos:
|
|
23
23
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
24
|
+
| Artefato | O que é |
|
|
25
|
+
|----------|-----------|
|
|
26
|
+
| **Snapshot** | Seleção de registros elegíveis, com informações detalhadas e rastreáveis. Cada inclusão tem uma justificativa explícita. |
|
|
27
|
+
| **Split** | Partição de treinamento/validação/teste segura contra vazamento de informações. Registros que compartilham a mesma família de sujeitos sempre ficam na mesma divisão. |
|
|
28
|
+
| **Export package** | Conjunto de dados autônomo: manifesto, metadados, imagens, divisões, descrição do conjunto de dados e checksums. |
|
|
29
|
+
| **Eval pack** | Tarefas de verificação compatíveis com as referências: cobertura de aspectos, prevenção de desvios indesejados, pontos de referência/referências, continuidade do sujeito. |
|
|
27
30
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
**Apenas local.** O style-dataset-lab se comunica com o ComfyUI em `localhost:8188` e nunca faz solicitações de rede externas. Sem telemetria, sem análise, sem envio de dados. A geração de imagens ocorre inteiramente na sua GPU. Os registros e os dados de referência permanecem no seu sistema de arquivos.
|
|
31
|
-
|
|
32
|
-
## O que o pacote npm oferece:
|
|
33
|
-
|
|
34
|
-
`npm install @mcptoolshop/style-dataset-lab` fornece:
|
|
31
|
+
Cada ativo no pipeline contém três elementos:
|
|
35
32
|
|
|
36
|
-
|
|
37
|
-
|
|
33
|
+
1. **Rastreabilidade** -- histórico completo de geração (checkpoint, LoRA, seed, sampler, cfg, tempo de execução)
|
|
34
|
+
2. **Conformidade com as referências** -- quais regras de estilo este ativo cumpre, não cumpre ou cumpre parcialmente
|
|
35
|
+
3. **Avaliação de qualidade** -- aprovado/rejeitado/limítrofe, com pontuações por dimensão
|
|
38
36
|
|
|
39
|
-
|
|
37
|
+
Funciona para arte de jogos, design de personagens, design de criaturas, arquitetura, conceitos de veículos/máquinas, e qualquer área onde a produção visual precisa seguir um padrão.
|
|
40
38
|
|
|
41
|
-
##
|
|
39
|
+
## Como começar
|
|
42
40
|
|
|
43
41
|
```bash
|
|
44
|
-
#
|
|
45
|
-
npm install @mcptoolshop/style-dataset-lab
|
|
42
|
+
# Install the CLI + pipeline
|
|
43
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
46
44
|
|
|
47
|
-
#
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
45
|
+
# Scaffold a new project
|
|
46
|
+
sdlab init my-project --domain character-design
|
|
47
|
+
|
|
48
|
+
# Validate the project structure
|
|
49
|
+
sdlab project doctor --project my-project
|
|
51
50
|
```
|
|
52
51
|
|
|
53
|
-
|
|
52
|
+
Domínios disponíveis: `game-art`, `character-design`, `creature-design`, `architecture`, `vehicle-mech` ou `generic`.
|
|
53
|
+
|
|
54
|
+
## Interface de linha de comando (CLI)
|
|
54
55
|
|
|
55
56
|
```bash
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
57
|
+
sdlab init <name> [--domain <domain>] # Scaffold a new project
|
|
58
|
+
sdlab project doctor [--project <name>] # Validate project config
|
|
59
|
+
|
|
60
|
+
sdlab generate <pack> [--project <name>] # Generate candidates via ComfyUI
|
|
61
|
+
sdlab generate:identity <packet> # Named-subject identity images
|
|
62
|
+
sdlab generate:controlnet # ControlNet-guided generation
|
|
63
|
+
sdlab generate:ipadapter # IP-Adapter reference-guided
|
|
64
|
+
|
|
65
|
+
sdlab curate <id> <status> <explanation> # Record review judgment
|
|
66
|
+
sdlab compare <a> <b> <winner> <reason> # Pairwise A-vs-B comparison
|
|
67
|
+
sdlab bind [--project <name>] # Bind records to constitution rules
|
|
68
|
+
sdlab painterly [--project <name>] # Post-processing style pass
|
|
69
|
+
|
|
70
|
+
sdlab snapshot create [--profile <name>] # Create frozen dataset snapshot
|
|
71
|
+
sdlab snapshot list # List all snapshots
|
|
72
|
+
sdlab snapshot diff <a> <b> # Compare two snapshots
|
|
73
|
+
sdlab eligibility audit # Audit record training eligibility
|
|
74
|
+
sdlab split build [--snapshot <id>] # Build train/val/test split
|
|
75
|
+
sdlab split audit <id> # Audit split for leakage + balance
|
|
76
|
+
sdlab card generate # Generate dataset card (md + JSON)
|
|
77
|
+
sdlab export build [--snapshot <id>] # Build versioned export package
|
|
78
|
+
sdlab eval-pack build # Build canon-aware eval pack
|
|
61
79
|
```
|
|
62
80
|
|
|
63
|
-
|
|
81
|
+
Todos os comandos aceitam `--project <nome>` (padrão: `star-freight`).
|
|
82
|
+
|
|
83
|
+
## Modelo do projeto
|
|
64
84
|
|
|
65
|
-
|
|
85
|
+
Cada projeto é um diretório autônomo dentro de `projects/`, com suas próprias referências, configuração e dados:
|
|
66
86
|
|
|
67
87
|
```
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
88
|
+
projects/
|
|
89
|
+
my-project/
|
|
90
|
+
project.json Project identity + generation defaults
|
|
91
|
+
constitution.json Rules array with rationale templates
|
|
92
|
+
lanes.json Subject lanes with detection patterns
|
|
93
|
+
rubric.json Scoring dimensions + thresholds
|
|
94
|
+
terminology.json Group vocabulary + detection order
|
|
95
|
+
canon/ Style constitution (markdown)
|
|
96
|
+
records/ Per-asset JSON (provenance + judgment + canon)
|
|
97
|
+
inputs/prompts/ Prompt packs (JSON)
|
|
98
|
+
outputs/ Generated images (gitignored)
|
|
99
|
+
comparisons/ A-vs-B preference judgments
|
|
100
|
+
snapshots/ Frozen dataset snapshots
|
|
101
|
+
splits/ Train/val/test partitions
|
|
102
|
+
exports/ Versioned export packages
|
|
103
|
+
eval-packs/ Canon-aware eval instruments
|
|
82
104
|
```
|
|
83
105
|
|
|
84
|
-
##
|
|
106
|
+
## Pipeline
|
|
85
107
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
108
|
+
```
|
|
109
|
+
canon → generate → curate → bind → snapshot → split → export → eval
|
|
110
|
+
| | | | | | | |
|
|
111
|
+
rules ComfyUI judgment rules frozen subject package verify
|
|
112
|
+
selection isolation
|
|
113
|
+
```
|
|
91
114
|
|
|
92
|
-
|
|
93
|
-
|
|
115
|
+
1. **Definir referências** -- escreva sua constituição de estilo e guia de revisão
|
|
116
|
+
2. **Gerar** -- o ComfyUI produz candidatos com rastreabilidade completa
|
|
117
|
+
3. **Curadoria** -- aprovar/rejeitar com pontuações por dimensão e modos de falha
|
|
118
|
+
4. **Associar** -- vincular cada ativo às regras de estilo, com veredictos de aprovação/rejeição/parcial
|
|
119
|
+
5. **Captura de instantâneo** -- congelar os registros elegíveis em uma seleção determinística e identificada
|
|
120
|
+
6. **Dividir** -- particionar em treinamento/validação/teste, com isolamento do sujeito e equilíbrio de aspectos
|
|
121
|
+
7. **Exportar** -- criar um pacote autônomo com manifesto, metadados, imagens e checksums
|
|
122
|
+
8. **Avaliar** -- gerar instrumentos de teste compatíveis com as referências para a verificação do modelo
|
|
94
123
|
|
|
95
|
-
|
|
96
|
-
npm run generate -- --game star-freight inputs/prompts/wave1.json
|
|
97
|
-
npm run generate -- --game star-freight inputs/prompts/wave1.json --dry-run
|
|
124
|
+
A conversão de formato para outros sistemas (TRL, LLaVA, Parquet, etc.) é gerenciada por [`repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset). O `sdlab` define a "verdade" do conjunto de dados; o `repo-dataset` a transforma em formatos especializados.
|
|
98
125
|
|
|
99
|
-
|
|
100
|
-
npm run generate:identity -- --game star-freight inputs/identity-packets/wave27a.json
|
|
126
|
+
## Modelos de domínio
|
|
101
127
|
|
|
102
|
-
|
|
103
|
-
npm run curate -- --game star-freight <asset_id> approved "explanation"
|
|
104
|
-
npm run curate -- --game star-freight <asset_id> rejected "explanation" --failures "too_clean"
|
|
128
|
+
Cada modelo de domínio é fornecido com definições de aspectos, regras de estilo, guias de pontuação e estruturas de terminologia projetadas para aquele contexto de produção:
|
|
105
129
|
|
|
106
|
-
|
|
107
|
-
|
|
130
|
+
| Domínio | Aspectos | Principais preocupações |
|
|
131
|
+
|--------|-------|-------------|
|
|
132
|
+
| **game-art** | personagem, ambiente, acessório, interface do usuário, nave, interior, equipamento | Silhueta na escala do jogo, diferenciação de facções, desgaste/envelhecimento |
|
|
133
|
+
| **character-design** | retrato, corpo inteiro, vista de 360 graus, folha de expressões, pose de ação | Precisão das proporções, lógica do figurino, leitura da personalidade, clareza dos gestos |
|
|
134
|
+
| **creature-design** | conceito, ortográfico, estudo de detalhes, ação, referência de escala, habitat | Plausibilidade anatômica, lógica evolutiva, distinção de silhuetas |
|
|
135
|
+
| **architecture** | exterior, interior, paisagem urbana, detalhe estrutural, ruína, paisagem | Plausibilidade estrutural, consistência dos materiais, perspectiva, coerência da época |
|
|
136
|
+
| **vehicle-mech** | exterior, cockpit, componente, esquema, folha de silhuetas, variação de dano | Lógica mecânica, linguagem de design funcional, pontos de acesso, descrição de danos. |
|
|
108
137
|
|
|
109
|
-
|
|
110
|
-
npm run canon-bind -- --game star-freight
|
|
138
|
+
## Produção do conjunto de dados
|
|
111
139
|
|
|
112
|
-
|
|
113
|
-
npm run compare -- --game star-freight <asset_a> <asset_b> a "A has better faction read"
|
|
140
|
+
A estrutura completa do conjunto de dados: instantâneo, divisão, exportação, avaliação.
|
|
114
141
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
142
|
+
```
|
|
143
|
+
snapshot --> split --> export --> eval-pack
|
|
144
|
+
| | | |
|
|
145
|
+
frozen subject package canon-aware
|
|
146
|
+
selection isolation (manifest, test instruments
|
|
147
|
+
+ lane metadata, (4 task types)
|
|
148
|
+
balance images,
|
|
149
|
+
checksums,
|
|
150
|
+
card)
|
|
118
151
|
```
|
|
119
152
|
|
|
120
|
-
|
|
153
|
+
**Instantâneos** congelam uma seleção determinística de registros elegíveis. Cada inclusão tem um registro de motivo. As "impressões digitais" de configuração garantem a reprodutibilidade.
|
|
154
|
+
|
|
155
|
+
**Divisões** atribuem registros às partições de treinamento/validação/teste, com isolamento de sujeitos (nenhuma família de sujeitos aparece em várias divisões) e distribuição equilibrada por "faixa". Um gerador de números pseudoaleatórios (PRNG) com semente garante resultados idênticos a partir da mesma semente.
|
|
156
|
+
|
|
157
|
+
Os **pacotes de exportação** são autônomos: manifesto, metadata.jsonl, imagens (com links simbólicos ou cópias), divisões, cartão do conjunto de dados (Markdown + JSON) e somas de verificação no formato BSD. Tudo o que é necessário para reconstruir o conjunto de dados do zero.
|
|
158
|
+
|
|
159
|
+
Os **pacotes de avaliação** são instrumentos de teste que levam em consideração o contexto e possuem quatro tipos de tarefas: cobertura de "faixas", desvio proibido, âncora/referência e continuidade do sujeito. Eles comprovam que a estrutura do conjunto de dados está sendo usada para futuras avaliações de modelos, e não apenas para descartar arquivos.
|
|
160
|
+
|
|
161
|
+
Exportação para formatos compatíveis através de [`repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) (TRL, LLaVA, Qwen2-VL, JSONL, Parquet, e mais). `repo-dataset` gerencia a conversão de formatos; `sdlab` detém a "verdade" do conjunto de dados.
|
|
162
|
+
|
|
163
|
+
## Exemplo do Star Freight
|
|
164
|
+
|
|
165
|
+
Clone o repositório para um exemplo completo e funcional: 1.182 registros, 28 "ondas" de prompts, 5 facções, 7 "faixas", 24 regras constitucionais e 892 ativos aprovados de um RPG de ficção científica.
|
|
121
166
|
|
|
122
167
|
```bash
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
168
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab
|
|
169
|
+
cd style-dataset-lab
|
|
170
|
+
|
|
171
|
+
# Validate the project
|
|
172
|
+
sdlab project doctor --project star-freight
|
|
127
173
|
|
|
128
|
-
#
|
|
129
|
-
|
|
174
|
+
# Run the full dataset spine
|
|
175
|
+
sdlab snapshot create --project star-freight # 839 eligible records
|
|
176
|
+
sdlab split build --project star-freight # ~80/10/10, zero leakage
|
|
177
|
+
sdlab export build --project star-freight # package with checksums
|
|
178
|
+
sdlab eval-pack build --project star-freight # 78 eval records
|
|
130
179
|
```
|
|
131
180
|
|
|
132
|
-
##
|
|
181
|
+
## Migrando da versão v1.x
|
|
133
182
|
|
|
134
|
-
|
|
183
|
+
A versão 2.0 renomeia `games/` para `projects/` e `--game` para `--project`:
|
|
135
184
|
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
prompts/ Prompt packs per wave (JSON: subjects, variations, defaults)
|
|
140
|
-
references/ IP-Adapter reference images
|
|
141
|
-
control-guides/ ControlNet guide overlays
|
|
142
|
-
identity-packets/ Named character identity spines
|
|
143
|
-
outputs/
|
|
144
|
-
candidates/ Raw generations (gitignored)
|
|
145
|
-
approved/ Curated approved (gitignored)
|
|
146
|
-
rejected/ Curated rejected (gitignored)
|
|
147
|
-
borderline/ Curated borderline (gitignored)
|
|
148
|
-
painterly/ Painterly-style variants (gitignored)
|
|
149
|
-
records/ Per-asset JSON (provenance + judgment + canon binding)
|
|
150
|
-
comparisons/ A-vs-B preference judgments
|
|
151
|
-
exports/ repo-dataset output (gitignored)
|
|
152
|
-
```
|
|
185
|
+
```bash
|
|
186
|
+
# Rename your data directory
|
|
187
|
+
mv games projects
|
|
153
188
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
checkpoint: dreamshaperXL_v21TurboDPMSDE.safetensors
|
|
158
|
-
lora: classipeintxl_v21.safetensors (weight: 1.0)
|
|
159
|
-
resolution: 1024x1024
|
|
160
|
-
steps: 8
|
|
161
|
-
cfg: 2.0
|
|
162
|
-
sampler: dpmpp_sde
|
|
163
|
-
scheduler: karras
|
|
164
|
-
speed: ~9s per image (RTX 5080)
|
|
189
|
+
# --game still works with a deprecation warning (removed in v3.0)
|
|
190
|
+
sdlab bind --game star-freight # works, prints warning
|
|
191
|
+
sdlab bind --project star-freight # canonical form
|
|
165
192
|
```
|
|
166
193
|
|
|
167
|
-
|
|
194
|
+
## Modelo de segurança
|
|
195
|
+
|
|
196
|
+
**Apenas local.** Comunica-se com o ComfyUI em `localhost:8188`. Sem telemetria, sem análises, sem requisições externas. As imagens permanecem na sua GPU e sistema de arquivos.
|
|
168
197
|
|
|
169
198
|
## Requisitos
|
|
170
199
|
|