@mcptoolshop/style-dataset-lab 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +37 -0
- package/LICENSE +21 -0
- package/README.es.md +142 -0
- package/README.fr.md +142 -0
- package/README.hi.md +142 -0
- package/README.it.md +142 -0
- package/README.ja.md +142 -0
- package/README.md +178 -0
- package/README.pt-BR.md +142 -0
- package/README.zh.md +142 -0
- package/SECURITY.md +28 -0
- package/package.json +45 -0
- package/scripts/bulk-curate-wave2-5.js +138 -0
- package/scripts/bulk-curate-waves11-18.js +413 -0
- package/scripts/canon-bind.js +437 -0
- package/scripts/compare.js +116 -0
- package/scripts/curate-wave25.js +209 -0
- package/scripts/curate.js +136 -0
- package/scripts/generate-controlnet.js +357 -0
- package/scripts/generate-identity.js +633 -0
- package/scripts/generate-ipadapter.js +339 -0
- package/scripts/generate.js +351 -0
- package/scripts/migrate-records.js +147 -0
- package/scripts/painterly-test.js +149 -0
- package/scripts/painterly.js +349 -0
package/README.ja.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="README.md">English</a> | <a href="README.zh.md">中文</a> | <a href="README.es.md">Español</a> | <a href="README.fr.md">Français</a> | <a href="README.hi.md">हिन्दी</a> | <a href="README.it.md">Italiano</a> | <a href="README.pt-BR.md">Português (BR)</a>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<p align="center">
|
|
6
|
+
<img src="https://raw.githubusercontent.com/mcp-tool-shop-org/brand/main/logos/style-dataset-lab/readme.png" width="400">
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/@mcptoolshop/style-dataset-lab"><img src="https://img.shields.io/npm/v/@mcptoolshop/style-dataset-lab" alt="npm"></a>
|
|
11
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
# style-dataset-lab
|
|
15
|
+
|
|
16
|
+
Visual dataset factory:VLMのファインチューニングのための、多様なトレーニングデータを生成、キュレーション、エクスポートします。
|
|
17
|
+
|
|
18
|
+
## これは何ですか?
|
|
19
|
+
|
|
20
|
+
「学習可能なビジュアル真実」を構築するためのツールキットです。各アセットには、以下の3つの情報が含まれています。
|
|
21
|
+
|
|
22
|
+
1. **画像ピクセル**:ComfyUIによって生成され、完全な情報(チェックポイント、LoRA、シード値、サンプラー、CFG)が付与されています。
|
|
23
|
+
2. **規範的な説明**:なぜこの画像が特定のスタイルに合致しているか、または合致していないかを、スタイルに関する規定に基づいて説明します。
|
|
24
|
+
3. **品質評価**:各次元のスコアと参照ルールに基づいて、承認/却下の判断がなされます。
|
|
25
|
+
|
|
26
|
+
生成されたデータは、[`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset)に連携し、10種類の形式(TRL、LLaVA、Qwen2-VL、Axolotl、LLaMA-Factory、ShareGPT、RLHFペア、DPO、キャプション、分類)で、多様なトレーニングデータを作成します。
|
|
27
|
+
|
|
28
|
+
## セキュリティモデル
|
|
29
|
+
|
|
30
|
+
**ローカル環境のみ**。style-dataset-labは、`localhost:8188`で動作するComfyUIとのみ通信し、外部ネットワークへのアクセスは一切行いません。テレメトリー、分析、データ送信機能は一切ありません。画像生成はすべて、お客様のGPU上で行われます。記録データと規範データは、お客様のファイルシステム上に保存されます。
|
|
31
|
+
|
|
32
|
+
## データセットの統計
|
|
33
|
+
|
|
34
|
+
| 指標 | 値 |
|
|
35
|
+
|--------|-------|
|
|
36
|
+
| キュレーションされたデータ | 1,182 |
|
|
37
|
+
| 総アセット数 | 2,571件(承認済み:893件、ペインタリーバリアント:887件) |
|
|
38
|
+
| プロンプトの種類 | 28 |
|
|
39
|
+
| ビジュアルカテゴリ | 18種類(衣装、船、内装、装備、環境、生物、ステーション、標識、照明、貨物、アーキテクチャ、生物、表面、日常生活、惑星、損傷/修理、異星生物、生活感のあるディテール) |
|
|
40
|
+
| ペア比較 | 人間による評価:6件、合成データ:71件 |
|
|
41
|
+
| 拒否の種類 | 30+ |
|
|
42
|
+
| IDパケットシステム | 派閥、ランク、視覚的な特徴を持つ名前付きキャラクター |
|
|
43
|
+
| エクスポート形式 | 10種類(@mcptoolshop/repo-dataset経由) |
|
|
44
|
+
|
|
45
|
+
## インストール
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
次に、プロジェクトをクローンするか、新しいデータセットの作業領域を初期化します。
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab my-dataset
|
|
55
|
+
cd my-dataset
|
|
56
|
+
npm install
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## ワークフロー
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# 1. Start ComfyUI
|
|
63
|
+
# (point it at your checkpoint + LoRA setup)
|
|
64
|
+
|
|
65
|
+
# 2. Generate candidates from a prompt pack
|
|
66
|
+
npm run generate -- inputs/prompts/wave1.json
|
|
67
|
+
npm run generate -- inputs/prompts/wave1.json --dry-run
|
|
68
|
+
|
|
69
|
+
# 3. Generate identity-packet characters
|
|
70
|
+
npm run generate:identity -- inputs/identity-packets/wave27a-identity-spine.json
|
|
71
|
+
|
|
72
|
+
# 4. Generate painterly variants of approved assets
|
|
73
|
+
npm run painterly -- <asset_id>
|
|
74
|
+
|
|
75
|
+
# 5. Curate -- approve, reject, or mark borderline
|
|
76
|
+
npm run curate -- <asset_id> approved "explanation" --scores "silhouette:0.9,palette:0.8"
|
|
77
|
+
npm run curate -- <asset_id> rejected "explanation" --failures "too_clean,wrong_material"
|
|
78
|
+
|
|
79
|
+
# 6. Bind canon explanations to assets
|
|
80
|
+
npm run canon-bind -- <asset_id>
|
|
81
|
+
|
|
82
|
+
# 7. Record pairwise comparisons
|
|
83
|
+
npm run compare -- <asset_a> <asset_b> a "A has better faction read because..."
|
|
84
|
+
|
|
85
|
+
# 8. Export training data via repo-dataset
|
|
86
|
+
npm run export
|
|
87
|
+
npm run inspect
|
|
88
|
+
npm run validate
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## ディレクトリ構造
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
canon/ Style constitution, review rubric, identity gates, species canon
|
|
95
|
+
inputs/
|
|
96
|
+
prompts/ Prompt packs per wave (JSON: subjects, variations, defaults)
|
|
97
|
+
references/ IP-Adapter reference images
|
|
98
|
+
control/ ControlNet control images
|
|
99
|
+
control-guides/ ControlNet guide overlays
|
|
100
|
+
identity-packets/ Named character identity spines (faction DNA, rank, visual anchors)
|
|
101
|
+
outputs/
|
|
102
|
+
candidates/ Raw generations (gitignored)
|
|
103
|
+
approved/ Curated approved (gitignored)
|
|
104
|
+
rejected/ Curated rejected (gitignored)
|
|
105
|
+
borderline/ Curated borderline (gitignored)
|
|
106
|
+
painterly/ Painterly-style variants (gitignored)
|
|
107
|
+
records/ Per-asset JSON (provenance + judgment + canon binding)
|
|
108
|
+
comparisons/ A-vs-B preference judgments
|
|
109
|
+
exports/ repo-dataset output (gitignored)
|
|
110
|
+
scripts/ generate, curate, compare, canon-bind, painterly, identity gen
|
|
111
|
+
workflows/ Reusable ComfyUI workflow templates
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## 生成設定
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
checkpoint: dreamshaperXL_v21TurboDPMSDE.safetensors
|
|
118
|
+
lora: classipeintxl_v21.safetensors (weight: 1.0)
|
|
119
|
+
resolution: 1024x1024
|
|
120
|
+
steps: 8
|
|
121
|
+
cfg: 2.0
|
|
122
|
+
sampler: dpmpp_sde
|
|
123
|
+
scheduler: karras
|
|
124
|
+
speed: ~9s per image (RTX 5080)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
追加の生成モード:ControlNet(ポーズ/深度ガイド)、IP-Adapter(参照ベース)、IDパケット(名前付きキャラクターの一貫性)。
|
|
128
|
+
|
|
129
|
+
## 必要なもの
|
|
130
|
+
|
|
131
|
+
- `localhost:8188`で動作する[ComfyUI](https://github.com/comfyanonymous/ComfyUI)
|
|
132
|
+
- DreamShaper XL Turbo チェックポイント + ClassipeintXL LoRA
|
|
133
|
+
- Node.js 20以上
|
|
134
|
+
- トレーニング用エクスポートのための[`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset)
|
|
135
|
+
|
|
136
|
+
## ライセンス
|
|
137
|
+
|
|
138
|
+
MIT
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
開発:<a href="https://mcp-tool-shop.github.io/">MCP Tool Shop</a>
|
package/README.md
ADDED
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="README.ja.md">日本語</a> | <a href="README.zh.md">中文</a> | <a href="README.es.md">Español</a> | <a href="README.fr.md">Français</a> | <a href="README.hi.md">हिन्दी</a> | <a href="README.it.md">Italiano</a> | <a href="README.pt-BR.md">Português (BR)</a>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<p align="center">
|
|
6
|
+
<img src="https://raw.githubusercontent.com/mcp-tool-shop-org/brand/main/logos/style-dataset-lab/readme.png" width="400">
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/@mcptoolshop/style-dataset-lab"><img src="https://img.shields.io/npm/v/@mcptoolshop/style-dataset-lab" alt="npm"></a>
|
|
11
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
# style-dataset-lab
|
|
15
|
+
|
|
16
|
+
Visual dataset factory -- generate, curate, and export multimodal training data for VLM fine-tuning.
|
|
17
|
+
|
|
18
|
+
## What this is
|
|
19
|
+
|
|
20
|
+
A **monorepo** toolkit for building **trainable visual truth** across multiple games. Each game gets its own data directory; scripts are shared. Every asset carries three things:
|
|
21
|
+
|
|
22
|
+
1. **Image pixels** -- generated by ComfyUI with full provenance (checkpoint, LoRA, seed, sampler, cfg)
|
|
23
|
+
2. **Canon explanation** -- why this is on-style or off-style, grounded in a style constitution
|
|
24
|
+
3. **Quality judgment** -- approved/rejected with per-dimension scores and cited rules
|
|
25
|
+
|
|
26
|
+
The output feeds into [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) to produce multimodal training data in 10 formats: TRL, LLaVA, Qwen2-VL, Axolotl, LLaMA-Factory, ShareGPT, RLHF pairs, DPO, captioning, and classification.
|
|
27
|
+
|
|
28
|
+
## Security model
|
|
29
|
+
|
|
30
|
+
**Local-only.** style-dataset-lab talks to ComfyUI on `localhost:8188` and never makes external network requests. No telemetry, no analytics, no phoning home. Image generation happens entirely on your GPU. Records and canon data stay on your filesystem.
|
|
31
|
+
|
|
32
|
+
## Dataset stats (Star Freight)
|
|
33
|
+
|
|
34
|
+
The included `games/star-freight/` dataset ships with the repo as a working example:
|
|
35
|
+
|
|
36
|
+
| Metric | Value |
|
|
37
|
+
|--------|-------|
|
|
38
|
+
| Curated records | 1,182 |
|
|
39
|
+
| Total assets | 2,571 (893 approved, 887 painterly variants) |
|
|
40
|
+
| Prompt waves | 28 |
|
|
41
|
+
| Visual categories | 18 (costumes, ships, interiors, equipment, environments, species, stations, signage, lighting, cargo, architecture, creatures, surfaces, daily life, planetary, damage/repair, alien biology, lived-in details) |
|
|
42
|
+
| Pairwise comparisons | 6 human + 71 synthetic |
|
|
43
|
+
| Distinct rejection types | 30+ |
|
|
44
|
+
| Identity packet system | Named characters with faction DNA, rank, and visual anchors |
|
|
45
|
+
| Export formats | 10 (via @mcptoolshop/repo-dataset) |
|
|
46
|
+
|
|
47
|
+
## Install
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Install scripts globally (optional)
|
|
51
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
52
|
+
|
|
53
|
+
# Clone the repo (includes star-freight game data)
|
|
54
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab
|
|
55
|
+
cd style-dataset-lab
|
|
56
|
+
npm install
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
To start a new game, create its directory structure:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
mkdir -p games/my-game/{canon,records,comparisons,inputs/prompts,outputs/{candidates,approved,rejected},exports}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Or use the existing `games/star-freight/` data to explore the workflow immediately.
|
|
66
|
+
|
|
67
|
+
## Monorepo structure
|
|
68
|
+
|
|
69
|
+
Each game lives in `games/<name>/` with its own canon, records, and assets. Scripts are shared and accept `--game <name>` (defaults to `star-freight`).
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
style-dataset-lab/
|
|
73
|
+
scripts/ Shared tooling (generate, curate, compare, etc.)
|
|
74
|
+
games/
|
|
75
|
+
star-freight/ Star Freight universe data
|
|
76
|
+
canon/ Style constitution, review rubric, species canon
|
|
77
|
+
records/ Per-asset JSON (provenance + judgment + canon)
|
|
78
|
+
comparisons/ A-vs-B preference judgments
|
|
79
|
+
inputs/ Prompt packs, identity packets, references
|
|
80
|
+
outputs/ Generated images (gitignored)
|
|
81
|
+
exports/ repo-dataset output (gitignored)
|
|
82
|
+
fractured-road/ (add your next game here)
|
|
83
|
+
canon/
|
|
84
|
+
records/
|
|
85
|
+
...
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
## Workflow
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
# 1. Start ComfyUI
|
|
92
|
+
# (point it at your checkpoint + LoRA setup)
|
|
93
|
+
|
|
94
|
+
# 2. Generate candidates from a prompt pack
|
|
95
|
+
npm run generate -- --game star-freight inputs/prompts/wave1.json
|
|
96
|
+
npm run generate -- --game star-freight inputs/prompts/wave1.json --dry-run
|
|
97
|
+
|
|
98
|
+
# 3. Generate identity-packet characters
|
|
99
|
+
npm run generate:identity -- --game star-freight inputs/identity-packets/wave27a.json
|
|
100
|
+
|
|
101
|
+
# 4. Generate painterly variants of approved assets
|
|
102
|
+
npm run painterly -- --game star-freight
|
|
103
|
+
|
|
104
|
+
# 5. Curate -- approve, reject, or mark borderline
|
|
105
|
+
npm run curate -- --game star-freight <asset_id> approved "explanation"
|
|
106
|
+
npm run curate -- --game star-freight <asset_id> rejected "explanation" --failures "too_clean"
|
|
107
|
+
|
|
108
|
+
# 6. Bind canon explanations to assets
|
|
109
|
+
npm run canon-bind -- --game star-freight
|
|
110
|
+
|
|
111
|
+
# 7. Record pairwise comparisons
|
|
112
|
+
npm run compare -- --game star-freight <asset_a> <asset_b> a "A has better faction read"
|
|
113
|
+
|
|
114
|
+
# 8. Export training data via repo-dataset
|
|
115
|
+
repo-dataset visual generate ./games/star-freight --format trl
|
|
116
|
+
repo-dataset visual inspect ./games/star-freight
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
## Adding a new game
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
mkdir -p games/my-game/{canon,records,comparisons,inputs/prompts,outputs/{candidates,approved,rejected},exports}
|
|
123
|
+
# Write your canon/constitution.md and canon/review-rubric.md
|
|
124
|
+
# Create prompt packs in inputs/prompts/
|
|
125
|
+
# Then generate, curate, and export as above with --game my-game
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
## Per-game directory layout
|
|
129
|
+
|
|
130
|
+
Each `games/<name>/` directory contains:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
canon/ Style constitution, review rubric, species canon, identity gates
|
|
134
|
+
inputs/
|
|
135
|
+
prompts/ Prompt packs per wave (JSON: subjects, variations, defaults)
|
|
136
|
+
references/ IP-Adapter reference images
|
|
137
|
+
control-guides/ ControlNet guide overlays
|
|
138
|
+
identity-packets/ Named character identity spines
|
|
139
|
+
outputs/
|
|
140
|
+
candidates/ Raw generations (gitignored)
|
|
141
|
+
approved/ Curated approved (gitignored)
|
|
142
|
+
rejected/ Curated rejected (gitignored)
|
|
143
|
+
borderline/ Curated borderline (gitignored)
|
|
144
|
+
painterly/ Painterly-style variants (gitignored)
|
|
145
|
+
records/ Per-asset JSON (provenance + judgment + canon binding)
|
|
146
|
+
comparisons/ A-vs-B preference judgments
|
|
147
|
+
exports/ repo-dataset output (gitignored)
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
## Generation setup
|
|
151
|
+
|
|
152
|
+
```yaml
|
|
153
|
+
checkpoint: dreamshaperXL_v21TurboDPMSDE.safetensors
|
|
154
|
+
lora: classipeintxl_v21.safetensors (weight: 1.0)
|
|
155
|
+
resolution: 1024x1024
|
|
156
|
+
steps: 8
|
|
157
|
+
cfg: 2.0
|
|
158
|
+
sampler: dpmpp_sde
|
|
159
|
+
scheduler: karras
|
|
160
|
+
speed: ~9s per image (RTX 5080)
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Additional generation modes: ControlNet (pose/depth-guided), IP-Adapter (reference-driven), and identity packets (named character consistency).
|
|
164
|
+
|
|
165
|
+
## Requirements
|
|
166
|
+
|
|
167
|
+
- [ComfyUI](https://github.com/comfyanonymous/ComfyUI) running on localhost:8188
|
|
168
|
+
- DreamShaper XL Turbo checkpoint + ClassipeintXL LoRA
|
|
169
|
+
- Node.js 20+
|
|
170
|
+
- [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) for training export
|
|
171
|
+
|
|
172
|
+
## License
|
|
173
|
+
|
|
174
|
+
MIT
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
Built by <a href="https://mcp-tool-shop.github.io/">MCP Tool Shop</a>
|
package/README.pt-BR.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="README.ja.md">日本語</a> | <a href="README.zh.md">中文</a> | <a href="README.es.md">Español</a> | <a href="README.fr.md">Français</a> | <a href="README.hi.md">हिन्दी</a> | <a href="README.it.md">Italiano</a> | <a href="README.md">English</a>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<p align="center">
|
|
6
|
+
<img src="https://raw.githubusercontent.com/mcp-tool-shop-org/brand/main/logos/style-dataset-lab/readme.png" width="400">
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/@mcptoolshop/style-dataset-lab"><img src="https://img.shields.io/npm/v/@mcptoolshop/style-dataset-lab" alt="npm"></a>
|
|
11
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
# style-dataset-lab
|
|
15
|
+
|
|
16
|
+
Ferramenta para criação de conjuntos de dados visuais – gera, organiza e exporta dados de treinamento multimodais para o ajuste fino de modelos de linguagem visuais (VLM).
|
|
17
|
+
|
|
18
|
+
## O que é isso
|
|
19
|
+
|
|
20
|
+
Um conjunto de ferramentas para construir a "verdade visual" que pode ser usada para treinamento. Cada elemento contém três coisas:
|
|
21
|
+
|
|
22
|
+
1. **Pixels da imagem** – gerados pelo ComfyUI com total rastreabilidade (checkpoint, LoRA, seed, sampler, cfg).
|
|
23
|
+
2. **Explicação detalhada** – por que este elemento está ou não de acordo com o estilo definido, com base em uma "constituição" de estilo.
|
|
24
|
+
3. **Avaliação de qualidade** – aprovado/rejeitado, com pontuações por dimensão e regras citadas.
|
|
25
|
+
|
|
26
|
+
A saída é integrada ao [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) para produzir dados de treinamento multimodais em 10 formatos: TRL, LLaVA, Qwen2-VL, Axolotl, LLaMA-Factory, ShareGPT, pares para RLHF, DPO, legendagem e classificação.
|
|
27
|
+
|
|
28
|
+
## Modelo de segurança
|
|
29
|
+
|
|
30
|
+
**Apenas local.** O style-dataset-lab se comunica com o ComfyUI em `localhost:8188` e nunca faz solicitações de rede externas. Sem telemetria, sem análise, sem envio de dados. A geração de imagens ocorre inteiramente na sua GPU. Os registros e os dados de referência permanecem no seu sistema de arquivos.
|
|
31
|
+
|
|
32
|
+
## Estatísticas do conjunto de dados
|
|
33
|
+
|
|
34
|
+
| Métrica | Valor |
|
|
35
|
+
|--------|-------|
|
|
36
|
+
| Registros organizados | 1,182 |
|
|
37
|
+
| Total de elementos | 2.571 (893 aprovados, 887 variações com estilo pictórico) |
|
|
38
|
+
| Variações de prompts | 28 |
|
|
39
|
+
| Categorias visuais | 18 (roupas, navios, interiores, equipamentos, ambientes, espécies, estações, sinalização, iluminação, carga, arquitetura, criaturas, superfícies, vida cotidiana, planetas, danos/reparos, biologia alienígena, detalhes realistas) |
|
|
40
|
+
| Comparativos | 6 criados por humanos + 71 criados sinteticamente |
|
|
41
|
+
| Tipos de rejeição distintos | 30+ |
|
|
42
|
+
| Sistema de pacotes de identidade | Personagens nomeados com DNA de facção, patente e referências visuais. |
|
|
43
|
+
| Formatos de exportação | 10 (via @mcptoolshop/repo-dataset) |
|
|
44
|
+
|
|
45
|
+
## Instalação
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Em seguida, clone um projeto ou inicialize um novo espaço de trabalho para conjuntos de dados:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab my-dataset
|
|
55
|
+
cd my-dataset
|
|
56
|
+
npm install
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## Fluxo de trabalho
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# 1. Start ComfyUI
|
|
63
|
+
# (point it at your checkpoint + LoRA setup)
|
|
64
|
+
|
|
65
|
+
# 2. Generate candidates from a prompt pack
|
|
66
|
+
npm run generate -- inputs/prompts/wave1.json
|
|
67
|
+
npm run generate -- inputs/prompts/wave1.json --dry-run
|
|
68
|
+
|
|
69
|
+
# 3. Generate identity-packet characters
|
|
70
|
+
npm run generate:identity -- inputs/identity-packets/wave27a-identity-spine.json
|
|
71
|
+
|
|
72
|
+
# 4. Generate painterly variants of approved assets
|
|
73
|
+
npm run painterly -- <asset_id>
|
|
74
|
+
|
|
75
|
+
# 5. Curate -- approve, reject, or mark borderline
|
|
76
|
+
npm run curate -- <asset_id> approved "explanation" --scores "silhouette:0.9,palette:0.8"
|
|
77
|
+
npm run curate -- <asset_id> rejected "explanation" --failures "too_clean,wrong_material"
|
|
78
|
+
|
|
79
|
+
# 6. Bind canon explanations to assets
|
|
80
|
+
npm run canon-bind -- <asset_id>
|
|
81
|
+
|
|
82
|
+
# 7. Record pairwise comparisons
|
|
83
|
+
npm run compare -- <asset_a> <asset_b> a "A has better faction read because..."
|
|
84
|
+
|
|
85
|
+
# 8. Export training data via repo-dataset
|
|
86
|
+
npm run export
|
|
87
|
+
npm run inspect
|
|
88
|
+
npm run validate
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## Estrutura de diretórios
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
canon/ Style constitution, review rubric, identity gates, species canon
|
|
95
|
+
inputs/
|
|
96
|
+
prompts/ Prompt packs per wave (JSON: subjects, variations, defaults)
|
|
97
|
+
references/ IP-Adapter reference images
|
|
98
|
+
control/ ControlNet control images
|
|
99
|
+
control-guides/ ControlNet guide overlays
|
|
100
|
+
identity-packets/ Named character identity spines (faction DNA, rank, visual anchors)
|
|
101
|
+
outputs/
|
|
102
|
+
candidates/ Raw generations (gitignored)
|
|
103
|
+
approved/ Curated approved (gitignored)
|
|
104
|
+
rejected/ Curated rejected (gitignored)
|
|
105
|
+
borderline/ Curated borderline (gitignored)
|
|
106
|
+
painterly/ Painterly-style variants (gitignored)
|
|
107
|
+
records/ Per-asset JSON (provenance + judgment + canon binding)
|
|
108
|
+
comparisons/ A-vs-B preference judgments
|
|
109
|
+
exports/ repo-dataset output (gitignored)
|
|
110
|
+
scripts/ generate, curate, compare, canon-bind, painterly, identity gen
|
|
111
|
+
workflows/ Reusable ComfyUI workflow templates
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Configuração de geração
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
checkpoint: dreamshaperXL_v21TurboDPMSDE.safetensors
|
|
118
|
+
lora: classipeintxl_v21.safetensors (weight: 1.0)
|
|
119
|
+
resolution: 1024x1024
|
|
120
|
+
steps: 8
|
|
121
|
+
cfg: 2.0
|
|
122
|
+
sampler: dpmpp_sde
|
|
123
|
+
scheduler: karras
|
|
124
|
+
speed: ~9s per image (RTX 5080)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Modos de geração adicionais: ControlNet (guiado por pose/profundidade), IP-Adapter (guiado por referência) e pacotes de identidade (consistência de personagens nomeados).
|
|
128
|
+
|
|
129
|
+
## Requisitos
|
|
130
|
+
|
|
131
|
+
- [ComfyUI](https://github.com/comfyanonymous/ComfyUI) em execução em localhost:8188
|
|
132
|
+
- Checkpoint DreamShaper XL Turbo + LoRA ClassipeintXL
|
|
133
|
+
- Node.js 20+
|
|
134
|
+
- [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) para exportação de treinamento
|
|
135
|
+
|
|
136
|
+
## Licença
|
|
137
|
+
|
|
138
|
+
MIT
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
Criado por <a href="https://mcp-tool-shop.github.io/">MCP Tool Shop</a
|
package/README.zh.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<a href="README.ja.md">日本語</a> | <a href="README.md">English</a> | <a href="README.es.md">Español</a> | <a href="README.fr.md">Français</a> | <a href="README.hi.md">हिन्दी</a> | <a href="README.it.md">Italiano</a> | <a href="README.pt-BR.md">Português (BR)</a>
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<p align="center">
|
|
6
|
+
<img src="https://raw.githubusercontent.com/mcp-tool-shop-org/brand/main/logos/style-dataset-lab/readme.png" width="400">
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/@mcptoolshop/style-dataset-lab"><img src="https://img.shields.io/npm/v/@mcptoolshop/style-dataset-lab" alt="npm"></a>
|
|
11
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
# style-dataset-lab
|
|
15
|
+
|
|
16
|
+
视觉数据集工厂 -- 用于生成、整理和导出多模态训练数据,以进行大型视觉语言模型(VLM)的微调。
|
|
17
|
+
|
|
18
|
+
## 这是什么
|
|
19
|
+
|
|
20
|
+
一个用于**构建可训练的视觉真理**的工具集。每个资源都包含以下三方面:
|
|
21
|
+
|
|
22
|
+
1. **图像像素** -- 由 ComfyUI 生成,并包含完整的溯源信息(检查点、LoRA、种子、采样器、CFG)。
|
|
23
|
+
2. **规范解释** -- 解释为什么该图像符合或不符合特定风格,并基于风格规范进行判断。
|
|
24
|
+
3. **质量评估** -- 经过批准/拒绝,并提供每个维度的评分以及引用规则。
|
|
25
|
+
|
|
26
|
+
输出结果通过 [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) 生成多模态训练数据,提供 10 种格式:TRL、LLaVA、Qwen2-VL、Axolotl、LLaMA-Factory、ShareGPT、RLHF 对、DPO、图像描述和分类。
|
|
27
|
+
|
|
28
|
+
## 安全模型
|
|
29
|
+
|
|
30
|
+
**仅本地运行。** style-dataset-lab 只与本地 `localhost:8188` 上的 ComfyUI 进行通信,不会进行任何外部网络请求。没有遥测数据,没有分析功能,不会向外部发送数据。图像生成完全在您的 GPU 上进行。记录和规范数据都存储在您的文件系统中。
|
|
31
|
+
|
|
32
|
+
## 数据集统计信息
|
|
33
|
+
|
|
34
|
+
| 指标 | 数值 |
|
|
35
|
+
|--------|-------|
|
|
36
|
+
| 整理记录 | 1,182 |
|
|
37
|
+
| 总资源数 | 2,571 (其中 893 个已批准,887 个为绘画风格变体) |
|
|
38
|
+
| 提示词组合 | 28 |
|
|
39
|
+
| 视觉类别 | 18 种 (包括:服装、船只、室内、设备、环境、物种、站点、标识、照明、货物、**架构**、生物、表面、日常生活、行星、损坏/修复、外星生物、细节)。 |
|
|
40
|
+
| 成对比较 | 6 个人工标注 + 71 个合成数据 |
|
|
41
|
+
| 不同类型的拒绝 | 30+ |
|
|
42
|
+
| 身份包系统 | 具有派系 DNA、等级和视觉特征的命名角色。 |
|
|
43
|
+
| 导出格式 | 10 种 (通过 `@mcptoolshop/repo-dataset`) |
|
|
44
|
+
|
|
45
|
+
## 安装
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
npm install -g @mcptoolshop/style-dataset-lab
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
然后克隆一个项目或初始化一个新的数据集工作区:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/mcp-tool-shop-org/style-dataset-lab my-dataset
|
|
55
|
+
cd my-dataset
|
|
56
|
+
npm install
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
## 工作流程
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# 1. Start ComfyUI
|
|
63
|
+
# (point it at your checkpoint + LoRA setup)
|
|
64
|
+
|
|
65
|
+
# 2. Generate candidates from a prompt pack
|
|
66
|
+
npm run generate -- inputs/prompts/wave1.json
|
|
67
|
+
npm run generate -- inputs/prompts/wave1.json --dry-run
|
|
68
|
+
|
|
69
|
+
# 3. Generate identity-packet characters
|
|
70
|
+
npm run generate:identity -- inputs/identity-packets/wave27a-identity-spine.json
|
|
71
|
+
|
|
72
|
+
# 4. Generate painterly variants of approved assets
|
|
73
|
+
npm run painterly -- <asset_id>
|
|
74
|
+
|
|
75
|
+
# 5. Curate -- approve, reject, or mark borderline
|
|
76
|
+
npm run curate -- <asset_id> approved "explanation" --scores "silhouette:0.9,palette:0.8"
|
|
77
|
+
npm run curate -- <asset_id> rejected "explanation" --failures "too_clean,wrong_material"
|
|
78
|
+
|
|
79
|
+
# 6. Bind canon explanations to assets
|
|
80
|
+
npm run canon-bind -- <asset_id>
|
|
81
|
+
|
|
82
|
+
# 7. Record pairwise comparisons
|
|
83
|
+
npm run compare -- <asset_a> <asset_b> a "A has better faction read because..."
|
|
84
|
+
|
|
85
|
+
# 8. Export training data via repo-dataset
|
|
86
|
+
npm run export
|
|
87
|
+
npm run inspect
|
|
88
|
+
npm run validate
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## 目录结构
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
canon/ Style constitution, review rubric, identity gates, species canon
|
|
95
|
+
inputs/
|
|
96
|
+
prompts/ Prompt packs per wave (JSON: subjects, variations, defaults)
|
|
97
|
+
references/ IP-Adapter reference images
|
|
98
|
+
control/ ControlNet control images
|
|
99
|
+
control-guides/ ControlNet guide overlays
|
|
100
|
+
identity-packets/ Named character identity spines (faction DNA, rank, visual anchors)
|
|
101
|
+
outputs/
|
|
102
|
+
candidates/ Raw generations (gitignored)
|
|
103
|
+
approved/ Curated approved (gitignored)
|
|
104
|
+
rejected/ Curated rejected (gitignored)
|
|
105
|
+
borderline/ Curated borderline (gitignored)
|
|
106
|
+
painterly/ Painterly-style variants (gitignored)
|
|
107
|
+
records/ Per-asset JSON (provenance + judgment + canon binding)
|
|
108
|
+
comparisons/ A-vs-B preference judgments
|
|
109
|
+
exports/ repo-dataset output (gitignored)
|
|
110
|
+
scripts/ generate, curate, compare, canon-bind, painterly, identity gen
|
|
111
|
+
workflows/ Reusable ComfyUI workflow templates
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## 生成设置
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
checkpoint: dreamshaperXL_v21TurboDPMSDE.safetensors
|
|
118
|
+
lora: classipeintxl_v21.safetensors (weight: 1.0)
|
|
119
|
+
resolution: 1024x1024
|
|
120
|
+
steps: 8
|
|
121
|
+
cfg: 2.0
|
|
122
|
+
sampler: dpmpp_sde
|
|
123
|
+
scheduler: karras
|
|
124
|
+
speed: ~9s per image (RTX 5080)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
其他生成模式:ControlNet(姿势/深度引导)、IP-Adapter(参考驱动)以及身份包(命名角色一致性)。
|
|
128
|
+
|
|
129
|
+
## 要求
|
|
130
|
+
|
|
131
|
+
- 运行在 `localhost:8188` 上的 [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
|
|
132
|
+
- DreamShaper XL Turbo 检查点 + ClassipeintXL LoRA
|
|
133
|
+
- Node.js 20+
|
|
134
|
+
- [`@mcptoolshop/repo-dataset`](https://github.com/mcp-tool-shop-org/repo-dataset) 用于训练数据导出
|
|
135
|
+
|
|
136
|
+
## 许可证
|
|
137
|
+
|
|
138
|
+
MIT
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
由 <a href="https://mcp-tool-shop.github.io/">MCP Tool Shop</a> 构建。
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
|---------|-----------|
|
|
7
|
+
| 0.x | Yes |
|
|
8
|
+
|
|
9
|
+
## Threat Model
|
|
10
|
+
|
|
11
|
+
style-dataset-lab is a **local-only** visual dataset factory. It connects to a ComfyUI server running on `localhost:8188` and does not make any external network calls.
|
|
12
|
+
|
|
13
|
+
**Attack surface:**
|
|
14
|
+
|
|
15
|
+
- **ComfyUI server** listens on localhost only. If exposed to a network, an attacker could inject workflow payloads. Keep the default localhost binding.
|
|
16
|
+
- **Untrusted model weights** (`.safetensors`, `.ckpt`, `.pth`, `.bin`) loaded by ComfyUI could contain malicious pickle payloads. Only load weights from sources you trust.
|
|
17
|
+
- **No telemetry.** This tool collects no usage data and makes no outbound requests beyond the local ComfyUI API.
|
|
18
|
+
- **No authentication.** The local ComfyUI API has no auth layer. This is acceptable for localhost but dangerous if network-exposed.
|
|
19
|
+
|
|
20
|
+
## Reporting a Vulnerability
|
|
21
|
+
|
|
22
|
+
If you discover a security issue, please email:
|
|
23
|
+
|
|
24
|
+
**64996768+mcp-tool-shop@users.noreply.github.com**
|
|
25
|
+
|
|
26
|
+
Expected response time: **72 hours**.
|
|
27
|
+
|
|
28
|
+
Please do not open public issues for security vulnerabilities.
|
package/package.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@mcptoolshop/style-dataset-lab",
|
|
3
|
+
"version": "1.1.0",
|
|
4
|
+
"description": "Visual dataset factory — generate, curate, and export multimodal training data for VLM fine-tuning",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"scripts": {
|
|
7
|
+
"generate": "node scripts/generate.js",
|
|
8
|
+
"curate": "node scripts/curate.js",
|
|
9
|
+
"compare": "node scripts/compare.js",
|
|
10
|
+
"canon-bind": "node scripts/canon-bind.js",
|
|
11
|
+
"painterly": "node scripts/painterly.js",
|
|
12
|
+
"generate:identity": "node scripts/generate-identity.js",
|
|
13
|
+
"export": "echo 'Usage: repo-dataset visual generate ./games/<game> --format trl'",
|
|
14
|
+
"inspect": "echo 'Usage: repo-dataset visual inspect ./games/<game>'"
|
|
15
|
+
},
|
|
16
|
+
"files": [
|
|
17
|
+
"scripts/",
|
|
18
|
+
"games/*/canon/",
|
|
19
|
+
"games/*/inputs/",
|
|
20
|
+
"package.json",
|
|
21
|
+
"README.md",
|
|
22
|
+
"LICENSE",
|
|
23
|
+
"SECURITY.md",
|
|
24
|
+
"CHANGELOG.md"
|
|
25
|
+
],
|
|
26
|
+
"keywords": [
|
|
27
|
+
"dataset",
|
|
28
|
+
"visual",
|
|
29
|
+
"training-data",
|
|
30
|
+
"comfyui",
|
|
31
|
+
"vlm",
|
|
32
|
+
"fine-tuning",
|
|
33
|
+
"multimodal"
|
|
34
|
+
],
|
|
35
|
+
"homepage": "https://mcp-tool-shop-org.github.io/style-dataset-lab/",
|
|
36
|
+
"repository": {
|
|
37
|
+
"type": "git",
|
|
38
|
+
"url": "https://github.com/mcp-tool-shop-org/style-dataset-lab.git"
|
|
39
|
+
},
|
|
40
|
+
"engines": {
|
|
41
|
+
"node": ">=20"
|
|
42
|
+
},
|
|
43
|
+
"author": "mcp-tool-shop",
|
|
44
|
+
"license": "MIT"
|
|
45
|
+
}
|