pixelkiln 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PROVIDERS.md +103 -33
- package/README.md +24 -13
- package/dist/cli.js +1347 -897
- package/dist/cli.js.map +1 -1
- package/dist/index.cjs +694 -242
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +95 -5
- package/dist/index.d.ts +95 -5
- package/dist/index.js +655 -204
- package/dist/index.js.map +1 -1
- package/docs/AGENTS.md +7 -5
- package/docs/ARCHITECTURE.md +15 -6
- package/docs/CLI.md +6 -4
- package/docs/COMFYUI.md +191 -0
- package/docs/GETTING_STARTED.md +14 -7
- package/docs/LIBRARY.md +9 -7
- package/docs/MANIFEST.md +54 -8
- package/docs/PROVIDER_BENCHMARK.md +113 -16
- package/docs/README.md +3 -2
- package/docs/RETRO_DIFFUSION.md +1 -1
- package/examples/comfyui/README.md +24 -0
- package/examples/comfyui/pixelkiln.manifest.json +35 -0
- package/examples/comfyui/workflow-api.json +59 -0
- package/package.json +2 -1
- package/schema/manifest.schema.json +4 -4
- package/skills/pixelkiln/SKILL.md +5 -3
- package/skills/pixelkiln/references/comfyui.md +46 -0
- package/skills/pixelkiln/references/mixed-providers.md +15 -7
- package/skills/pixelkiln/references/pixellab.md +5 -3
- package/skills/pixelkiln/references/retro-diffusion.md +7 -0
package/docs/AGENTS.md
CHANGED
|
@@ -48,22 +48,24 @@ not a replacement:
|
|
|
48
48
|
| PixelKiln library/CLI | Budgets, state, provenance, review, recovery, audit, and packaging. |
|
|
49
49
|
| PixelLab adapter | The current production and live-tested generation backend. |
|
|
50
50
|
| Retro Diffusion adapter | Experimental backend; authenticated paid single-still lifecycle plus mocked advanced-workflow tests. |
|
|
51
|
+
| ComfyUI adapter | Experimental self-hosted still-image backend using a committed API-format workflow. |
|
|
51
52
|
|
|
52
53
|
PixelKiln's core is provider-neutral, but PixelLab remains the only production
|
|
53
54
|
and paid-generation-tested adapter. Retro Diffusion generation support is
|
|
54
55
|
experimental. Paid RD Fast and RD Plus single-candidate stills have passed from
|
|
55
56
|
quote through validated download and recovery. Multi-candidate, tileset, GIF,
|
|
56
57
|
and spritesheet workflows still need representative live smoke tests. See
|
|
57
|
-
[
|
|
58
|
+
[provider comparison](../PROVIDERS.md) before choosing a provider for a
|
|
58
59
|
new project or a large environment asset. Once chosen, follow
|
|
59
60
|
[Set up PixelLab](./PIXELLAB.md) or
|
|
60
|
-
[Set up Retro Diffusion](./RETRO_DIFFUSION.md)
|
|
61
|
+
[Set up Retro Diffusion](./RETRO_DIFFUSION.md), or
|
|
62
|
+
[Set up ComfyUI](./COMFYUI.md).
|
|
61
63
|
|
|
62
64
|
The installed skill keeps the shared safety workflow in `SKILL.md` and loads a
|
|
63
|
-
focused reference only when needed: PixelLab, Retro Diffusion, or a
|
|
64
|
-
uses
|
|
65
|
+
focused reference only when needed: PixelLab, Retro Diffusion, ComfyUI, or a
|
|
66
|
+
project that uses more than one. Mixed-provider repositories should use separate manifests and
|
|
65
67
|
lockfiles so each plan and budget keeps its provider-specific unit. The
|
|
66
|
-
[provider comparison](../PROVIDERS.md#use-
|
|
68
|
+
[provider comparison](../PROVIDERS.md#use-multiple-providers-in-one-project) has a
|
|
67
69
|
complete layout and command example.
|
|
68
70
|
|
|
69
71
|
## Recommended first prompt
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -5,7 +5,7 @@ PixelKiln separates provider mechanics from the project state machine:
|
|
|
5
5
|
```text
|
|
6
6
|
manifest + lock + planning + review + recovery + artifact pipelines
|
|
7
7
|
──────────────────── Provider interface ─────────────────────────
|
|
8
|
-
PixelLabProvider RetroDiffusionProvider FakeProvider
|
|
8
|
+
PixelLabProvider RetroDiffusionProvider ComfyUIProvider FakeProvider
|
|
9
9
|
```
|
|
10
10
|
|
|
11
11
|
Everything above the provider boundary is backend-neutral. URL shapes, auth
|
|
@@ -18,7 +18,10 @@ The committed manifest is intent. Resolution combines one style and one asset,
|
|
|
18
18
|
loads/reference-hashes style images, applies overrides, chooses a provider-
|
|
19
19
|
supported generator, and computes a deterministic spec hash. The hash excludes
|
|
20
20
|
project root, output location, and tags but includes every pixel-affecting
|
|
21
|
-
setting.
|
|
21
|
+
setting. A provider may resolve local files before hashing. ComfyUI uses this
|
|
22
|
+
hook to parse and hash workflow JSON without making a network request. The
|
|
23
|
+
runtime graph can then be submitted without putting a machine-specific path in
|
|
24
|
+
the stable identity.
|
|
22
25
|
|
|
23
26
|
See [manifest reference](./MANIFEST.md).
|
|
24
27
|
|
|
@@ -126,7 +129,13 @@ an undefined method.
|
|
|
126
129
|
an experimental still, tileset, and animation adapter. Authenticated RD Fast
|
|
127
130
|
and RD Plus single-candidate still lifecycles have passed end to end. Its
|
|
128
131
|
multi-candidate, tileset, GIF, and spritesheet paths retain mocked coverage
|
|
129
|
-
pending paid live smokes. `
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
132
|
+
pending paid live smokes. `ComfyUIProvider` is an experimental self-hosted
|
|
133
|
+
adapter. It resolves and hashes an API-format workflow offline, submits an
|
|
134
|
+
input-bound clone, polls local history, and stores portable `comfyui://` output
|
|
135
|
+
references so a lockfile does not retain a workstation hostname. It supports
|
|
136
|
+
one still-image output node today. A core-node Stable Diffusion 1.5 graph has
|
|
137
|
+
passed single-image generation, four-candidate queue detection, and cache-only
|
|
138
|
+
recovery on Apple MPS. `FakeProvider` implements the same contract
|
|
139
|
+
in memory, which keeps the pipeline testable without credentials or network
|
|
140
|
+
access. See [library API](./LIBRARY.md) and
|
|
141
|
+
[provider comparison](../PROVIDERS.md).
|
package/docs/CLI.md
CHANGED
|
@@ -10,9 +10,10 @@ separated values work too. This strict parsing prevents a misspelled filter
|
|
|
10
10
|
from widening a paid run.
|
|
11
11
|
|
|
12
12
|
The manifest's top-level `provider` field selects the provider for `plan`,
|
|
13
|
-
`doctor`, and pipeline commands. It defaults to `pixellab
|
|
14
|
-
`retrodiffusion` adapter
|
|
15
|
-
|
|
13
|
+
`doctor`, and pipeline commands. It defaults to `pixellab`. The experimental
|
|
14
|
+
`retrodiffusion` adapter supports still-image `map`/`pixflux`, `tiles` sheets,
|
|
15
|
+
and `animation` GIF/spritesheet work. The experimental `comfyui` adapter runs
|
|
16
|
+
committed API-format `map` workflows on a self-hosted server.
|
|
16
17
|
|
|
17
18
|
## Everyday pipeline
|
|
18
19
|
|
|
@@ -141,7 +142,8 @@ or download artwork.
|
|
|
141
142
|
### `balance`
|
|
142
143
|
|
|
143
144
|
Show the manifest-selected provider's remaining balance and cost unit. Reports
|
|
144
|
-
a capability error when an installed provider has no
|
|
145
|
+
a capability error when an installed provider, such as local ComfyUI, has no
|
|
146
|
+
balance endpoint.
|
|
145
147
|
|
|
146
148
|
### `status`
|
|
147
149
|
|
package/docs/COMFYUI.md
ADDED
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
# Set up ComfyUI
|
|
2
|
+
|
|
3
|
+
PixelKiln can run a committed ComfyUI workflow on a self-hosted server. This
|
|
4
|
+
adapter is experimental. It supports still-image `map` jobs, one or more review
|
|
5
|
+
candidates, local provenance, and cache-backed recovery. A core-node Stable
|
|
6
|
+
Diffusion 1.5 workflow has passed single-image generation and a four-candidate
|
|
7
|
+
review queue on Apple MPS. A higher-quality SDXL workflow has also passed four
|
|
8
|
+
building and environment renders. ComfyUI Cloud is not part of this release.
|
|
9
|
+
|
|
10
|
+
## Start ComfyUI
|
|
11
|
+
|
|
12
|
+
Install and start ComfyUI using its
|
|
13
|
+
[official installation guide](https://docs.comfy.org/installation/overview).
|
|
14
|
+
PixelKiln connects to `http://127.0.0.1:8188` by default. Set a different URL
|
|
15
|
+
only when ComfyUI listens elsewhere:
|
|
16
|
+
|
|
17
|
+
```dotenv
|
|
18
|
+
COMFYUI_BASE_URL=http://127.0.0.1:8188
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
No API key is required for the standard local server. Keep an unauthenticated
|
|
22
|
+
server on loopback. If you expose it to another machine, put authentication and
|
|
23
|
+
TLS in front of it, then point `COMFYUI_BASE_URL` at that protected endpoint.
|
|
24
|
+
|
|
25
|
+
Check the connection without generating an image:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
pixelkiln doctor
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## Export an API-format workflow
|
|
32
|
+
|
|
33
|
+
Build and test the workflow in ComfyUI first. Enable developer mode in ComfyUI
|
|
34
|
+
settings, then use **Save (API Format)**. Commit the exported JSON beside the
|
|
35
|
+
manifest or in a project workflow directory. PixelKiln reads the file during
|
|
36
|
+
planning and hashes its parsed JSON, so a node or model change marks dependent
|
|
37
|
+
assets stale even if the filename stays the same.
|
|
38
|
+
|
|
39
|
+
The output node must expose an `images` array in ComfyUI history. A standard
|
|
40
|
+
`SaveImage` node does this. Record these node IDs and input names from the
|
|
41
|
+
exported JSON:
|
|
42
|
+
|
|
43
|
+
- the positive text encoder's prompt input;
|
|
44
|
+
- the latent image width, height, and batch-size inputs;
|
|
45
|
+
- the sampler seed input, when the PixelKiln style declares a seed;
|
|
46
|
+
- the final `SaveImage` node.
|
|
47
|
+
|
|
48
|
+
Node IDs are workflow-specific. Do not copy IDs from an example without
|
|
49
|
+
checking the exported file.
|
|
50
|
+
|
|
51
|
+
The repository includes a working core-node
|
|
52
|
+
[smoke project](../examples/comfyui/README.md). It uses the public checkpoint
|
|
53
|
+
from ComfyUI's official first-generation guide to test plumbing, not to claim
|
|
54
|
+
pixel-art quality.
|
|
55
|
+
|
|
56
|
+
## Install the tested quality stack
|
|
57
|
+
|
|
58
|
+
The committed quality benchmark uses two public model files:
|
|
59
|
+
|
|
60
|
+
| File | ComfyUI folder | SHA-256 | License named by the model card |
|
|
61
|
+
|---|---|---|---|
|
|
62
|
+
| [`sd_xl_base_1.0.safetensors`](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors) | `models/checkpoints` | `31e35c80fc4829d14f90153f4c74cd59c90b779f6afe05a74cd6120b893f7e5b` | CreativeML Open RAIL++-M |
|
|
63
|
+
| [`pixel-art-xl.safetensors`](https://huggingface.co/nerijs/pixel-art-xl/blob/main/pixel-art-xl.safetensors) | `models/loras` | `4234637cb80c998f41e348e6a6cb6bc20d8d038b2b0f256b6129b3b5e353eef7` | CreativeML OpenRAIL-M |
|
|
64
|
+
|
|
65
|
+
Download each file into the named folder, verify its checksum, then confirm the
|
|
66
|
+
checkpoint and LoRA appear in ComfyUI. The files are about 6.9 GB and 171 MB.
|
|
67
|
+
They are not bundled with PixelKiln. Read both model cards before distributing
|
|
68
|
+
the models or their outputs.
|
|
69
|
+
|
|
70
|
+
The benchmark renders at 1024×1024, where SDXL has enough room to compose the
|
|
71
|
+
scene, then uses ComfyUI's core `ImageScale` node with `nearest-exact` to write
|
|
72
|
+
the requested 256px or 384px PNG. Asset width and height are therefore bound to
|
|
73
|
+
the scale node, not the latent node. This is the useful trick: keep the model at
|
|
74
|
+
its working resolution while PixelKiln still validates the exact game-ready
|
|
75
|
+
output dimensions.
|
|
76
|
+
|
|
77
|
+
You can reproduce the four samples with the committed
|
|
78
|
+
[ComfyUI benchmark project](../benchmarks/provider-environments/comfyui/README.md).
|
|
79
|
+
|
|
80
|
+
## Configure the manifest
|
|
81
|
+
|
|
82
|
+
Put dimensions on each asset. Put workflow configuration under
|
|
83
|
+
`providerOptions.comfyui` on the style:
|
|
84
|
+
|
|
85
|
+
```jsonc
|
|
86
|
+
{
|
|
87
|
+
"name": "my-game",
|
|
88
|
+
"provider": "comfyui",
|
|
89
|
+
"styles": {
|
|
90
|
+
"local-environment": {
|
|
91
|
+
"generator": "map",
|
|
92
|
+
"outDir": "assets/generated/environments",
|
|
93
|
+
"seed": 31415,
|
|
94
|
+
"providerOptions": {
|
|
95
|
+
"comfyui": {
|
|
96
|
+
"workflowFile": "workflows/pixel-environment-api.json",
|
|
97
|
+
"outputNodeId": "9",
|
|
98
|
+
"numImages": 4,
|
|
99
|
+
"bindings": {
|
|
100
|
+
"prompt": { "nodeId": "6", "input": "text" },
|
|
101
|
+
"width": { "nodeId": "5", "input": "width" },
|
|
102
|
+
"height": { "nodeId": "5", "input": "height" },
|
|
103
|
+
"batchSize": { "nodeId": "5", "input": "batch_size" },
|
|
104
|
+
"seed": { "nodeId": "3", "input": "seed" }
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
},
|
|
110
|
+
"assets": {
|
|
111
|
+
"cliffside_fortress": {
|
|
112
|
+
"prompt": "a fortified monastery carved into a mountain cliff",
|
|
113
|
+
"width": 768,
|
|
114
|
+
"height": 512
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
| Option | Meaning |
|
|
121
|
+
|---|---|
|
|
122
|
+
| `workflowFile` | API-format workflow JSON, relative to the manifest. |
|
|
123
|
+
| `outputNodeId` | Node whose completed history contains the final `images` array. |
|
|
124
|
+
| `numImages` | Expected candidates, from 1 to 16. Defaults to 1. |
|
|
125
|
+
| `bindings.prompt` | Workflow input replaced with the resolved PixelKiln prompt. |
|
|
126
|
+
| `bindings.width` / `height` | Inputs replaced with the asset dimensions. |
|
|
127
|
+
| `bindings.batchSize` | Input replaced with `numImages`. |
|
|
128
|
+
| `bindings.seed` | Optional sampler seed input. Required when the style declares `seed`. |
|
|
129
|
+
|
|
130
|
+
PixelKiln refuses missing nodes and inputs during the offline plan. It also
|
|
131
|
+
clones the workflow before applying bindings, so one asset cannot mutate the
|
|
132
|
+
next asset's request.
|
|
133
|
+
|
|
134
|
+
## Plan, generate, and review
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
pixelkiln doctor --dry-run
|
|
138
|
+
pixelkiln plan
|
|
139
|
+
pixelkiln gen --style local-environment --only cliffside_fortress --budget 0
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Local ComfyUI work uses the `free` PixelKiln cost unit and therefore requires a
|
|
143
|
+
zero budget. That means there is no metered provider charge. It does not claim
|
|
144
|
+
that GPU time, electricity, hosted hardware, or model licenses are free.
|
|
145
|
+
|
|
146
|
+
When `numImages` is greater than one, `pixelkiln pick` opens the same local
|
|
147
|
+
candidate review used by hosted providers. The lockfile stores the ComfyUI
|
|
148
|
+
prompt ID, output node, workflow hash, and selected output. Durable source
|
|
149
|
+
references use `comfyui://` rather than embedding a workstation hostname.
|
|
150
|
+
`pixelkiln restore` first uses the validated local content cache, then resolves
|
|
151
|
+
the portable reference against the current `COMFYUI_BASE_URL`.
|
|
152
|
+
|
|
153
|
+
## Current boundary
|
|
154
|
+
|
|
155
|
+
- Supported generator: `map`.
|
|
156
|
+
- Supported output: PNG images returned by one output node.
|
|
157
|
+
- Supported dimensions: 16–4096px per edge, subject to the workflow, model,
|
|
158
|
+
sampler, VRAM, and node constraints.
|
|
159
|
+
- PixelKiln `styleImages` and `palette` are rejected. Put image references,
|
|
160
|
+
ControlNet, LoRA, palette, and other controls inside the committed workflow.
|
|
161
|
+
- Video, animation, masks, multiple output nodes, uploads, and ComfyUI Cloud
|
|
162
|
+
authentication are not implemented.
|
|
163
|
+
- PixelKiln does not install checkpoints or custom nodes. Every machine running
|
|
164
|
+
the project must provide the models and nodes named by the workflow. The
|
|
165
|
+
benchmark workflows use only core ComfyUI nodes.
|
|
166
|
+
|
|
167
|
+
For large mountains, buildings, and backgrounds, model choice and working
|
|
168
|
+
resolution matter more than the provider label. The SDXL benchmark produced a
|
|
169
|
+
coherent 384px cliff fortress and two layered environments where the starter
|
|
170
|
+
SD1.5 workflow did not. It still returned opaque isolated assets and thousands
|
|
171
|
+
of source colors, so transparency and palette reduction remain separate art
|
|
172
|
+
pipeline steps. Benchmark the exact committed workflow before assigning it a
|
|
173
|
+
production batch.
|
|
174
|
+
|
|
175
|
+
## Troubleshooting
|
|
176
|
+
|
|
177
|
+
- **Connection refused:** start ComfyUI or correct `COMFYUI_BASE_URL`.
|
|
178
|
+
- **Missing node or input:** export the workflow again and update the manifest
|
|
179
|
+
bindings to match its API JSON.
|
|
180
|
+
- **Prompt validation failed:** open the workflow in ComfyUI and check missing
|
|
181
|
+
custom nodes, checkpoints, VAEs, LoRAs, and invalid node values.
|
|
182
|
+
- **Wrong image count:** make the bound batch input and `numImages` describe the
|
|
183
|
+
same final output count. PixelKiln fails the job rather than recording a
|
|
184
|
+
partial candidate set.
|
|
185
|
+
- **Out of memory:** lower asset dimensions or batch size, or change the
|
|
186
|
+
workflow. PixelKiln's 4096px ceiling is a schema limit, not a promise that a
|
|
187
|
+
particular machine can render that canvas.
|
|
188
|
+
|
|
189
|
+
ComfyUI's official server route reference documents the `/prompt`,
|
|
190
|
+
`/history/{prompt_id}`, `/view`, and `/system_stats` endpoints used by this
|
|
191
|
+
adapter: <https://docs.comfy.org/development/comfyui-server/comms_routes>.
|
package/docs/GETTING_STARTED.md
CHANGED
|
@@ -9,8 +9,9 @@ checks need provider access.
|
|
|
9
9
|
## Requirements
|
|
10
10
|
|
|
11
11
|
- Node.js 20 or newer
|
|
12
|
-
- A credential for the
|
|
13
|
-
PixelLab or `RD_API_KEY` for
|
|
12
|
+
- A credential for the selected hosted provider: `PIXELLAB_API_KEY` for
|
|
13
|
+
PixelLab or `RD_API_KEY` for Retro Diffusion. Self-hosted ComfyUI needs a
|
|
14
|
+
reachable server instead of an API key.
|
|
14
15
|
|
|
15
16
|
Install the published package in the project that owns the art:
|
|
16
17
|
|
|
@@ -45,8 +46,8 @@ cp examples/minimal/pixelkiln.manifest.json pixelkiln.manifest.json
|
|
|
45
46
|
```
|
|
46
47
|
|
|
47
48
|
Set each style's generator, dimensions, prompt prefix/suffix, reference images,
|
|
48
|
-
and output directory. Then add one manifest entry per asset.
|
|
49
|
-
in `.env.local` beside the manifest:
|
|
49
|
+
and output directory. Then add one manifest entry per asset. For PixelLab, put
|
|
50
|
+
the credential in `.env.local` beside the manifest:
|
|
50
51
|
|
|
51
52
|
```dotenv
|
|
52
53
|
PIXELLAB_API_KEY=...
|
|
@@ -66,12 +67,18 @@ Multi-candidate, tileset, GIF, and spritesheet live runs remain, so PixelLab
|
|
|
66
67
|
remains the production adapter. See
|
|
67
68
|
[Manifest reference](MANIFEST.md#experimental-retro-diffusion) for
|
|
68
69
|
provider options and current limits, or
|
|
69
|
-
[
|
|
70
|
+
[provider comparison](../PROVIDERS.md) for selection guidance.
|
|
71
|
+
For self-hosted generation, set `provider` to `comfyui`, commit an API-format
|
|
72
|
+
workflow, and bind the inputs PixelKiln may replace. Local ComfyUI jobs use a
|
|
73
|
+
zero `free` budget, which describes the lack of a metered provider charge, not
|
|
74
|
+
the cost of hardware or electricity. See [Set up ComfyUI](COMFYUI.md).
|
|
75
|
+
|
|
70
76
|
The provider setup guides give the shortest complete path for
|
|
71
|
-
[PixelLab](PIXELLAB.md)
|
|
77
|
+
[PixelLab](PIXELLAB.md), [Retro Diffusion](RETRO_DIFFUSION.md), and
|
|
78
|
+
[ComfyUI](COMFYUI.md).
|
|
72
79
|
If one repository needs both, use separate provider-specific manifests and
|
|
73
80
|
lockfiles. See
|
|
74
|
-
[Use
|
|
81
|
+
[Use multiple providers in one project](../PROVIDERS.md#use-multiple-providers-in-one-project).
|
|
75
82
|
|
|
76
83
|
Before spending anything, validate and price the selected work:
|
|
77
84
|
|
package/docs/LIBRARY.md
CHANGED
|
@@ -29,10 +29,11 @@ if (plan.actionable.length) {
|
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
Planning performs no provider calls and spends nothing. Passing a provider lets
|
|
32
|
-
its
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
spec
|
|
32
|
+
its `supports()` and `estimate()` methods determine cost unit and candidate
|
|
33
|
+
count. A provider may also resolve local files before hashing. The ComfyUI
|
|
34
|
+
adapter uses that hook to parse and hash a workflow JSON file without contacting
|
|
35
|
+
the server. A resolved spec has the fully inherited style and asset settings
|
|
36
|
+
plus its deterministic spec hash.
|
|
36
37
|
|
|
37
38
|
## Audit and gate generated art
|
|
38
39
|
|
|
@@ -116,8 +117,9 @@ See [TILES.md](./TILES.md) for file contracts and engine details.
|
|
|
116
117
|
## Provider integrations
|
|
117
118
|
|
|
118
119
|
`Provider` is the capability boundary. Generation pipelines accept that
|
|
119
|
-
interface rather than importing PixelLab directly; `
|
|
120
|
-
|
|
120
|
+
interface rather than importing PixelLab directly; `PixelLabProvider`,
|
|
121
|
+
`RetroDiffusionProvider`, and `ComfyUIProvider` are built in, while
|
|
122
|
+
`FakeProvider` implements it in memory for deterministic tests.
|
|
121
123
|
|
|
122
124
|
```ts
|
|
123
125
|
import { FakeProvider, fetchAssets, poll, submit } from "pixelkiln"
|
|
@@ -131,7 +133,7 @@ await fetchAssets(provider, specs, lock, lockPath)
|
|
|
131
133
|
|
|
132
134
|
Provider-backed operations mutate the supplied lock object; persist at the
|
|
133
135
|
workflow boundary with `saveLock`. See
|
|
134
|
-
[
|
|
136
|
+
[provider comparison](../PROVIDERS.md) before selecting or implementing
|
|
135
137
|
another backend, especially its optional capabilities and cost units.
|
|
136
138
|
|
|
137
139
|
`submit` validates adapter estimates again at the spending boundary and returns
|
package/docs/MANIFEST.md
CHANGED
|
@@ -32,7 +32,7 @@ Unknown properties are rejected at every level.
|
|
|
32
32
|
|---|---|---|
|
|
33
33
|
| `$schema` | no | Editor schema URL/path. It does not affect generation identity. |
|
|
34
34
|
| `name` | yes | Project/account tag namespace. |
|
|
35
|
-
| `provider` | no | Provider registry id. Defaults to `pixellab`; `retrodiffusion`
|
|
35
|
+
| `provider` | no | Provider registry id. Defaults to `pixellab`; `retrodiffusion` and `comfyui` are experimental. |
|
|
36
36
|
| `styles` | yes | Map of style id to inherited generation/output settings. |
|
|
37
37
|
| `assets` | yes | Map of stable asset id to subject and per-asset overrides. |
|
|
38
38
|
|
|
@@ -49,7 +49,7 @@ not merely a label edit.
|
|
|
49
49
|
| `promptPrefix` | `""` | Prepended to every participating asset prompt. |
|
|
50
50
|
| `promptSuffix` | `""` | Appended to every participating asset prompt. |
|
|
51
51
|
| `styleImages` | `[]` | `{ "path": "..." }` reference images. Paths are manifest-relative. |
|
|
52
|
-
| `size` | integer
|
|
52
|
+
| `size` | integer 16–8192 | Square size. Each provider and generator applies its own narrower limits. For PixelLab `1dir`, a style reference's dimensions take precedence. |
|
|
53
53
|
| `view` | string | PixelLab `map`: `low top-down`, `high top-down`, or `side`. Other generators interpret this separately. |
|
|
54
54
|
| `outline` | string | PixelLab `map`: `single color outline`, `selective outline`, or `lineless`. |
|
|
55
55
|
| `shading` | string | PixelLab `map`: `flat shading`, `basic shading`, `medium shading`, or `detailed shading`. |
|
|
@@ -112,10 +112,9 @@ the correct extension and validates the correct structure.
|
|
|
112
112
|
|
|
113
113
|
`promptStyle` accepts a live Retro Diffusion still-style selector,
|
|
114
114
|
`numImages` accepts 1–16 candidates, and `removeBg` overrides
|
|
115
|
-
`noBackground`. The Retro Diffusion API accepts 16–512px output
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
limits. RD Pro and user styles accept up to nine reference images. Costs are
|
|
115
|
+
`noBackground`. The Retro Diffusion API accepts 16–512px output. Selected
|
|
116
|
+
styles can impose smaller limits. RD Pro and user styles accept up to nine
|
|
117
|
+
reference images. Costs are
|
|
119
118
|
planned in USD and checked again with Retro Diffusion's free authoritative
|
|
120
119
|
quote endpoint before the paid request is sent. Authenticated single-candidate
|
|
121
120
|
RD Fast and RD Plus paths have passed from quote through validated output and
|
|
@@ -175,14 +174,61 @@ For a Wang-style tileset sheet:
|
|
|
175
174
|
`rd_tile__tile_variation` requires one style image. Provider-specific size and
|
|
176
175
|
input constraints are checked during the free planning phase.
|
|
177
176
|
|
|
177
|
+
## Experimental ComfyUI
|
|
178
|
+
|
|
179
|
+
ComfyUI runs a committed API-format workflow on a self-hosted server. The
|
|
180
|
+
workflow file is resolved relative to the manifest and its parsed content is
|
|
181
|
+
part of the spec hash.
|
|
182
|
+
|
|
183
|
+
```jsonc
|
|
184
|
+
{
|
|
185
|
+
"name": "my-game",
|
|
186
|
+
"provider": "comfyui",
|
|
187
|
+
"styles": {
|
|
188
|
+
"local": {
|
|
189
|
+
"generator": "map",
|
|
190
|
+
"outDir": "assets/generated/local",
|
|
191
|
+
"seed": 31415,
|
|
192
|
+
"providerOptions": {
|
|
193
|
+
"comfyui": {
|
|
194
|
+
"workflowFile": "workflows/pixel-api.json",
|
|
195
|
+
"outputNodeId": "9",
|
|
196
|
+
"numImages": 4,
|
|
197
|
+
"bindings": {
|
|
198
|
+
"prompt": { "nodeId": "6", "input": "text" },
|
|
199
|
+
"width": { "nodeId": "5", "input": "width" },
|
|
200
|
+
"height": { "nodeId": "5", "input": "height" },
|
|
201
|
+
"batchSize": { "nodeId": "5", "input": "batch_size" },
|
|
202
|
+
"seed": { "nodeId": "3", "input": "seed" }
|
|
203
|
+
}
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
},
|
|
208
|
+
"assets": {
|
|
209
|
+
"mountain": {
|
|
210
|
+
"prompt": "a snowbound mountain pass",
|
|
211
|
+
"width": 768,
|
|
212
|
+
"height": 512
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
}
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Node IDs come from the exported workflow; they are not stable across unrelated
|
|
219
|
+
workflows. The current adapter supports `map`, PNG output from one node, 1–16
|
|
220
|
+
candidates, and dimensions from 16–4096px. It rejects manifest `styleImages`
|
|
221
|
+
and `palette`; keep those controls inside the workflow. See
|
|
222
|
+
[Set up ComfyUI](COMFYUI.md) for the complete procedure and limits.
|
|
223
|
+
|
|
178
224
|
## Asset fields
|
|
179
225
|
|
|
180
226
|
| Field | Type/default | Meaning |
|
|
181
227
|
|---|---|---|
|
|
182
228
|
| `prompt` | string, required | Subject-specific prompt. It may be empty only during existing-art onboarding. |
|
|
183
229
|
| `category` | string | Human grouping metadata. |
|
|
184
|
-
| `width` | integer 16–
|
|
185
|
-
| `height` | integer 16–
|
|
230
|
+
| `width` | integer 16–8192 | Per-asset width override. Each provider applies its own ceiling. |
|
|
231
|
+
| `height` | integer 16–8192 | Per-asset height override. Each provider applies its own ceiling. |
|
|
186
232
|
| `size` | integer 32–256 | Per-asset square size override. |
|
|
187
233
|
| `file` | string | Filename/path override beneath the style output root. |
|
|
188
234
|
| `styles` | string array, `[]` | If non-empty, generate this asset only in the named styles. |
|
|
@@ -1,8 +1,11 @@
|
|
|
1
1
|
# Environment provider benchmark
|
|
2
2
|
|
|
3
|
-
This benchmark compares PixelLab and Retro Diffusion on
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
This benchmark compares PixelLab and Retro Diffusion on five game-art briefs,
|
|
4
|
+
with a four-brief ComfyUI extension using a named SDXL stack. Three briefs use
|
|
5
|
+
256×256 output; two use 384×384 to test larger buildings and environment
|
|
6
|
+
backgrounds. The hosted providers have two attempts per brief. ComfyUI has one
|
|
7
|
+
attempt on each supported brief. Prompt text and seed numbers match, but seeds
|
|
8
|
+
are not portable between models.
|
|
6
9
|
|
|
7
10
|
The benchmark tests the adapters that PixelKiln ships. It does not rank every
|
|
8
11
|
model or endpoint sold by either provider.
|
|
@@ -14,10 +17,12 @@ model or endpoint sold by either provider.
|
|
|
14
17
|
| Mountain observatory | `map`, high top-down view | `rd_plus__isometric_asset` | Isolated building on a snowy ridge |
|
|
15
18
|
| River gate | `map`, low top-down view | `rd_plus__topdown_asset` | Isolated landmark spanning water |
|
|
16
19
|
| Alpine valley | `pixflux`, background kept | `rd_plus__environment` | Full scenic background |
|
|
20
|
+
| Cliffside fortress | `map`, high top-down view | `rd_plus__isometric_asset` | Large isolated building complex |
|
|
21
|
+
| Volcanic pass | `pixflux`, background kept | `rd_plus__environment` | Full scenic background with reusable depth planes |
|
|
17
22
|
|
|
18
|
-
Both manifests
|
|
19
|
-
|
|
20
|
-
|
|
23
|
+
Both manifests use seeds `31415` and `27182`. The provider-specific route or
|
|
24
|
+
style is allowed to do its job. No image was picked, edited, cropped, or
|
|
25
|
+
post-processed.
|
|
21
26
|
|
|
22
27
|
PixelLab rejected `view: "isometric"` on the `map` endpoint with HTTP 422. The
|
|
23
28
|
successful observatory attempts use the supported `high top-down` view while
|
|
@@ -30,6 +35,9 @@ The committed manifests and lockfiles are here:
|
|
|
30
35
|
- [PixelLab lockfile](../benchmarks/provider-environments/pixellab/pixelkiln.lock.json)
|
|
31
36
|
- [Retro Diffusion manifest](../benchmarks/provider-environments/retrodiffusion/pixelkiln.manifest.json)
|
|
32
37
|
- [Retro Diffusion lockfile](../benchmarks/provider-environments/retrodiffusion/pixelkiln.lock.json)
|
|
38
|
+
- [ComfyUI benchmark project](../benchmarks/provider-environments/comfyui/README.md)
|
|
39
|
+
- [ComfyUI manifest](../benchmarks/provider-environments/comfyui/pixelkiln.manifest.json)
|
|
40
|
+
- [ComfyUI lockfile](../benchmarks/provider-environments/comfyui/pixelkiln.lock.json)
|
|
33
41
|
|
|
34
42
|
## Mountain observatory
|
|
35
43
|
|
|
@@ -93,15 +101,94 @@ For this brief, PixelLab wins on prompt coverage, graphic clarity, consistency,
|
|
|
93
101
|
and cost. Retro Diffusion wins if the desired result is a closer, more cinematic
|
|
94
102
|
scene.
|
|
95
103
|
|
|
104
|
+
## Cliffside fortress at 384×384
|
|
105
|
+
|
|
106
|
+
Prompt: `a large fortified monastery built into a sheer mountain cliff,
|
|
107
|
+
isometric three-quarter view, central stone keep, two side towers, terraced
|
|
108
|
+
stairs, copper roofs, isolated with no scenery`
|
|
109
|
+
|
|
110
|
+
| PixelLab A | PixelLab B | Retro Diffusion A | Retro Diffusion B |
|
|
111
|
+
|---|---|---|---|
|
|
112
|
+
|  |  |  |  |
|
|
113
|
+
|
|
114
|
+
The larger canvas helped both providers. PixelLab used most of the frame and
|
|
115
|
+
kept the cliff, stairs, central keep, and tower structure legible. Attempt B is
|
|
116
|
+
the clearest match for a fortified monastery. Both outputs still include an
|
|
117
|
+
opaque gray field, and their 246 and 249 colors would need deliberate cleanup
|
|
118
|
+
for a tightly controlled palette.
|
|
119
|
+
|
|
120
|
+
Retro Diffusion improved markedly over its 256×256 observatory attempts. Both
|
|
121
|
+
results read as substantial cliffside compounds, and attempt B makes good use
|
|
122
|
+
of the full canvas. They are ready-to-place transparent cutouts with 75% and
|
|
123
|
+
52% transparent pixels and only 55 and 49 colors. PixelLab is more reliable on
|
|
124
|
+
the exact architectural brief. Retro Diffusion is closer to a finished modular
|
|
125
|
+
map asset.
|
|
126
|
+
|
|
127
|
+
## Volcanic pass at 384×384
|
|
128
|
+
|
|
129
|
+
Prompt: `a wide volcanic mountain pass at dawn, layered black peaks, glowing
|
|
130
|
+
lava river, basalt fortress in the middle distance, smoke plumes, full-bleed
|
|
131
|
+
parallax background with open sky`
|
|
132
|
+
|
|
133
|
+
| PixelLab A | PixelLab B | Retro Diffusion A | Retro Diffusion B |
|
|
134
|
+
|---|---|---|---|
|
|
135
|
+
|  |  |  |  |
|
|
136
|
+
|
|
137
|
+
PixelLab produced broader compositions with open sky and visibly separated
|
|
138
|
+
mountain planes. Attempt A includes the smoke plume and a clear volcano; attempt
|
|
139
|
+
B simplifies the scene into a graphic basin. Neither attempt includes a
|
|
140
|
+
recognizable fortress. Attempt A also contains a generated signature-like mark
|
|
141
|
+
in the lower-right corner, so it is not usable without cleanup. The files use
|
|
142
|
+
44 and 26 colors.
|
|
143
|
+
|
|
144
|
+
Retro Diffusion made the pass and lava river unmistakable in both attempts. Its
|
|
145
|
+
narrow canyon framing is strong for a scene the player enters, but it leaves
|
|
146
|
+
less open sky and fewer obvious planes for a distant backdrop. It also dropped
|
|
147
|
+
the fortress and most of the smoke detail. The files use 26 and 25 colors.
|
|
148
|
+
|
|
149
|
+
None of these four files is a finished parallax package. They are flattened,
|
|
150
|
+
opaque scenes. PixelLab gives an artist clearer depth bands to cut apart; Retro
|
|
151
|
+
Diffusion gives the stronger single-frame canyon. A production workflow should
|
|
152
|
+
generate or extract the sky, distant peaks, middle ground, and foreground as
|
|
153
|
+
separate assets.
|
|
154
|
+
|
|
155
|
+
## ComfyUI SDXL extension
|
|
156
|
+
|
|
157
|
+
The local extension uses
|
|
158
|
+
[SDXL Base 1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
|
|
159
|
+
with [Pixel Art XL](https://huggingface.co/nerijs/pixel-art-xl). Both committed
|
|
160
|
+
workflows use core ComfyUI nodes, generate at 1024×1024, and finish with
|
|
161
|
+
`nearest-exact` reduction. There was no manual image edit or candidate choice.
|
|
162
|
+
|
|
163
|
+
| Mountain observatory, 256px | Cliffside fortress, 384px | Alpine valley, 256px | Volcanic pass, 384px |
|
|
164
|
+
|---|---|---|---|
|
|
165
|
+
|  |  |  |  |
|
|
166
|
+
|
|
167
|
+
The observatory and fortress are the strongest architectural results in this
|
|
168
|
+
small sample. The fortress has a clear entrance, tower hierarchy, stairs, and a
|
|
169
|
+
single readable footprint. The alpine scene keeps the river, village, tree
|
|
170
|
+
line, and distant ridges separate. The volcanic scene is dramatic and legible,
|
|
171
|
+
but it drops the requested basalt fortress, the same prompt-coverage failure
|
|
172
|
+
seen in both hosted providers.
|
|
173
|
+
|
|
174
|
+
The trade-off is production cleanup. All four ComfyUI PNGs are opaque. The two
|
|
175
|
+
isolated files use 16,811 and 31,572 RGB colors, while the backgrounds use
|
|
176
|
+
36,248 and 50,985. They look pixelated because of the LoRA and nearest-exact
|
|
177
|
+
reduction, but they are not indexed, low-palette sprites. Add explicit
|
|
178
|
+
background removal and palette quantization when the target art direction
|
|
179
|
+
requires them.
|
|
180
|
+
|
|
96
181
|
## Cost and operational results
|
|
97
182
|
|
|
98
183
|
| Provider | Successful images | Charged amount | Final balance |
|
|
99
184
|
|---|---:|---:|---:|
|
|
100
|
-
| PixelLab |
|
|
101
|
-
| Retro Diffusion |
|
|
185
|
+
| PixelLab | 10 | 10 generations | 4,411 generations |
|
|
186
|
+
| Retro Diffusion | 10 | $0.744 | $9.73 |
|
|
187
|
+
| ComfyUI | 4 | 0 `free` PixelKiln units | No account balance |
|
|
102
188
|
|
|
103
189
|
PixelLab charged one generation per image. Retro Diffusion quoted and charged
|
|
104
|
-
$0.058
|
|
190
|
+
$0.058 for each 256px RD Plus image and $0.099 for each 384px RD Plus image;
|
|
191
|
+
PixelKiln's hard ceiling rounds the latter to $0.10 per image.
|
|
105
192
|
|
|
106
193
|
The run also caught two integration details:
|
|
107
194
|
|
|
@@ -111,24 +198,34 @@ The run also caught two integration details:
|
|
|
111
198
|
$0.057768 to $0.058. PixelKiln now rounds offline estimates up to the live
|
|
112
199
|
quote precision, so planning remains a safe ceiling.
|
|
113
200
|
|
|
114
|
-
|
|
115
|
-
PNG cache entries
|
|
201
|
+
All three manifests now pass `doctor` and report a current plan. The hosted
|
|
202
|
+
projects have ten healthy PNG cache entries each; the ComfyUI extension has
|
|
203
|
+
four.
|
|
116
204
|
|
|
117
205
|
## Recommendation
|
|
118
206
|
|
|
119
207
|
For large isolated buildings or landmarks, start with PixelLab when prompt
|
|
120
208
|
coverage matters most. Budget for background cleanup. Start with Retro
|
|
121
|
-
Diffusion when a transparent,
|
|
122
|
-
|
|
209
|
+
Diffusion when a transparent, low-color asset matters more than capturing every
|
|
210
|
+
noun in a complex prompt. At 384×384, Retro Diffusion can fill the frame with a
|
|
211
|
+
substantial structure rather than the compact cutouts seen in the first brief.
|
|
123
212
|
|
|
124
213
|
For full scenic backgrounds, start with PixelLab Pixflux. These two attempts
|
|
125
214
|
were cheaper and more faithful to the brief. Try Retro Diffusion when you want
|
|
126
215
|
foreground framing and a closer illustrated scene.
|
|
127
216
|
|
|
128
|
-
|
|
217
|
+
Try the tested ComfyUI SDXL stack when local control and larger compositions
|
|
218
|
+
matter more than ready-to-place transparency. It produced the best large
|
|
219
|
+
building in this run and a strong layered valley, with no provider charge. It
|
|
220
|
+
also took roughly 90 seconds per 1024px render on the tested Apple MPS machine,
|
|
221
|
+
and every output still needs palette review.
|
|
222
|
+
|
|
223
|
+
Do not ask any provider for one giant finished level. Generate terrain,
|
|
129
224
|
background, buildings, landmarks, and foreground pieces separately. Compose
|
|
130
225
|
them in the engine, then use integer nearest-neighbor scaling for display.
|
|
131
226
|
|
|
132
227
|
This sample is useful, not definitive. Two attempts expose obvious tendencies,
|
|
133
|
-
but they do not measure every style, prompt family, or model update.
|
|
134
|
-
|
|
228
|
+
but they do not measure every style, prompt family, or model update. The new
|
|
229
|
+
volcanic brief also shows why prompt coverage needs review at the object level:
|
|
230
|
+
all four images lost the requested fortress. Rerun the committed manifests when
|
|
231
|
+
either provider changes its models.
|