pixelkiln 0.6.0 → 0.8.0

package/PROVIDERS.md

@@ -1,18 +1,19 @@
-# PixelLab and Retro Diffusion
+# Provider comparison
 
-PixelKiln can route one manifest through either PixelLab or Retro Diffusion.
+PixelKiln can route one manifest through PixelLab, Retro Diffusion, or a
+self-hosted ComfyUI server.
 The project model keeps planning, hard budgets, lockfile provenance, human
 review, recovery, and packaging. Each adapter owns its service's
 authentication, pricing, validation, request lifecycle, and output formats.
 
 PixelLab remains the default so existing manifests and spec hashes remain
-compatible. Select Retro Diffusion with the manifest's top-level `provider`
-field and keep its settings under `providerOptions.retrodiffusion`.
+compatible. Select another backend with the manifest's top-level `provider`
+field and keep its settings under the matching `providerOptions` key.
 
-Ready to configure a project? Use [Set up PixelLab](./docs/PIXELLAB.md) for the
-production provider or
-[Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md) for the experimental
-adapter. This page focuses on choosing between them.
+Ready to configure a project? Use [Set up PixelLab](./docs/PIXELLAB.md),
+[Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md), or
+[Set up ComfyUI](./docs/COMFYUI.md). This page focuses on choosing between
+them.
 
 ## Support status
 
@@ -20,6 +21,7 @@ adapter. This page focuses on choosing between them.
 |---|---|---|---|
 | PixelLab | Production; paid generation and account workflows live-tested | `PIXELLAB_API_KEY` | generations |
 | Retro Diffusion | Experimental; authenticated paid still generation, download, provenance, and recovery live-tested; advanced workflows pending | `RD_API_KEY` | USD |
+| ComfyUI | Experimental; local single-image generation, four-candidate queue, provenance, and cache-only recovery live-tested on Apple MPS | none; optional `COMFYUI_BASE_URL` | free |
 | FakeProvider | Test-only deterministic lifecycle | none | free |
 
 Live tests now cover single-candidate RD Fast and RD Plus stills from cost quote
@@ -28,6 +30,10 @@ The RD Plus run covered isometric-asset, top-down-asset, and environment styles.
 Retro Diffusion's multi-candidate review, tileset, GIF, and spritesheet paths
 have mocked integration coverage but still need representative paid live runs.
 
+ComfyUI's `free` unit means PixelKiln cannot identify a metered provider
+charge. It does not count hardware, hosting, electricity, or model-license
+costs.
+
 ## PixelLab vs. Retro Diffusion
 
 This comparison describes the adapters PixelKiln ships today, not every feature
@@ -52,17 +58,39 @@ is the desired look, or a USD quote is easier to budget. For a production batch,
 run one representative asset through the selected provider before expanding the
 scope.
 
+## Where ComfyUI fits
+
+ComfyUI is different from the hosted providers. PixelKiln does not choose a
+model or hide the graph. You commit an API-format workflow and declare the exact
+prompt, size, batch, and optional seed inputs PixelKiln may replace.
+
+| Decision | ComfyUI through PixelKiln |
+|---|---|
+| Best fit today | Private assets, local models, custom graph control, and teams that already maintain ComfyUI workflows |
+| PixelKiln generator | `map` stills |
+| Output | One PNG output node, with 1–16 review candidates |
+| Cost model | `0 free`; local compute and hosting are outside PixelKiln's estimate |
+| Reproducibility | Workflow content is hashed; model files, custom-node versions, and runtime settings must still be managed outside PixelKiln |
+| Account lifecycle | Read-only connectivity check; no balance, remote object listing, tagging, or purge |
+| Confidence | Full mocked coverage plus a live core-node single-image lifecycle, four-candidate queue, and cache-only restore on Apple MPS; pixel-art model benchmark pending |
+
+ComfyUI is the strongest option when workflow ownership matters more than a
+managed service. It is also the easiest provider to make irreproducible by
+accident. Two machines can share the same graph but differ in checkpoint bytes,
+custom-node versions, or sampler behavior. Commit the workflow and document the
+external model stack used to validate it.
+
 ## Large environments, mountains, and buildings
 
 Start by deciding whether the result is an isolated map object or a complete
 background. That distinction matters more than raw canvas size.
 
-| Asset type | PixelLab through PixelKiln | Retro Diffusion through PixelKiln |
-|---|---|---|
-| Isolated house, building, mountain, or landmark | Start with `map`: arbitrary dimensions up to 400×400 and a measured one-generation cost. Live benchmark outputs had opaque backgrounds, so plan for cleanup. Use `1dir` only when references or candidate variety justify 20–40 generations and a square canvas. | Start with `rd_plus__topdown_asset`, `rd_plus__isometric_asset`, or `rd_tile__scene_object`, depending on perspective. `rd_tile__scene_object` is specifically intended for 64–384px objects placed on tile maps. |
-| Full scenic background | Use `pixflux` with `noBackground: false` when an exact palette matters, or `map` for a simple scene. Current PixelKiln routes top out at 400×400. | `rd_plus__environment` targets one-point-perspective scenes; `rd_plus__topdown_map` targets 3/4 top-down maps. These styles support up to 384×384. |
-| Style consistency across a set | `1dir` accepts a style reference and returns size-dependent candidates, but it is more expensive and capped at the square-object range. | RD Pro accepts up to nine references and has stronger prompt following, but its common styles top out at 256×256 and cost $0.18 per image. Environment-specific RD Plus styles trade references for a larger 384px canvas. |
-| Very large final scene | Generate reusable objects, terrain, and background layers separately; assemble them deterministically and integer-upscale the result. | Use the same layered approach. The API has a 512px overall ceiling, but the useful environment and scene-object styles currently cap at 384px. |
+| Asset type | PixelLab | Retro Diffusion | ComfyUI |
+|---|---|---|---|
+| Isolated house, building, mountain, or landmark | Start with `map`: arbitrary dimensions up to 400×400 and a measured one-generation cost. Live benchmark outputs had opaque backgrounds, so plan for cleanup. Use `1dir` only when references or candidate variety justify 20–40 generations and a square canvas. | Start with `rd_plus__topdown_asset`, `rd_plus__isometric_asset`, or `rd_tile__scene_object`, depending on perspective. `rd_tile__scene_object` is intended for 64–384px objects placed on tile maps. | Choose a checkpoint or LoRA trained for the intended perspective, then keep background removal or segmentation in the workflow. The adapter accepts up to 4096px per edge, but useful size depends on the model and VRAM. |
+| Full scenic background | Use `pixflux` with `noBackground: false` when an exact palette matters, or `map` for a simple scene. Current PixelKiln routes top out at 400×400. | `rd_plus__environment` targets one-point-perspective scenes; `rd_plus__topdown_map` targets 3/4 top-down maps. These styles support up to 384×384. | A custom workflow can use composition controls, tiled diffusion, or a generate-then-downscale path. PixelKiln does not configure those nodes; it binds the final size and records the graph hash. |
+| Style consistency across a set | `1dir` accepts a style reference and returns size-dependent candidates, but it is more expensive and capped at the square-object range. | RD Pro accepts up to nine references and has stronger prompt following, but its common styles top out at 256×256 and cost $0.18 per image. Environment-specific RD Plus styles trade references for a larger 384px canvas. | LoRAs, reference adapters, ControlNet, and shared latent settings can live in the committed workflow. Reproducibility also depends on external model and custom-node versions. |
+| Very large final scene | Generate reusable objects, terrain, and background layers separately; assemble them deterministically and integer-upscale the result. | Use the same layered approach. The API has a 512px overall ceiling, but the useful environment and scene-object styles currently cap at 384px. | The graph can tile, upscale, or composite beyond hosted-provider limits, but memory and seam quality become workflow concerns. Prefer reusable layers unless the scene truly needs one render. |
 
 For production environments, prefer a kit over a monolith: seamless terrain,
 separate landmarks/buildings, foreground occluders, and a distant backdrop.
@@ -77,12 +105,14 @@ edge cleanliness, tiling/layerability, prompt adherence, and usable results per
 provider unit. Seeds are provider-specific, so equal seed numbers do not make
 the outputs directly reproducible across services.
 
-See the [environment provider benchmark](./docs/PROVIDER_BENCHMARK.md) for the
-twenty generated images, prompts, manifests, measured costs, and review. The
+See the [hosted-provider environment benchmark](./docs/PROVIDER_BENCHMARK.md)
+for the twenty generated images, prompts, manifests, measured costs, and review. The
 384×384 additions test a larger cliffside building and a full volcanic
-background.
+background. ComfyUI needs a separate benchmark because model and workflow are
+part of the provider configuration; comparing an unnamed graph would not be
+repeatable.
 
-## Use both providers in one project
+## Use multiple providers in one project
 
 One manifest selects one provider. PixelKiln does not currently support a
 provider override on an individual style or asset. The boundary is deliberate:
@@ -90,7 +120,7 @@ one command constructs one account adapter, and one `--budget` must have one
 meaning. PixelLab generations and Retro Diffusion dollars cannot share a safe
 ceiling.
 
-A repository can still use both providers today. Give each provider its own
+A repository can still use multiple providers today. Give each provider its own
 manifest, lockfile, and output directory:
 
 ```text
@@ -99,6 +129,8 @@ art/
   pixelkiln.pixellab.lock.json
   pixelkiln.retrodiffusion.manifest.json
   pixelkiln.retrodiffusion.lock.json
+  pixelkiln.comfyui.manifest.json
+  pixelkiln.comfyui.lock.json
 pixelkiln.workspace.json
 ```
 
@@ -110,16 +142,20 @@ pixelkiln gen --manifest art/pixelkiln.pixellab.manifest.json --lock art/pixelki
 
 pixelkiln plan --manifest art/pixelkiln.retrodiffusion.manifest.json --lock art/pixelkiln.retrodiffusion.lock.json
 pixelkiln gen --manifest art/pixelkiln.retrodiffusion.manifest.json --lock art/pixelkiln.retrodiffusion.lock.json --budget <usd>
+
+pixelkiln plan --manifest art/pixelkiln.comfyui.manifest.json --lock art/pixelkiln.comfyui.lock.json
+pixelkiln gen --manifest art/pixelkiln.comfyui.manifest.json --lock art/pixelkiln.comfyui.lock.json --budget 0
 ```
 
-Register both manifests in the workspace catalog for aggregate status and
+Register the manifests in the workspace catalog for aggregate status and
 complete claim checks. Keep the provider lockfiles separate. Package their
 reviewed outputs independently, or combine explicit files with `pixelkiln pack
 --inputs <file> --out <path>`.
 
 This is a useful split when PixelLab handles prompt-sensitive buildings and
-account recovery while Retro Diffusion handles environment-styled backdrops,
-clean cutouts, or native animation. Retro Diffusion is not a higher-resolution
+account recovery, Retro Diffusion handles environment-styled backdrops, clean
+cutouts, or native animation, and ComfyUI handles private or project-specific
+model workflows. Retro Diffusion is not a higher-resolution
 route through PixelKiln today: its useful environment styles cap at 384×384,
 while PixelLab `map` reaches 400×400. Its advantage is the model/style and
 output type, not raw dimensions.
@@ -171,28 +207,30 @@ estimate and hard budget remain enforced.
 
 ## What to build next
 
-The highest-value next feature is native per-style provider routing, not a third
-adapter. The benchmark now shows a useful split: PixelLab follows dense
-building briefs more reliably, while Retro Diffusion returns cleaner cutouts
-and strong close environment framing. Two manifests make that combination
-possible, but awkward. One manifest should be able to send a building style to
-PixelLab and a background or animation style to Retro Diffusion.
+The first ComfyUI release now has a live, versioned core-node reference
+workflow. The remaining confidence gap is visual rather than transport: choose
+and pin a pixel-art model or LoRA, then benchmark a large building and scenic
+background against the hosted-provider briefs. A human still needs to complete
+the live four-candidate selection and fetch path.
 
-This needs provider-keyed budgets and confirmations, not a provider field added
-in isolation. A safe plan must keep `4 generations` and `$0.40` separate,
-construct and rate-limit each adapter independently, and require an explicit
-provider for account-wide commands. The lockfile already records the provider
-on each entry, so its identity model is ready for the change.
+Native per-style provider routing remains the highest-value orchestration
+feature. One manifest should be able to send a building style to PixelLab, an
+animation style to Retro Diffusion, and a private model workflow to ComfyUI.
+That requires provider-keyed budgets and confirmations, independently
+rate-limited adapters, and an explicit provider for account-wide commands. The
+lockfile already records the provider on each entry.
 
-After that, Scenario is the best next hosted provider candidate.
+Scenario remains the best next hosted provider candidate. The implementation
+scope and acceptance criteria are tracked in
+[GitHub issue #52](https://github.com/gfargo/pixelkiln/issues/52).
 
 | Candidate | What it adds | Fit with PixelKiln | Main cost or risk | Priority |
 |---|---|---|---|---:|
 | Scenario | Custom-trained style models, references, image editing, background removal, upscaling, and managed assets | Async jobs, asset IDs, and free `dryRun` cost estimates map closely to PixelKiln's plan/submit/poll/download lifecycle | API access requires a paid plan; auth uses both an API key and secret, so the provider factory must describe more than one credential | 1 |
-| ComfyUI local/cloud | Reproducible workflow graphs, broad model choice, local GPU execution, and a cloud path using a compatible API | Workflow JSON could become a durable provenance input; jobs and output downloads already resemble the current provider boundary | Local cost has no universal provider unit; the cloud API is marked experimental and requires a subscription | 2 |
+| ComfyUI Cloud | Managed execution of workflow graphs without running a local GPU | Could reuse part of the workflow model, but authentication, endpoints, billing, and lifecycle must remain separate from the local adapter | Treating cloud as a base-URL swap would hide real security and cost differences | 2 |
 | fal | A large hosted model catalog, including pixel-art style controls, LoRAs, editing, upscaling, and background removal | Queue-based requests and model schemas are accessible through one client | Model-specific schemas and prices move the adapter toward a marketplace abstraction rather than one stable art workflow | 3 |
 
-Scenario deserves the first spike because its [custom generation API](https://docs.scenario.com/get-started/generation/third-party-model-generation)
+Scenario is the next hosted spike because its [custom generation API](https://docs.scenario.com/get-started/generation/third-party-model-generation)
 returns an asynchronous job ID, its [generation surface](https://docs.scenario.com/get-started/documentation/key-capabilities-at-a-glance)
 supports custom models and image references, and its
 [Compute Unit guidance](https://help.scenario.com/articles/7934059476-api-usage-and-credits-compute-units)
@@ -200,25 +238,17 @@ documents free cost preflights. That combination adds something the current
 providers do not: a project-specific visual model with a cost check that can be
 captured before submission.
 
-ComfyUI should follow. Its [local server API](https://docs.comfy.org/development/overview)
-would cover private or offline GPU workflows, while the
-[cloud API](https://docs.comfy.org/development/cloud/overview) accepts the same
-workflow-shaped requests and exposes asynchronous jobs. The adapter needs an
-explicit budget policy before implementation. A local run could report planned
-image count and measured execution time, but it must not pretend those values
-are a portable dollar estimate.
-
 Recommended order:
 
-1. Add per-style provider selection, provider-keyed budgets, and mixed-provider
+1. Select and benchmark a pinned pixel-art ComfyUI workflow, including a human-reviewed candidate fetch.
+2. Add per-style provider selection, provider-keyed budgets, and mixed-provider
    integration tests.
-2. Finish live Retro Diffusion multi-candidate, tileset, GIF, and spritesheet
+3. Finish live Retro Diffusion multi-candidate, tileset, GIF, and spritesheet
    smoke tests.
-3. Build a narrow Scenario still-image spike with dry-run cost, submit, poll,
+4. Build the Scenario still-image spike from issue #52 with dry-run cost, submit, poll,
    download, and one custom-model or reference-image benchmark.
-4. Define local and cloud budget semantics for ComfyUI, then prototype one
-   versioned workflow.
-5. Consider general raster marketplaces only with explicit nearest-neighbor,
+5. Design ComfyUI Cloud as a separate authenticated and billable adapter.
+6. Consider general raster marketplaces only with explicit nearest-neighbor,
    palette, transparency, and reproducibility checks.
 
 Midjourney is not an adapter target without an official public API. Automating

package/README.md

@@ -20,9 +20,13 @@ live-tested backend. An experimental Retro Diffusion adapter supports native
 pixel-art stills, candidate batches, tileset sheets, animated GIFs, and PNG
 spritesheets. Authenticated RD Fast and RD Plus single-candidate still paths
 have passed from quote through validated output and recovery. Multi-candidate,
-tileset, GIF, and spritesheet live runs remain. See
-[PixelLab vs. Retro Diffusion](./PROVIDERS.md) for the trade-offs, including
-large environment and building workflows.
+tileset, GIF, and spritesheet live runs remain. An experimental ComfyUI adapter
+runs committed API-format still-image workflows on a self-hosted server. Its
+core-node smoke project has passed live generation, candidate queueing, and
+cache-only recovery on Apple MPS. An SDXL plus Pixel Art XL workflow also has
+four baseline samples and four refined transparency and palette samples. See
+[provider comparison](./PROVIDERS.md) for the trade-offs, including large
+environment and building workflows.
 `FakeProvider` exercises the same contract deterministically in tests.
 
 > **Release status:** PixelKiln is published on npm. Merges to `main` use
@@ -98,7 +102,7 @@ cp examples/minimal/pixelkiln.manifest.json ../my-game/pixelkiln.manifest.json
 cd ../my-game
 ```
 
-Put the provider credential in `.env.local` beside the manifest:
+Put a hosted provider's credential in `.env.local` beside the manifest:
 
 ```dotenv
 # PixelLab (the default provider)
@@ -106,6 +110,9 @@ PIXELLAB_API_KEY=...
 
 # Or Retro Diffusion when `provider` is `retrodiffusion`
 RD_API_KEY=...
+
+# Self-hosted ComfyUI needs no key; override its local URL only when needed
+COMFYUI_BASE_URL=http://127.0.0.1:8188
 ```
 
 Validate locally, inspect exact work/cost, then generate with a hard ceiling:
@@ -130,9 +137,10 @@ pixelkiln plan
 ```
 
 See [Getting started](./docs/GETTING_STARTED.md) for new and existing projects.
-Use [Set up PixelLab](./docs/PIXELLAB.md) or
-[Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md) for provider-specific
-credentials, manifest examples, and current limits.
+Use [Set up PixelLab](./docs/PIXELLAB.md),
+[Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md), or
+[Set up ComfyUI](./docs/COMFYUI.md) for provider-specific configuration,
+manifest examples, and current limits.
 
 ## Agent skill
 
@@ -176,10 +184,11 @@ Styles are namespaces. Adding a second style re-derives the same asset ids into
 a separate output directory and separate lock keys without clobbering the first
 set. Generator choice, reference-image bytes, dimensions, palette, seed, and
 prompt settings participate in deterministic spec identity. A manifest may
-select `retrodiffusion` instead and pass namespaced `providerOptions`; see
+select another provider and pass namespaced `providerOptions`; see
 [Set up PixelLab](./docs/PIXELLAB.md),
 [Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md), and
-[PixelLab vs. Retro Diffusion](./PROVIDERS.md) for configuration, costs,
+[Set up ComfyUI](./docs/COMFYUI.md). The
+[provider comparison](./PROVIDERS.md) covers costs,
 current confidence, and limitations.
 
 The schema rejects unknown fields and invalid generator combinations before
@@ -226,8 +235,9 @@ replace a hard palette or reference-image constraint. See
 
 Generator names describe PixelKiln workflows; their exact capabilities and
 prices depend on the selected provider. Retro Diffusion also supports the
-provider-specific `animation` generator. Compare the adapters in
-[PixelLab vs. Retro Diffusion](./PROVIDERS.md).
+provider-specific `animation` generator. ComfyUI currently supports `map`
+through an operator-supplied workflow. Compare the adapters in the
+[provider comparison](./PROVIDERS.md).
 
 ## Derived artifacts
 
@@ -332,11 +342,12 @@ writes, and offline provenance verification. See [Library API](./docs/LIBRARY.md
 | [Getting started](./docs/GETTING_STARTED.md) | First project, existing-art onboarding, everyday workflow, and what to commit. |
 | [Set up PixelLab](./docs/PIXELLAB.md) | Production-provider credentials, manifest, generators, and account workflows. |
 | [Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md) | Experimental-provider credentials, styles, formats, cost checks, and limits. |
+| [Set up ComfyUI](./docs/COMFYUI.md) | Experimental self-hosted server, workflow bindings, local cost semantics, and limits. |
 | [CLI reference](./docs/CLI.md) | Every command, flag, JSON mode, and exit contract. |
 | [Manifest reference](./docs/MANIFEST.md) | Every style/asset field and generator constraint. |
 | [Agent workflows](./docs/AGENTS.md) | Official skill install, operating model, and provider-aware safety. |
 | [Generators](./docs/GENERATORS.md) | Capability choice, measured costs, palettes, style references, and tiles. |
-| [Environment provider benchmark](./docs/PROVIDER_BENCHMARK.md) | Twenty matched outputs comparing buildings, landmarks, large backgrounds, cost, and file readiness. |
+| [Environment provider benchmark](./docs/PROVIDER_BENCHMARK.md) | Twenty-eight outputs comparing providers, large scenes, transparency, palette size, and file readiness. |
 | [Derived artifacts](./docs/ARTIFACTS.md) | Pack, mount, export, provenance, ownership, transactions, and recovery. |
 | [Recovery](./docs/RECOVERY.md) | Restore, caches, adopt, salvage, claims, and purge safety. |
 | [Quality gates](./docs/QUALITY.md) | Plan, doctor, audit, cache, JSON, and CI. |
@@ -344,7 +355,7 @@ writes, and offline provenance verification. See [Library API](./docs/LIBRARY.md
 | [Library API](./docs/LIBRARY.md) | Public TypeScript contracts and examples. |
 | [Tiles](./docs/TILES.md) | Structural outputs and generic/Tiled/Godot formats. |
 | [Endpoint research](./docs/ENDPOINTS.md) | Measured PixelLab API behavior and recipes. |
-| [PixelLab vs. Retro Diffusion](./PROVIDERS.md) | Provider selection, costs, supported workflows, confidence, and limitations. |
+| [Provider comparison](./PROVIDERS.md) | Provider selection, costs, supported workflows, confidence, and limitations. |
 
 The [public documentation site](https://pixelkiln.griffen.codes/docs) is built by
 the application in [`website/`](./website/README.md). It reads these Markdown