pixelkiln 0.86.1 → 0.86.2

package/PROVIDERS.md

@@ -27,7 +27,7 @@ them.
 | PixelLab | Production; paid generation and account workflows live-tested | `PIXELLAB_API_KEY` | generations |
 | Retro Diffusion | Experimental; authenticated paid still generation, download, provenance, and recovery live-tested; advanced workflows pending | `RD_API_KEY` | USD |
 | ComfyUI | Experimental; local generation, review, recovery, and image-to-image revision smoke live-tested on Apple MPS; inpaint/outpaint remain mock-tested | none; optional `COMFYUI_BASE_URL` | free |
-| Scenario | Experimental; BFL Flux 2 Dev authentication, CU preflight, paid single/two-output generation, review, download, and durable recovery live-tested | `SCENARIO_SDK_API_KEY` and `SCENARIO_SDK_API_SECRET` | compute-units |
+| Scenario | Experimental; BFL Flux 2 Dev authentication, CU preflight, paid single/two-output generation, review, download, and durable recovery live-tested; reference-image edits, tool models, and multi-model chains run once each | `SCENARIO_SDK_API_KEY` and `SCENARIO_SDK_API_SECRET` | compute-units |
 | FakeProvider | Test-only deterministic lifecycle | none | free |
 
 Live tests now cover single-candidate RD Fast and RD Plus stills from cost quote
@@ -267,9 +267,11 @@ to ComfyUI. Provider-keyed budgets, independent orchestration, lock-authoritativ
 recovery, and explicit account-provider selection ship with it.
 
 The Scenario still-image adapter from
-[GitHub issue #52](https://github.com/gfargo/pixelkiln/issues/52) now has mocked
-edge coverage and a paid BFL Flux 2 Dev lifecycle smoke. It remains
-experimental until more model schemas and representative art briefs pass.
+[GitHub issue #52](https://github.com/gfargo/pixelkiln/issues/52) has mocked edge
+coverage, a paid BFL Flux 2 Dev lifecycle smoke, and a broader evaluation (ten
+more models, six tool models, reference-image edits, multi-model chains, and a real
+game's art) recorded in [Set up Scenario](./docs/SCENARIO.md#where-the-evidence-is).
+It remains experimental until more model schemas and representative art briefs pass.
 
 | Candidate | What it adds | Fit with PixelKiln | Main cost or risk | Priority |
 |---|---|---|---|---:|

package/README.md

@@ -366,7 +366,7 @@ verification. See [Library API](./docs/LIBRARY.md).
 | [Set up PixelLab](./docs/PIXELLAB.md) | Production-provider credentials, manifest, generators, and account workflows. |
 | [Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md) | Experimental-provider credentials, styles, formats, cost checks, and limits. |
 | [Set up ComfyUI](./docs/COMFYUI.md) | Self-hosted stills, revisions, ordered frame sets, per-asset inputs, and quality limits. |
-| [Set up Scenario](./docs/SCENARIO.md) | Experimental hosted models, two-part credentials, CU preflight, review, and durable downloads. |
+| [Set up Scenario](./docs/SCENARIO.md) | Experimental hosted models, two-part credentials, CU preflight, reference-image edits, multi-model chains, measured findings, and durable downloads. |
 | [Versioned recipes](./docs/RECIPES.md) | Pinned workflow packs, model hashes, manifest templates, and quality contracts. |
 | [Controlled revisions](./docs/REVISIONS.md) | Image-to-image, inpaint, palette cleanup, animation, and interpolation from a parent; masks, fail-closed readiness, provenance, and ComfyUI bindings. |
 | [CLI reference](./docs/CLI.md) | Every command, flag, JSON mode, and exit contract. |

package/docs/AGENTS.md

@@ -72,7 +72,7 @@ replacement:
 | PixelLab adapter | The current production and live-tested generation backend: stills, tiles, UI panels and elements, `objectPro` objects, controlled revisions, and the character family (four engines, states, loops, mirrors, portraits, outfits, adoption). |
 | Retro Diffusion adapter | Experimental backend; authenticated paid single-still lifecycle plus mocked advanced-workflow tests. |
 | ComfyUI adapter | Experimental self-hosted still, controlled-revision, and atomic frame-set backend; PixelKiln automates lineage and mechanical quality checks while a person retains visual approval. |
-| Scenario adapter | Experimental hosted still-image backend; BFL Flux 2 Dev quote, paid single/two-output generation, human review, and durable recovery live-tested. |
+| Scenario adapter | Experimental hosted still-image backend with reference-image edits and cross-style chains; BFL Flux 2 Dev quote, paid generation, human review, and durable recovery live-tested, plus ten more models, six tool models, and a real game's art run once each. |
 
 PixelKiln's core is provider-neutral, but PixelLab remains the only production
 adapter. Retro Diffusion generation support is experimental. Paid RD Fast and

package/docs/ARCHITECTURE.md

@@ -288,7 +288,7 @@ coverage. Live Apple MPS runs cover a small Stable
 Diffusion 1.5 plumbing smoke and
 the SDXL Base plus Pixel Art XL composition benchmark, including four-candidate
 review, cache recovery, and native-grid refinement. `ScenarioProvider` is an experimental hosted still-image
-adapter. It keeps offline planning conservative, records the provider's free
+adapter that also uploads references for `styleImages` and `image-to-image` revisions. It keeps offline planning conservative, records the provider's free
 submit-time Compute Unit quote, and stores durable `scenario://` job and asset
 references that refresh signed URLs. Authentication and CU preflight have
 passed live with BFL Flux 2 Dev, along with paid single/two-output jobs, human

package/docs/CLI.md

@@ -17,7 +17,7 @@ work accordingly. The experimental `retrodiffusion` adapter supports
 still-image `map`/`pixflux`, `tiles` sheets, and `animation` GIF/spritesheet
 work. The experimental `comfyui` adapter runs committed API-format `map` and
 ordered still-frame workflows on a self-hosted server. The experimental `scenario` adapter runs
-hosted still models with Compute Unit preflight and durable asset recovery.
+hosted still models, with reference-image edits, Compute Unit preflight, and durable asset recovery.
 
 Which command:
 

package/docs/GETTING_STARTED.md

@@ -73,7 +73,7 @@ remains the production adapter. See
 provider options and current limits, or
 [provider comparison](../PROVIDERS.md) for selection guidance.
 
-For experimental Scenario still generation, use both credentials:
+For experimental Scenario still generation and reference-image edits, use both credentials:
 
 ```dotenv
 SCENARIO_SDK_API_KEY=...

package/docs/SCENARIO.md

@@ -1,16 +1,29 @@
 # Set up Scenario
 
 [Scenario](https://www.scenario.com/) is an experimental hosted provider for
-project-specific models and third-party image models. PixelKiln currently
-supports still PNG generation through Scenario's universal model endpoint,
-free Compute Unit preflight, asynchronous jobs, multi-output review, and
-refreshable asset downloads.
-
-Every code path in the adapter has mocked coverage. BFL Flux 2 Dev has also passed
-live authentication, cost preflight, paid single- and two-output generation,
-human review, PNG download, and provider-backed recovery. Start every untested
-model with one disposable asset and a small ceiling. This integration remains
-experimental because Scenario model schemas vary.
+project-specific models and third-party image models. PixelKiln supports still
+PNG generation through Scenario's universal model endpoint, free Compute Unit
+preflight, asynchronous jobs, multi-output review, and refreshable asset
+downloads. It also uploads your images as references, so a style's
+`styleImages` and `image-to-image` revisions work, and a manifest can chain
+several Scenario models across styles with
+[`revision.fromStyle`](./REVISIONS.md#across-styles). The adapter handles
+still images only; audio models are reachable through the same account but not
+through PixelKiln.
+
+Every code path in the adapter has mocked coverage. BFL Flux 2 Dev passed live
+authentication, cost preflight, paid single- and two-output generation, human
+review, PNG download, and provider-backed recovery. A later evaluation ran ten
+more text-to-image models, six tool models, reference-image edits, multi-model
+chains, and a real game's art; its findings are on this page and its benchmark
+projects are listed in [Where the evidence is](#where-the-evidence-is). Start
+every untested model with one disposable asset and a small ceiling. This
+integration remains experimental because Scenario model schemas vary.
+
+The short version of what that evaluation found: price does not predict fit;
+large or illustrative art (landmarks, covers, flat backdrops) comes out well
+through render, Pixelate, cutout, and `refine`; small native-size sprites, tiles,
+and characters do not; and a reference image keeps an existing design.
 
 ## Requirements
 
@@ -283,6 +296,39 @@ grid giving the target width, produced a convincing pixel scene with 31 colours.
 It renders legible title text, which PixelLab does not, and the text survives
 Pixelate. Before pixelating it is a painterly illustration, not pixel art.
 
+A [structures pilot](../benchmarks/provider-scenario-game-pilot/README.md#structures-pilot-90-cu)
+repeated this on five more buildings from five biomes at 8 CU each (18 CU with GPT
+Image 2): all seven chains landed on the sprite's exact size at high confidence
+with 63 colours and far more detail. Treat each result as a new design: the
+building changed style, the footprint drifted by up to 25%, Klein baked a ground patch
+under most (GPT did not), and a thin lattice tower at 128 px came out noisy. Asking for
+twice the grid worked as an image but gave only medium confidence.
+
+To keep a building's own design, send the existing sprite as a reference and ask for
+the same design redrawn in finer detail (a placed `source` asset as the parent of an
+`image-to-image` revision, the sprite upscaled 4× on white). Against the current
+sprites, silhouette overlap rose from 0.39 to 0.66 with text alone to 0.93 to 1.00
+with a Klein reference (0.77 and 0.81 with GPT Image 2), and the footprint matched.
+The result is the same sprite with finer texture and shading, not a new look and not a
+bigger one. Klein shifted colours; GPT held them better at a lower silhouette match.
+
+**Backdrops** (wide, flat landform strips) need the prompt to say so. The game's
+original prompt, which ends "isometric 2:1 perspective", came back as a ridge and a
+treeline lying on a diagonal ground plane. A prompt with its isometric wording
+removed and "flat side-on front-facing elevation view like a stage backdrop strip,
+perfectly horizontal composition spanning the full width of the image edge to
+edge, NOT isometric, no three-quarter view, no diagonal ground plane, the base
+line straight and horizontal and the form fading to nothing at its foot, isolated
+on a plain white background, no sky" gave a front-facing horizontal band for an
+alpine ridge and a swamp treeline. At a 1024×448 render with `pixelGridSize: 128`
+(a 256-cell grid on that 1024 px input) both came back as the game's exact
+256×112 grid at high confidence, 45 to 47 colours. The 1 CU Klein alpine ridge
+was a cleaner single massif than the 11 CU GPT one, which was a wide band of many
+peaks. Birefnet did not handle the fade into mist: it left a speckled grey fringe
+along the foot where the form dissolves into the white background, so a fade
+needs another route (keying luminance to alpha locally, or rendering on a known
+tint).
+
 Do not use this for small native-size props. A winter pine and an autumn tree at
 32 px came back as different trees, and Birefnet left holes in the canopy. A
 free recolour of the original, or a PixelLab generation at the native size, is
@@ -338,6 +384,57 @@ and post steps beside the file.
 Music models (ACE-Step, MusicGen, ElevenLabs Music, Lyria 3.5) quoted 10 to 30
 CU; Sonilo and Lyria 3 Pro need the `cu-pro-q3-25` plan. Music was not generated.
 
+### Calling the API directly: the dry run and the legacy endpoint
+
+Two facts for anyone calling Scenario outside PixelKiln, because both cost compute
+units to find:
+
+- **The free dry run is only the query parameter** `?dryRun=true`. A `dryRun: true`
+  in the JSON body is ignored, and the request submits and bills a real job. A probe of
+  the legacy `/generate/txt2img` endpoint made that mistake and billed 10 CU for a
+  FLUX.1 LoRA image, and jobs there cannot be cancelled ("Cannot cancel this type of
+  job"). The adapter always sends the query parameter.
+- **LoRA models do not use `/generate/custom/{modelId}`.** A public pixel-art LoRA
+  (Flux Retro Aesthetics) answered "Custom models only are supported for this
+  endpoint" there. The older `/generate/txt2img` endpoint accepted it at 10 CU an
+  image, returned a JPEG (which the adapter refuses), and is absent from the current
+  API reference. The adapter does not call it, and the same-quality Klein render costs
+  1 CU.
+
+### Budget and billing notes
+
+- **The adapter cannot read your balance.** `gen` enforces the budget you pass
+  (`scenario does not expose an account balance; enforcing its run budget`) and
+  the per-style `maxComputeUnits`, but the remaining balance is only on Scenario's
+  dashboard. Whether the allowance renews depends on the plan and billing
+  setup; a dashboard showing "No monthly reset scheduled" means it may not.
+- **Check the dashboard against your own tally.** In our evaluation a hand total
+  of quoted and billed amounts came to about 715 CU while the dashboard's balance
+  had fallen by 753 CU, about 5% more. The cause was not identified, so budget with a
+  margin.
+- **Quotes were exact where the adapter made them.** Every quote the adapter took
+  matched the billed amount in the lockfile, including failed assets that the
+  adapter refused after billing (a JPEG or WebP result is billed even though no
+  file lands).
+- **Flat prices.** GPT Image 2 `quality: medium` was 11 CU at any canvas size (`high`
+  44), ElevenLabs sound effects 30 CU at any length, and tool models 2 to 10 CU.
+
+### Where the evidence is
+
+Each is a committed project with its manifest, lockfile, outputs, and README:
+
+| Project | What it ran | CU |
+|---|---|---:|
+| `benchmarks/provider-scenario-smoke` | The first live run: FLUX.2 Dev, one and two outputs, recovery | 48 |
+| `benchmarks/provider-scenario-bakeoff` | Ten models on one brief | 33 |
+| `benchmarks/provider-scenario-edit` | One reference-image edit | 1 |
+| `benchmarks/provider-scenario-tools` | Birefnet, Pixel Snapper, Pixelate, three upscalers, and the refine gate | 34 |
+| `benchmarks/provider-scenario-pipeline` | Generate, pixelate, cut out, refine a character, with a sweep of Pixelate grid sizes | 41 |
+| `benchmarks/provider-scenario-chain` | The same chain declared with `revision.fromStyle` and run by one `gen` | 8 |
+| `benchmarks/provider-scenario-game-pilot` | A real game's landmarks, covers, backdrops, structures, and sound effects (settings and measurements only; the game's art is not included) | about 630 |
+
+Scratch projects for the game pilot are not committed, so its README is the record.
+
 ## Plan before spending
 
 Scenario prices depend on model, size, steps, and output count, so PixelKiln
@@ -490,10 +587,13 @@ quote, final billing, selected asset ID, output index, and output hash without
 retaining credentials or signed query strings. Once cached, `restore` prefers
 the validated local content cache.
 
-Scenario account listing, adoption, salvage, tagging, deletion, balance, image
-uploads, training, background removal, and upscaling are not part of this first
-adapter. PixelKiln reports unavailable account commands instead of pretending
-they succeeded.
+Scenario account listing, adoption, salvage, tagging, deletion, balance, and
+training are not part of the adapter. Image uploads exist only to send
+references (see [Reference images and edits](#reference-images-and-edits)), and
+background removal, pixel snapping, and upscaling run as ordinary models through
+`image-to-image` revisions ([Tool models](#tool-models-as-image-to-image)).
+PixelKiln reports unavailable account commands instead of pretending they
+succeeded.
 
 ## Mixed-provider projects
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pixelkiln",
-  "version": "0.86.1",
+  "version": "0.86.2",
   "description": "Manifest-driven pixel-art generation, review, recovery, and packaging with deterministic provenance.",
   "type": "module",
   "sideEffects": false,

package/skills/pixelkiln/references/scenario.md

@@ -27,7 +27,7 @@ schemas remain unverified. Require a one-asset smoke before widening a batch.
   `referenceArray` for a model that names it differently. Other revision modes
   and `strength` are refused.
 
-Scenario model parameters vary. The MVP supplies prompt, 16–4096px dimensions
+Scenario model parameters vary. The adapter supplies prompt, 16–4096px dimensions
 (each model has tighter bounds, reported by the free dry run), one to four outputs, and optional seed. Put only documented
 model inputs in `parameters`; PixelKiln rejects attempts to override its prompt,
 dimensions, seed, output count, project routing, or CU ceiling.
@@ -45,5 +45,14 @@ See `docs/SCENARIO.md` for settings and measurements.
 PixelKiln validates the job, bytes, hashes, and recovery path. It does not prove
 the output is good pixel art. Inspect one result at 1× before approving a batch.
 
+When calling Scenario outside PixelKiln (audio, say), the free dry run is only the
+query parameter `?dryRun=true`: a `dryRun: true` in the JSON body is ignored and
+submits and bills a real job. LoRA models need the legacy `/generate/txt2img`
+endpoint (10 CU, JPEG output, not callable by the adapter). The adapter cannot read
+the CU balance; check the dashboard and budget with a margin (a hand tally ran about
+5% under the dashboard). A reference image keeps an existing design (silhouette
+overlap 0.93 to 1.00 against 0.39 to 0.66 from text alone), though some models shift
+its colours; use it before a text-only prompt when the look must be kept.
+
 See `docs/SCENARIO.md` for setup, manifest examples, unsupported account
 operations, and the live validation checklist.