@oriro/orirocli 0.1.11 → 0.1.12

package/skills/craft/image-generation-engineer/SKILL.md

@@ -1,183 +1,183 @@
----
-watermark: ORIRO
-disable-model-invocation: true
-name: image-generation-engineer
-provider: ORIRO.ai
-copyright: Copyright (c) 2026 ORIRO.ai
-description: >
-  World-class image-generation engineering — build, train, fine-tune, serve, and improve any
-  text-to-image / diffusion system. Activate for ANY task involving: diffusion models (DDPM,
-  DDIM, flow matching, rectified flow), architectures (U-Net, DiT, MM-DiT, latent diffusion),
-  FLUX.1 / FLUX.2 (schnell / dev / Kontext), Stable Diffusion / SDXL / SD3.5, Z-Image, VAE,
-  text conditioning (CLIP, OpenCLIP, T5-XXL), LoRA / DreamBooth / ControlNet fine-tuning,
-  samplers (DDIM, DPM-Solver++, LCM, adversarial distillation), classifier-free guidance,
-  FP8 / GGUF / INT8 quantization for image models, FID / CLIP-score evaluation, serving image
-  models (diffusers, vLLM, BentoML, Modal H100, Cloud Run), Nano Banana / gemini-2.5-flash-image,
-  or building ORIRO's own image engine. Use whenever the user wants to generate images, train or
-  fine-tune an image model, choose an image model, speed up image inference, evaluate image
-  quality, or deploy text-to-image — even if they never say "diffusion" or "FLUX". Last
-  researched: June 3 2026.
-metadata:
-  type: reference
----
-
-# Image Generation Engineer
-
-How to build, train, serve, and improve any image-generation system — from the diffusion math up
-to production serving. This skill is the standing reference for every visual-generative task on
-<user>'s projects. Consult it before acting; load the relevant `references/` file for depth.
-
-The guiding principle: **don't reinvent what ships today.** Most tasks start by _using_ an existing
-open model (FLUX.1 [schnell], Apache 2.0) or the user's own API key (Nano Banana / Gemini), and only
-descend into custom architecture/training when there's a real reason. The decision tree below routes
-you; the reference files hold the depth.
-
----
-
-## PART 0 — QUICK DECISION TREE (read this first, every time)
-
-```
-WHAT ARE YOU BUILDING?
-│
-├── Need images NOW, no training budget?
-│   → FLUX.1 [schnell] (Apache 2.0, 12B, 1-4 steps, commercial OK)
-│   → Serve via diffusers + vLLM or BentoML
-│
-├── Need to fine-tune an existing model?
-│   → LoRA on FLUX.1 [dev] (fastest, cheapest, best quality)
-│   → DreamBooth for concept injection
-│   → ControlNet adapter for structural conditioning
-│
-├── Building from scratch (research / custom architecture)?
-│   → Start: DDPM on small dataset (MNIST/CIFAR) — understand the math
-│   → Scale: Latent Diffusion (VAE + U-Net) on real images
-│   → Advance: DiT (replace U-Net with Transformer)
-│   → State of art: Flow Matching + MM-DiT (FLUX/SD3 architecture)
-│
-├── Need text conditioning?
-│   → CLIP text encoder (OpenCLIP ViT-L/14, Apache 2.0)
-│   → T5-XXL for nuanced language (SD3/FLUX use both CLIP+T5)
-│   → OpenCLIP BigG/14 for highest fidelity
-│
-├── Need inference speed?
-│   → DDIM sampler (50→10 steps, same quality)
-│   → LCM distillation (4-8 steps)
-│   → Latent adversarial distillation (1-4 steps, FLUX [schnell])
-│   → FP8 quantization (2× speed, half memory, negligible quality loss)
-│
-└── Need to serve at scale?
-    → vLLM 0.17.1 (continuous batching, FP8, multi-GPU)
-    → BentoML (unified API, auto-scaling)
-    → Modal H100 (per-request GPU, zero idle cost)
-```
-
----
-
-## Reference map — where the depth lives
-
-Load the file that matches the task. Each is self-contained; don't read them all up front.
-
-| File                                  | Covers                                                                                                                                                                                     | Read when                                                                                                       |
-| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
-| `references/foundations.md`           | Diffusion math: forward/reverse process, noise schedules, DDIM, flow matching / rectified flow, classifier-free guidance                                                                   | You're implementing the training objective, a sampler, or debugging why generations are noisy/blurry/off-prompt |
-| `references/architectures.md`         | U-Net, DiT, MM-DiT; VAE / latent space; CLIP & T5 text conditioning (single + dual encoder)                                                                                                | You're choosing or building a model backbone, a VAE, or a text-conditioning stack                               |
-| `references/training.md`              | Data pipeline, full training loop + config, LoRA fine-tuning, ControlNet                                                                                                                   | You're training or fine-tuning anything                                                                         |
-| `references/inference-and-serving.md` | Sampler comparison, quantization (FP8/GGUF/INT8), full inference pipeline, FID/CLIP eval, serving stacks (Modal, Cloud Run, vLLM, BentoML), open-model reference table, diffusers patterns | You're optimizing inference, evaluating quality, or deploying                                                   |
-
-For the deep math/architecture detail the decision tree references (e.g. the exact DDIM update, MM-DiT block, VAE loss), the canonical text lives in those files — pull it in rather than reconstructing from memory.
-
----
-
-## ORIRO build plan — Use → Adapt → Build → Own
-
-ORIRO's image strategy is phased so we ship value immediately at $0 and only take on training cost
-when it pays for itself. Honor the OR-NO-TOKENS / zero-cost-to-ORIRO rule throughout: user BYOK and
-open Apache-2.0 models first, ORIRO-paid keys never.
-
-```
-Phase 1 — USE (ship NOW):
-  Nano Banana via the USER'S Google AI Studio API key. $0 to ORIRO, 500 free/day, immediate.
-  Model: gemini-2.5-flash-image. No training, no infra — just proxy the user's key.
-
-Phase 2 — ADAPT (months 3-6):
-  Fine-tune FLUX.1 [schnell] (Apache 2.0) on ORIRO style via LoRA on Modal H100 (~$50/run).
-  Custom styles: ORIRO aesthetic, UI mockups, avatars. Serve on Modal, $0.001-0.003/image.
-
-Phase 3 — BUILD (months 6-12):
-  Train a small DiT (~500M params) on a curated dataset. Specialize: UI/UX mockups, product
-  design, avatars. ORIRO-specific fine-tune on ORIRO user outputs (DPO flywheel). Host on
-  Cloud Run or Modal.
-
-Phase 4 — OWN (year 2+):
-  Full custom LDM or flow-matching model on ORIRO data + licensed public data. Independent of
-  any external provider. Image generation becomes a product line.
-```
-
-**Minimum viable engine (build now):** an `image` route where the user supplies their own Google AI
-Studio key and ORIRO proxies the request (never ORIRO's billing). Enhance prompts with a style map:
-
-```
-photorealistic → "photorealistic, 8K, professional photography"
-illustration   → "digital illustration, clean lines, vibrant colors"
-ui-mockup      → "clean UI mockup, Figma style, modern design system"
-icon           → "app icon, clean vector style, 1024×1024"
-game-asset     → "game asset, pixel art / 3D render, game-ready"
-enhanced_prompt = f"{prompt}, {STYLE_PROMPTS[style]}"
-```
-
-### Quality bar — do not ship below these
-
-For ANY ORIRO image engine, measure and gate on:
-
-- **FID ≤ 30** (acceptable for production; ≤10 excellent, matches real images)
-- **CLIP Score ≥ 0.28** (adequate text-image alignment)
-- **User rating ≥ 4.0/5.0** (in-app ratings)
-- **Generation speed ≤ 10s** (acceptable UX)
-- **Cost ≤ $0.01/image** (sustainable economics)
-
-Below these thresholds: do not ship to production.
-
----
-
-## PART 10 — COMMON MISTAKES (and how to avoid them)
-
-These are the high-frequency failure modes. Most "my model is blurry / off-prompt / OOMing" reports
-trace back to one of these — check here before deep debugging.
-
-```
-MISTAKE 1: Training in pixel space for large images
-  FIX: Always use latent diffusion (VAE encoder first).  RULE: resolution > 128×128 → latent space.
-
-MISTAKE 2: Linear noise schedule for high resolution
-  FIX: cosine (standard) or sigmoid (best for 512+) schedule for 256+ px.
-
-MISTAKE 3: Only MSE loss for the VAE
-  FIX: add perceptual (LPIPS) + adversarial loss.  Pure MSE → blurry; perceptual → sharp edges.
-
-MISTAKE 4: Forgetting CFG during inference
-  FIX: guidance_scale 5-15 for text-to-image.  EXCEPTION: flow-matching models (FLUX [schnell]) = 0.
-
-MISTAKE 5: Attention-only LoRA targets
-  FIX: include FFN modules. q,k,v,o + gate_proj/up_proj/down_proj (or ff.net.*) = complete.
-
-MISTAKE 6: Not normalizing images to [-1, 1]
-  FIX: T.Normalize([0.5,0.5,0.5],[0.5,0.5,0.5]).  Models expect [-1,1], not [0,1].
-
-MISTAKE 7: FP32 for training large models
-  FIX: BF16 mixed precision — bfloat16 is more stable than fp16 (which overflows) and 2× faster.
-
-MISTAKE 8: No EMA during training
-  FIX: keep an exponential moving average of weights (decay 0.9999). EMA weights = better at inference.
-
-MISTAKE 9: Fixed-resolution dataset
-  FIX: multi-aspect-ratio bucketing (SDXL technique). Bucket by AR → no padding → ~20% faster training.
-
-MISTAKE 10: Evaluating only on the training distribution
-  FIX: hold-out test set + out-of-distribution prompts. FID on held-out + CLIP score on diverse prompts.
-```
-
----
-
-_Researched June 3 2026. Sources: arxiv (DDPM/DDIM/DiT/FLUX papers), HuggingFace Diffusers docs,
-Black Forest Labs FLUX repo, ICLR 2026 architecture-evolution blogpost, BentoML model guide, OpenCLIP
-GitHub. Update this skill whenever a new SOTA architecture ships._
+---
+watermark: ORIRO
+disable-model-invocation: true
+name: image-generation-engineer
+provider: ORIRO.ai
+copyright: Copyright (c) 2026 ORIRO.ai
+description: >
+  World-class image-generation engineering — build, train, fine-tune, serve, and improve any
+  text-to-image / diffusion system. Activate for ANY task involving: diffusion models (DDPM,
+  DDIM, flow matching, rectified flow), architectures (U-Net, DiT, MM-DiT, latent diffusion),
+  FLUX.1 / FLUX.2 (schnell / dev / Kontext), Stable Diffusion / SDXL / SD3.5, Z-Image, VAE,
+  text conditioning (CLIP, OpenCLIP, T5-XXL), LoRA / DreamBooth / ControlNet fine-tuning,
+  samplers (DDIM, DPM-Solver++, LCM, adversarial distillation), classifier-free guidance,
+  FP8 / GGUF / INT8 quantization for image models, FID / CLIP-score evaluation, serving image
+  models (diffusers, vLLM, BentoML, Modal H100, Cloud Run), Nano Banana / gemini-2.5-flash-image,
+  or building ORIRO's own image engine. Use whenever the user wants to generate images, train or
+  fine-tune an image model, choose an image model, speed up image inference, evaluate image
+  quality, or deploy text-to-image — even if they never say "diffusion" or "FLUX". Last
+  researched: June 3 2026.
+metadata:
+  type: reference
+---
+
+# Image Generation Engineer
+
+How to build, train, serve, and improve any image-generation system — from the diffusion math up
+to production serving. This skill is the standing reference for every visual-generative task on
+<user>'s projects. Consult it before acting; load the relevant `references/` file for depth.
+
+The guiding principle: **don't reinvent what ships today.** Most tasks start by _using_ an existing
+open model (FLUX.1 [schnell], Apache 2.0) or the user's own API key (Nano Banana / Gemini), and only
+descend into custom architecture/training when there's a real reason. The decision tree below routes
+you; the reference files hold the depth.
+
+---
+
+## PART 0 — QUICK DECISION TREE (read this first, every time)
+
+```
+WHAT ARE YOU BUILDING?
+│
+├── Need images NOW, no training budget?
+│   → FLUX.1 [schnell] (Apache 2.0, 12B, 1-4 steps, commercial OK)
+│   → Serve via diffusers + vLLM or BentoML
+│
+├── Need to fine-tune an existing model?
+│   → LoRA on FLUX.1 [dev] (fastest, cheapest, best quality)
+│   → DreamBooth for concept injection
+│   → ControlNet adapter for structural conditioning
+│
+├── Building from scratch (research / custom architecture)?
+│   → Start: DDPM on small dataset (MNIST/CIFAR) — understand the math
+│   → Scale: Latent Diffusion (VAE + U-Net) on real images
+│   → Advance: DiT (replace U-Net with Transformer)
+│   → State of art: Flow Matching + MM-DiT (FLUX/SD3 architecture)
+│
+├── Need text conditioning?
+│   → CLIP text encoder (OpenCLIP ViT-L/14, Apache 2.0)
+│   → T5-XXL for nuanced language (SD3/FLUX use both CLIP+T5)
+│   → OpenCLIP BigG/14 for highest fidelity
+│
+├── Need inference speed?
+│   → DDIM sampler (50→10 steps, same quality)
+│   → LCM distillation (4-8 steps)
+│   → Latent adversarial distillation (1-4 steps, FLUX [schnell])
+│   → FP8 quantization (2× speed, half memory, negligible quality loss)
+│
+└── Need to serve at scale?
+    → vLLM 0.17.1 (continuous batching, FP8, multi-GPU)
+    → BentoML (unified API, auto-scaling)
+    → Modal H100 (per-request GPU, zero idle cost)
+```
+
+---
+
+## Reference map — where the depth lives
+
+Load the file that matches the task. Each is self-contained; don't read them all up front.
+
+| File                                  | Covers                                                                                                                                                                                     | Read when                                                                                                       |
+| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
+| `references/foundations.md`           | Diffusion math: forward/reverse process, noise schedules, DDIM, flow matching / rectified flow, classifier-free guidance                                                                   | You're implementing the training objective, a sampler, or debugging why generations are noisy/blurry/off-prompt |
+| `references/architectures.md`         | U-Net, DiT, MM-DiT; VAE / latent space; CLIP & T5 text conditioning (single + dual encoder)                                                                                                | You're choosing or building a model backbone, a VAE, or a text-conditioning stack                               |
+| `references/training.md`              | Data pipeline, full training loop + config, LoRA fine-tuning, ControlNet                                                                                                                   | You're training or fine-tuning anything                                                                         |
+| `references/inference-and-serving.md` | Sampler comparison, quantization (FP8/GGUF/INT8), full inference pipeline, FID/CLIP eval, serving stacks (Modal, Cloud Run, vLLM, BentoML), open-model reference table, diffusers patterns | You're optimizing inference, evaluating quality, or deploying                                                   |
+
+For the deep math/architecture detail the decision tree references (e.g. the exact DDIM update, MM-DiT block, VAE loss), the canonical text lives in those files — pull it in rather than reconstructing from memory.
+
+---
+
+## ORIRO build plan — Use → Adapt → Build → Own
+
+ORIRO's image strategy is phased so we ship value immediately at $0 and only take on training cost
+when it pays for itself. Honor the OR-NO-TOKENS / zero-cost-to-ORIRO rule throughout: user BYOK and
+open Apache-2.0 models first, ORIRO-paid keys never.
+
+```
+Phase 1 — USE (ship NOW):
+  Nano Banana via the USER'S Google AI Studio API key. $0 to ORIRO, 500 free/day, immediate.
+  Model: gemini-2.5-flash-image. No training, no infra — just proxy the user's key.
+
+Phase 2 — ADAPT (months 3-6):
+  Fine-tune FLUX.1 [schnell] (Apache 2.0) on ORIRO style via LoRA on Modal H100 (~$50/run).
+  Custom styles: ORIRO aesthetic, UI mockups, avatars. Serve on Modal, $0.001-0.003/image.
+
+Phase 3 — BUILD (months 6-12):
+  Train a small DiT (~500M params) on a curated dataset. Specialize: UI/UX mockups, product
+  design, avatars. ORIRO-specific fine-tune on ORIRO user outputs (DPO flywheel). Host on
+  Cloud Run or Modal.
+
+Phase 4 — OWN (year 2+):
+  Full custom LDM or flow-matching model on ORIRO data + licensed public data. Independent of
+  any external provider. Image generation becomes a product line.
+```
+
+**Minimum viable engine (build now):** an `image` route where the user supplies their own Google AI
+Studio key and ORIRO proxies the request (never ORIRO's billing). Enhance prompts with a style map:
+
+```
+photorealistic → "photorealistic, 8K, professional photography"
+illustration   → "digital illustration, clean lines, vibrant colors"
+ui-mockup      → "clean UI mockup, Figma style, modern design system"
+icon           → "app icon, clean vector style, 1024×1024"
+game-asset     → "game asset, pixel art / 3D render, game-ready"
+enhanced_prompt = f"{prompt}, {STYLE_PROMPTS[style]}"
+```
+
+### Quality bar — do not ship below these
+
+For ANY ORIRO image engine, measure and gate on:
+
+- **FID ≤ 30** (acceptable for production; ≤10 excellent, matches real images)
+- **CLIP Score ≥ 0.28** (adequate text-image alignment)
+- **User rating ≥ 4.0/5.0** (in-app ratings)
+- **Generation speed ≤ 10s** (acceptable UX)
+- **Cost ≤ $0.01/image** (sustainable economics)
+
+Below these thresholds: do not ship to production.
+
+---
+
+## PART 10 — COMMON MISTAKES (and how to avoid them)
+
+These are the high-frequency failure modes. Most "my model is blurry / off-prompt / OOMing" reports
+trace back to one of these — check here before deep debugging.
+
+```
+MISTAKE 1: Training in pixel space for large images
+  FIX: Always use latent diffusion (VAE encoder first).  RULE: resolution > 128×128 → latent space.
+
+MISTAKE 2: Linear noise schedule for high resolution
+  FIX: cosine (standard) or sigmoid (best for 512+) schedule for 256+ px.
+
+MISTAKE 3: Only MSE loss for the VAE
+  FIX: add perceptual (LPIPS) + adversarial loss.  Pure MSE → blurry; perceptual → sharp edges.
+
+MISTAKE 4: Forgetting CFG during inference
+  FIX: guidance_scale 5-15 for text-to-image.  EXCEPTION: flow-matching models (FLUX [schnell]) = 0.
+
+MISTAKE 5: Attention-only LoRA targets
+  FIX: include FFN modules. q,k,v,o + gate_proj/up_proj/down_proj (or ff.net.*) = complete.
+
+MISTAKE 6: Not normalizing images to [-1, 1]
+  FIX: T.Normalize([0.5,0.5,0.5],[0.5,0.5,0.5]).  Models expect [-1,1], not [0,1].
+
+MISTAKE 7: FP32 for training large models
+  FIX: BF16 mixed precision — bfloat16 is more stable than fp16 (which overflows) and 2× faster.
+
+MISTAKE 8: No EMA during training
+  FIX: keep an exponential moving average of weights (decay 0.9999). EMA weights = better at inference.
+
+MISTAKE 9: Fixed-resolution dataset
+  FIX: multi-aspect-ratio bucketing (SDXL technique). Bucket by AR → no padding → ~20% faster training.
+
+MISTAKE 10: Evaluating only on the training distribution
+  FIX: hold-out test set + out-of-distribution prompts. FID on held-out + CLIP score on diverse prompts.
+```
+
+---
+
+_Researched June 3 2026. Sources: arxiv (DDPM/DDIM/DiT/FLUX papers), HuggingFace Diffusers docs,
+Black Forest Labs FLUX repo, ICLR 2026 architecture-evolution blogpost, BentoML model guide, OpenCLIP
+GitHub. Update this skill whenever a new SOTA architecture ships._