@m0saic/knowledge 0.2.0

package/docs/templates/construction-strategy.md

@@ -0,0 +1,329 @@
+# Skill: Template Construction Strategy (Human-Friendly vs Engine-Native)
+
+How to decide between human-readable layout and engine-native power, choose splits
+vs overlays, and build templates that are stable, predictable, and extensible.
+Not every valid DSL construct is ideal for every template.
+
+> **Source of truth:** Existing templates in [https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src/m0saic](https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src/m0saic) — categories (2026-07-26): `alpine`, `benchmark`, `brand`, `charts`, `code`, `collage`, `demos`, `dsl-tutorial`, `forensic`, `github`, `hero`, `language`, `media`, `meta`, `primitives`, `science`, `social`, `theming`, `wireframe` (`ls` the folder — this list drifts). `charts/bar-graph/v2` is the canonical example of programmatic construction.
+
+Companion docs: the placement recipes are
+[`geometry-recipes.md`](geometry-recipes.md); layout-intent verification is
+[`layout-contract.md`](layout-contract.md).
+
+## Two construction styles
+
+**Human-friendly layout style** — uses `N(...)` / `N[...]` splits directly,
+`1{...}` or group overlays, avoids complex `0`-run overlay staging, keeps the
+string short and readable:
+
+    2(
+      3[1,1,1],
+      3[1,1,1]
+    )
+
+Ideal for hand-written templates, teaching examples, simple grids, standard UI
+layouts.
+
+**Engine-native composition style** — heavy overlay stacking, rect-capture
+overlays, deep nesting, generated programmatically, possibly thousands of
+characters, geometry derived algorithmically. Ideal for dictionary-generated
+layouts, glyph rendering, debug visualizers, signal-driven demos, high-precision
+rectangle emission.
+
+Choose engine-native only when demonstrating DSL power, encoding rect sets
+programmatically, building art/glyph systems, or constructing debug tools.
+**"Data visualizations" is NOT a blanket license for overlay-stack style.** A
+chart's chrome (title, axis labels, ticks, legend, plot frame) is *real
+geometry* — see the Hard Rule below. Only the data mark that needs per-frame
+animation (a line draw-on, a count-up) is exempt and lives as an overlay.
+
+## Choosing between splits and overlays
+
+Use **splits** when defining structural geometry, siblings need shared partition
+logic, the layout is conceptually hierarchical, or you want stable structural
+identity. Use **overlays** when layering paint, regions are independent, you are
+composing rect instructions, you want depth isolation, or you are building glyphs
+or masks. Splits define space; overlays define drawing.
+
+## Hard rule: real geometry first
+
+> **The Rect Thesis.** To render anything at known bounds: carve an m0 CELL sized
+> exactly to those bounds (splits / `weightedSplit` / `placeRects`, or nest a
+> primitive like `@m0saic/primitives/grid` for repeated thin rects), then **fill
+> it** (a lavfi color) or **mask it** (an inline-mask path). The cell is the
+> size+position; the source just fills the cell it lands in. This is the core of
+> m0saic — see [`../m0saic-thesis.md`](../m0saic-thesis.md).
+
+**If you know a region's rectangle at author time, encode it as a real m0 rect
+(splits / `weightedSplit` / `placeRects`) — NOT as a full-canvas source
+positioned by `drawtext` `xExpr`/`yExpr` or an inline mask.**
+
+A template's job is **JS pixel-math → real rectangles → m0 DSL geometry.** When an
+agent instead computes pixels and bakes them into full-frame drawtext/mask
+sources, the output "renders right" but the document *carries no geometry* — which
+defeats the structure preview, Render Hero (its whole job is to show the `{}`
+frame tree), the layout corpus, and the entire point of the DSL.
+
+- Splits / `placeRects` define **WHERE**; the source fills the cell it lands in.
+- Full-frame overlay stacks + in-source positioning are the **FALLBACK**, reserved
+  for content that genuinely cannot be a static rect — e.g. an animated line
+  sweep, or per-frame-varying geometry.
+- **Smell test:** if your `m0` is `buildOverlayStack(N)` with `N == element
+  count` and every source is full-canvas, you have pixel-math baked into
+  drawtext, not geometry. Preview + Render Hero will be empty/garbage. Rebuild
+  with real cells.
+- **Reuse first-party primitives** for standard chrome — gridlines are
+  `@m0saic/primitives/grid/v2` (v1 is DEPRECATED since 2026-07-07: its per-pixel
+  basis silently drops when nested — the layout contract's presence check is what
+  catches that class of failure, see
+  [`layout-contract.md`](layout-contract.md)); do not re-emit gridline rects by
+  hand.
+
+### The animation exemption is narrow — even animated pieces are tightly placed
+
+The "animated / per-frame-varying" exemption above is **NOT** a license for
+full-canvas overlays. A piece that animates (a line sliver, a marker, an area)
+still has a KNOWN bounding box — you generated its path, so you know its extent.
+A masked source whose `bounds` == the full canvas is the lie: it tells the
+document "this element is everywhere," which wrecks selection/hit-testing (one
+sliver's box swallows every click beneath it), Render Hero, and the layout
+corpus. **Complexity lives in the m0 string (geometry), not hidden inside
+full-frame masks.** Place each animated piece as a tight rect:
+
+1. Compute the piece's bbox (incl. stroke / radius + ~1px anti-alias pad).
+2. Snap the bbox **OUTWARD** to a bounded basis grid (≤200 cells/split) so the
+   cell ⊇ the shape and never clips it.
+3. `placeRects()` the pieces (input in paint order — greedy first-fit packs them
+   into the fewest overlay layers; areas under lines under points).
+4. Generate the mask path in **cell-local** coords (absolute − cell origin); set
+   the mask `bounds` to the cell's px size → `scaleX==scaleY==1`, pixel-exact, no
+   clip. Per-piece `overlay.alpha` (`fadeInExpr`) works the same inside a placed
+   cell as on a full-frame overlay.
+
+> Worked example: `charts/line-chart/v1/line-series.ts` (`placePiece`) — the
+> animated line/area/points place as tight bbox cells (quantized to the same
+> frame basis as the chrome), not full-canvas masked overlays. That template was
+> first authored as ~78 full-frame overlays (one per title/label/gridline)
+> positioned by `xExpr`, then rebuilt as a `weightedSplit` frame with only the
+> animated line left as an overlay. Stat-card (`weightedSplit` bands) is the
+> canonical small example.
+
+### The placement recipes
+
+How to make the placed rects composable — precision that does NOT track the
+canvas — is the recipe book at [`geometry-recipes.md`](geometry-recipes.md):
+
+- Recipe 1: `SNAP_PX` snap-outward for tight-mask cells (seams + GCD collapse).
+- Recipe 2: `placeInsetPieces` / `placeOptimizedPieces` for independent chrome.
+- Recipe 3: gutterless ratio grid + `latticeCellInset` for uniform grids
+  (incl. the `gridCellInset` → `latticeCellInset` migration).
+- Plus: the routing table, the feasibility/precision/quantization floors, and the
+  `placeRect` escape hatch for must-be-exact elements.
+- Recipe 4 (the lattice, `@m0saic/template-utils` `lattice/`): weighted bands go
+  through `weightedSplit(latticeWeights(weights, { cap: 120 }), axis, { claimants })`
+  — `latticeWeights` judges the GCD-reduced total, never emits more slots than
+  the input had, keeps symmetry (`[pad, mid, pad]`) and alternation
+  (items/gutters), bounds drift to max(2 %, 2 slots, 1 slot of the target) and
+  otherwise returns the input untouched so the convention reports it. **Never
+  round each weight to a cap on its own** — the sum lands on cap ± (bands − 1),
+  which is how the whole Alpine pack sat on 121. When an intent must register
+  with the cell a split will actually produce (a chip inset, a tick font cap, a
+  nested child's slot), predict it with `quantizedSections(totalPx, weights)` —
+  the parser's outside-in remainder, an EDGE section absorbs it first — and hand
+  children 5-smooth slots (`nearestSmooth`, `snapSlot`, or the hero's `insetSlot`
+  pattern that nudges the rect until the predicted cell is smooth). Self-framed
+  soups take the pitch from `divisors(side)` nearest `side/K` (`ceilToSmooth`
+  fallback on a rough side), never `round(side/K)` + `ceil`. The rule behind it
+  (`latticeSmooth`, throw) and the `lattice` declarations:
+  [`reference/template-flags.md`](reference/template-flags.md).
+
+### Don't size/position with `size` expressions — carve the cell
+
+To make an element small/short (a tick, a rule, a swatch, a badge), DON'T keep
+the source full-tile and shrink it with `fitMode:"content"` + `size:{wExpr,hExpr}`
++ `placement:{fit:"contain"}`. `fit` is only `contain`/`cover` — both **scale**:
+a source small in BOTH axes upscales to fill its tile (blocks / distortion). It
+only "stays small" when one dimension is already full (so a full-height 1px rule
+via `wExpr:"max(1,TW*f)", hExpr:"TH"` is the one acceptable size-expr case).
+
+**Carve the cell instead.** Geometry decides size+position; the source fills.
+
+- **Smell test:** writing `size:{wExpr,hExpr}` to make something SMALL in both
+  axes → you're scaling, not placing. Carve a rect.
+- Worked example: line-chart **tick marks**. First attempt sized stubs with
+  size-exprs → rendered as full-cell blocks (contain upscaled them). Fix: carve a
+  narrow RAIL cell beside the labels and nest a short `grid` — its thin
+  line-cells ARE the tick rects, exact and aligned. (`chrome.ts`)
+
+### Solid-colour tile sources — use `makeColorTile`
+
+`@m0saic/template-utils` exports `makeColorTile(color, opts?)` as the shared factory for "paint this cell with a solid colour" sources (https://github.com/m0saic-project/m0saic-packages/blob/main/packages/template-utils/src/sources/makeColorTile.ts#L52). Adoption is now broad — `brand/logo/v1`/`v2`/`v3`, the whole alpine pack (via `alpine/_shared/alpine-card.ts`), and ~20 other templates use it. New templates should follow the same convention so the engine pipeline stays uniform. (Grep `makeColorTile` in https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src for the live consumer list.)
+
+```ts
+import { makeColorTile } from "@m0saic/template-utils";
+
+makeColorTile("#050314");
+makeColorTile(color, { overlay: { alpha, enable, xExpr, yExpr, startAtSec, blendMode } });
+makeColorTile(color, { mask });
+makeColorTile(color, { overlay: { enable }, mask });
+```
+
+Returns a `MosaicLavfiSource` using ffmpeg's `color=` generator. Prefer this over hand-rolling a `MosaicTextSource` with empty literal text + `visual.backgroundColor`:
+
+- `color=` is essentially free per cell — no rasterized intermediate to share, so N independent tiles cost no more than one shared source with N refs would (and refs to lavfi at nested depth aren't wired in v1 anyway — see `rendering-model-contract.md` Rule 9's ref-source note).
+- Lavfi sources are valid `MosaicRefSource` targets at root depth without needing the `renderMode: { kind: "image" }` escape hatch.
+- Single API across templates — no per-template variants of `makeSolidColorTile` to drift.
+
+Verify the export with: `grep -n "export function makeColorTile" https://github.com/m0saic-project/m0saic-packages/blob/main/packages/template-utils/src/sources/makeColorTile.ts.
+
+### Drawing shapes: SVG inline-mask on a **color** source
+
+The Rect Thesis above says a cell is either *filled* or *masked*. This is how to mask
+one — arrows, triangles, badge silhouettes, any non-rect shape.
+
+**Two wrong reaches, in order of temptation:**
+
+1. **A drawtext glyph** (`▲` / `▼` / an emoji). ffmpeg silently drops these as tofu.
+   Nothing errors; the shape is just gone.
+2. **A `text` source with a `backgroundColor` rect + an inline-mask.** This *works*,
+   but routes a shape through the text path and distorts easily.
+
+**Do this instead** — a lavfi colour source (via `makeColorTile` above), masked by
+an SVG path:
+
+```ts
+makeColorTile(color, { mask: { kind: "inline-mask", localPath, bounds } })
+```
+
+No drawtext involvement, and it renders in **both** the engine and the Make preview
+(LavfiTile applies masks). m0 controls the rect; SVG draws the shape into it. ffmpeg is
+excellent at *composition* and the wrong tool for *drawing* — SVG is the specialist.
+
+#### ⚠️ The load-bearing rule: `bounds` aspect MUST match the cell's aspect
+
+The engine scales the authored path to the tile:
+
+```
+scaleX = tileW / bounds.width
+scaleY = tileH / bounds.height
+```
+
+Those are computed **independently**. If `bounds` aspect ≠ tile aspect the scale is
+**anisotropic** and the shape distorts — a right angle stops being a right angle. Author
+the path in the cell's actual pixel space (or any matching ratio) so `scaleX === scaleY`.
+Get that right and the shape is undistorted *and* crisp at any resolution: sharp
+rasterizes at tile res, so it stays vector-clean.
+
+This one rule is the whole difference between a crisp triangle and a smeared one, and
+it fails silently — the render succeeds, it just looks wrong.
+
+#### Two supporting habits
+
+- **Keep the rect's split coarse.** Sizing the shape's cell with pixel-exact
+  `weightedSplit` weights can push the literal split past ~200 cells and hit
+  engine split-rounding. Percent-ish weights are enough — the mask does the precision.
+- **Author variants in the template, select by prop.** Up / down / neutral arrows are
+  three authored paths; the template picks one from a `direction` prop. Don't compute
+  path geometry at render time.
+
+#### Check the shared helpers before authoring a path
+
+`@m0saic/dsl-stdlib` ships a small shape library at
+https://github.com/m0saic-dsl/m0/blob/main/packages/dsl-stdlib/src/libraries/mask-shapes — each returns a ready `M0cMaskEntry`
+(path + correctly-authored `bounds`, so the aspect rule above is handled for you):
+
+| Helper | Signature |
+|---|---|
+| `circleMask` | `(width, height = width)` |
+| `roundedRectMask` | `(width, height, radius…)` |
+| `pillMask` | `(width, height)` |
+
+**Prefer these over a hand-authored path** — they're where the bounds discipline lives.
+
+There is **no `triangleMask`** yet; `charts/stat-card/v1` authors its arrow paths inline
+and leaves a `// (Future: a triangleMask helper in dsl-stdlib/mask-shapes.)` marker at
+`stat-card.ts:300`. If you need a second triangle, add the helper rather than
+copy-pasting the path — that's the signal the library is missing one.
+
+### Canvas / card backgrounds: prefer `document.backgroundColor`
+
+For a full-frame background fill, set `backgroundColor` on the returned
+`MosaicDocument` — **don't** add a dedicated base layer via a `1{...}` overlay wrapper.
+
+A base-layer overlay makes the doc **two layers** (L0 = surface, L1 = content). The
+engine renders that correctly, but the Make-page preview then has to composite the
+upper layer over the lower one, and **that compositing is best-effort** — it can show
+black, or a stray cell background, where the lower layer should show through the
+upper's transparent areas. The render is right and the preview lies, which is the worst
+failure mode: the preview is the surface you and the human review on.
+
+`document.backgroundColor` keeps the card a **single DSL layer**. The engine fills the
+canvas wherever sources don't cover; the preview paints the same colour straight onto
+`.gp-canvas` with no layer compositing. Render and preview agree, and the m0 is lighter
+(no wrapper, no base tile source).
+
+**Reach for a base tile only when `backgroundColor` can't express it** — rounded
+corners, a stroke, per-region fills. It's a flat rect fill, nothing more.
+
+**Two adjacent things, easily confused:**
+
+- **`doc.backgroundImage`** (`MosaicBackgroundImage`) is the sibling for a *baked image*
+  background — bottom layer, beneath every source, **on top of** `backgroundColor`.
+  Carries `opacity` (ghost a reference at `0.3`), `fit` (default `"cover"`;
+  `"contain"` letterboxes against `backgroundColor`), and `focusX`/`focusY` crop
+  anchors with the same semantics as `placement.focusX`. This is the mechanism behind
+  the agent protocol's reference-background bake — build chrome on top of a reference
+  render, section by section.
+- **`source.visual.backgroundColor`** is a *different field on a different object* — it
+  fills one source's tile, not the canvas. Same name, unrelated scope. Check which one
+  you're reading.
+
+### Text needs a WIDTH, chrome needs `min(H, 0.75·W)`
+
+`drawtext` never shrinks a string and `fit:"contain"` only picks the anchor, so a
+font sized from a bar's HEIGHT clips on any canvas narrower than the design
+aspect. Fit every text under `textEmUnits(text) × fontSize × em` with a budget of
+`cell × 0.94 − 2px` (quantization takes 1–2px per split level), prefer a degrade
+ladder (shrink → compact wording → drop) over ellipsis at the floor, and size
+chrome bars off `min(H, 0.75·W)` rather than a fraction of H (a 173px header slab
+on 1080×1920 otherwise). The full rule set, em table, and the 7-canvas gate sweep:
+[`layout-contract.md`](layout-contract.md) §"Text".
+
+### Verifying layout intent survived
+
+The m0 is exact but disposable — stableKeys and tile order are per-string and
+can't carry authored intent, and quantization squashes are invisible to
+string-level tools. The label-keyed layout contract (`withLayoutContract` /
+`checkLayout` / `assertLayout`) plus the three resolve-only audits close that
+loop: [`layout-contract.md`](layout-contract.md).
+
+## Avoid overengineering
+
+Just because the DSL allows `0{...}`, deep overlay stacks, logical owners `-{}`,
+and nested containers everywhere does not mean every template should use them.
+Prefer the simplest construct that expresses the layout clearly.
+
+Weighted splits: prefer small basis values (10, 20, 32, 64); avoid extreme counts
+unless intentional; ensure the render resolution supports the split granularity;
+remember feasibility constraints. Only use `0`-runs for precise weighted
+geometry, advanced prefix-overlay patterns, or algorithmic generation — avoid
+them for everyday layouts.
+
+A good template has predictable geometry, works at intended resolutions, uses
+overlays intentionally, avoids fragile structural side-effects and unnecessary
+deep nesting — and is deterministic, has safe defaults, and passes validation
+cleanly.
+
+## Checklist
+
+When designing a template:
+
+- [ ] Do I know each region's rect at author time? If so, is it a REAL m0 cell (not a full-frame `xExpr`/mask overlay)? (See the Hard Rule.)
+- [ ] Is this primarily structural layout or paint layering?
+- [ ] Could this be simpler with direct splits?
+- [ ] Am I using overlays intentionally?
+- [ ] Is the string readable if humans will maintain it?
+- [ ] Is this dictionary-level generation where readability does not matter?
+- [ ] Does it validate and remain feasible at target resolution?
+- [ ] Is every split count above 12 5-smooth (2ᵃ3ᵇ5ᶜ) — basis capped with `latticeWeights` / `precision`, never per-weight rounding? The gate (`latticeSmooth`, throw) and `m0saic doctor` will refuse it otherwise; a raster declares `lattice: { mode: "bitmap" }`, a content count `lattice.allow` with its reason.
+- [ ] Did I route through the right placement recipe ([`geometry-recipes.md`](geometry-recipes.md)) instead of hand-rolling pixel math?

package/docs/templates/data-pipeline.md

@@ -0,0 +1,324 @@
+# Data pipeline: fetchers, adapters, handles, and wiring
+
+The chain: a FETCHER template publishes data → the resolver threads it across
+pipeline steps / sibling tiles → a CONSUMER template reads it from ctx → a cron
+Job (or the CLI / Make page) renders the whole thing from a `.mosaicx`.
+Copy-adaptable reference pair: `@m0saic/meta/fixture-fetcher/v1` +
+`@m0saic/meta/upstream-echo/v1` (see Reference implementations).
+
+## The three patterns
+
+All three ride the runtime upstream collection (`collectUpstream`,
+the render engine source (not published) — pure, sync; the runner invokes it
+per step transition before the step's `render`):
+
+- **Data-fetcher** — a step-0 template whose render returns a doc publishing
+  structured data via `MosaicDataSource`; downstream reads `ctx.upstreamData[alias]`
+  / `ctx.upstreamVariables`.
+- **Adapter** — a small pure template that reads upstream data, transforms it,
+  and re-publishes under a different alias. Cross-pack composition.
+- **Handle** — a producer self-stamps `{ stepIndex, flattenedStableKey }` into
+  its variables; a downstream `MosaicRefSource` spreads the handle to mirror the
+  producer's rendered pixels across the step boundary — no re-render, no re-encode.
+
+Fetchers/adapters carry *data* across the back-edge; handles carry *pixel pointers*.
+
+## Publish channels × read views
+
+A producer publishes through three channels on its returned doc:
+
+| # | Channel | Flat view | Namespaced view | Publication entry |
+|---|---|---|---|---|
+| 1 | Doc-level `doc.variables` | ✅ merges | — | — (no provenance) |
+| 2 | `MosaicDataSource` WITHOUT `alias` | ✅ merges | — | ✅ |
+| 3 | `MosaicDataSource` WITH `alias` | ✅ merges | ✅ `upstreamData[alias]` | ✅ |
+
+Plus one authored (not template-published) source: `pipeline.variables`, the
+pipeline-level seed — flat view only, visible from step 0 onward. (Data flow
+WITHIN one template is plain function returns; the ctx views below exist for
+crossing a step/tile boundary.)
+
+- **`upstreamVariables` — the global bank.** Flat last-write-wins union of everything
+  above. Grounded example: THEMING — a theme step publishes tokens once
+  (`{ chromeDark: "#111", accent: "#e33", … }`); every downstream template reads
+  `ctx.upstreamVariables.chromeDark`; swap the one producer (or the seed) to re-skin.
+- **`upstreamData` — alias-namespaced blocks.** Populated ONLY by aliased
+  `MosaicDataSource`s. The consumer must know the namespace; that's the point — it's
+  a contract. The alias stays FIXED across versions (part of `outputsSchema`);
+  varying data goes in the payload, never in the key. Consumers declare needs in
+  `upstreamDataSchema`, so drift is a resolve-time error and keys can't shadow.
+- **`upstreamPublications` — the provenance escape hatch.** Lossless ordered list,
+  one entry per data source, stamped with `stepIndex`, `tileStableKey`, and
+  `templateId`. Same-alias colliders BOTH survive here. Same back-edge scope as the
+  merged views, just unmerged — not a wider window; doc-level `variables` mint no
+  publication. Use for geometry-addressed reads
+  (`publications.find(p => p.tileStableKey === "r/fc0")`) when "which tile said
+  this" is the real question; otherwise prefer the alias. Rule of thumb: alias
+  anything another pack might read; doc-level variables are fine for single-producer
+  flows, themes, and handles; publications are the audit channel, not the read path.
+
+## Author a fetcher (producer)
+
+- `defineMosaicTemplate` with `capabilities: { tier: "capability", caps: {…} }`.
+  Without that tier the Level 1 gate STRIPS `ctx.secrets` / `ctx.connections` before
+  your render runs — long the #1 silent failure. It is no longer silent: a core-tier
+  template that READS a handle the host actually supplied gets a `console.warn`
+  naming the template, the field and the declaration that would grant it, plus a
+  `warn`-level `capability_denied_read` template log on `ctx.telemetry`. The read
+  still returns `undefined` and the ctx key set is unchanged — the tripwire is a
+  non-enumerable getter, so only a real read trips it. Both handles are withheld
+  together, which is the tell: `ctx` keys `[analysis, cache, media, mode, output,
+  target, telemetry]` with `connections`/`secrets` absent means TIER, not a
+  missing host connection. (Raw template objects that skip `defineMosaicTemplate`
+  are not gated — don't rely on that.) Pure-derivation fetchers (data from props
+  alone) can stay `core`.
+- The render return, sketched (the load-bearing fields):
+
+      // m0: "1" — one cell, for the carrier
+      // sources: [ { type: "lavfi", color: "#000000" },             // carrier
+      //   { type: "data", alias: asAliasId("athleteData"),
+      //     variables: { name, country, stats } } ]                 // the publish
+      // sidecars: { athleteData: data }                             // disk mirror
+
+- **Include a 1-cell renderable carrier** (the lavfi above). A strictly data-only
+  doc cannot be planned when a `.mosaicx` wrapper's cell references it; inline
+  data-only step files DO work (the planner skips them), but the carrier makes the
+  fetcher safe in both shapes. Data sources occupy no m0 cell — the planner filters
+  them from layout — so the m0 counts renderables only.
+- Secrets: `ctx.secrets.get("env:NAME")` (CLI/jobs resolve env vars; desktop
+  also resolves `keychain:` refs). NEVER put cleartext in variables/sidecars —
+  publish derived facts only. Connections: `ctx.connections.get("publisher@profile")`
+  (CLI needs `M0SAIC_CONNECTIONS_FILE=<path to connections.json>`).
+- Declare `outputsSchema` — all-optional keys unless you can always deliver;
+  required keys are validated post-render (VARIABLES_SCHEMA_MISMATCH, error).
+  Mirror the payload to `sidecars.<key>` — lands as `<output>.<key>.json` next
+  to every render, your byte-assertable proof. Opt out only for sensitive data.
+- Naming: announce the role in the id — `<scope>/<pack>/<name>-fetcher/v1` (or
+  `/intel/`, `/prologue/`); `role: "data-fetcher"` is a real `TEMPLATE_ROLES`
+  value (the opt-in, warning-only `TEMPLATE_ROLE_NAMING_MISMATCH` pins the hint).
+
+## Author a consumer
+
+- Core tier is fine — consumers of capability-produced data stay core; the data
+  is opaque input, so the consumer stays deterministic given it.
+- Read via the `@m0saic/template-utils` upstream helpers — NOT raw ctx fields
+  (optional + generic-typed; they decay into casts). Full set at
+  https://github.com/m0saic-project/m0saic-packages/blob/main/packages/template-utils/src/data/upstream.ts#L36-135` (verified 2026-07-27):
+  `hasUpstream`, `listUpstreamAliases`, `hasUpstreamBlock`, `getUpstreamBlock`,
+  `requireUpstreamBlock`, `getUpstreamVariables`, `getUpstreamVariable`,
+  `getUpstreamPublications`, `findUpstreamPublications`.
+
+      if (!hasUpstream(ctx)) return renderMissingCard();          // wired at all?
+      const athlete = requireUpstreamBlock(ctx, "athleteData");   // throws w/ available aliases
+      const soft = getUpstreamBlock(ctx, "designTokens");         // undefined when absent
+      const accent = getUpstreamVariable<string>(ctx, "accent") ?? "#e33"; // flat union
+      listUpstreamAliases(ctx)                                    // sorted, for diagnostics
+      findUpstreamPublications(ctx, { tileStableKey: "r/fc0" })   // provenance escape hatch
+
+  Helpers never transform values, and "no upstream threaded" stays
+  distinguishable from "threaded but empty" (`undefined` vs `{}`).
+- Declare `upstreamDataSchema` / `upstreamVariablesSchema` — required keys make
+  mis-wiring a resolve-time ERROR. Standalone renders (no resolver) skip the
+  check, so always handle absent upstream with a deterministic, VISIBLE missing
+  state (à la upstream-echo's red card); `requireUpstreamBlock` is the fail-fast
+  complement for consumers meaningless without their producer.
+
+## Adapter (re-emission)
+
+Sketch — core tier, `role: "adapter"` (id-hint `adapter`|`bridge`):
+
+      // render(_props, ctx):
+      //   const upstream = ctx.upstreamData?.teamAData;
+      //   return doc with sources: [{ type: "data", alias: asAliasId("teamBData"),
+      //     variables: { athlete: { name: upstream?.hero?.fullName, … } } }]
+
+Why the template layer, not the type layer: the impulse is to let `MosaicRefSource`
+target a `MosaicDataSource` (ref relays data). We deliberately don't — a ref carries
+pixel-affecting props (`placement`, `effects`, `mask`, …) that would be silently
+meaningless on a data target, and a render() can transform / filter / merge far
+beyond static renames. Enforced structurally: data sources occupy no m0 cell, so a
+ref at a data source's "slot" surfaces `MOSAIC_REF_NOT_FOUND`
+(`MOSAIC_DATA_SOURCE_VISIBLE_USE` stays registered in `@m0saic/types` for ABI
+stability but is emitted from nowhere). Write an adapter for: cross-pack renames,
+subset filtering, merging two aliased sources, computed fields. Writing the same
+one-key rename a fifth time is the signal to propose a re-emission variant.
+
+## Handle (cross-step pixel reuse)
+
+Sketch — producer stamps a self-describing pointer; consumer spreads it into a ref:
+
+      // producer (owns the geometry, so the key is correct by construction):
+      //   variables: { introHero: { stepIndex: ctx.pipelineStep!.index,
+      //                             flattenedStableKey: "intro_hero" } }
+      // consumer:
+      //   sources: [{ type: "ref", ...ctx.upstreamVariables!.introHero,
+      //     placement: { fit: "cover" }, playback: { loopMode: "freeze" } }]
+
+The producer self-stamps (never the consumer hardcoding) because only the producer
+knows its `stepIndex` and cell stableKeys at author time — v2 can rename
+"intro_hero" without breaking consumers. Channel choice: doc-level `variables`
+(flat — single-producer flows) or an aliased `MosaicDataSource` (namespace-safe —
+consumer reads `ctx.upstreamData.introHero`); both spread into the ref identically.
+Data sources sit beside renderables without occupying cells, so ONE producer can
+publish pixels + a data block + a handle in one doc.
+
+Resolution matrix (engine behavior, 3c.4):
+
+| # | Locality | Size | Duration | Behavior |
+|---|---|---|---|---|
+| 1  | Same doc   | match  | match  | Precursor reuse, identity. Ref's `placement` applies. |
+| 2  | Same doc   | differ | match  | Precursor reuse + `placement.fit` + offset. |
+| 3a | Cross-step | match  | match  | Precursor reuse across step boundary; `stepIndex` disambiguates. |
+| 3b | Cross-step | differ | match  | Precursor reuse + `placement.fit`; target in earlier step. |
+| 3c | Cross-step | (either) | differ | `playback.loopMode` (`"loop"`/`"freeze"`/`"cut"`) + `clipStartMs`/`clipDurationMs`. |
+
+All five reuse the producer's intermediate precursor file. Each consumer's own
+`placement` / `playback` / `effects` / `mask` / `visual` / `audio` / `overlay`
+decorate the mirrored stream independently — N consumers, N decoration chains,
+one decode. Cross-step pixel-format / fps / alpha bridging is engine-internal
+(the ref consumer's chain inserts `scale=` / `fps=` / `setpts=` / `format=` as
+needed — https://github.com/m0saic-project/m0saic-packages/blob/main/packages/types/src/source/source.ts#L1198-1201`).
+Handle vs fetcher + re-render: handle when you want **the same rendered pixels**
+back (cheapest, frame-identical); fetcher + downstream re-render when you want
+**the same logical content rendered differently** per step. They compose.
+
+## Wire the pipeline (.mosaicx)
+
+TWO equivalent shapes. Both resolve to the same `mosaic_pipeline`; the fixtures
+render byte-identical mp4s + sidecars, and the e2e suite gates that.
+
+**Flat (preferred for new chains)** —
+the CLI source (not published):
+`kind: "mosaicx_pipeline"` at the root, steps each wrap ONE invocation. No `m0`,
+no `sources`, no `children` — a pipeline root carries no layout of its own.
+Stamp `size`/`fps`/`durationMs` on the pipeline (the self-describing-artifact
+rule) where the wrapped form stamps them on the wrapper document.
+
+**Wrapped (legacy; still fully supported)** —
+the CLI source (not published):
+mosaicx_document → single `{type:"mosaic", ref}` source → `mosaic_pipeline` child
+whose steps each wrap ONE invocation. The resolver PROMOTES the pipeline child to
+the top-level renderable.
+
+Either way the steps read:
+
+    step 0: fetcher invocation, "intermediate": true   ← required; its carrier
+            renders to workspace but never reaches the deliverable
+    step 1+: consumer invocations — each sees (earlier steps ∪ earlier
+            siblings); strict back-edge, no forward references
+
+The wrapped form is promoted to the top level automatically; the flat form is
+already there. Invocation ids differ between the two (they are slugs of the path
+into the source tree, so the wrapper contributes a `children.<ref>.` segment) —
+internal handles only, not deliverable names.
+
+Cross-TILE also works without a pipeline: an earlier sibling invocation's
+published data reaches later siblings in the same doc.
+
+The flat root is what makes a chain visible to every surface that iterates a
+**top-level** pipeline's steps. As of 2026-08-14 the CLI (`m0saic make`,
+`m0saic resolve`) accepts both roots; Compose rejects `mosaicx_pipeline` at load
+and `m0saic open --make` can't open one (it needs a top-level invocation).
+`MosaicXPipeline` carries `runner` but deliberately NOT `vocab` / `agent` /
+`suggestedOutput` — those are read only by surfaces that require a top-level
+`template_invocation` source.
+
+## Run it
+
+- CLI: `M0SAIC_FIXTURE_SECRET=… m0saic make thing.mosaicx -w 640 -h 360 -o out.mp4`
+  (env vars ARE the CLI secret store; sidecars land next to out.mp4)
+- Jobs: input kind `"mosaicx-file"` (UI infers it from the extension). Cron jobs
+  auto-suffix colliding outputs; `validateJob` is strict — unknown template ids
+  / schema drift FAIL validation.
+- Make page: open the .mosaicx and render — same lowering.
+- Smoke: `npm run smoke:test-templates -w @m0saic/cli -- fixture-pipeline`
+
+### Jobs window tokens (host-injected time)
+
+A cron/scheduled Job interpolates window tokens into invocation props at TRIGGER
+time, before `resolveMosaicx` runs — the template still receives a plain explicit
+`{startISO,endISO}` window and stays clock-free. Token set (UTC): `{{todayISO}}`,
+`{{lastFullWeekStartISO}}`, `{{lastFullWeekEndISO}}` — the last COMPLETE Mon..Sun
+week strictly before the trigger instant. Exact-substring replacement inside
+string values only.
+
+- the Mosaic Desktop / Web app source (not published) — `windowTokenMap(triggerInstant)`
+  (reuses `lastFullWeek` from the github pack's week-math) +
+  `interpolateWindowTokens` (nested string walk).
+- `runJob.js` threads the trigger instant and interpolates BOTH a `.mosaicx` doc
+  (before resolve) and template `mergedProps` (before render).
+- `validateJob.js` mirrors it; unrecognized `{{…}}` → `UNKNOWN_WINDOW_TOKEN` warning
+  (never an error; unknown tokens pass through verbatim).
+
+Timezone: v1 is UTC. The cron `tz` schedules WHEN a job fires but does NOT shift
+the tokens — a weekly Monday job always renders the just-completed week.
+Coupling note: `windowTokens.js` imports week math from a template pack; the
+cleaner long-term home is `@m0saic/platform` — flagged for a future hoist.
+
+## Diagnostics you'll meet
+
+| Code | Means | Fix |
+|---|---|---|
+| VARIABLES_SCHEMA_MISMATCH (error) | upstream payload ≠ declared schema, or producer under-delivered outputsSchema | fix the wiring/alias/types |
+| PIPELINE_DATA_ONLY_STEP_NOT_INTERMEDIATE (error) | data-only step not marked intermediate | add `"intermediate": true` |
+| MOSAIC_DATA_SOURCE_OUTSIDE_PIPELINE (warning) | data source on a non-pipeline root — nobody can read it | move it into a step / invocation child |
+| VARIABLES_NOT_YET_IMPLEMENTED (warning) | raw authored pipeline sets step variables; only the resolver can thread | run it via .mosaicx invocations |
+| MOSAICX_RESOLVE_FAILED (jobs validate, error) | an invocation failed (unknown id / render threw) | fix the template id / props |
+
+## The github/ connector pack (the shipped exemplar)
+
+https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src/m0saic/github (verified 2026-07-27): `connection.ts`
+(registerHostConnection "github@default" + /rate_limit probe), `client.ts`
+(injectable-fetch, Link pagination, 202-retry, per-run call budget, committer-date
+windowing), `week-math.ts` (pure ISO-week / Mon-start helpers),
+`repo-facts-fetcher/v1` (capability, publishes `githubRepoFacts`),
+`weekly-pulse-adapter/v1` (core, PURE, publishes `weeklyPulse`), `weekly-pulse/v1`
+(self-contained app-runnable capability consumer — the clock-rule exception below),
+`__fixtures__/` (a frozen real week + record-facts.mjs). No generic connector
+substrate until connector #2.
+
+**Coverage record (never silently zero).** A fetcher hitting a rate-limited /
+partial API publishes an explicit `coverage` block (authenticated?,
+commitDetails full/partial/none, which KPIs are real; every cap in
+`coverage.truncated`). The adapter selects KPIs ADAPTIVELY from it — a mirror
+repo drops PR/issue KPIs instead of emitting zeros; degraded basis is LABELED.
+
+**The clock rule (hosts inject time).** Templates NEVER read the wall clock;
+identical facts replay → byte-identical publish; all tests run replay or stubbed
+fetch; CI never touches the network. Scheduling is the host's job (window tokens
+above). One documented exception: an app-runnable capability template's live-mode
+default window (`capability-templates.md` §"App-runnable capability templates").
+
+**Secret handling.** Token → request headers ONLY. `coverage.authenticated` is the
+only signal a token was used; the cleartext never enters variables, sidecars,
+diagnostics, errors, or logs. tokenRef resolution order: `props.tokenRef` (when it
+resolves) → connection keychain via `mintConnectionSecretRef` → anonymous.
+
+## Forward references
+
+- **TemplateRole** (layer 1 shipped) — `data-fetcher` and `adapter` are real
+  `TEMPLATE_ROLES` values (https://github.com/m0saic-project/m0saic-packages/blob/main/packages/types/src/template/template.ts#L1169);
+  `handle-producer` is NOT a role. Render-return-shape enforcement (3j) parked.
+- **Schema validation** (Phase 3h) — once template generics (`<P, O, U, D, S>`)
+  thread end-to-end, producer/consumer mismatches become type-level errors.
+- **Cross-step mirror render** (Phase 3c.4 — ✅ **SHIPPED**, re-verified
+  2026-07-27) — the handle pattern's ref consumer mirrors precursor pixels across
+  step boundaries for real: 20 `ref-*` capability fixtures exercise it in the
+  always-on E2E gate (the CLI source (not published)), and
+  the render engine source (not published) asserts
+  `MOSAIC_REF_NOT_YET_IMPLEMENTED` is NOT emitted. **Do not treat the handle
+  pattern as non-functional — it is one of the best-tested surfaces in the
+  repo.** (Fetchers/adapters produce no pixels — no visual-golden mandate.)
+- **Sibling docs** — `recursion-nested-rendering.md`: `renderNestedTemplate` (how
+  a parent invokes producer/fetcher templates); `philosophy-and-contract.md`: the
+  `capabilities.tier` contract; `emission-patterns.md`: the sidecar/telemetry
+  *mechanism* the fetcher role applies.
+
+## Reference implementations
+
+- Producer: https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src/m0saic/meta/fixture-fetcher/v1
+- Consumer: https://github.com/m0saic-project/m0saic-packages/blob/main/packages/templates/src/m0saic/meta/upstream-echo/v1 (self-evidencing green/red status card)
+- Wiring (flat root): the CLI source (not published)
+- Wiring (wrapped root): the CLI source (not published)
+  — the same chain; the e2e matrix asserts both produce identical deliverables
+- Headless jobs usage: the Mosaic Desktop / Web app source (not published) (stub pattern + cron-style run)