@oh-my-pi/snapcompact 15.11.4 → 15.11.7

package/CHANGELOG.md

@@ -2,6 +2,28 @@
 
 ## [Unreleased]
 
+## [15.11.7] - 2026-06-12
+
+### Added
+
+- Added `SHAPE_VARIANTS`, the catalog of research-eval frame variants the native renderer reproduces faithfully (`8x8r`/`8x8u`/`6x6u`/`5x8` × `sent`/`bw`), with `ShapeVariantName`, `SHAPE_VARIANT_NAMES`, and the `isShapeVariantName` guard
+- `resolveShape(api, variant?)` now accepts an explicit variant name (or `"auto"`); forced variants keep their geometry but are re-priced for the target provider's image billing (token estimate and OpenAI `original` detail hint)
+- Added the six research-eval winning frame variants to `SHAPE_VARIANTS`: `6x12-dim` (Claude fable), `8x13-bw` (Opus), `8on16-bw` (GPT grid runner-up), `doc-8on16-bw` (GPT), `doc-8on16-sent` (GLM), and `doc-8on16-sent-dim` (Gemini/Kimi), backed by new `Shape` fields `stretch` (disable Lanczos stretch: natural glyphs on a larger cell pitch), `columns` (two word-wrapped newspaper columns), `stopwordDim`, and the X.org `6x12`/`8x13` fonts
+- Added `dimStopwords()`, which prints high-frequency function words in dim ink via zero-width markers (skipping spans that are already dim), and `wrap()`, the greedy word-wrap used to typeset doc-layout pages; `geometry`/`render`/`renderMany`/`frames`/`compact` understand doc shapes (wrap once, paginate into `2 * rows`-line pages), and compaction frames persist `columns`/`stopwordDim` for mixed-shape detection
+- `resolveShape` now takes a `ShapeTarget` (`{ api, id }` — a pi-ai `Model` works as-is) and detects the ideal shape from the **model id**, not just the wire API: a Claude routed through Vertex or an OpenAI-compatible gateway keeps its Claude shape, with billing still priced by the API family actually carrying the request. `idealShapeVariant(modelId)` exposes the model-line table; unmeasured models fall back to the API family's winner
+- `resolveShape` now also resolves an ideal **frame size** per model line, and billing estimates come from verified per-family formulas instead of flat 1568px constants: Anthropic bills 28px patches capped at 4,784 visual tokens (+5% margin), Gemini 3.x bills a fixed 1,120-token `media_resolution` budget per image at any pixel size, and OpenAI bills 32px patches × 1.2 under the 10,000-patch `detail: "original"` budget. High-res Claude lines (Opus 4.7+, Fable, Mythos — native 2576px-edge ingestion) get 1932px frames (same recall and cost, a third fewer frames); Gemini gets 2048px frames (+70% chars per frame at the same bill); GPT and Kimi stay at 1568px (area-proportional billing and a model-side 1792px processor cap, respectively). `idealShapeVariant` now returns an `IdealShape` (`{ variant, frameSize? }`)
+- Added per-provider image-count budgets: `PROVIDER_IMAGE_BUDGETS`, `DEFAULT_PROVIDER_IMAGE_BUDGET`, `providerImageBudget()`, and `providerFrameBudget()` (the image budget clamped to `MAX_FRAMES`). OpenRouter is capped at its measured hard limit of 8 images per request (excess images are silently dropped with no error); unknown providers get a safe floor of 5
+- Added `Archive.textTail`: archive content past the frame budget is no longer dropped — `compact()` stops rendering at the budget and keeps the newest unframed slice as verbatim text on the summary (capped at two frame capacities with middle elision, counted into `truncatedChars` when elided). The tail persists in `preserveData` and is folded back into frames by the next compaction
+
+### Changed
+
+- Frames are no longer padded to a square: the native renderer clips each PNG's height to the text rows actually printed, so a partially filled frame (typically the newest) bills only the pixel rows it uses
+- **Changed the OpenAI default shape from `6x6u-sent` to `8on16-bw`.** A production-regime mono eval (gpt-5.5, the full 800k-char SQuAD flow in one request, n=50) scored the old dense default f1 .602 vs .851 for `8on16-bw` rendered by the production pipeline, at near-equal total cost (the dense cells burned the frame savings on reasoning tokens); chunked exp14 had already scored `8on16-bw` .906. `SHAPES.openaiDense` is renamed to `SHAPES.openai`
+- **Changed the Google default shape from `8x8r-sent` to `doc-8on16-sent-dim`.** Production-rendered mono eval on gemini-3.5-flash (400k chars, one request, n=25): f1 .900 vs .853 for the repeated grid at lower cost, agreeing with the chunked round-2 winner
+- **Changed the Anthropic default shape from `8x8r-bw` to `6x12-dim`.** Production mono eval on claude-fable (400k chars, one request, n=25): f1 .840 vs .877 for the repeated grid — within noise — at 37% lower cost (12 frames instead of 21 per 400k chars), with clean completions in every probe; opus reads the same trade (.800 vs .833 at 42% lower cost)
+- `normalize()` now keeps line structure: whitespace runs containing a line break collapse to `NEWLINE_GLYPH` (U+2588 FULL BLOCK, drawn by the native renderer as a pitch-black cell one character wide) instead of a plain space; leading/trailing breaks are trimmed, and the frame-reading prompt explains the marker
+- `normalize()` now skips characters the fonts cannot render instead of printing `?` blanks: whole ANSI escape sequences are stripped, and bare control characters, zero-width format characters (ZWSP, BOM, directional marks), combining marks, and lone surrogates are dropped without occupying a cell; `?` remains the fallback for unsupported graphic characters only
+
 ## [15.11.4] - 2026-06-12
 
 ### Breaking Changes

package/README.md

@@ -0,0 +1,70 @@
+# @oh-my-pi/snapcompact
+
+Bitmap-frame context compression for vision-capable LLMs.
+
+Instead of asking an LLM to summarize discarded conversation history, snapcompact serializes it and renders the text into dense PNG frames of pixel-font glyphs that vision models read back directly. The whole pass is local and deterministic — no LLM call, no API key, no latency beyond rendering. Rasterization and PNG encoding happen in native code (`@oh-my-pi/pi-natives`).
+
+Built for [oh-my-pi](https://github.com/can1357/oh-my-pi)'s compaction pipeline, but the rendering API works on arbitrary text.
+
+## How it works
+
+1. Discarded history is serialized to compact text (`serializeConversation`), with per-tool-result and per-argument character caps.
+2. Text is normalized for the bundled bitmap fonts (`normalize`): ANSI sequences stripped, whitespace collapsed, newline runs folded into a single full-block glyph so line structure survives.
+3. Pages of text are rasterized into PNG frames (`render` / `renderMany`). Frame width is fixed per shape; height hugs the rows actually printed, so a partially filled frame never bills blank pixel rows.
+4. Frames persist in the compaction entry's `preserveData` and are re-attached to the summary message on every context rebuild.
+
+Frame shapes are provider-aware, chosen by SQuAD recall evals (see `research/`) against real provider billing:
+
+| Reader | Default shape | Notes |
+| --- | --- | --- |
+| Anthropic | `6x12-dim` | X.org 6x12 glyphs, stopwords dimmed gray; high-res Claude lines get 1932px frames |
+| Google | `doc-8on16-sent-dim` @2048 | Two newspaper columns, sentence-hue ink; Gemini bills a fixed per-image budget, so larger frames are free chars |
+| OpenAI | `8on16-bw` | 8x13 glyphs on a patch-aligned 16px pitch, sent at `detail: "original"` |
+| Unknown | Anthropic shape | Per-provider image-count budgets guard against gateways that silently drop frames |
+
+`resolveShape({ api, id })` matches the model id, not just the wire API — a Claude routed through Vertex or OpenRouter keeps its Claude shape, priced for the gateway actually carrying the request.
+
+## Install
+
+```sh
+bun add @oh-my-pi/snapcompact
+```
+
+Ships TypeScript source directly (no build step); requires Bun ≥ 1.3.14.
+
+## Usage
+
+Render arbitrary text into LLM image blocks:
+
+```ts
+import { renderMany, frames, resolveShape } from "@oh-my-pi/snapcompact";
+
+const images = renderMany(longText, { model }); // ImageContent[], first page first
+const count = frames(longText, { model });      // frame count without rendering
+const shape = resolveShape(model);              // eval-optimal Shape for the reader
+```
+
+Run a full compaction pass over prepared messages:
+
+```ts
+import { compact } from "@oh-my-pi/snapcompact";
+
+const result = await compact(preparation, { model, maxFrames: 8 });
+// result.summary        — text summary with <files> operations block
+// result.preserveData   — frame archive, re-attachable via getPreservedArchive() + images()
+```
+
+## API surface
+
+- **Compaction**: `compact`, `CompactionPreparation`, `CompactionResult`, `getPreservedArchive`, `images`
+- **Rendering**: `render`, `renderMany`, `frames`, `geometry`
+- **Shapes**: `SHAPES`, `SHAPE_VARIANTS`, `resolveShape`, `idealShapeVariant`, `isShape`, `isShapeVariantName`
+- **Text**: `serializeConversation`, `normalize`, `dimStopwords`, `wrap`
+- **Budgets**: `providerImageBudget`, `providerFrameBudget`, `MAX_FRAMES`, `FRAME_TOKEN_ESTIMATE`
+- **File ops**: `createFileOps`, `computeFileLists`, `upsertFileOperations`
+
+## References
+
+- [Monorepo README](https://github.com/can1357/oh-my-pi#readme)
+- [Compaction architecture](../../docs/compaction.md)
+- [CHANGELOG](./CHANGELOG.md)

package/dist/types/snapcompact.d.ts

@@ -2,27 +2,40 @@
  * Snapcompact compaction: archive conversation history as dense bitmap images.
  *
  * Instead of asking an LLM to summarize discarded history, the serialized
- * conversation is rendered into square PNG frames of pixel-font text that
- * vision models read back directly, like an archivist at a snapcompact frame
- * reader.
+ * conversation is rendered into PNG frames of pixel-font text that vision
+ * models read back directly, like an archivist at a snapcompact frame
+ * reader. Frames are `frameSize` wide; their height hugs the text rows
+ * actually printed, so a partially filled frame never bills blank rows.
  *
  * The frame shape is provider-aware, following the snapcompact SQuAD evals
  * (`packages/snapcompact`, 200k-token monolithic runs):
  *
- * - **Anthropic** (`8x8r-bw`): unscii-8 square cells, black ink, every line
- *   printed twice with the copy on a pale highlight band. Read at F1 parity
- *   with raw text at ~2x lower cost; the colored variants drew refusals at
- *   scale, the repeated plain shape did not.
- * - **Google** (`8x8r-sent`): same repeated grid with six-hue sentence
- *   coloring (0.90 F1 at ~2.9x lower cost on gemini-3.5-flash).
- * - **OpenAI** (`6x6u-sent`): OpenAI bills a flat ~2.9k tokens per image, so
- *   image count is the only cost lever — unscii-8 Lanczos-stretched to 6x6
- *   cells packs the most readable chars per frame. Frames request
- *   `detail: "original"`; the default `auto` downscale destroys 6px glyphs.
- * - **Unknown providers** default to the Anthropic shape (most
- *   refusal-robust). Gateways that resize images (e.g. OpenRouter normalizes
- *   visual payloads to a fixed token budget) defeat any shape — optical
- *   context fails silently there.
+ * - **Anthropic** (`6x12-dim`): X.org 6x12 glyphs, black ink, stopwords
+ *   dimmed gray — recall within noise of the repeated `8x8r-bw` grid at
+ *   ~40% lower cost; `8x8r-bw` remains the max-recall choice via the shape
+ *   setting. Opus 4.7+/Fable/Mythos ingest high-res natively (2576px edge,
+ *   4,784 visual-token cap, no flag needed), so those lines get 1932px
+ *   frames: same recall and cost, a third fewer frames. Older Claude lines
+ *   downscale past 1568px and keep the standard frame.
+ * - **Google** (`doc-8on16-sent-dim` @2048): two word-wrapped newspaper
+ *   columns of 8x13 glyphs, sentence-hue ink, dimmed stopwords. Gemini 3.x
+ *   bills a fixed `media_resolution` budget per image (default 1,120
+ *   tokens) regardless of pixels, so the 2048px frame carries +70% chars at
+ *   the same bill (f1 .88 vs .90 at 1568). `ULTRA_HIGH` doubles the budget
+ *   and reads 3072px frames, but loses on chars/$ — deliberately unused.
+ * - **OpenAI** (`8on16-bw`): 8x13 glyphs on a patch-aligned 16px pitch,
+ *   black ink (gpt-5.5 mono F1 .867 vs .602 for the previous `6x6u-sent`).
+ *   Patch billing (32px × 1.2, 10k-patch budget at `detail: "original"`) is
+ *   area-proportional, so resolution cannot improve chars/$ — 1568 stays.
+ *   `detail: "high"` would downgrade (2,500-patch cap); `original` is sent.
+ * - **Unknown providers** default to the Anthropic shape. Gateways can
+ *   defeat any shape silently: OpenRouter enforces a per-model image cap
+ *   (measured: 8 images for glm-4.6v — frames past the cap are dropped with
+ *   no error, billed tokens plateau exactly at 8x frame cost). The same
+ *   frames routed direct to the vendor read fine (glm f1 .20 -> .78), so
+ *   `providerImageBudget` caps per-request images per provider (OpenRouter
+ *   8, unknown 5) and `compact()` keeps any archive overflow as a text tail
+ *   on the summary instead of rendering frames that would be dropped.
  *
  * The whole pass is local and deterministic — no LLM call, no API key, no
  * latency beyond rendering. Rasterization and PNG encoding happen in native
@@ -34,14 +47,22 @@ import type { Api, ImageContent, Message, Model } from "@oh-my-pi/pi-ai";
 /** One eval-validated frame shape: font, cell, ink, repetition, and size. */
 export interface Shape {
     /** Bundled font in the native renderer. */
-    font: "5x8" | "8x8";
+    font: "5x8" | "8x8" | "6x12" | "8x13";
     /** Target cell advance in pixels; differing from the font's natural cell
      *  renders via Lanczos stretch (anti-aliased RGB frame). */
     cellWidth: number;
     /** Target cell pitch in pixels. */
     cellHeight: number;
+    /** `false` → glyphs drawn at natural size on the cell pitch (8on16);
+     *  `true`/`undefined` → legacy auto Lanczos stretch when cell ≠ natural. */
+    stretch?: boolean;
     /** Ink: `sent` cycles six hues at sentence boundaries; `bw` is black. */
     variant: "sent" | "bw";
+    /** Print stopwords in dim ink (research `dim`/`sent-dim` variants). */
+    stopwordDim?: boolean;
+    /** 1/undefined = row-major grid; 2 = two word-wrapped newspaper columns
+     *  (research `doc`). */
+    columns?: number;
     /** Each text line is printed this many times; copies after the first sit
      *  on a pale highlight band (redundancy coding). */
     lineRepeat: number;
@@ -52,55 +73,192 @@ export interface Shape {
     /** Resolution hint attached to frame images (OpenAI-only). */
     imageDetail?: ImageContent["detail"];
 }
-/** Eval-validated shapes, keyed by the provider family they won on. */
-export declare const SHAPES: {
-    /** `8x8r-bw`: unscii square, black ink, lines doubled on highlight bands. */
-    readonly anthropic: {
+/** Geometry half of a {@link Shape}: everything except provider billing. */
+export type ShapeGeometry = Omit<Shape, "frameTokenEstimate" | "imageDetail">;
+/**
+ * Frame variants exercised by the SQuAD evals in `research/` that the native
+ * renderer reproduces faithfully, keyed by their research names. Font codes:
+ * `8x8u` unscii square cell, `8x8r` unscii with every line printed twice
+ * (redundancy coding), `6x6u` unscii Lanczos-squeezed to 6x6 (densest
+ * readable cell), `5x8` the X.org legacy font on its 2576px frame, `6x12`
+ * and `8x13` the X.org misc fonts, `8on16` 8x13 glyphs on an 8x16 cell pitch
+ * (no stretch, extra leading), `doc-` prefixed shapes a two-column
+ * word-wrapped newspaper layout. Ink: `sent` cycles six hues at sentence
+ * boundaries, `bw` is plain black, `-dim` suffix prints stopwords in gray.
+ */
+export declare const SHAPE_VARIANTS: {
+    readonly "8x8r-bw": {
         readonly font: "8x8";
         readonly cellWidth: 8;
         readonly cellHeight: 8;
         readonly variant: "bw";
         readonly lineRepeat: 2;
         readonly frameSize: 1568;
-        readonly frameTokenEstimate: 3300;
     };
-    /** `8x8r-sent`: the repeated grid with sentence-hue ink. */
-    readonly google: {
+    readonly "8x8r-sent": {
         readonly font: "8x8";
         readonly cellWidth: 8;
         readonly cellHeight: 8;
         readonly variant: "sent";
         readonly lineRepeat: 2;
         readonly frameSize: 1568;
-        readonly frameTokenEstimate: 1100;
     };
-    /** `6x6u-sent`: unscii stretched to 6x6 — densest readable cell, fewest
-     *  frames (OpenAI bills per image, ~2.9k tokens flat). */
-    readonly openaiDense: {
+    readonly "8x8u-bw": {
+        readonly font: "8x8";
+        readonly cellWidth: 8;
+        readonly cellHeight: 8;
+        readonly variant: "bw";
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "8x8u-sent": {
+        readonly font: "8x8";
+        readonly cellWidth: 8;
+        readonly cellHeight: 8;
+        readonly variant: "sent";
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "6x6u-bw": {
+        readonly font: "8x8";
+        readonly cellWidth: 6;
+        readonly cellHeight: 6;
+        readonly variant: "bw";
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "6x6u-sent": {
         readonly font: "8x8";
         readonly cellWidth: 6;
         readonly cellHeight: 6;
         readonly variant: "sent";
         readonly lineRepeat: 1;
         readonly frameSize: 1568;
-        readonly frameTokenEstimate: 2900;
-        readonly imageDetail: "original";
     };
-    /** Original 5x8 X.org shape (pre-shape-table sessions rendered this). */
-    readonly legacy: {
+    readonly "5x8-bw": {
+        readonly font: "5x8";
+        readonly cellWidth: 5;
+        readonly cellHeight: 8;
+        readonly variant: "bw";
+        readonly lineRepeat: 1;
+        readonly frameSize: 2576;
+    };
+    readonly "5x8-sent": {
         readonly font: "5x8";
         readonly cellWidth: 5;
         readonly cellHeight: 8;
         readonly variant: "sent";
         readonly lineRepeat: 1;
         readonly frameSize: 2576;
-        readonly frameTokenEstimate: 3300;
     };
+    readonly "6x12-dim": {
+        readonly font: "6x12";
+        readonly cellWidth: 6;
+        readonly cellHeight: 12;
+        readonly variant: "bw";
+        readonly stopwordDim: true;
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "8x13-bw": {
+        readonly font: "8x13";
+        readonly cellWidth: 8;
+        readonly cellHeight: 13;
+        readonly variant: "bw";
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "8on16-bw": {
+        readonly font: "8x13";
+        readonly cellWidth: 8;
+        readonly cellHeight: 16;
+        readonly stretch: false;
+        readonly variant: "bw";
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "doc-8on16-bw": {
+        readonly font: "8x13";
+        readonly cellWidth: 8;
+        readonly cellHeight: 16;
+        readonly stretch: false;
+        readonly variant: "bw";
+        readonly columns: 2;
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "doc-8on16-sent": {
+        readonly font: "8x13";
+        readonly cellWidth: 8;
+        readonly cellHeight: 16;
+        readonly stretch: false;
+        readonly variant: "sent";
+        readonly columns: 2;
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+    readonly "doc-8on16-sent-dim": {
+        readonly font: "8x13";
+        readonly cellWidth: 8;
+        readonly cellHeight: 16;
+        readonly stretch: false;
+        readonly variant: "sent";
+        readonly stopwordDim: true;
+        readonly columns: 2;
+        readonly lineRepeat: 1;
+        readonly frameSize: 1568;
+    };
+};
+/** Research name of one renderable frame variant. */
+export type ShapeVariantName = keyof typeof SHAPE_VARIANTS;
+/** All variant names, in declaration order (for settings enums). */
+export declare const SHAPE_VARIANT_NAMES: readonly ShapeVariantName[];
+/** Runtime guard for variant names loaded from config. */
+export declare function isShapeVariantName(value: unknown): value is ShapeVariantName;
+/** Eval-validated shapes, keyed by the provider family they won on. */
+export declare const SHAPES: {
+    /** `6x12-dim`: X.org 6x12 glyphs, black ink with stopwords dimmed gray.
+     *  Production mono eval on claude-fable: f1 .840 vs .877 for the repeated
+     *  `8x8r-bw` grid (within noise at n=25) at 37% lower cost — 12 frames
+     *  instead of 21 per 400k chars. Never refused in any run. */
+    anthropic: Shape;
+    /** `doc-8on16-sent-dim`: two word-wrapped columns, sentence hues, dimmed
+     *  stopwords. Production mono eval on gemini-3.5-flash: f1 .900 vs .853
+     *  for the repeated grid, at lower cost; also the chunked round-2 winner. */
+    google: Shape;
+    /** `8on16-bw`: 8x13 X.org glyphs on a 16px pitch, black ink. Mono eval on
+     *  gpt-5.5 (200k-token single request, n=50): f1 .851 vs .602 for the
+     *  previous `6x6u-sent` default at near-equal total cost; chunked exp14
+     *  scored it .906. */
+    openai: Shape;
+    /** Original 5x8 X.org shape (pre-shape-table sessions rendered this). */
+    legacy: Shape;
 };
 /** Runtime guard for shape overrides loaded from config or preserve data. */
 export declare function isShape(value: unknown): value is Shape;
-/** Pick the eval-optimal frame shape for a provider API. */
-export declare function resolveShape(api?: Api): Shape;
+/** One model line's ideal format: variant plus an optional frame-size
+ *  override when the line reads larger frames at no extra cost. */
+export interface IdealShape {
+    variant: ShapeVariantName;
+    frameSize?: number;
+}
+/** Eval-ideal format for a model id, or undefined when unmeasured. */
+export declare function idealShapeVariant(modelId: string): IdealShape | undefined;
+/** What will read the frames: the wire API (billing) and model id (shape). */
+export interface ShapeTarget {
+    api?: Api;
+    id?: string;
+}
+/**
+ * Pick the frame shape for a reader. An explicit `variant` (anything but
+ * `"auto"`) forces that geometry; otherwise the model id selects the
+ * eval-winning shape — and frame size — for its model line, falling back to
+ * the API family's winner when the model is unmeasured. Billing (token
+ * estimate, detail hint) always follows the API family actually carrying
+ * the request, computed for the resolved frame size. Accepts a full pi-ai
+ * `Model` or any `{ api, id }` subset.
+ */
+export declare function resolveShape(model?: ShapeTarget, variant?: ShapeVariantName | "auto"): Shape;
 /** Legacy frame edge in pixels (the 5x8 shape's eval-validated size). New
  *  shapes carry their own `frameSize`. */
 export declare const FRAME_SIZE = 2576;
@@ -111,6 +269,21 @@ export declare const MAX_FRAMES = 8;
 /** Conservative per-frame token estimate used for context budgeting
  *  (upper bound across shapes: Anthropic bills 1568*1568/750 ≈ 3,278). */
 export declare const FRAME_TOKEN_ESTIMATE = 3300;
+/**
+ * Per-request image-count budgets by provider id. Routers and smaller
+ * providers enforce hard caps and silently DROP images past them (measured:
+ * OpenRouter caps at 8 — images 9+ vanish with no error and billed tokens
+ * plateau at 8x frame cost). First-party APIs allow far more; their values
+ * are conservative policy caps well under the measured hard limits
+ * (Anthropic 100, OpenAI 500, Gemini ~2500).
+ */
+export declare const PROVIDER_IMAGE_BUDGETS: Record<string, number>;
+/** Safe floor for unknown providers (strictest mainstream measured: Groq ~5). */
+export declare const DEFAULT_PROVIDER_IMAGE_BUDGET = 5;
+/** Per-request image budget for `provider`; unknown providers get the floor. */
+export declare function providerImageBudget(provider: string | undefined): number;
+/** Archive frame budget for `provider`: its image budget clamped to {@link MAX_FRAMES}. */
+export declare function providerFrameBudget(provider: string | undefined): number;
 /** Key under `CompactionEntry.preserveData` holding the frame archive. */
 export declare const PRESERVE_KEY = "snapcompact";
 /** One developed snapcompact frame: a base64 PNG plus its reading geometry. */
@@ -118,7 +291,7 @@ export interface Frame {
     /** Base64-encoded PNG. */
     data: string;
     mimeType: string;
-    /** Characters per row in the frame grid. */
+    /** Characters per row in the frame grid (per-column width on doc frames). */
     cols: number;
     /** Text rows in the frame grid (unique lines, not repeated copies). */
     rows: number;
@@ -128,6 +301,10 @@ export interface Frame {
     font?: Shape["font"];
     variant?: Shape["variant"];
     lineRepeat?: number;
+    /** 2 on two-column doc frames; absent on row-major grid frames. */
+    columns?: number;
+    /** True when stopwords were printed in dim ink. */
+    stopwordDim?: boolean;
     /** Resolution hint forwarded to the provider when re-attaching. */
     detail?: ImageContent["detail"];
 }
@@ -139,11 +316,18 @@ export interface Archive {
     totalChars: number;
     /** Characters dropped so far to respect the frame budget. */
     truncatedChars: number;
+    /** Most recent slice of archived history that exceeded the frame budget,
+     *  kept verbatim as normalized text (dim markers and newline glyphs
+     *  included). Shipped as plain text in the compaction summary and folded
+     *  back into frames by the next compaction. */
+    textTail?: string;
 }
 export interface Geometry {
+    /** Characters per row (per-column line width when `columns === 2`). */
     cols: number;
     rows: number;
-    /** Characters that fit one frame (cols * rows). */
+    /** Characters that fit one frame (nominal upper bound on doc shapes,
+     *  where real consumption is wrap-dependent). */
     capacity: number;
 }
 export interface Options<TMessage = Message> extends SerializeOptions {
@@ -233,15 +417,39 @@ export interface SerializeOptions {
     dimToolResults?: boolean;
 }
 export declare function serializeConversation(messages: Message[], options?: SerializeOptions): string;
+/** Printed in place of newline runs: the native renderer fills this cell
+ *  entirely with pitch-black ink, so line structure survives whitespace
+ *  collapsing at a one-cell cost. */
+export declare const NEWLINE_GLYPH = "\u2588";
 /**
- * Prepare text for printing: collapse whitespace runs (incl. newlines) to
- * single spaces — the eval's "paragraph breaks collapsed to spaces" format —
- * then fold everything outside the fonts' ASCII + Latin-1 coverage to ASCII
- * approximations (`?` as the last resort).
+ * Prepare text for printing: strip ANSI escape sequences, collapse horizontal
+ * whitespace runs to single spaces and newline-bearing runs to one
+ * {@link NEWLINE_GLYPH} (drawn as a pitch-black cell), then fold everything
+ * outside the fonts' ASCII + Latin-1 coverage to ASCII approximations.
+ * Unrenderable control/format/combining characters are dropped without
+ * occupying a cell; `?` remains the fallback for unsupported graphic
+ * characters. The zero-width ink toggles {@link DIM_ON}/{@link DIM_OFF} pass
+ * through untouched.
  */
 export declare function normalize(text: string): string;
+/**
+ * Wrap each maximal alphabetic run that is a stopword in {@link DIM_ON} /
+ * {@link DIM_OFF} so it prints in dim gray ink. Spans that are already dim
+ * (e.g. archived tool output) pass through untouched — wrapping there would
+ * terminate the enclosing dim span early. Markers are zero-width, so the
+ * visible glyph count is unchanged.
+ */
+export declare function dimStopwords(text: string): string;
+/**
+ * Greedy word-wrap, no mid-word breaks (hard split only for width+ words) —
+ * ported verbatim from `research/exp14_bestgpt.py` `wrap()`. Zero-width dim
+ * markers count toward word length here; serialized history places them at
+ * word boundaries, so the drift is at most one cell per affected line.
+ */
+export declare function wrap(text: string, width: number): string[];
 export declare function geometry(shape: Shape, size?: number): Geometry;
-/** Render one snapcompact frame from already-normalized text. */
+/** Render one snapcompact frame from already-normalized text. Doc shapes
+ *  (`columns === 2`) expect one page of `\n`-joined pre-wrapped lines. */
 export declare function render(text: string, shape: Shape, size?: number): RenderedFrame;
 /** Options for {@link renderMany} and {@link frames}. */
 export interface RenderManyOptions {
@@ -260,7 +468,9 @@ export interface RenderManyOptions {
  * Empty/whitespace-only input yields no frames.
  */
 export declare function renderMany(text: string, options?: RenderManyOptions): ImageContent[];
-/** Frames needed to hold `text` at the given shape/size, without rendering. */
+/** Frames needed to hold `text` at the given shape/size, without rendering.
+ *  For doc shapes this wraps the text once and counts pages of `2 * rows`
+ *  lines; for grid shapes it divides by the frame capacity. */
 export declare function frames(text: string, options?: Pick<RenderManyOptions, "shape" | "model" | "frameSize">): number;
 /** Validate and extract a persisted frame archive from `preserveData`. */
 export declare function getPreservedArchive(preserveData: Record<string, unknown> | undefined): Archive | undefined;
@@ -271,7 +481,10 @@ export declare function images(archive: Archive): ImageContent[];
  * the discarded history, prints it onto PNG frames in the provider-optimal
  * shape, merges previously archived frames (oldest dropped beyond the
  * budget), and produces a deterministic summary explaining how to read the
- * frames.
+ * frames. Pages past the frame budget are never rendered (providers with
+ * hard image caps silently drop excess frames on the wire) — the newest
+ * unrendered slice survives verbatim as a text tail on the summary and is
+ * folded back into frames by the next compaction.
  *
  * Frames archived under a different shape (provider switches, legacy 5x8
  * sessions) are kept as-is — each frame carries its own geometry, and the

package/package.json

@@ -1,7 +1,7 @@
 {
 	"type": "module",
 	"name": "@oh-my-pi/snapcompact",
-	"version": "15.11.4",
+	"version": "15.11.7",
 	"description": "Bitmap-frame context compression for vision-capable LLMs",
 	"homepage": "https://omp.sh",
 	"author": "Can Boluk",
@@ -31,9 +31,9 @@
 		"fmt": "biome format --write ."
 	},
 	"dependencies": {
-		"@oh-my-pi/pi-ai": "15.11.4",
-		"@oh-my-pi/pi-natives": "15.11.4",
-		"@oh-my-pi/pi-utils": "15.11.4"
+		"@oh-my-pi/pi-ai": "15.11.7",
+		"@oh-my-pi/pi-natives": "15.11.7",
+		"@oh-my-pi/pi-utils": "15.11.7"
 	},
 	"devDependencies": {
 		"@types/bun": "^1.3.14"
@@ -43,6 +43,7 @@
 	},
 	"files": [
 		"src",
+		"README.md",
 		"CHANGELOG.md",
 		"dist/types"
 	],

package/src/prompts/snapcompact-summary.md

@@ -1,6 +1,6 @@
 Prior conversation history has been archived verbatim onto {{frameCount}} snapcompact frame{{#if multipleFrames}}s{{/if}} — the bitmap image{{#if multipleFrames}}s{{/if}} attached below{{#if multipleFrames}}, ordered oldest to newest{{/if}}.
 
-Reading a frame: monospace {{fontCell}} pixel font on a white background, {{cols}} characters per row, {{rows}} text rows per frame; read left to right, top to bottom. Text flows continuously with no word wrap, so words may break across row ends. Whitespace runs (including newlines) were collapsed to single spaces. {{#if sentenceInk}}Ink color cycles through six colors, advancing at sentence boundaries — a color change marks a new sentence.{{else}}Glyphs are plain black ink.{{/if}}{{#if dimmedToolResults}} Tool output is printed in dim gray ink — gray text is archived tool output, not conversation.{{/if}}{{#if lineRepeated}} Every text line is printed twice in a row — first on the white background, then repeated on a pale yellow band. The copies are identical: read each line once and use the duplicate only to double-check hard glyphs.{{/if}} Roles are tagged inline as [User]:, [Assistant]:, [Assistant thinking]:, [Assistant tool calls]:, and [Tool result]:.
+Reading a frame: monospace {{fontCell}} pixel font on a white background, {{#if docColumns}}typeset as two word-wrapped newspaper columns of {{cols}} characters by {{rows}} lines each — read the left column top to bottom, then the right column{{else}}{{cols}} characters per row, {{rows}} text rows per frame; read left to right, top to bottom. Text flows continuously with no word wrap, so words may break across row ends{{/if}}. Horizontal whitespace runs were collapsed to single spaces; line breaks print as a solid black cell (one character wide) — treat each as a newline. {{#if sentenceInk}}Ink color cycles through six colors, advancing at sentence boundaries — a color change marks a new sentence.{{else}}Glyphs are plain black ink.{{/if}}{{#if stopwordDimmed}} Common function words (the, of, and, …) are printed in dim gray; content words carry the full ink.{{/if}}{{#if dimmedToolResults}} Tool output is printed in dim gray ink — gray text is archived tool output, not conversation.{{/if}}{{#if lineRepeated}} Every text line is printed twice in a row — first on the white background, then repeated on a pale yellow band. The copies are identical: read each line once and use the duplicate only to double-check hard glyphs.{{/if}} Roles are tagged inline as [User]:, [Assistant]:, [Assistant thinking]:, [Assistant tool calls]:, and [Tool result]:.
 {{#if mixedShapes}}
 
 Older frames may use a different font, grid, or ink coloring than described above; the reading order is always the same (left to right, top to bottom, oldest frame first).
@@ -15,3 +15,10 @@ The earliest frame begins with "[Summary of earlier history]" — a condensed di
 {{/if}}
 
 Total archived: {{totalChars}} characters. Consult the frames whenever you need exact earlier details (user wording, decisions, file paths, tool output). If a region is hard to read, re-derive the fact from the workspace (re-read files, re-run commands) rather than guessing.
+{{#if textTail}}
+
+The frame budget ran out before the newest part of the archive. That remainder continues below as plain text — it is newer than every frame and ends where the live conversation resumes.
+
+[Archived history, continued as text]
+{{textTail}}
+{{/if}}