@gui-chat-plugin/sequence 0.4.1 → 0.5.1

package/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## 0.5.1
+
+- **Markdown slides fit in movies too.** A dense Markdown slide shrank to fit in the View, whose page
+  runs the fitting, but a movie made from it (MulmoChat's makeMovie: an `html_tailwind` beat) cut
+  it off at the top and bottom. `SLIDE_FIT_SCRIPT` is the same fitting, for the host to give the
+  movie's page (MulmoCast's html_tailwind beat takes it as its `script`). The Markdown slide's box
+  no longer shrinks as a flex item: MulmoCast's page makes the body a flex column, where it did,
+  and its text was fitted into the shrunken box, much too small. Checked in a MulmoChat movie: a
+  14-item slide with a formula on every line fits, its title and last line in view.
+
+## 0.5.0
+
+- **Chart slides**: `presentSlide` takes `chart`, a Chart.js configuration (`{ type, data,
+options }`, as MulmoCast's chart beats take it), for numbers to compare, a trend or proportions.
+  The View draws it under the slide's title on a sandboxed page of its own
+  (`chartSlideDocument`), with the same kind of policy as an HTML slide's: Chart.js from jsDelivr,
+  checked by its hash, and the configuration as data. Chart.js's own types only
+  (`SLIDE_CHART_TYPES`). The parameter is JSON text: as an object without properties, Gemini
+  Live used it for 2 charts in 4 asked for, drawing the others in HTML; as JSON text, 3 in 3. An
+  object is read too.
+- **Markdown slides with TeX math**: `presentSlide` takes `markdown`, with math between `$…$` or
+  `$$…$$`, for equations, definitions, short lists and small tables. `execute()` makes it an HTML
+  slide (`markdownSlideHtml`), so Views and movies show it as one; the record and the result keep
+  the text too, in `markdown`. Math is MathML, drawn by the browser with nothing to load, with the
+  thin spaces around function names and the matrix column gaps Chrome's MathML leaves out. The
+  slide's type shrinks until it fits. New dependencies: marked and KaTeX, imported only when a
+  Markdown slide is made.
+- The prompt tells the model which kind a slide should be: a picture for a scene, a chart for
+  numbers, Markdown for math and text, HTML for designed layouts and diagrams.
+- An HTML slide's page shrinks a box marked `data-fit` until it fits (its script's hash changed).
+
 ## 0.4.1
 
 - **Slides are explained, not announced.** Asked only to "explain" the slide on the screen, OpenAI's

package/README.md

@@ -3,11 +3,11 @@
 Tools that show things one step at a time, as pictures, for
 [GUI Chat Protocol](https://github.com/receptron/gui-chat-protocol) hosts:
 
-| Tool               | Shows                                                                                                                                                                              |
-| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `presentSlide`     | One slide of a spoken slideshow (mode `"presentation"`), or one step of a step-by-step guide the user follows along with (mode `"steps"`): a generated picture, or a slide in HTML |
-| `defineStoryboard` | A story's cast: a reference sheet for each recurring character                                                                                                                     |
-| `presentPanel`     | One panel of the story, with its characters drawn as in their sheets; an interactive story offers choices                                                                          |
+| Tool               | Shows                                                                                                                                                                                                                   |
+| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `presentSlide`     | One slide of a spoken slideshow (mode `"presentation"`), or one step of a step-by-step guide the user follows along with (mode `"steps"`): a generated picture, a chart, or a slide in Markdown (with TeX math) or HTML |
+| `defineStoryboard` | A story's cast: a reference sheet for each recurring character                                                                                                                                                          |
+| `presentPanel`     | One panel of the story, with its characters drawn as in their sheets; an interactive story offers choices                                                                                                               |
 
 From MulmoChat and MulmoGlass, tested there by voice on OpenAI Realtime, Gemini Live and Grok, and
 in text chat.
@@ -60,11 +60,47 @@ overlap (a host that runs `execute()` on a server gets them as concurrent reques
 claimed before anything is awaited, so an identical one waits for it and a later step is held while
 it is drawn.
 
-## HTML slides
-
-A slide is a generated picture (`imagePrompt`) or HTML (`html`), the model's choice per slide: a
-picture for a scene or an object, HTML for words, numbers, lists, comparisons and diagrams, which
-picture models draw badly. The model writes the inside of `<body>` for a 1280x720 canvas, with
+## Slides that aren't pictures
+
+A slide is a generated picture (`imagePrompt`), a chart (`chart`), Markdown (`markdown`) or HTML
+(`html`), the model's choice per slide: a picture for a scene or an object; a chart for numbers to
+compare, a trend or proportions; Markdown for equations (TeX math), definitions, short lists and
+small tables; HTML for designed layouts, comparisons, timelines and diagrams. Picture models draw
+words, numbers and math badly. A slide sent with more than one is one kind: `html` first, then
+`markdown`, then `chart`, then the picture.
+
+### Markdown slides
+
+`markdown` is GitHub-flavored Markdown with TeX math between `$…$` (inline) or `$$…$$` (a
+displayed equation), `\(…\)` and `\[…\]` too. A dollar starts math only before a non-space and
+ends it only after one, not before a digit, so "$5 and $10" stays money; `\$` is a dollar sign.
+`execute()` makes it into an HTML slide (`markdownSlideHtml`, with [marked](https://marked.js.org)
+and [KaTeX](https://katex.org), imported only when a Markdown slide is made), so it is shown, saved
+and made into a movie as one: the result and the record have the HTML in `html` and the text as the
+model wrote it in `markdown`. Math is MathML (KaTeX's `mathml` output), which the browser draws with
+no stylesheet or font to load: it works offline and in a movie as on the screen. Function names
+(`\log`, `\sin`) get TeX's thin spaces, and matrix cells padding, which Chrome's MathML doesn't give
+them. TeX that KaTeX can't read is shown as written, in red. The slide's type is sized by how much it
+says, and the page shrinks it until it fits. A page that shows the slide's HTML without the View's
+page (a movie's) fits it with `SLIDE_FIT_SCRIPT`: MulmoCast's `html_tailwind` beat takes it as its
+`script`.
+
+### Chart slides
+
+`chart` is a [Chart.js](https://www.chartjs.org) configuration, `{ type, data, options }`, as
+MulmoCast's `chart` beats take it (their `chartData`), of one of Chart.js's own types
+(`SLIDE_CHART_TYPES`: bar, line, pie, doughnut, radar, polarArea, scatter, bubble). The tool advertises it as JSON text (an object without properties, Gemini Live
+used for 2 charts in 4 asked for; JSON text for 3 in 3), and an object is read too; one of another type, or without `data`, is refused, and the model is told
+why. The record and the result keep it in `chart`. `chartSlideDocument(title, chart)` (core entry)
+makes the page: the slide's title over the chart, which Chart.js (from jsDelivr, one version,
+checked by its hash) draws growing into place. The configuration is in a JSON data block, so it
+can't run anything, and its policy, like an HTML slide's, lets the page send nothing out and run no
+script but its own and Chart.js. Offline, the page says the chart couldn't be drawn.
+
+### HTML slides
+
+The model writes the inside of `<body>` for a 1280x720 canvas, with
+Tailwind CSS classes, and animates it with [MulmoCast](https://github.com/receptron/mulmocast-cli)'s The model writes the inside of `<body>` for a 1280x720 canvas, with
 Tailwind CSS classes, and animates it with [MulmoCast](https://github.com/receptron/mulmocast-cli)'s
 declarative `data-animation` attributes: `animate` (`data-opacity`, `data-translate-x/y`,
 `data-scale`, `data-rotate`, `data-width`/`data-height`), `counter` and `typewriter`, each with
@@ -94,7 +130,7 @@ that an HTML step is on the screen; a guide step asked for again is then shown a
 
 Saved through `files.artifacts`, with their pictures' paths:
 
-- `slideshows/<id>.json`: `{ id, title, mode, totalSlides, slides: { "1": { title, imagePrompt, imagePath } } }`; an HTML slide has `html`, an `imagePrompt` of `""` and no `imagePath`
+- `slideshows/<id>.json`: `{ id, title, mode, totalSlides, slides: { "1": { title, imagePrompt, imagePath } } }`; a slide that isn't a picture has an `imagePrompt` of `""` and no `imagePath`, and has `html` (an HTML slide), `html` and `markdown` (a Markdown slide) or `chart` (a chart slide)
 - `storyboards/<id>.json`: `{ id, title, style, totalPanels, interactive, characters: [{ name, description, imagePath }], panels: { "1": { caption, characters, imagePrompt, imagePath, choices } } }`
 
 IDs are 12 hex digits. The results name them, so a later tool call can refer to a slideshow or a

package/dist/core/chartSlide.d.ts

@@ -0,0 +1,14 @@
+/** The chart types a slide can have: Chart.js's own, with no plugin. */
+export declare const SLIDE_CHART_TYPES: readonly ["bar", "line", "pie", "doughnut", "radar", "polarArea", "scatter", "bubble"];
+/** A Chart.js configuration: `type`, `data` and optionally `options`. */
+export type SlideChart = Record<string, unknown> & {
+    type: (typeof SLIDE_CHART_TYPES)[number];
+    data: Record<string, unknown>;
+};
+/** CHART_PAGE_SCRIPT's hash, which the policy allows (checked by a test). */
+export declare const CHART_PAGE_SCRIPT_HASH = "sha256-TfZZxJX+yWgzYLy7zqsMa85OJEaC+zNLIpKLDZVnUqA=";
+/** JSON that can sit in a <script> data block: no "<", so no "</script>"
+ *  and no "<!--" can end it early. */
+export declare const scriptSafeJson: (value: unknown) => string;
+/** The page a chart slide is shown as. */
+export declare function chartSlideDocument(title: string, chart: SlideChart): string;

package/dist/core/definitions.d.ts

@@ -1,4 +1,5 @@
 import type { ToolDefinition } from "gui-chat-protocol";
+import { type SlideChart } from "./chartSlide";
 export declare const PRESENT_SLIDE = "presentSlide";
 export declare const DEFINE_STORYBOARD = "defineStoryboard";
 export declare const PRESENT_PANEL = "presentPanel";
@@ -13,16 +14,22 @@ export interface SlideArgs {
     slide: number;
     totalSlides: number;
     title: string;
-    /** The picture's prompt; "" for an HTML slide. */
+    /** The picture's prompt; "" for a slide that isn't a picture. */
     imagePrompt: string;
-    /** An HTML slide's body; absent for a picture. */
+    /** An HTML slide's body. */
     html?: string;
+    /** A Markdown slide's text, with TeX math. */
+    markdown?: string;
+    /** A chart slide's Chart.js configuration. */
+    chart?: SlideChart;
     mode: SlideMode;
 }
-export declare const SLIDE_ARGS_ERROR = "presentSlide needs slide and totalSlides (whole numbers, slide from 1 to totalSlides), and either an imagePrompt or html";
-export declare const SLIDE_HTML_TOO_LONG = "presentSlide's html is too long: keep a slide under 60000 characters";
+export declare const SLIDE_ARGS_ERROR = "presentSlide needs slide and totalSlides (whole numbers, slide from 1 to totalSlides), and one of imagePrompt, html, markdown or chart";
+export declare const SLIDE_HTML_TOO_LONG = "presentSlide's slide is too long: keep its html, markdown or chart under 60000 characters";
+export declare const SLIDE_CHART_INVALID: string;
 /** The slide arguments, or what is wrong with them (said to the model). A
- *  slide with html is an HTML slide, whatever imagePrompt says. */
+ *  slide is one kind: html first, then markdown, then chart, then a
+ *  picture, whatever else was sent with it. */
 export declare function parseSlideArgs(args: Record<string, unknown>): SlideArgs | string;
 /** What the model does once a slide is on the screen. */
 export declare const slideShownInstructions: ({ slide, totalSlides, title, mode, }: SlideArgs) => string;
@@ -35,19 +42,27 @@ export interface Slideshow {
     totalSlides: number;
     slides: Record<string, {
         title: string;
-        /** "" for an HTML slide. */
+        /** "" for a slide that isn't a picture. */
         imagePrompt: string;
         imagePath?: string;
-        /** An HTML slide's body (slideHtmlDocument makes it a page). */
+        /** An HTML slide's body (slideHtmlDocument makes it a page); for a
+         *  Markdown slide, the HTML made of it. */
         html?: string;
+        /** A Markdown slide's text, as the model wrote it. */
+        markdown?: string;
+        /** A chart slide's Chart.js configuration (chartSlideDocument). */
+        chart?: SlideChart;
     }>;
 }
-/** A shown slide's result data: a picture, or an HTML slide's body. */
+/** A shown slide's result data: a picture, an HTML slide's body (a
+ *  Markdown slide's included, with its markdown), or a chart. */
 export interface SlideData {
     imageData?: string;
     imagePath?: string;
     html?: string;
-    /** The picture's prompt; "" for an HTML slide. */
+    markdown?: string;
+    chart?: SlideChart;
+    /** The picture's prompt; "" for a slide that isn't a picture. */
     prompt: string;
     slideshowId: string;
     slide: number;

package/dist/core/host.d.ts

@@ -13,11 +13,14 @@ export declare function imageOf(result: ToolResult | null | undefined): {
 };
 /** An HTML slide's body, when the result is one. */
 export declare function htmlOf(result: ToolResult | null | undefined): string;
-/** Whether a result shows something: a picture or an HTML slide. */
+/** A chart slide's configuration as text, when the result is one. */
+export declare function chartOf(result: ToolResult | null | undefined): string;
+/** Whether a result shows something: a picture, an HTML slide (a Markdown
+ *  slide is one) or a chart. */
 export declare const shownContent: (result: ToolResult) => boolean;
 /** Whether two results show the same picture: by its saved path when both
  *  have one (a host may send the current result without its image data),
- *  else by the picture itself, or the same HTML slide. A host that sends the
+ *  else by the picture itself, or the same HTML slide or chart. A host that sends the
  *  current result by its path only can't tell an HTML slide is on the
  *  screen: one asked for again is shown again. */
 export declare function samePicture(a: ToolResult | null | undefined, b: ToolResult | null | undefined): boolean;

package/dist/core/index.d.ts

@@ -4,6 +4,8 @@ export declare const presentSlidePluginCore: SequencePluginCore;
 export declare const defineStoryboardPluginCore: SequencePluginCore;
 export declare const presentPanelPluginCore: SequencePluginCore;
 export * from "./definitions";
-export { MAX_SLIDE_HTML, SLIDE_ANIMATION_KINDS, SLIDE_AUTO_END_SECONDS, SLIDE_CSS_ANIMATIONS_FINISHED, SLIDE_HEIGHT, SLIDE_WIDTH, slideHtmlDocument, } from "./slideHtml";
+export { MAX_SLIDE_HTML, SLIDE_ANIMATION_KINDS, SLIDE_AUTO_END_SECONDS, SLIDE_CSS_ANIMATIONS_FINISHED, SLIDE_FIT_SCRIPT, SLIDE_HEIGHT, SLIDE_WIDTH, slideHtmlDocument, } from "./slideHtml";
+export { SLIDE_CHART_TYPES, chartSlideDocument, type SlideChart, } from "./chartSlide";
+export { markdownSlideHtml } from "./markdownSlide";
 export { SLIDESHOWS_DIR, STORYBOARDS_DIR, loadRecord } from "./records";
 export type { SequenceContext } from "./host";

package/dist/core/markdownSlide.d.ts

@@ -0,0 +1,9 @@
+/** The box's first font size, in px, from how much the slide says: a few
+ *  lines are large, many smaller. The page shrinks it further when the slide
+ *  still doesn't fit; a movie, which doesn't run the page's script, has this
+ *  one. */
+export declare function markdownFontSize(markdown: string): number;
+/** KaTeX's MathML, spaced as TeX spaces it where Chrome doesn't. */
+export declare const spaceFunctionNames: (mathml: string) => string;
+/** A Markdown slide's HTML: the body of an HTML slide. */
+export declare function markdownSlideHtml(markdown: string): Promise<string>;

package/dist/core/slideHtml.d.ts

@@ -11,8 +11,16 @@ export declare const SLIDE_ANIMATION_KINDS: readonly ["animate", "counter", "typ
  *  it runs to the end of its beat, which the narration decides and the View
  *  doesn't know. */
 export declare const SLIDE_AUTO_END_SECONDS = 8;
+/** A script that fits a slide's data-fit boxes, as the View's page does, for
+ *  a page that shows the slide's HTML without it: a host making a movie of a
+ *  Markdown slide runs it in the movie's page (MulmoCast's html_tailwind
+ *  beat takes it as its script), or dense text is cut off there. It fits at
+ *  once and again when the page has loaded. */
+export declare const SLIDE_FIT_SCRIPT = "(() => {\n  const fit = () => {\n    for (const el of document.querySelectorAll(\"[data-fit]\")) {\n      let size = parseFloat(getComputedStyle(el).fontSize);\n      while (size > 12 && (el.scrollHeight > el.clientHeight + 1 || el.scrollWidth > el.clientWidth + 1)) {\n        size -= 1;\n        el.style.fontSize = size + \"px\";\n      }\n    }\n  };\n  fit();\n  addEventListener(\"load\", fit);\n})();";
 /** PAGE_SCRIPT's hash, which the policy allows (checked by a test). */
-export declare const PAGE_SCRIPT_HASH = "sha256-0zsqCvkCfq5Z1iJCn46NiAtSAyIr/JDkyNs/Fae1J+Y=";
+export declare const PAGE_SCRIPT_HASH = "sha256-bNBxv6ehYc2PHOgmTReFSYsrxiGvq6aqqg9m5RwS7Lc=";
+/** Text as HTML text or an attribute's value. */
+export declare const escapeHtml: (text: string) => string;
 /** The slide's HTML, made safe to show:
  *  - the tags that act on the page from outside the body are plain text:
  *    <meta> (a refresh navigates the frame, a policy of its own), <base> and