@pinecall/dot 0.11.23 → 0.11.24

package/docs/macros.md

@@ -0,0 +1,166 @@
+# Macros
+
+> **Experimental.** `@pinecall/dot` may change in any release. This page is true of the version it ships with.
+
+A macro is a guided walk, a JSON file in the agent's folder (`macros/<name>.json`). The agent says
+each step as written; the page runs each step's moves as their words are said. The same file is
+what a visitor plays with no call (a *walk*, recorded in the agent's voice). [Write a tutor
+agent](write-a-tutor.md) shows where it goes; this page is the format.
+
+```json
+{
+  "name": "plans",
+  "title": "Plans and prices",
+  "when": "the visitor asks what it costs, or which plan to choose",
+  "steps": [
+    {
+      "say": "Here are our three plans: one monthly price instead of a surprise bill.",
+      "beats": [
+        { "word": "Here", "do": "scroll_to", "find": ["region", "Pricing"] },
+        { "word": "three", "do": "highlight", "find": ["region", "Pricing"] }
+      ]
+    },
+    {
+      "say": "Family Care is the one most families pick.",
+      "beats": [
+        { "word": "Family", "do": "highlight", "find": ["article", "Family Care plan"] },
+        { "word": "pick", "do": "point", "find": ["article", "Family Care plan"] }
+      ]
+    }
+  ]
+}
+```
+
+## The file
+
+| field | |
+|---|---|
+| `name` | what the model names it in `runMacro`. The key it is given in `macros({ plans })` when left out |
+| `title` | the visitor's button: "Plans and prices". `name` when left out |
+| `when` | when to run it, in the words the model reads: "the visitor asks what it costs" |
+| `pages` | the paths of the pages it is for: `["/security"]`. Left out, it is for every page. See below |
+| `steps` | at least one |
+
+A step:
+
+| field | |
+|---|---|
+| `say` | the sentence, said exactly as written |
+| `beats` | the moves hung on its words; may be empty |
+| `code` | `{ "title": "agent.ts", "text": "…" }` opens a window of code beside the page; `null` closes it. A step that names none leaves the window as it is |
+| `lines` | `[from, to]`, from 1: lights those lines of the window and dims the rest, and the pointer goes to them |
+
+A beat:
+
+| field | |
+|---|---|
+| `word` | the word of `say` it runs on. Case and punctuation do not count: `"Whitening,"` is `whitening` |
+| `do` | the move, below |
+| `find` | `[role, name]`: the element, as the page's snapshot names it. Needed by every move but `unhighlight`, `here` and `home` |
+| `nth` | which time the word is said in the step, from 1 (the first by default), for a word that repeats |
+
+## The moves
+
+| `do` | |
+|---|---|
+| `point` | the pointer flies to the element, the face beside it, and rests under its first words. Scrolls first if it is off the screen |
+| `highlight` | dims the rest of the page and lights the element, following it as the page scrolls |
+| `unhighlight` | takes the light away |
+| `scroll_to` | scrolls the page, eased, until the element sits in the upper middle of the screen, below the site's own header (a fixed or sticky bar along the top, or the page's `scroll-padding-top`). A thing that fits is shown whole; a section taller than the screen shows from its start, just under the header |
+| `here` | a ring at the pointer, twice |
+| `home` | the pointer goes back to its corner |
+
+A macro never types: a walk shows, and the agent writes into a field only what the visitor said.
+
+## Walks for some pages
+
+A site with several pages usually has a walk for each, and a visitor on `/security` should not be
+offered the tour of the home. Name the pages in the file:
+
+```json
+{ "name": "keys", "title": "Where your keys live", "pages": ["/security"], "steps": [ … ] }
+```
+
+- **The menu** lists only the walks for the page the visitor is on (a trailing slash does not
+  matter: `/security` and `/security/` are one page).
+- **A call** is the same: the page tells the agent its path, and the agent offers and runs only
+  the walks for it. A walk that names no page is for all of them, so name the home's
+  walks too (`"pages": ["/"]`) if they are only for the home.
+- **Each page has its own published document**, so a visitor downloads only the walks for the page
+  they are on: `walks` for the home (with the walks that name no page), `walks-security` for `/security`,
+  `walks-developers-sdk` for `/developers/sdk`. The page reads its own and, where the agent published none
+  for it, `walks`. A document is 64 KiB at most; a page with a great many walks can pass it, and is split
+  by naming the pages.
+- **A page that never said its path** (an older page) is offered every walk.
+- A path is exact: `/developers` is not `/developers/sdk`. List both.
+
+## Naming elements: `find`
+
+`find` is `[role, name]`: the first element of that role whose name *contains* the text (case does
+not matter). Roles and names are the accessibility tree's, not the markup's:
+
+| role | what it is | its name |
+|---|---|---|
+| `region` | a `<section>` with `aria-label` or `aria-labelledby` | that label |
+| `heading` | `<h1>`–`<h6>` | its text |
+| `article` | `<article>` | its `aria-label` |
+| `button`, `link`, `tab` | a `<button>` or `summary`, an `<a href>`, `role="tab"` | its text or `aria-label` |
+| `group` | `role="group"`, or `<details>` | its `aria-label` |
+| `textbox`, `combobox`, `checkbox`, `radio` | fields | their label |
+| `img` | `<img>`, `<figure>` | its `alt` |
+| `form` | `<form>` | its `aria-label` |
+
+Three habits keep a walk alive:
+
+1. **Name the parts you point at.** A plain `<div>` has no name, so a macro cannot find it: give
+   it `role="group"` and an `aria-label`, or point at its heading instead. This is the whole
+   reason a beat "does nothing".
+2. **Use a part of the name, not the whole.** `["region", "Pricing"]` finds "Pricing, three plans"
+   too; a label you reword later still matches.
+3. **A beat that finds nothing is skipped**, quietly, in a walk and in a call. The walk goes on, so a
+   renamed section loses a move, not the walk.
+
+## Writing sentences that play well
+
+The sentence is read aloud by the agent's voice, and a walk is recorded once in it.
+
+- **Short sentences, one idea each.** The visitor can stop at any step, and the agent is told how far
+  they got.
+- **Put the beat on the word that names the thing**, not before it: `"Family"` for "Family Care is
+  the one most families pick", so the pointer arrives as the name is said. Avoid a word that is also
+  said earlier in the sentence unless you mean it, or give `nth`.
+- **Light, then point, then let go.** `highlight` on the first word, `point` a few words later,
+  `unhighlight` before you move to a part of the page the light is not on.
+- **Move the light before the page moves.** A step that goes to another section lights it first, on
+  the same word as the scroll (`highlight`, then `scroll_to`, both on that word): the light then comes
+  in with the section. A light left on a section the page scrolls away from dims the whole screen.
+- **Write numbers and symbols as you would say them**: "nineteen dollars a month", not "$19/mo".
+- **Check it with your ears.** `pinecall walks play plans` plays the recorded walk in the terminal;
+  a sentence cut short or a name said strangely is a lexicon entry (`pinecall lexicon`), after which
+  that sentence is recorded again.
+
+## What is checked, and when
+
+`macros({ … })` checks every file when the agent's class loads, and refuses one that is not a JSON
+object, has no steps, a step that says nothing, a beat that is not `{ word, do, find?, nth? }` or
+uses `type`, a word its sentence never says, or `code`/`lines` of the wrong shape. The message names
+the macro and the step:
+
+```
+macro plans: step 2 never says "Familia"
+```
+
+What it cannot check is the page: whether `find` names something that exists. That is found by
+running it.
+
+## How the agent runs one
+
+In a call, `runMacro("plans")` sends each step to the page and then says it. The model names a walk
+the way it heard it: `runMacro("site tour")` runs `tour` (every word of a macro's name is in what was
+asked), and an agent with a single macro runs it whatever it is called. The page binds the step to the reply that
+follows and runs each beat as its word is said, by the voice's own timings. When the visitor speaks,
+the walk stops and the agent answers them; the result tells it how far it got.
+
+With no call, the page plays the recorded sentence and runs the same beats on the audio's clock,
+the words lit as they are said. A call opened after a walk played is told which one and how far, so
+the agent goes on from there instead of saying it again.

package/docs/write-a-tutor.md

@@ -0,0 +1,223 @@
+# Write a tutor agent
+
+> **Experimental.** `@pinecall/dot` may change in any release. This page is true of the version it ships with.
+
+A tutor is an agent that is also a person on your site: a dot in the corner that talks, and a
+pointer of its own that shows the visitor where things are. It never clicks or sends anything for
+them. This page takes you from an empty agent to one that answers a call, gives guided walks, and
+draws your own tools' results on the screen. Each step runs on its own, so stop where you have what
+you need.
+
+You need a Pinecall agent project (`pinecall new`, the [tutorial](https://docs.pinecall.io/start/tutorial/))
+and a site you may edit.
+
+## 1. The agent extends `Tutor`
+
+```bash
+pnpm add @pinecall/dot @pinecall/agents
+```
+
+```tsx
+// agents/pip/agent.tsx
+import { macros, Tutor } from "@pinecall/dot/agent";
+
+/**
+ * You are Pip, Brightside Dental's front desk, on the visitor's screen. Short sentences: everything you
+ * say is read aloud. Show them where things are on the page; never say you clicked something for them.
+ */
+export default class Pip extends Tutor {
+  static override macros = macros({});
+}
+```
+
+That is a tutor. `Tutor` adds the pointer's tools (`point`, `highlight`, `scrollTo`, `fill`, `home`
+and the rest), the page's events, and a view that prints the screen the visitor sees. If your class
+has a `render()` of its own, end it with `{this.seeing()}`, or the model never reads the page:
+
+```tsx
+override render() {
+  return (
+    <>
+      <p>Ask for the visitor's name before anything else.</p>
+      {this.seeing()}
+    </>
+  );
+}
+```
+
+A call that no tutor's page joined (the phone, a chat, a site with no dot) is the agent's alone:
+the pointer's tools stay hidden and the view says nothing of a screen. One class serves both.
+
+Hold it in the sandbox with `pinecall start`, as any agent.
+
+## 2. The site draws the dot
+
+The quickest way is the widget:
+
+```html
+<script type="module" src="https://cloud.pinecall.io/widget/pinecall-widget.js"></script>
+<pinecall-widget agent="pip" look="dot" tutor token-url="/pinecall/token"></pinecall-widget>
+```
+
+`token-url` is a door of yours that mints a seat for the visitor with your org's key (never put the
+key on the page; [tokens](https://docs.pinecall.io/api/tokens/)). Add `walks="sandbox"` while you
+build, to read the sandbox's walks.
+
+To own the code, build it from the two packages the widget uses:
+
+```ts
+import { room } from "@pinecall/room";
+import { tutor } from "@pinecall/dot";
+
+let pip: ReturnType<typeof tutor> | null = null;
+const call = room({
+  tokens: (scope, resume) => ticket(scope, resume),   // your door's answer
+  gateway: "https://cloud.pinecall.io",
+  onUi: (data) => void pip?.take(data),
+});
+pip = tutor(call, {
+  label: "Talk to Pip",
+  onClick: () => { pip?.wake(); void call.start("talk"); },   // inside the click: audio starts only there
+});
+```
+
+Open the page, click the dot and ask "what's on this page?". The agent reads the page as roles and
+names, the way Playwright's aria snapshot does, so **name the parts you want it to find**: a
+`<section>` needs an `aria-label` or `aria-labelledby` to be a region; a `<div>` you want to point
+at needs `role="group"` and an `aria-label`. [The README](../README.md#the-page-as-the-agent-reads-it)
+shows what the agent sees.
+
+## 3. Write a macro
+
+A macro is a guided walk: sentences the agent says as written, with moves hung on their words. Save
+it as `agents/pip/macros/tour.json`:
+
+```json
+{
+  "name": "tour",
+  "title": "Site tour",
+  "when": "the visitor asks to be shown around",
+  "steps": [
+    {
+      "say": "Up here are our services. Whitening is the one most people ask about.",
+      "beats": [
+        { "word": "Up", "do": "scroll_to", "find": ["region", "Services"] },
+        { "word": "Whitening", "do": "highlight", "find": ["article", "Whitening"] }
+      ]
+    }
+  ]
+}
+```
+
+and give it to the class:
+
+```tsx
+import tour from "./macros/tour.json" with { type: "json" };
+
+static override macros = macros({ tour });
+```
+
+Now "show me around" in a call runs it, and `pinecall start` records it as a walk (step 4). The
+[macro reference](macros.md) has every field, the moves, and how to write one that holds up.
+
+## 4. Walks a visitor plays with no call
+
+As the agent starts, `Tutor` records each sentence of each macro once, in the agent's own voice,
+and publishes them. The terminal says so:
+
+```
+pip  walk tour · 5 sentences · 0:38 · cartesia sonic-3 …
+```
+
+A visitor then plays them from the dot's menu with no call and nothing spent. Turn it on by naming
+where the walks are published:
+
+```html
+<pinecall-widget … walks></pinecall-widget>          <!-- production's; walks="sandbox" for the sandbox's -->
+```
+```ts
+tutor(call, { walks: "https://cloud.pinecall.io/v1/agents/pip/public/walks" });   // ?env=sandbox for the sandbox's
+```
+
+`pinecall walks list` shows what an agent published and `pinecall walks play tour` plays it in your
+terminal. A sentence you did not edit costs nothing the next time; one you edit is recorded again.
+
+**Show some walks on some pages.** Name the pages in the macro (`"pages": ["/security"]`) and the
+menu, and the agent in a call, offer it only there; with no `pages` a walk is for every page
+([Macros](macros.md#walks-for-some-pages)).
+
+**Tie a walk to the page.** A section marked `data-pinecall-walk="tour"` makes the menu mark that
+walk while the visitor is on that section. A button of yours marked `data-pinecall-play="tour"` and
+`hidden` is shown while that walk can play, and a click plays it:
+
+```html
+<section id="services" data-pinecall-walk="tour" aria-labelledby="services-h">…</section>
+<button data-pinecall-play="tour" hidden>Show me around</button>
+```
+
+## 5. Say one thing without a macro
+
+Mark any element with the line it should say:
+
+```html
+<article data-pinecall-say="Family Care covers up to four people for thirty-nine dollars a month.">
+```
+
+The agent says it, with the pointer on the element, when the visitor asks about it or presses the ▶
+that shows on it during a call. See [Lines on the page](../README.md#lines-on-the-page).
+
+## 6. Draw your own tools' results
+
+An agent has tools of its own: free times, a booking, a piece of code. Their results are on the
+call's log, so the page says how to draw each by the tool's name, on a card above the dot:
+
+```ts
+tutor(call, {
+  views: {
+    freeSlots: (host, result, tools) => {
+      const { times } = result as { times: string[] };
+      host.innerHTML = times.map((time) => `<button>${time}</button>`).join("");
+      host.onclick = (event) => tools.send((event.target as HTMLElement).textContent ?? "");
+    },
+  },
+});
+```
+
+With the widget, set `widget.views = { … }` on the element. A view may draw anything (React included:
+`createRoot(host)`, and return `() => root.unmount()` to take it down). `tools.send(text)` says
+something to the agent in writing, as the visitor; `tools.hide()` puts the card away.
+
+**Two things to get right:**
+
+- **The log must carry the tools.** A visitor's seat reads the call's log through the *public*
+  projection, which holds no tool runs. Your token door has to ask for the tenant's: `log: "tenant"`
+  when it mints (`mint(key, { …, log: "tenant" })` in `@pinecall/room/server`). Without it no card
+  ever appears and nothing says why.
+- **The log masks what the agent declared private.** A tool argument listed under `pii` reads
+  as `***` there. To fill a form with what the visitor said, have the tool tell the page, to this
+  visitor alone:
+
+  ```ts
+  if (this.onScreen()) this.call.send("pinecall.ui", { type: "form.fill", name, email });
+  ```
+
+  and handle it in `onUi` (`tutor.take(data)` is `false` for a message that is not its own):
+
+  ```ts
+  onUi: (data) => { if (pip?.take(data)) return; if (isFormFill(data)) fillMyForm(data); }
+  ```
+
+## 7. Try it, then ship it
+
+1. `pinecall start` holds the agent in the sandbox and records its walks there.
+2. Serve your site, with the dot's walks pointing at the sandbox (`walks="sandbox"`), and talk to it.
+   Everything a laptop does stays in the sandbox.
+3. `pinecall deploy` runs it in production, where it records its walks as it starts; the dot's
+   `walks` then points at production's.
+
+## What the tutor will not do
+
+It never clicks, submits a form or opens another page for the visitor. It writes into a field only
+what the visitor told it, never into a password, a card, a file or a hidden field. The page's words
+reach the model fenced as data, never as instructions: put what other people wrote (reviews,
+comments) under `data-pinecall-skip` so the agent does not read it at all.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@pinecall/dot",
-  "version": "0.11.23",
-  "description": "The talking dot: a face that speaks with the agent's voice, and a tutor that walks the page with a pointer of its own",
+  "version": "0.11.24",
+  "description": "EXPERIMENTAL. The talking dot: a face that speaks with the agent's voice, and a tutor that walks the page with a pointer of its own",
   "license": "Apache-2.0",
   "author": "Pinecall <hello@pinecall.io>",
   "repository": {
@@ -24,13 +24,14 @@
     }
   },
   "files": [
-    "dist"
+    "dist",
+    "docs"
   ],
   "dependencies": {
-    "@pinecall/room": "^0.11.23"
+    "@pinecall/room": "^0.11.24"
   },
   "peerDependencies": {
-    "@pinecall/agents": "^0.11.23"
+    "@pinecall/agents": "^0.11.24"
   },
   "peerDependenciesMeta": {
     "@pinecall/agents": {
@@ -43,7 +44,7 @@
     "livekit-client": "^2.22.3",
     "typescript": "7.0.2",
     "vitest": "^5.0.0",
-    "@pinecall/agents": "^0.11.23"
+    "@pinecall/agents": "^0.11.24"
   },
   "engines": {
     "node": ">=24"