@chalky/chalky 0.1.5 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +8 -2
  2. package/agent/SYSTEM.md +2 -1
  3. package/agent/extensions/app/README.md +15 -11
  4. package/agent/extensions/app/skills/test-your-app/SKILL.md +23 -0
  5. package/agent/extensions/board/browser.js +5 -9
  6. package/agent/extensions/card/index.ts +18 -18
  7. package/agent/extensions/file/browser.js +46 -7
  8. package/agent/extensions/file/index.ts +3 -2
  9. package/agent/extensions/frame/README.md +2 -2
  10. package/agent/extensions/frame/index.ts +16 -7
  11. package/agent/extensions/history/index.ts +3 -1
  12. package/agent/extensions/note/README.md +3 -1
  13. package/agent/extensions/note/browser.js +33 -7
  14. package/agent/extensions/note/guide.md +2 -1
  15. package/agent/extensions/note/index.ts +36 -13
  16. package/agent/extensions/page/browser.js +453 -0
  17. package/agent/extensions/page/index.ts +70 -1
  18. package/agent/extensions/page/page.html +47 -0
  19. package/agent/extensions/page/skills/interact-with-page/SKILL.md +59 -0
  20. package/agent/extensions/screen/browser.js +6 -19
  21. package/agent/extensions/version/browser.js +9 -0
  22. package/agent/extensions/version/index.ts +73 -21
  23. package/agent/skills/design/SKILL.md +1 -1
  24. package/agent/skills/show-me/SKILL.md +66 -0
  25. package/package.json +2 -1
  26. package/server.ts +169 -7
  27. package/node_modules/xlsx/CHANGELOG.md +0 -361
  28. package/node_modules/xlsx/LICENSE +0 -201
  29. package/node_modules/xlsx/README.md +0 -41
  30. package/node_modules/xlsx/bin/xlsx.njs +0 -310
  31. package/node_modules/xlsx/bower.json +0 -22
  32. package/node_modules/xlsx/dist/LICENSE +0 -201
  33. package/node_modules/xlsx/dist/cpexcel.d.ts +0 -39
  34. package/node_modules/xlsx/dist/cpexcel.full.mjs +0 -1502
  35. package/node_modules/xlsx/dist/cpexcel.js +0 -1506
  36. package/node_modules/xlsx/dist/shim.min.js +0 -2
  37. package/node_modules/xlsx/dist/xlsx.core.min.js +0 -18
  38. package/node_modules/xlsx/dist/xlsx.core.min.map +0 -1
  39. package/node_modules/xlsx/dist/xlsx.extendscript.js +0 -28259
  40. package/node_modules/xlsx/dist/xlsx.full.min.js +0 -24
  41. package/node_modules/xlsx/dist/xlsx.full.min.map +0 -1
  42. package/node_modules/xlsx/dist/xlsx.mini.min.js +0 -10
  43. package/node_modules/xlsx/dist/xlsx.mini.min.map +0 -1
  44. package/node_modules/xlsx/dist/xlsx.zahl.js +0 -4
  45. package/node_modules/xlsx/dist/xlsx.zahl.mjs +0 -2
  46. package/node_modules/xlsx/dist/zahl.d.ts +0 -4
  47. package/node_modules/xlsx/package.json +0 -183
  48. package/node_modules/xlsx/types/index.d.ts +0 -1065
  49. package/node_modules/xlsx/types/tsconfig.json +0 -15
  50. package/node_modules/xlsx/xlsx.js +0 -28103
  51. package/node_modules/xlsx/xlsx.mjs +0 -28228
  52. package/node_modules/xlsx/xlsxworker.js +0 -14
@@ -28,6 +28,9 @@
28
28
  Extending without editing this file: an extension's browser.js (extensions/<name>/browser.js; GET /page/scripts lists them);
29
29
  each script's default export gets pageApi: on(type, fn), send(obj) (-> the extensions' page handlers),
30
30
  slot("top" | "status" | "side"), tool(button) -> the row under that palette tool (for its sub-tools), note(text) -> the line, load(name), block(name, props) (blocks from extensions/card/blocks),
31
+ notice(content, onClose) -> {el, set(...content), close(), open}: a notice in the column at the top right, newest on
32
+ top (content: a string, a node, or a list of them; a <button class="go"> is the main action); its ✕ calls onClose,
33
+ then closes it; an action closes it with close(). Notices of every extension go here, so they never overlap.
31
34
  command(name, fn, description): "/name rest" typed in the input calls fn(rest) instead of sending (other "/..." text
32
35
  is sent); typing "/" lists the commands above the input, narrowing as you type.
33
36
  provide(name, obj) / use(name) -> promise of obj: what one browser plugin offers others (scripts load in any
@@ -82,6 +85,24 @@
82
85
  #status .sub:not(:has(> :not([hidden]))) { display: none; } /* only hidden buttons (or none): no row */
83
86
  #status .tool .badge { top: -4px; right: -4px; } /* on the corner, off the icon */
84
87
  #side { position: fixed; top: 0; right: 0; z-index: 20; display: flex; flex-direction: column; align-items: flex-end; }
88
+ /* notices (pageApi.notice): a column at the top right, newest on top; above cards and frames */
89
+ #notices { position: fixed; top: 14px; right: 16px; z-index: 200000; width: min(340px, calc(100vw - 32px));
90
+ display: flex; flex-direction: column; pointer-events: none; }
91
+ #notices .notice { display: grid; grid-template-rows: 1fr; pointer-events: auto;
92
+ transition: grid-template-rows .32s cubic-bezier(.3,.7,.3,1), opacity .25s, transform .32s cubic-bezier(.3,.7,.3,1); }
93
+ #notices .notice.out { grid-template-rows: 0fr; opacity: 0; transform: translateX(28px); } /* arriving or leaving: no height, so the rest slide */
94
+ #notices .notice > div { min-height: 0; overflow: hidden; }
95
+ #notices .notice-box { position: relative; margin-bottom: 8px; padding: 10px 34px 10px 14px; border-radius: 14px; display: flex;
96
+ flex-wrap: wrap; align-items: center; gap: 8px 10px; font-size: 13px; color: var(--text);
97
+ background: var(--glass-strong, #fff); border: 1px solid var(--edge, #eee); box-shadow: 0 8px 26px rgba(0,0,0,.12);
98
+ backdrop-filter: var(--blur); -webkit-backdrop-filter: var(--blur); }
99
+ #notices .notice-box button { border: 0; border-radius: 8px; padding: 5px 11px; font: inherit; font-size: 12px; cursor: pointer;
100
+ background: var(--line, #eee); color: var(--text); }
101
+ #notices .notice-box button.go { background: var(--accent, #0071e3); color: #fff; }
102
+ #notices .notice-box button.notice-x { position: absolute; top: 6px; right: 6px; width: 22px; height: 22px; padding: 0;
103
+ background: none; color: var(--muted); font-size: 13px; line-height: 22px; }
104
+ #notices .notice-box button.notice-x:hover { background: var(--hover, #eee); color: var(--text); }
105
+ @media (prefers-reduced-motion: reduce) { #notices .notice { transition: opacity .15s; } }
85
106
  #rail button, #status > button, #status > details > summary, #status > .tool > button, #status .sub > button {
86
107
  width: 38px; height: 38px; display: grid; place-items: center; border-radius: 12px; color: var(--muted);
87
108
  position: relative; list-style: none; cursor: pointer; transition: background 0.15s, color 0.15s; }
@@ -212,6 +233,7 @@
212
233
  </form>
213
234
  <nav id="status" data-name="tools"></nav>
214
235
  <aside id="side"></aside>
236
+ <div id="notices" aria-live="polite"></div>
215
237
  <script>
216
238
  const log = document.getElementById("log"), to = document.getElementById("to");
217
239
  const ws = new WebSocket((location.protocol === "https:" ? "wss://" : "ws://") + location.host + "/ws");
@@ -402,6 +424,31 @@
402
424
  command: (name, fn, description = "") => { commands[name] = { fn, description }; },
403
425
  provide: (name, obj) => { offered[name] = obj; for (const ok of wanted[name] || []) ok(obj); delete wanted[name]; },
404
426
  use: (name) => name in offered ? Promise.resolve(offered[name]) : new Promise((ok) => (wanted[name] ??= []).push(ok)),
427
+ notice: (content, onClose) => { // a notice at the top right, newest on top; ✕ calls onClose, then it goes
428
+ const item = Object.assign(document.createElement("div"), { className: "notice out" });
429
+ const inner = document.createElement("div"), box = Object.assign(document.createElement("div"), { className: "notice-box" });
430
+ const x = Object.assign(document.createElement("button"), { className: "notice-x", textContent: "✕", title: "Close" });
431
+ let open = true;
432
+ const handle = {
433
+ el: box,
434
+ set: (...nodes) => box.replaceChildren(...nodes, x), // new content, no new animation
435
+ close: () => {
436
+ if (!open) return;
437
+ open = false;
438
+ item.classList.add("out");
439
+ setTimeout(() => item.remove(), 360);
440
+ },
441
+ get open() { return open; },
442
+ };
443
+ x.onclick = () => { onClose?.(); handle.close(); };
444
+ handle.set(...[content].flat());
445
+ inner.append(box);
446
+ item.append(inner);
447
+ document.getElementById("notices").prepend(item);
448
+ item.offsetHeight; // laid out closed first, so opening animates
449
+ item.classList.remove("out");
450
+ return handle;
451
+ },
405
452
  tool: (button) => { // puts button in the palette with a row under it; returns the row, for its sub-tools
406
453
  const sub = Object.assign(document.createElement("div"), { className: "sub" });
407
454
  const wrap = Object.assign(document.createElement("span"), { className: "tool" });
@@ -0,0 +1,59 @@
1
+ ---
2
+ name: interact-with-page
3
+ description: Use the page the user has open the way they would, while they watch - read what's on screen, click, type, press keys, drag - in any card, app or part of the page. Use to check what the user sees, show them something, fill something in for them, reproduce what they pointed at, or try something you built.
4
+ ---
5
+ # Using the user's page
6
+
7
+ The page_run tool (load it: tool_search "page_run") runs JavaScript in the page the user has open, as it is right
8
+ now. With card=<id> it runs inside that card's app: `document` is the app's page. The code is the body of an async
9
+ function: `await` works, and you `return` what you want back.
10
+
11
+ ## Act with the helpers
12
+ They act as a person would: your pointer moves to the thing first, what you touch gets an outline with your name,
13
+ and typing goes letter by letter, so the user can follow.
14
+ - `await click(t)`
15
+ - `await type(t, text)`: replaces what the field holds (`{add: true}` adds to it). A dropdown picks the option with
16
+ that text; a slider, date or colour field is set to that value.
17
+ - `await press(key, t?)`: Enter, Escape, Tab, ArrowDown, "Ctrl+z"... to t, else to what has focus. Enter in a form
18
+ field submits the form.
19
+ - `await drag(t, dx, dy)` by pixels, or `await drag(t, other)` onto another thing. A draggable thing (a file row)
20
+ is dragged and dropped as the browser does it, so drop targets get it.
21
+ - `read(t?)`: what's on screen as short lines: headings, controls with their value and state ([selected],
22
+ [checked], [disabled]), short texts. Only what's in view; `read(null, {all: true})` for the whole page. The
23
+ conversation (the chat log) is left out: you know it; `read("#log")` if you need it.
24
+ - `await wait(ms)`, `await wait(".row.new")` (until it appears), `await wait(() => condition)`.
25
+
26
+ t is an element, a CSS selector, or the visible text: a button's words, a field's placeholder or label. Or a spot
27
+ `{x: 600, y: 300}` in window px (screen_see's picture, or an element's getBoundingClientRect()): whatever is on top
28
+ there gets it, as with a real mouse. Use a spot for empty space (`await drag({x: 400, y: 300}, 300, 200)` draws a
29
+ frame there) and for tools that work through a layer over the page (Point: click the tool, then the spot).
30
+ ```
31
+ page_run(card="habit-tracker", code=`await type("New habit", "Drink water");
32
+ await click("Add habit");
33
+ return read()`)
34
+ ```
35
+ Plain DOM code is fine for reading and checking precisely: `getComputedStyle(el)`, `el.scrollWidth > el.clientWidth`
36
+ (text cut off), state the app exposes. For acting, use the helpers: a bare `el.click()` happens at once, and the user
37
+ only sees an outline afterwards.
38
+
39
+ ## Look after each step, cheaply
40
+ One step, or a few, per page_run, ending with a small read: `return read()` or the one value you need. You see what
41
+ happened before you decide the next step. A read is about a hundred tokens; a picture (screen_see) is over a thousand
42
+ and stays in your conversation, so look at a picture when the look matters (layout, colours, clipped or overlapping
43
+ things), not after every step. From bash the same is `curl -X POST --data '{"code": "...", "card": "<id>"}' {page}/page/run`,
44
+ for a loop or a script.
45
+
46
+ ## The user is watching, and can take over
47
+ While you use the page a bar says so, with a Stop button. If the user clicks or types on the page, page_run answers
48
+ "the user took over": stop there, don't retry, and say in a sentence what you were doing. When they press
49
+ "Let <you> continue" you get a message: read the page again first, it may have changed.
50
+
51
+ ## Care
52
+ - It's the live page: clicks really happen and data really changes. Touch only what your task needs, and don't send
53
+ messages through the chat box.
54
+ - The page's Stop button and rollbacks refuse code: they are the user's.
55
+ - Events from code are not quite real input: hover styles, native dropdown lists, file pickers, and copy and paste
56
+ don't respond. For those, and for other window sizes, use the browser-check skill
57
+ (a separate headless browser).
58
+ - Errors the page logs while your code runs come back with the answer: read them.
59
+ - An answer marked "not on the board the user is looking at" ran on a card they can't see right now.
@@ -6,18 +6,10 @@
6
6
  // frame() the screen right now, as a canvas (null when not shared)
7
7
  // picture(full, box) crop a frame to box (window px {x, y, w, h}), upload it (POST /screen/shot): {path, url}
8
8
  // An agent can ask to see the screen (its tool screen_see, event screen_ask): if the tab is shared, a picture
9
- // of the whole window goes back at once; else a card asks "Share / No" first, since the agent's ask is no click
9
+ // of the whole window goes back at once; else a notice asks "Share" (✕: no) first, since the agent's ask is no click
10
10
  // (screen_reply {id, shot, window, main}).
11
11
  export default async function (page) {
12
12
  const area = page.slot("top");
13
- const style = document.createElement("style");
14
- style.textContent = `
15
- .screen-ask { position: fixed; top: 76px; left: 50%; transform: translateX(-50%); z-index: 100; display: flex; align-items: center; gap: 10px;
16
- padding: 8px 8px 8px 14px; border-radius: 14px; background: var(--glass-strong, #fff); border: 1px solid var(--edge, #eee);
17
- box-shadow: 0 8px 30px rgba(0, 0, 0, .12); font-size: 13px; color: var(--text); }
18
- .screen-ask button { border: 0; border-radius: 9px; padding: 5px 12px; font: inherit; cursor: pointer; background: var(--line, #eee); color: var(--text); }
19
- .screen-ask button.yes { background: var(--accent, #0071e3); color: #fff; }`;
20
- document.head.append(style);
21
13
 
22
14
  let stream = null, video = null;
23
15
  async function share() {
@@ -53,8 +45,8 @@ export default async function (page) {
53
45
  // ---- an agent asks to see the screen ----
54
46
  const asks = {}; // id -> its card, while the agent waits
55
47
  async function answer(id) {
56
- for (const card of Object.values(asks)) card.remove(); // not in the picture
57
- await new Promise((ok) => setTimeout(ok, 300)); // a fresh frame from the shared tab
48
+ for (const card of Object.values(asks)) card.close(); // not in the picture
49
+ await new Promise((ok) => setTimeout(ok, 400)); // it has slid away; a fresh frame from the shared tab
58
50
  const full = frame();
59
51
  const s = full && await picture(full, { x: 0, y: 0, w: innerWidth, h: innerHeight }).catch(() => null);
60
52
  const m = area.getBoundingClientRect(); // window (0, 0) in main-space px (as note/browser.js)
@@ -63,14 +55,9 @@ export default async function (page) {
63
55
  }
64
56
  page.on("screen_ask", ({ id, agent }) => {
65
57
  if (shared()) return answer(id);
66
- const card = Object.assign(document.createElement("div"), { className: "screen-ask" });
67
- const yes = Object.assign(document.createElement("button"), { className: "yes", textContent: "Share" });
68
- const no = Object.assign(document.createElement("button"), { textContent: "No" });
69
- card.append(`${agent} wants to see your screen`, yes, no);
58
+ const yes = Object.assign(document.createElement("button"), { className: "go", textContent: "Share" });
70
59
  yes.onclick = async () => { await share(); answer(id); };
71
- no.onclick = () => page.send({ type: "screen_reply", id, shot: null });
72
- asks[id] = card;
73
- document.body.append(card);
60
+ asks[id] = page.notice([`${agent} wants to see your screen`, yes], () => page.send({ type: "screen_reply", id, shot: null })); // ✕: no
74
61
  });
75
- page.on("screen_ask_done", ({ id }) => { asks[id]?.remove(); delete asks[id]; }); // answered (here or in another tab), or timed out
62
+ page.on("screen_ask_done", ({ id }) => { asks[id]?.close(); delete asks[id]; }); // answered (here or in another tab), or timed out
76
63
  }
@@ -3,6 +3,7 @@
3
3
  // with dom-to-image-more, then each same-site app frame's own page pasted in place (the library leaves frames
4
4
  // blank), crops to the cards plus a margin, and uploads it: PUT /version/<sha>/shot (the first page wins).
5
5
  // Can't be drawn: video, and pictures or frames from other sites (those parts stay blank).
6
+ // Also: when Chalky was updated at this start (GET /version/update), a notice says what changed or why it couldn't.
6
7
  const MARGIN = 32;
7
8
  let lib;
8
9
  const library = () => lib ??= new Promise((ok, no) => { // loaded once, on the first picture
@@ -51,6 +52,14 @@ function shoot(sha) { // one picture at a time
51
52
  export default async function (page) {
52
53
  if (new URLSearchParams(location.search).has("shot")) return; // the old headless view
53
54
  page.on("version", (ev) => setTimeout(() => shoot(ev.sha), 2000)); // after the page has redrawn what changed
55
+ fetch("/version/update").then((r) => r.json()).then((u) => {
56
+ if (!u) return;
57
+ const kept = u.conflicts?.length;
58
+ page.notice(u.error ? [`Chalky couldn't update to ${u.release}, so it runs as before. ${u.error}`]
59
+ : [`Updated to ${u.release}: ${u.files} file${u.files === 1 ? "" : "s"} changed.`
60
+ + (kept ? ` ${kept} kept your change; agent1 is merging the update into ${kept === 1 ? "it" : "them"}.` : "")
61
+ + " To undo it, go back a version in Versions."]);
62
+ }).catch(() => {});
54
63
  try { // the current version has no picture yet (made while no page was open): take it now
55
64
  const current = (await (await fetch("/version")).json()).find((v) => v.current);
56
65
  if (current && !current.shot) setTimeout(() => shoot(current.short), 3000); // once the cards are drawn
@@ -1,14 +1,21 @@
1
1
  // Version history of agent/ and built/ together: what the agents are and what they made, so one rollback puts both
2
2
  // back. The repo and the rule for what it keeps (not logins, endpoints or installed packages) are the host's
3
3
  // (host.history, server.ts: /recover uses them too); this extension commits, lists and rolls back, one operation at
4
- // a time. Its first commit is both folders as they were the first time, tagged "original".
5
- // - When an agent's turn is over (pi's agent_settled) and either folder changed: commit, author = that agent, message =
6
- // its reload reason, else what it was asked (Python: what it first said, often just "done"), else what it said.
4
+ // a time. Its first commit is both folders as they were the first time, tagged "original" (in an installed home,
5
+ // server.ts makes the first one: "Started with Chalky <version>"; it also merges each new release, as a version).
6
+ // - When an agent's turn is over (pi's agent_settled) and either folder changed: commit, author = that agent. The
7
+ // message's first line (the label) says what changed, made here from facts, not from what was said: the card
8
+ // extension's words for the agent's card changes ("showed trip"), then the other changed things named from
9
+ // their paths (NAMES: "new app planner", "changed extension card"), at most 3, then its reload reason, if any.
10
+ // The second line: the first line of what it was asked (detail, not shown in the list).
7
11
  // Event "version" to the pages ({agent, sha, message, files, restart: false}: they refresh what changed).
8
12
  // - Pictures for the panel: taken by the open page itself (browser.js here, with dom-to-image-more; no
9
13
  // headless browser), after each new version and, when a page opens, for the current version if it has none.
10
14
  // In data/version/shots/<sha>.png. A version made while no page was open and changed again before one opened: none.
11
- // - GET /version commits, newest first ({sha, short, author, date, message, current, shot})
15
+ // - GET /version the user's versions, newest first ({sha, short, author, date, message, current, shot}):
16
+ // host.history.versions(), so not the releases an update merged in
17
+ // GET /version/update this start's update of Chalky (server.ts's host.update), to the first page that
18
+ // asks (browser.js: a notice); later: null. {release, files, conflicts} or {release, error}
12
19
  // GET /version/<sha>/shot that commit's picture (404 if none; HEAD too)
13
20
  // PUT /version/<sha>/shot the page's picture (PNG, at most 10 MB; {saved: false} if it has one: the first page wins)
14
21
  // POST /version/<sha>/rollback that commit's files as a NEW commit (history is never rewritten, so a rollback
@@ -23,12 +30,13 @@ import git from "isomorphic-git";
23
30
  import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
24
31
 
25
32
  const host = (globalThis as any).chalky;
26
- const { repo, kept, stage } = host.history;
33
+ const { repo, stage, restore, versions } = host.history;
27
34
  const SHOTS = path.join(host.data, "version", "shots");
28
35
  const MAX_SHOT = 10 * 1024 * 1024;
29
36
  const who = (name: string) => ({ name, email: `${name}@chalky` });
30
37
 
31
- const shared = host.shared.get("version") ?? { queue: Promise.resolve(), turnFrom: {} as Record<string, number> };
38
+ const shared = host.shared.get("version") ?? { queue: Promise.resolve(), turnFrom: {} as Record<string, number>, cards: {} as Record<string, string[]> };
39
+ shared.cards ??= {}; // (shared state made by an older copy of this file, before a reload)
32
40
  host.shared.set("version", shared);
33
41
  function queued<T>(fn: () => Promise<T>): Promise<T> { // history operations one at a time (agents can finish together)
34
42
  const run = shared.queue.then(fn);
@@ -49,33 +57,71 @@ if (!shared.creating) shared.creating = queued(async () => {
49
57
 
50
58
  const head = async () => { await shared.queue; return git.resolveRef({ ...repo, ref: "HEAD" }); }; // after queued work (e.g. the first commit)
51
59
 
52
- async function commit(author: string, message: string) { // the changed files, or [] if nothing changed
60
+ async function commit(author: string, words: string | ((changed: string[]) => Promise<string>)) { // the changed files, or [] if nothing changed
53
61
  const changed = await stage();
54
62
  if (!changed.length) return [];
63
+ const message = typeof words === "string" ? words : await words(changed);
55
64
  let sha: string;
56
65
  try { sha = await git.commit({ ...repo, message, author: who(author), disallowEmpty: true }); }
57
66
  catch (e: any) { if (e.code === "EmptyCommitError") return []; throw e; } // nothing really changed
58
- host.broadcast("version", { agent: author, sha: sha.slice(0, 7), message, files: changed, restart: false }); // pages take its picture
67
+ host.broadcast("version", { agent: author, sha: sha.slice(0, 7), message: message.split("\n")[0], files: changed, restart: false }); // pages take its picture
59
68
  return changed;
60
69
  }
61
70
 
71
+ // A version's label: what changed, in words. The things a path is part of (anything else: the path itself):
72
+ const NAMES: [RegExp, string][] = [
73
+ [/^agent\/extensions\/([^/]+)\//, "extension $1"],
74
+ [/^agent\/skills\/([^/]+)\//, "skill $1"],
75
+ [/^agent\/SYSTEM\.md$/, "instructions"],
76
+ [/^agent\/settings\.json$/, "settings"],
77
+ [/^built\/apps\/([^/]+)\//, "app $1"],
78
+ [/^built\/card\.json$/, "cards"], // left out when the card extension's words say what changed
79
+ [/^built\/board\.json$/, "boards"],
80
+ [/^built\//, ""], // another file: its path in built/
81
+ ];
82
+ function nameOf(p: string) { // "agent/extensions/card/browser.js" -> "extension card"; "built/notes/a.txt" -> "notes/a.txt"
83
+ for (const [re, name] of NAMES) {
84
+ const m = p.match(re);
85
+ if (m) return name ? m[0].replace(re, name) : p.slice(m[0].length);
86
+ }
87
+ return p;
88
+ }
89
+ function describe(files: { path: string, how: "new" | "changed" | "removed" }[], cards: string[], reason?: string) {
90
+ const hows = new Map<string, Set<string>>(); // thing -> how its files changed
91
+ for (const f of files) { const t = nameOf(f.path); if (t !== "cards" || !cards.length) hows.set(t, (hows.get(t) ?? new Set()).add(f.how)); }
92
+ const parts = [...new Set(cards), ...[...hows].map(([t, h]) => `${h.size === 1 ? [...h][0] : "changed"} ${t}`)];
93
+ const label = parts.length > 3 ? `${parts.slice(0, 3).join("; ")} and ${parts.length - 3} more` : parts.join("; ");
94
+ return reason ? `${label}: ${reason.split("\n")[0]}` : label;
95
+ }
96
+
62
97
  const textOf = (m: any) => typeof m.content === "string" ? m.content
63
98
  : (m.content ?? []).filter((c: any) => c.type === "text").map((c: any) => c.text).join("\n");
64
99
 
65
100
  export default function (pi: ExtensionAPI) {
66
- pi.on("agent_start", (_e, ctx) => { shared.turnFrom[pi.getSessionName()!] = ctx.sessionManager.getEntries().length; });
101
+ // the card extension's words for each card change, kept per agent while its turn runs (the user's: not an agent's)
102
+ pi.on("session_start", () => host.api.card?.watch("version", (_id: string, by?: string, what?: string) => {
103
+ if (what && by) shared.cards[by]?.push(what);
104
+ }));
105
+ pi.on("agent_start", (_e, ctx) => {
106
+ const agent = pi.getSessionName()!;
107
+ shared.turnFrom[agent] = ctx.sessionManager.getEntries().length;
108
+ shared.cards[agent] = [];
109
+ });
67
110
  pi.on("agent_settled", (_e, ctx) => {
68
111
  const agent = pi.getSessionName()!;
69
112
  const turn = ctx.sessionManager.getEntries().slice(shared.turnFrom[agent] ?? 0).filter((e: any) => e.type === "message").map((e: any) => e.message);
70
113
  const reason = host.shared.get(`reload-reason ${agent}`);
71
114
  host.shared.delete(`reload-reason ${agent}`);
72
- let message = "";
73
- for (const text of [reason ?? "", ...turn.filter((m: any) => m.role === "user").map(textOf), ...turn.filter((m: any) => m.role === "assistant").map(textOf)]) {
74
- const line = text.split("\n").map((l: string) => l.trim()).find((l: string) => l && !/^\[\w+ (from|to) /.test(l));
75
- if (line) { message = line; break; }
76
- }
115
+ const cards = shared.cards[agent] ?? [];
116
+ delete shared.cards[agent];
117
+ const asked = turn.filter((m: any) => m.role === "user").map(textOf)
118
+ .flatMap((t: string) => t.split("\n").map((l) => l.trim())).find((l: string) => l && !/^\[\w+ (from|to) /.test(l)) ?? "";
77
119
  return queued(async () => {
78
- const changed = await commit(agent, message.slice(0, 100) || `changes by ${agent}`);
120
+ const changed = await commit(agent, async (files) => {
121
+ const before = new Set(await git.listFiles({ ...repo, ref: "HEAD" }));
122
+ const label = describe(files.map((p) => ({ path: p, how: !fs.existsSync(path.join(host.home, p)) ? "removed" : before.has(p) ? "changed" : "new" })), cards, reason);
123
+ return [label || `changes by ${agent}`, asked.slice(0, 200)].filter(Boolean).join("\n\n");
124
+ });
79
125
  // files in agent/ that aren't part of an agent, there now (a deleted one is the agent moving it: no new warning)
80
126
  const stray = changed.filter((p) => p.startsWith("agent/") && !host.isAgentPart(p.slice("agent/".length)) && fs.existsSync(path.join(host.home, p)));
81
127
  if (stray.length && host.agents.has(agent))
@@ -85,9 +131,14 @@ export default function (pi: ExtensionAPI) {
85
131
  });
86
132
  });
87
133
 
134
+ host.routes.set("/version/update", (_req: any, res: any) => {
135
+ const u = host.update && !host.update.told ? host.update : null;
136
+ if (u) u.told = true;
137
+ res.writeHead(200, { "content-type": "application/json" }).end(JSON.stringify(u));
138
+ });
88
139
  host.routes.set("/version", async (_req: any, res: any) => {
89
140
  const now = await head(); // waits for queued work, so the log is complete
90
- const out = (await git.log(repo)).map(({ oid, commit: c }) => ({ sha: oid, short: oid.slice(0, 7), author: c.author.name,
141
+ const out = (await versions()).map(({ oid, commit: c }) => ({ sha: oid, short: oid.slice(0, 7), author: c.author.name,
91
142
  date: new Date(c.author.timestamp * 1000).toISOString(), message: c.message.split("\n")[0], current: oid === now,
92
143
  shot: fs.existsSync(path.join(SHOTS, `${oid}.png`)) })); // the page takes the current one's picture if it has none
93
144
  res.writeHead(200, { "content-type": "application/json" }).end(JSON.stringify(out));
@@ -123,11 +174,10 @@ export default function (pi: ExtensionAPI) {
123
174
  if (what !== "rollback" || req.method !== "POST") return res.writeHead(404).end();
124
175
  const subject = `${sha.slice(0, 7)} ${(await git.readCommit({ ...repo, oid: sha })).commit.message.split("\n")[0]}`;
125
176
  await host.api.team?.stop();
126
- const files = await queued(async () => {
177
+ let files: string[];
178
+ try { files = await queued(async () => {
127
179
  await commit("user", "uncommitted changes before rollback"); // nothing is ever lost
128
- // that commit's agent/ and built/ into the folders and the index, removing files it doesn't have; the branch
129
- // stays where it is. What the history doesn't keep (logins, endpoints) isn't touched.
130
- await git.checkout({ ...repo, ref: sha, noUpdateHead: true, force: true, filepaths: kept.filepaths });
180
+ await restore(sha); // that commit's agent/ and built/ (server.ts)
131
181
  const files = await stage();
132
182
  // git keeps files, not folders: a folder whose files the rollback removed stays, empty (an extension folder
133
183
  // without index.ts would show in the map). Remove those, up to the first one that isn't empty.
@@ -136,7 +186,9 @@ export default function (pi: ExtensionAPI) {
136
186
  }
137
187
  await git.commit({ ...repo, message: `rollback to ${subject}`, author: who("user") });
138
188
  return files;
139
- });
189
+ }); } catch (e: any) { // an answer, not a request left hanging
190
+ return res.writeHead(500, { "content-type": "application/json" }).end(JSON.stringify({ error: String(e?.message ?? e) }));
191
+ }
140
192
  for (const name of host.agents.keys()) await host.reloadAgent(name);
141
193
  if (files.includes("built/card.json")) { host.shared.get("card")?.load(); host.api.history?.add("user", `rolled back to ${subject}`); }
142
194
  for (const name of host.agents.keys())
@@ -120,4 +120,4 @@ in it share a palette, a way of shading and a level of detail.
120
120
 
121
121
  ## 11. Check it
122
122
  Load browser-check and look at a screenshot: overlaps, clipped text, labels colliding, things outside their box.
123
- Fix what you see once, then stop polishing; the user will point at what they want changed.
123
+ Fix everything you see before you show it: the user marks up details, not missing basics.
@@ -0,0 +1,66 @@
1
+ ---
2
+ name: show-me
3
+ description: Answer "how do I…", "what does this do", "what can I do here" questions about Chalky itself by showing, not telling - explain each thing in a note pinned on it, use the page's own tools in front of the user with your pointer (a spare board, a test file), then hand over. Use when the user asks how to use Chalky, a tool, a button or a panel, or asks to be shown or walked through something on the page.
4
+ ---
5
+ # Showing the user how Chalky works
6
+
7
+ The user learns a tool by seeing it used on their own screen. So answer a how-question by doing it on the page,
8
+ slowly, while they watch: say what the thing is, right on it, use it as they would, show what came out, then let
9
+ them try. A text card or a long reply describing the tools is the thing to avoid: it is what they asked you not to
10
+ do. Don't build a tour app either: they asked about the page they have, not about what you could build.
11
+
12
+ ## How to show
13
+ - **Show what they asked about.** "How do frames work?" is one demo, shown fully. "What can I do?" is a short tour:
14
+ two or three tools, then offer the rest. "Show me all the tools" is all of them: keep each one short, one step
15
+ per tool (below). Ask nothing first: start, they can stop you.
16
+ - **One step per tool, in a tour.** Send a tool's calls together, in one step: delete the last tool's note, pin a
17
+ note on this tool's button, and one page_run that uses it once. For example, for Frames:
18
+ `note_done("n4", delete: true)`, `note_add(on: {element: "frames"}, text: "Frames: drag a box where something
19
+ should go and write what goes in it; Implement sends them to an agent. I'm drawing one now.")`, and `page_run`
20
+ that clicks the tool, drags a box at an empty spot and types in it. A tool that is better explained than used
21
+ (Dictate needs their microphone; Roll back is theirs to press) gets only the note.
22
+ - **Explain on the thing, in a note.** Before using a tool, pin a note on it (note_add) with a sentence or two: what
23
+ it is for and what you are about to do. The user reads it right where your pointer goes; your text between tool
24
+ calls never reaches the page. When you move on to the next thing, delete it (note_done(id, delete: true)) so only
25
+ one explanation is on the page at a time. For a quick pointer without words to read ("this one"), circle it in ink
26
+ instead (doodle_mark), and clear the mark (doodle_clear) when you move on.
27
+ - **Act with your pointer.** Use page_run's helpers (interact-with-page skill): click, type, drag, press. They move
28
+ your pointer, so the user can follow. One or two actions per page_run, ending with a small read, so you see what
29
+ happened before the next step. Check results with read() or a line of DOM code; look at the screen (screen_see)
30
+ only when the look itself matters.
31
+ - **Use spots for empty space.** Tools that work on the page itself (frames, the pen, text, the eraser, Point) need a
32
+ place, not a button: give a spot {x, y} in window px. `await drag({x: 700, y: 300}, 300, 180)` draws a frame or a
33
+ pen stroke there, `await click({x, y})` points or places, `await type({x, y}, "words")` clicks there and types
34
+ (into the box the click opened, or as the Text tool's words). Choose spots on the empty part of the board: first
35
+ close a panel that covers it (messages, versions: click its rail button), then check the spot, e.g.
36
+ `document.elementFromPoint(x, y).closest('#top') && !document.elementFromPoint(x, y).closest('[data-card]')`.
37
+ - **Never on their work.** Demo on a spare board (click "+" on the boards shelf) or with a test file you make.
38
+ Don't change, move or delete their cards, files or notes to show something.
39
+ - **Hand over, then tidy.** End with their turn: say in a sentence what to do next ("drag a box on the empty space
40
+ and type in it"), with the tool left on for them. Delete your explaining notes and clear your marks. Leave no
41
+ other tool on. Then ask whether to keep what the demo made (the board, the test file, a frame) or remove it, and
42
+ do what they say.
43
+ - **Point at what you must not press.** Rollback, Stop and deleting their things are theirs: put a note on the
44
+ button saying what it does. (The page refuses a rollback from your code anyway.)
45
+ - **If they take over** (page_run says so), stop: they are trying it. When they hand back, read the page again
46
+ before going on; they may have done the next step themselves.
47
+ - **Keep the reply short.** The page showed it. Your final reply says in two or three sentences what they saw and
48
+ what they can try.
49
+
50
+ ## The page, for showing
51
+ Names are the data-name to use in note_add / doodle_mark (on: {element: "..."}) and page_run (`[data-name=...]`).
52
+ The toolbar's order changes: find things by name, never by position.
53
+
54
+ | What | Name | What it does, and how to show it |
55
+ |---|---|---|
56
+ | Point / notes | note | Click anything (or draw around an area) and write: a note pinned to that thing, a thread with an agent. Show: click the tool, click a spot on a thing, type, Enter. |
57
+ | Doodle pen | pen | Draw and write on the page, click through it; Enter in the message line sends the drawing as steps (don't send it). Show: click the tool, drag at a spot. |
58
+ | Text | text | Words placed on the page, sent with the doodle. Show: click the tool, type at a spot. |
59
+ | Eraser | eraser | Takes away lines, words or unsent frames. Show: click the tool, drag over what you drew. |
60
+ | Hide marks | show-marks | Hides or shows all marks. |
61
+ | Frames | frames | Boxes the user lays out for agents to fill. Show: click the tool, drag a box at a spot, type in its box, then Implement (implement) sends them to an agent. |
62
+ | Files | files | Opens the files panel: everything in the drops folder. A row dragged onto the page opens a note there; a click opens a preview (spreadsheets, PDF, Word, slides, pictures). An empty folder shows an empty panel: write a test file into it first. |
63
+ | Microphone | dictate | Speech to text in the message line. |
64
+ | Messages / Versions | messages, versions | The left rail: the conversation; and the history of the page, with a picture of each version and Roll back. |
65
+ | Boards shelf | boards, new-board | Numbered boards; "+" makes a new one and goes to it. |
66
+ | Message line | message-line | Where the user types to the agents. Don't send messages through it. |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@chalky/chalky",
3
- "version": "0.1.5",
3
+ "version": "0.1.6",
4
4
  "description": "Chalky on pi: agents build what you see in the browser, and you point at it and mark it up",
5
5
  "license": "MIT",
6
6
  "type": "module",
@@ -29,6 +29,7 @@
29
29
  },
30
30
  "dependencies": {
31
31
  "@earendil-works/pi-coding-agent": "1.4.0",
32
+ "diff3": "0.0.3",
32
33
  "dom-to-image-more": "3.11.0",
33
34
  "dompurify": "3.4.17",
34
35
  "isomorphic-git": "1.44.2",