@chalky/chalky 0.1.3 → 0.1.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -6
- package/{app → agent}/SYSTEM.md +16 -11
- package/agent/extensions/app/README.md +68 -0
- package/{app/extensions/ui → agent/extensions/app}/agents.js +22 -16
- package/agent/extensions/app/guide.md +5 -0
- package/agent/extensions/app/index.ts +66 -0
- package/{app/extensions/ui/skills/agent-apps → agent/extensions/app/skills/app-agents}/SKILL.md +17 -18
- package/agent/extensions/board/browser.js +359 -0
- package/agent/extensions/board/guide.md +4 -0
- package/agent/extensions/board/index.ts +100 -0
- package/agent/extensions/card/README.md +88 -0
- package/agent/extensions/card/blocks/Button.js +34 -0
- package/{app/extensions/ui/components → agent/extensions/card/blocks}/Column.js +2 -2
- package/agent/extensions/card/blocks/Markdown.js +37 -0
- package/{app/extensions/ui/components/Card.js → agent/extensions/card/blocks/Panel.js} +3 -3
- package/{app/extensions/ui/components → agent/extensions/card/blocks}/Row.js +2 -2
- package/{app/extensions/ui/components → agent/extensions/card/blocks}/TextField.js +3 -3
- package/{app/extensions/ui → agent/extensions/card}/browser.js +155 -128
- package/agent/extensions/card/guide.md +13 -0
- package/agent/extensions/card/index.ts +370 -0
- package/agent/extensions/command/browser.js +13 -0
- package/agent/extensions/command/guide.md +2 -0
- package/agent/extensions/command/index.ts +94 -0
- package/agent/extensions/doodle/README.md +57 -0
- package/agent/extensions/doodle/browser.js +1013 -0
- package/agent/extensions/doodle/guide.md +22 -0
- package/agent/extensions/doodle/index.ts +274 -0
- package/agent/extensions/file/browser.js +143 -0
- package/agent/extensions/file/guide.md +5 -0
- package/{app/extensions/files → agent/extensions/file}/index.ts +34 -16
- package/{app/extensions/frames → agent/extensions/frame}/README.md +13 -13
- package/{app/extensions/frames → agent/extensions/frame}/browser.js +24 -19
- package/agent/extensions/frame/guide.md +3 -0
- package/agent/extensions/frame/index.ts +81 -0
- package/{app/extensions/ui_history → agent/extensions/history}/browser.js +18 -18
- package/{app/extensions/ui_history → agent/extensions/history}/index.ts +32 -33
- package/{app → agent}/extensions/mcp/browser.js +1 -1
- package/agent/extensions/mcp/guide.md +3 -0
- package/{app → agent}/extensions/mcp/index.ts +2 -4
- package/agent/extensions/model/browser.js +363 -0
- package/{app → agent}/extensions/model/index.ts +8 -4
- package/agent/extensions/model/login.ts +215 -0
- package/agent/extensions/note/README.md +35 -0
- package/agent/extensions/note/browser.js +356 -0
- package/agent/extensions/note/guide.md +8 -0
- package/agent/extensions/note/index.ts +192 -0
- package/{app → agent}/extensions/page/index.ts +2 -2
- package/{app → agent}/extensions/page/page.html +55 -49
- package/agent/extensions/prompt/index.ts +38 -0
- package/{app → agent}/extensions/reload/index.ts +3 -3
- package/{app → agent}/extensions/screen/browser.js +4 -4
- package/{app → agent}/extensions/screen/index.ts +11 -11
- package/{app/extensions/steps → agent/extensions/step}/index.ts +1 -1
- package/{app/extensions/swarm → agent/extensions/team}/README.md +19 -19
- package/{app/extensions/swarm → agent/extensions/team}/guide.md +4 -4
- package/{app/extensions/swarm → agent/extensions/team}/index.ts +41 -66
- package/{app/extensions/versions → agent/extensions/version}/browser.js +10 -10
- package/{app/extensions/versions → agent/extensions/version}/index.ts +39 -54
- package/{app → agent}/extensions/viewer/README.md +4 -4
- package/{app → agent}/extensions/viewer/browser.js +8 -8
- package/agent/extensions/viewer/guide.md +3 -0
- package/{app → agent}/extensions/viewer/index.ts +22 -17
- package/agent/extensions/voice/browser.js +72 -0
- package/agent/extensions/voice/index.ts +39 -0
- package/{app/extensions/ui → agent/extensions/voice}/voice.js +5 -5
- package/agent/settings.json +8 -0
- package/{app/extensions/ui/skills/ui-check → agent/skills/browser-check}/SKILL.md +4 -4
- package/{app/extensions/ui/skills/ui-check → agent/skills/browser-check}/scripts/check.mjs +1 -1
- package/{app/extensions/ui/skills/ui-design → agent/skills/design}/SKILL.md +14 -14
- package/agent/skills/extension-map/SKILL.md +56 -0
- package/{app → agent}/skills/write-skill/SKILL.md +1 -1
- package/bin/chalky.mjs +3 -3
- package/package.json +8 -8
- package/server.ts +162 -63
- package/app/extensions/appdata/README.md +0 -10
- package/app/extensions/appdata/guide.md +0 -4
- package/app/extensions/appdata/index.ts +0 -47
- package/app/extensions/doodle/README.md +0 -15
- package/app/extensions/doodle/browser.js +0 -388
- package/app/extensions/doodle/guide.md +0 -6
- package/app/extensions/doodle/index.ts +0 -54
- package/app/extensions/files/browser.js +0 -96
- package/app/extensions/files/guide.md +0 -7
- package/app/extensions/frames/guide.md +0 -7
- package/app/extensions/frames/index.ts +0 -76
- package/app/extensions/mcp/guide.md +0 -5
- package/app/extensions/point/README.md +0 -22
- package/app/extensions/point/browser.js +0 -245
- package/app/extensions/point/guide.md +0 -6
- package/app/extensions/point/index.ts +0 -97
- package/app/extensions/prompt/index.ts +0 -19
- package/app/extensions/ui/README.md +0 -69
- package/app/extensions/ui/components/App.js +0 -28
- package/app/extensions/ui/components/Button.js +0 -28
- package/app/extensions/ui/components/Markdown.js +0 -37
- package/app/extensions/ui/guide.md +0 -7
- package/app/extensions/ui/index.ts +0 -271
- package/app/extensions/viewer/guide.md +0 -5
- package/app/mcp.json +0 -3
- package/app/skills/plugin-map/SKILL.md +0 -49
- package/app/ui/apps/.keep +0 -0
- /package/{app/extensions/ui/components → agent/extensions/card/blocks}/Table.js +0 -0
- /package/{app/extensions/ui/components → agent/extensions/card/blocks}/Text.js +0 -0
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
<!-- The page the user sees, served fresh on every load by the page extension (extensions/page/; edits show on reload).
|
|
3
3
|
Design (pivot 2026-10-14, see the Chalky repo's NOTES.md "Direction"): what the agents build fills the screen; talk is secondary.
|
|
4
4
|
Layout, every part fixed over the full window:
|
|
5
|
-
#top main space: what the agents built (the
|
|
5
|
+
#top main space: what the agents built (the card extension draws its cards here). Scrolls.
|
|
6
6
|
#rail left edge: icons that open a panel; clicking the open one closes it.
|
|
7
7
|
#panel a glass panel over the main space: #messages (the conversation) or #versions (history,
|
|
8
8
|
a screenshot of each version, rollback; the user's way to undo the agent's own changes: keep it working).
|
|
@@ -14,9 +14,12 @@
|
|
|
14
14
|
A tool with sub-tools (frames' Implement) is a <span class="tool">: its button, then a
|
|
15
15
|
smaller row (.sub) that hangs directly under it, hidden while none of its buttons shows (pageApi.tool).
|
|
16
16
|
#side right edge: empty; a browser plugin fills it (the files plugin's file panel).
|
|
17
|
+
Names (data-name) an agent's pen can point at: tools (the palette), rail, messages, versions, message-line; the
|
|
18
|
+
palette's buttons get theirs from their plugins (pen, text, eraser, show-marks, note, frames, files, dictate).
|
|
17
19
|
Look: glass layers on one soft background, tokens in :root (colours, glass, blur, sizes); icons are inline
|
|
18
20
|
SVG from ICONS (data-icon="name").
|
|
19
|
-
Messages: only words are full text (the user's, and an agent's reply);
|
|
21
|
+
Messages: only words are full text (the user's, and an agent's reply); a message the page wrote for the user
|
|
22
|
+
("[note on the page] ...") shows their words, the whole text in its data-raw (plugins find their ids there); everything else an agent does
|
|
20
23
|
(reasoning, tool calls, results, messages between agents) is a small icon: hover to peek, click to read.
|
|
21
24
|
App agents (status "app": part of something built, e.g. a simulation's members) stay out of the messages,
|
|
22
25
|
the picker and the working rows (one row per app): their app shows their talk.
|
|
@@ -24,7 +27,7 @@
|
|
|
24
27
|
"status", "message", "version", errors. Sending: {to, text} is a message, {type: "stop"} stops all.
|
|
25
28
|
Extending without editing this file: an extension's browser.js (extensions/<name>/browser.js; GET /page/scripts lists them);
|
|
26
29
|
each script's default export gets pageApi: on(type, fn), send(obj) (-> the extensions' page handlers),
|
|
27
|
-
slot("top" | "status" | "side"), tool(button) -> the row under that palette tool (for its sub-tools), note(text), load(name),
|
|
30
|
+
slot("top" | "status" | "side"), tool(button) -> the row under that palette tool (for its sub-tools), note(text) -> the line, load(name), block(name, props) (blocks from extensions/card/blocks),
|
|
28
31
|
command(name, fn, description): "/name rest" typed in the input calls fn(rest) instead of sending (other "/..." text
|
|
29
32
|
is sent); typing "/" lists the commands above the input, narrowing as you type.
|
|
30
33
|
provide(name, obj) / use(name) -> promise of obj: what one browser plugin offers others (scripts load in any
|
|
@@ -32,11 +35,11 @@
|
|
|
32
35
|
<meta charset="utf-8">
|
|
33
36
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
34
37
|
<title>Chalky</title>
|
|
35
|
-
<!-- Libraries that
|
|
38
|
+
<!-- Libraries that blocks import by name (the host's dependencies, served by the card extension). Must come before any module. -->
|
|
36
39
|
<script type="importmap">
|
|
37
40
|
{ "imports": {
|
|
38
|
-
"marked": "/
|
|
39
|
-
"dompurify": "/
|
|
41
|
+
"marked": "/card/node_modules/marked/lib/marked.esm.js",
|
|
42
|
+
"dompurify": "/card/node_modules/dompurify/dist/purify.es.mjs"
|
|
40
43
|
} }
|
|
41
44
|
</script>
|
|
42
45
|
<style>
|
|
@@ -113,17 +116,17 @@
|
|
|
113
116
|
mask-image: var(--fade); -webkit-mask-image: var(--fade); }
|
|
114
117
|
#log:hover { --fade: linear-gradient(transparent, #000 56px); }
|
|
115
118
|
#log > :first-child { margin-top: auto; } /* few messages: they sit at the bottom (flex-end would break scrolling) */
|
|
116
|
-
.
|
|
119
|
+
.group { margin: 0 0 22px; animation: rise 0.4s ease-out; }
|
|
117
120
|
@keyframes rise { from { opacity: 0; transform: translateY(14px); } }
|
|
118
121
|
.who { display: flex; align-items: center; gap: 6px; font-size: 11.5px; font-weight: 500; letter-spacing: 0.01em;
|
|
119
122
|
color: var(--c, var(--muted)); }
|
|
120
123
|
.who svg { width: 16px; height: 16px; }
|
|
121
124
|
.who time { margin-left: auto; font-weight: 400; color: var(--muted); opacity: 0; transition: opacity 0.2s; }
|
|
122
|
-
.
|
|
125
|
+
.group:hover .who time { opacity: 1; }
|
|
123
126
|
.text { margin: 3px 0 0 22px; color: var(--soft); white-space: pre-wrap; overflow-wrap: anywhere; letter-spacing: -0.005em; }
|
|
124
|
-
.
|
|
127
|
+
.group:not([data-who="you"]) .text { color: var(--text); }
|
|
125
128
|
.text .to { color: var(--muted); font-size: 12px; margin-right: 4px; }
|
|
126
|
-
.text >
|
|
129
|
+
.text > block-markdown, .text > block-markdown > p:first-child { display: inline; } /* first line next to .to */
|
|
127
130
|
.steps { display: flex; flex-wrap: wrap; gap: 1px; margin: 4px 0 0 19px; }
|
|
128
131
|
.step { color: var(--muted); padding: 3px; border-radius: 6px; opacity: 0.8; }
|
|
129
132
|
.step svg { width: 14px; height: 14px; }
|
|
@@ -187,9 +190,9 @@
|
|
|
187
190
|
</style>
|
|
188
191
|
<body data-panel="messages">
|
|
189
192
|
<div id="top"></div>
|
|
190
|
-
<nav id="rail">
|
|
191
|
-
<button id="show-messages" data-icon="messages" title="Messages"><span class="badge" id="unread"></span></button>
|
|
192
|
-
<button id="show-versions" data-icon="versions" title="Versions"></button>
|
|
193
|
+
<nav id="rail" data-name="rail">
|
|
194
|
+
<button id="show-messages" data-name="messages" data-icon="messages" title="Messages"><span class="badge" id="unread"></span></button>
|
|
195
|
+
<button id="show-versions" data-name="versions" data-icon="versions" title="Versions"></button>
|
|
193
196
|
</nav>
|
|
194
197
|
<aside id="panel">
|
|
195
198
|
<section id="messages"><div id="log"></div></section>
|
|
@@ -201,13 +204,13 @@
|
|
|
201
204
|
</section>
|
|
202
205
|
</aside>
|
|
203
206
|
<div id="working" hidden><div id="busy"></div><button id="stop" data-icon="stop" title="Stop all agents"></button></div>
|
|
204
|
-
<form id="f">
|
|
207
|
+
<form id="f" data-name="message-line">
|
|
205
208
|
<div id="cmds" hidden></div>
|
|
206
209
|
<select id="to" title="Send to" hidden></select>
|
|
207
210
|
<input id="text" placeholder="Tell the agents what to build or change" autocomplete="off">
|
|
208
211
|
<button data-icon="send" title="Send"></button>
|
|
209
212
|
</form>
|
|
210
|
-
<nav id="status"></nav>
|
|
213
|
+
<nav id="status" data-name="tools"></nav>
|
|
211
214
|
<aside id="side"></aside>
|
|
212
215
|
<script>
|
|
213
216
|
const log = document.getElementById("log"), to = document.getElementById("to");
|
|
@@ -247,15 +250,15 @@
|
|
|
247
250
|
to.hidden = to.options.length < 2;
|
|
248
251
|
}
|
|
249
252
|
|
|
250
|
-
// ----
|
|
251
|
-
//
|
|
252
|
-
//
|
|
253
|
+
// ---- Block runtime: the catalog of blocks (/card/blocks/<Name>.js), each a custom element <block-name> with
|
|
254
|
+
// update(props, card). The page draws replies with it; browser parts use page.load / page.block (the card
|
|
255
|
+
// extension draws the agents' cards from the same catalog).
|
|
253
256
|
let catalog = null;
|
|
254
|
-
const loading = {}; // name -> promise: true once <
|
|
257
|
+
const loading = {}; // name -> promise: true once <block-name> is defined, false if there is no such block
|
|
255
258
|
const loaded = {}; // name -> version loaded (custom elements can't be redefined: changes need a reload)
|
|
256
259
|
async function fetchCatalog() { // false: the agent didn't answer (e.g. restarting); the catalog stays as it was
|
|
257
260
|
try {
|
|
258
|
-
const r = await fetch("/
|
|
261
|
+
const r = await fetch("/card/blocks");
|
|
259
262
|
if (r.ok) { catalog = await r.json(); return true; }
|
|
260
263
|
} catch {}
|
|
261
264
|
catalog ??= {};
|
|
@@ -265,32 +268,32 @@
|
|
|
265
268
|
return loading[name] ??= (async () => {
|
|
266
269
|
if (!catalog || !(name in catalog)) await fetchCatalog(); // maybe just written
|
|
267
270
|
if (!(name in catalog)) return false;
|
|
268
|
-
const tag = `
|
|
269
|
-
if (!customElements.get(tag)) customElements.define(tag, (await import(`/
|
|
271
|
+
const tag = `block-${name.toLowerCase()}`;
|
|
272
|
+
if (!customElements.get(tag)) customElements.define(tag, (await import(`/card/blocks/${name}.js?v=${catalog[name]}`)).default);
|
|
270
273
|
loaded[name] = catalog[name];
|
|
271
274
|
return true;
|
|
272
275
|
})().catch((e) => { delete loading[name]; throw e; }); // a broken file: try again next time
|
|
273
276
|
}
|
|
274
|
-
async function
|
|
275
|
-
try { if (!(await load(name))) return null; } catch (e) { console.warn(`
|
|
276
|
-
const el = document.createElement(`
|
|
277
|
+
async function block(name, props = {}) { // a ready element, or null if the block is missing/broken
|
|
278
|
+
try { if (!(await load(name))) return null; } catch (e) { console.warn(`block ${name}:`, e); return null; }
|
|
279
|
+
const el = document.createElement(`block-${name.toLowerCase()}`);
|
|
277
280
|
el.update(props, { children: [], mount() {}, set() {}, action() {} });
|
|
278
281
|
return el;
|
|
279
282
|
}
|
|
280
283
|
// After an agent saves a version (end of its turn), show what it changed without a manual refresh:
|
|
281
|
-
// an
|
|
282
|
-
// (the browser can't redefine a custom element). New
|
|
284
|
+
// an app's files -> reload that app's frame; this page, a browser.js or a block in use -> reload the page
|
|
285
|
+
// (the browser can't redefine a custom element). New blocks need nothing: they load on first use.
|
|
283
286
|
async function refreshAfter(files) {
|
|
284
287
|
if (!(await fetchCatalog())) return reloadWhenUp(); // can't tell what changed: reload once the agent answers
|
|
285
288
|
for (const [n, p] of Object.entries(loading)) p.then((ok) => { if (!ok) delete loading[n]; }); // maybe created now
|
|
286
289
|
const changed = Object.keys(loaded).filter((n) => catalog[n] !== loaded[n]);
|
|
287
|
-
if (changed.length || files.some((f) => f === "extensions/page/page.html" || f.endsWith("/browser.js"))) return location.reload();
|
|
288
|
-
const apps = files.filter((f) => f.startsWith("
|
|
290
|
+
if (changed.length || files.some((f) => f === "agent/extensions/page/page.html" || f.endsWith("/browser.js"))) return location.reload();
|
|
291
|
+
const apps = files.filter((f) => f.startsWith("built/apps/")).map((f) => `/built/apps/${f.split("/")[2]}/`);
|
|
289
292
|
for (const frame of document.querySelectorAll("iframe"))
|
|
290
293
|
if (apps.some((a) => new URL(frame.src, location.href).pathname.startsWith(a))) frame.contentWindow.location.reload();
|
|
291
294
|
}
|
|
292
295
|
|
|
293
|
-
// ---- Messages:
|
|
296
|
+
// ---- Messages: groups of consecutive messages from one speaker (an agent, or "you"). A group has a
|
|
294
297
|
// header (robot in the agent's colour, name, time), then words as text and other work as rows of icons.
|
|
295
298
|
let ts = null; // time of the event being shown (set per event)
|
|
296
299
|
let unread = 0; // replies that arrived while the messages panel was closed
|
|
@@ -300,11 +303,11 @@
|
|
|
300
303
|
if (atBottom) log.scrollTop = log.scrollHeight;
|
|
301
304
|
}
|
|
302
305
|
|
|
303
|
-
function
|
|
306
|
+
function group(who) {
|
|
304
307
|
const last = log.lastElementChild;
|
|
305
308
|
if (last?.dataset.who === who) return last;
|
|
306
309
|
const b = document.createElement("div");
|
|
307
|
-
b.className = "
|
|
310
|
+
b.className = "group";
|
|
308
311
|
b.dataset.who = who;
|
|
309
312
|
if (who !== "you") b.style.setProperty("--c", colour(who));
|
|
310
313
|
b.innerHTML = `<div class="who">${icon(who === "you" ? "person" : "robot")}<span></span><time></time></div>`;
|
|
@@ -318,18 +321,19 @@
|
|
|
318
321
|
const div = document.createElement("div");
|
|
319
322
|
div.className = "text";
|
|
320
323
|
if (toWhom) div.append(Object.assign(document.createElement("span"), { className: "to", textContent: `→ ${toWhom}` }));
|
|
321
|
-
const span = document.createElement("span"); // plain at once; swapped for the Markdown
|
|
324
|
+
const span = document.createElement("span"); // plain at once; swapped for the Markdown block when ready
|
|
322
325
|
span.textContent = words;
|
|
323
326
|
div.append(span);
|
|
324
|
-
follow(() =>
|
|
325
|
-
|
|
327
|
+
follow(() => group(who).append(div));
|
|
328
|
+
block("Markdown", { text: words }).then((el) => el && follow(() => span.replaceWith(el)));
|
|
326
329
|
if (who !== "you" && document.body.dataset.panel !== "messages")
|
|
327
330
|
document.getElementById("unread").textContent = ++unread;
|
|
331
|
+
return div;
|
|
328
332
|
}
|
|
329
333
|
|
|
330
334
|
function step(agent, kind, title, content, error = false) { // one icon in the agent's row; click to read it
|
|
331
335
|
follow(() => {
|
|
332
|
-
const b =
|
|
336
|
+
const b = group(agent);
|
|
333
337
|
let row = b.lastElementChild;
|
|
334
338
|
if (row.classList.contains("detail")) row = row.previousElementSibling; // the row whose step is open
|
|
335
339
|
if (!row.classList.contains("steps")) b.append((row = Object.assign(document.createElement("div"), { className: "steps" })));
|
|
@@ -351,8 +355,10 @@
|
|
|
351
355
|
});
|
|
352
356
|
}
|
|
353
357
|
|
|
354
|
-
function note(words) { // a quiet line: joined, saved a version, errors, ...
|
|
355
|
-
|
|
358
|
+
function note(words) { // a quiet line: joined, saved a version, errors, ... (returns it, for a plugin to act on)
|
|
359
|
+
const line = Object.assign(document.createElement("div"), { className: "note", textContent: words });
|
|
360
|
+
follow(() => log.append(line));
|
|
361
|
+
return line;
|
|
356
362
|
}
|
|
357
363
|
|
|
358
364
|
// ---- Working: animated dots for each agent that isn't idle (from "status" events), Stop beside them
|
|
@@ -391,8 +397,8 @@
|
|
|
391
397
|
send: (obj) => ws.send(JSON.stringify(obj)), // arrives at the extensions' page handlers
|
|
392
398
|
slot: (id) => document.getElementById(id), // "top": the main space; "status": the tool palette (top); "side": right edge
|
|
393
399
|
note, // note(text): a quiet line in the messages
|
|
394
|
-
load, // load(name): define
|
|
395
|
-
|
|
400
|
+
load, // load(name): define block <block-name>; resolves false if there is no such block
|
|
401
|
+
block, // block(name, props): a ready element, or null
|
|
396
402
|
command: (name, fn, description = "") => { commands[name] = { fn, description }; },
|
|
397
403
|
provide: (name, obj) => { offered[name] = obj; for (const ok of wanted[name] || []) ok(obj); delete wanted[name]; },
|
|
398
404
|
use: (name) => name in offered ? Promise.resolve(offered[name]) : new Promise((ok) => (wanted[name] ??= []).push(ok)),
|
|
@@ -409,7 +415,7 @@
|
|
|
409
415
|
});
|
|
410
416
|
|
|
411
417
|
const lastFrom = {}; // agent -> who its current turn's answer goes to
|
|
412
|
-
const asked = {}; //
|
|
418
|
+
const asked = {}; // team_ask / team_tell tool-call ids
|
|
413
419
|
|
|
414
420
|
ws.onmessage = (e) => {
|
|
415
421
|
const ev = JSON.parse(e.data);
|
|
@@ -433,11 +439,11 @@
|
|
|
433
439
|
const m = /^\[(\w+) from ([^\s\]]+)[^\]]*\][^\n]*\n?/.exec(content);
|
|
434
440
|
const from = m ? m[2] : "user", words = m ? content.slice(m[0].length) : content;
|
|
435
441
|
if (ev.reply_to) lastFrom[a] = ev.reply_to; // set on the message that starts a turn
|
|
436
|
-
// a message the page wrote for the user ("[note on the
|
|
442
|
+
// a message the page wrote for the user ("[note on the page] <their words>" + lines for the agent):
|
|
437
443
|
// show their words; the lines meant for the agent go behind an icon
|
|
438
444
|
const tagged = from === "user" && /^\[([^\]]+)\] ?(.*)/.exec(words);
|
|
439
445
|
if (tagged) {
|
|
440
|
-
text("you", tagged[2] || tagged[1], agents.size > 1 ? a : null);
|
|
446
|
+
text("you", tagged[2] || tagged[1], agents.size > 1 ? a : null).dataset.raw = words; // all of it, for plugins (doodle: its group)
|
|
441
447
|
if (words.includes("\n")) step("you", "inbox", `${tagged[1]}: what ${a} got`, words);
|
|
442
448
|
} else if (from === "user") text("you", words, agents.size > 1 ? a : null);
|
|
443
449
|
// from another agent: already shown as its "asks"/"tells" step, or as its answer
|
|
@@ -454,12 +460,12 @@
|
|
|
454
460
|
for (const c of ev.tool_calls || []) {
|
|
455
461
|
let args = {};
|
|
456
462
|
try { args = JSON.parse(c.function.arguments); } catch {}
|
|
457
|
-
if (c.function.name === "
|
|
458
|
-
else if (c.function.name === "
|
|
463
|
+
if (c.function.name === "team_ask") { asked[c.id] = true; step(a, "ask", `${a} asks ${args.to}`, args.text || ""); }
|
|
464
|
+
else if (c.function.name === "team_tell") { asked[c.id] = true; step(a, "ask", `${a} tells ${args.to}`, args.text || ""); }
|
|
459
465
|
else step(a, "tool", `${a} → ${c.function.name}`, c.function.arguments);
|
|
460
466
|
}
|
|
461
467
|
}
|
|
462
|
-
//
|
|
468
|
+
// a team_ask result is the other agent's answer, already shown as its reply; a team_tell result is
|
|
463
469
|
// just "sent" (errors still shown)
|
|
464
470
|
// a result may be a list of parts (view_image): text, and [image <path>] for an image
|
|
465
471
|
const res = typeof ev.content === "string" ? ev.content
|
|
@@ -562,7 +568,7 @@
|
|
|
562
568
|
};
|
|
563
569
|
|
|
564
570
|
async function loadVersions() {
|
|
565
|
-
try { versions = await (await fetch("/
|
|
571
|
+
try { versions = await (await fetch("/version")).json(); } catch { return; } // no versions plugin
|
|
566
572
|
sel = Math.max(0, versions.findIndex((v) => v.current));
|
|
567
573
|
renderWheel();
|
|
568
574
|
}
|
|
@@ -587,7 +593,7 @@
|
|
|
587
593
|
detail.textContent = `${v.short} · ${v.author} · ${new Date(v.date).toLocaleString()}` + (v.current ? " · current" : "");
|
|
588
594
|
rollbackBtn.disabled = v.current;
|
|
589
595
|
preview.hidden = !v.shot; // a version made while no page was open has no picture
|
|
590
|
-
if (v.shot) preview.src = `/
|
|
596
|
+
if (v.shot) preview.src = `/version/${v.sha}/shot`;
|
|
591
597
|
}
|
|
592
598
|
|
|
593
599
|
function select(i) {
|
|
@@ -602,7 +608,7 @@
|
|
|
602
608
|
if (!confirm(`Roll back to "${v.message}" (${v.short})?\n\nThe agent restarts and busy agents are interrupted. ` +
|
|
603
609
|
`Later changes stay in the history, so you can roll forward again.`)) return;
|
|
604
610
|
note(`rolling back to ${v.short}…`);
|
|
605
|
-
await fetch(`/
|
|
611
|
+
await fetch(`/version/${v.sha}/rollback`, { method: "POST" });
|
|
606
612
|
};
|
|
607
613
|
|
|
608
614
|
openPanel(SHOT ? "" : document.body.dataset.panel);
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
// The prompt after SYSTEM.md (pi's documented way to replace its default prompt, plain text): one section per
|
|
2
|
+
// extension, its guide.md, then <paths>. A guide.md is found by looking in each extension folder, as the page finds
|
|
3
|
+
// browser.js, so an extension has nothing to register. Read again on every input (an edited guide shows at once).
|
|
4
|
+
// In a guide: {agent} {built} {data} {workspace} {page} become the full paths and address, {name} and {role} this
|
|
5
|
+
// agent's, and README.md the full path of that extension's README.md. pi adds the skills list and <cwd> itself.
|
|
6
|
+
// READMEs and skills keep the placeholders as written (agents edit them; <paths> says what each one is).
|
|
7
|
+
import fs from "node:fs";
|
|
8
|
+
import path from "node:path";
|
|
9
|
+
import { getDocsPath, getExamplesPath, getReadmePath, type ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
10
|
+
|
|
11
|
+
const host = (globalThis as any).chalky;
|
|
12
|
+
const EXTENSIONS = path.dirname(import.meta.dirname);
|
|
13
|
+
|
|
14
|
+
export default function (pi: ExtensionAPI) {
|
|
15
|
+
pi.on("before_agent_start", (event) => {
|
|
16
|
+
const name = pi.getSessionName() ?? "", page = `http://127.0.0.1:${host.port}`;
|
|
17
|
+
const values: Record<string, string> = { agent: host.agent, built: host.built, data: host.data, workspace: host.workspace,
|
|
18
|
+
page, name, role: host.agents.get(name)?.role ?? "" };
|
|
19
|
+
const sections = event.systemPromptOptions.sections;
|
|
20
|
+
for (const ext of fs.readdirSync(EXTENSIONS).sort()) {
|
|
21
|
+
const guide = path.join(EXTENSIONS, ext, "guide.md");
|
|
22
|
+
if (!fs.existsSync(guide)) continue;
|
|
23
|
+
sections[ext] = fs.readFileSync(guide, "utf8").replace(/\{(\w+)\}/g, (m, k) => values[k] ?? m)
|
|
24
|
+
.replaceAll("README.md", path.join(EXTENSIONS, ext, "README.md"));
|
|
25
|
+
}
|
|
26
|
+
sections.paths = [
|
|
27
|
+
`{agent} = ${host.agent}: you: your prompt, settings, extensions/ and skills/`,
|
|
28
|
+
`{built} = ${host.built}: what you make for the user: card.json (the cards), board.json (the boards), apps/<name>/ (the apps)`,
|
|
29
|
+
` {agent} and {built} are versioned together: each turn that changes them is saved, and the user can roll both back.`,
|
|
30
|
+
`{data} = ${host.data}: what is kept while running (conversations, what users typed into cards and apps, notes; not versioned)`,
|
|
31
|
+
`{workspace} = ${host.workspace}: your working folder (<cwd>), for project work; drops/ in it holds the user's files`,
|
|
32
|
+
`{page} = ${page}: the user's page; a file in {built} is at {page}/built/<its path in {built}>`,
|
|
33
|
+
`The host: ${host.root}/server.ts`,
|
|
34
|
+
`pi's docs: ${getDocsPath()}/extensions.md for extensions, examples in ${getExamplesPath()}/extensions/; the rest in `
|
|
35
|
+
+ `${getReadmePath()} and ${getDocsPath()}/. Read pi's .md files completely and follow their links.`,
|
|
36
|
+
].join("\n");
|
|
37
|
+
});
|
|
38
|
+
}
|
|
@@ -1,8 +1,8 @@
|
|
|
1
|
-
//
|
|
1
|
+
// Reload: an agent's edits to itself take effect. The tool only asks; when this agent's turn
|
|
2
2
|
// has ended, the host calls pi's session.reload() (extensions, skills and prompt load again; the conversation stays).
|
|
3
3
|
// Not pi's reload-runtime.ts example: its pi.sendUserMessage("/reload-runtime") reaches the model as text in 1.4.0
|
|
4
4
|
// (expandPromptTemplates defaults to false), and the model kept calling the tool again.
|
|
5
|
-
// No process restart, so no start.sh (a rollback: the
|
|
5
|
+
// No process restart, so no start.sh (a rollback: the version extension, which reloads every agent).
|
|
6
6
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
7
7
|
import { Type } from "typebox";
|
|
8
8
|
|
|
@@ -13,7 +13,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
13
13
|
async execute(_id, a) {
|
|
14
14
|
const host = (globalThis as any).chalky, name = pi.getSessionName();
|
|
15
15
|
host.reloadAfterTurn.set(name, false); // not quiet: the page notes the reload
|
|
16
|
-
host.shared.set(`reload-reason ${name}`, a.reason); // the
|
|
16
|
+
host.shared.set(`reload-reason ${name}`, a.reason); // the version extension's commit message for this turn
|
|
17
17
|
return { content: [{ type: "text", text: "Reload scheduled for when your turn ends. End your turn now; don't call reload again." }],
|
|
18
18
|
details: undefined };
|
|
19
19
|
},
|
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
//
|
|
2
|
-
// of what the user sees. Offered as page.provide("screen", {...});
|
|
1
|
+
// The page side of the screen extension: this tab, shared once (getDisplayMedia), for every extension that needs a picture
|
|
2
|
+
// of what the user sees. Offered as page.provide("screen", {...}); an extension gets it with await page.use("screen"):
|
|
3
3
|
// share() ask the browser to share this tab (only works right after a click: transient activation);
|
|
4
4
|
// resolves true if shared. The user can stop sharing any time (the browser's bar).
|
|
5
5
|
// shared() whether the tab is shared now
|
|
6
6
|
// frame() the screen right now, as a canvas (null when not shared)
|
|
7
7
|
// picture(full, box) crop a frame to box (window px {x, y, w, h}), upload it (POST /screen/shot): {path, url}
|
|
8
|
-
// An agent can ask to see the screen (its tool
|
|
8
|
+
// An agent can ask to see the screen (its tool screen_see, event screen_ask): if the tab is shared, a picture
|
|
9
9
|
// of the whole window goes back at once; else a card asks "Share / No" first, since the agent's ask is no click
|
|
10
10
|
// (screen_reply {id, shot, window, main}).
|
|
11
11
|
export default async function (page) {
|
|
@@ -57,7 +57,7 @@ export default async function (page) {
|
|
|
57
57
|
await new Promise((ok) => setTimeout(ok, 300)); // a fresh frame from the shared tab
|
|
58
58
|
const full = frame();
|
|
59
59
|
const s = full && await picture(full, { x: 0, y: 0, w: innerWidth, h: innerHeight }).catch(() => null);
|
|
60
|
-
const m = area.getBoundingClientRect(); // window (0, 0) in main-space px (as
|
|
60
|
+
const m = area.getBoundingClientRect(); // window (0, 0) in main-space px (as note/browser.js)
|
|
61
61
|
page.send({ type: "screen_reply", id, shot: s?.path || null, window: { width: innerWidth, height: innerHeight },
|
|
62
62
|
main: { x: area.scrollLeft - m.left, y: area.scrollTop - m.top } });
|
|
63
63
|
}
|
|
@@ -1,9 +1,9 @@
|
|
|
1
|
-
// Screen: what the user sees, as pictures
|
|
1
|
+
// Screen: what the user sees, as pictures. The
|
|
2
2
|
// browser half shares the tab once (for every extension: doodle's steps, an agent's look) and uploads pictures
|
|
3
|
-
// cropped from it. This half keeps them and gives agents the tool
|
|
4
|
-
// (their size, scroll, panels, running apps;
|
|
5
|
-
// POST /screen/shot a png (at most MAX_BYTES): saved in
|
|
6
|
-
// event "screen_ask" {id, agent} to every open page, when an agent calls
|
|
3
|
+
// cropped from it. This half keeps them and gives agents the tool screen_see: the user's window as it is now
|
|
4
|
+
// (their size, scroll, panels, running apps; browser-check's headless Chrome is a fresh window of its own).
|
|
5
|
+
// POST /screen/shot a png (at most MAX_BYTES): saved in data/screen, {path} back
|
|
6
|
+
// event "screen_ask" {id, agent} to every open page, when an agent calls screen_see
|
|
7
7
|
// page message screen_reply {id, shot, window: {width, height}, main: {x, y}}: the first answers
|
|
8
8
|
// (shot null: the user said no; main: window (0, 0) in main-space px)
|
|
9
9
|
// event "screen_ask_done" {id} to every page when answered or after ASK_WAIT s, so the asks go away
|
|
@@ -14,14 +14,14 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
|
14
14
|
import { Type } from "typebox";
|
|
15
15
|
|
|
16
16
|
const host = (globalThis as any).chalky;
|
|
17
|
-
const SHOTS = path.join(host.
|
|
17
|
+
const SHOTS = path.join(host.data, "screen");
|
|
18
18
|
const MAX_BYTES = 8 * 1024 * 1024;
|
|
19
19
|
const PNG = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
|
20
|
-
const ASK_WAIT = 90; // s for the user to answer
|
|
20
|
+
const ASK_WAIT = 90; // s for the user to answer screen_see
|
|
21
21
|
|
|
22
22
|
const isShot = (p: unknown) => typeof p === "string" && path.dirname(p) === SHOTS && fs.existsSync(p);
|
|
23
23
|
host.api.screen = { isShot };
|
|
24
|
-
// id -> resolve, while
|
|
24
|
+
// id -> resolve, while screen_see waits for a page (on the host: the tool is per agent, the page handler one)
|
|
25
25
|
const asks: Map<string, (msg: any) => void> = host.shared.get("screen-asks") ?? new Map();
|
|
26
26
|
host.shared.set("screen-asks", asks);
|
|
27
27
|
|
|
@@ -50,9 +50,9 @@ const result = (text: string, png?: string) => ({ details: undefined, content: [
|
|
|
50
50
|
|
|
51
51
|
export default function (pi: ExtensionAPI) {
|
|
52
52
|
pi.registerTool({
|
|
53
|
-
name: "
|
|
53
|
+
name: "screen_see", label: "Screen see", exposure: "deferred",
|
|
54
54
|
description: "See the user's screen as they see it right now: a picture of their whole window (their size, scroll, "
|
|
55
|
-
+ "open panels, apps as they are running).
|
|
55
|
+
+ "open panels, apps as they are running). browser-check's headless Chrome is a fresh window of its own, not theirs: when "
|
|
56
56
|
+ "the user talks about what they see, or before you say something looks right to them, look here. The first "
|
|
57
57
|
+ "time, the user is asked to share the tab (they may say no); then it's instant.",
|
|
58
58
|
parameters: Type.Object({}),
|
|
@@ -74,7 +74,7 @@ export default function (pi: ExtensionAPI) {
|
|
|
74
74
|
const mx = Math.round(m.x ?? 0), my = Math.round(m.y ?? 0);
|
|
75
75
|
return result(`the user's screen right now: their window is ${w.width}x${w.height} px; the picture is the whole window, `
|
|
76
76
|
+ `${pw}x${ph} px (${+k.toFixed(3)} picture px per window px). Window (0, 0) is main-space (${mx}, ${my}): a spot in `
|
|
77
|
-
+ `the picture is main-space x = ${mx} + picture x / ${+k.toFixed(3)}, the same for y (what
|
|
77
|
+
+ `the picture is main-space x = ${mx} + picture x / ${+k.toFixed(3)}, the same for y (what card_show's place uses).`, reply.shot);
|
|
78
78
|
},
|
|
79
79
|
});
|
|
80
80
|
}
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
//
|
|
1
|
+
// Step: a limit on steps (model calls) per input: pi's loop has none, and a model that
|
|
2
2
|
// keeps calling tools would run on (it did once: 55 rounds). At the limit the run is stopped and the agent says so.
|
|
3
3
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
4
4
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Your team (the
|
|
1
|
+
# Your team (the team extension)
|
|
2
2
|
Your name and role are in your team section.
|
|
3
3
|
You work with other AI agents as peers; there is no manager.
|
|
4
4
|
|
|
@@ -6,30 +6,30 @@ You work with other AI agents as peers; there is no manager.
|
|
|
6
6
|
- What you receive starts with a header: "[<kind> from <name> · <time>]". <name> is "user" (the human),
|
|
7
7
|
another agent, or a harness part ("system"). Kinds:
|
|
8
8
|
message (from the user or the harness), question (the sender is waiting for your answer),
|
|
9
|
-
|
|
9
|
+
request (the sender went on working), reply (the answer to a request you sent),
|
|
10
10
|
fyi (an update from an agent working for you; its final answer still comes).
|
|
11
|
-
- Your final answer in a turn is delivered for you: to the sender of a message, question or
|
|
11
|
+
- Your final answer in a turn is delivered for you: to the sender of a message, question or request; after a
|
|
12
12
|
reply, to whoever you were working for (the header says where). Say it once, there; don't repeat it.
|
|
13
|
-
-
|
|
13
|
+
- team_tell(to, text): send a request and keep working. If the agent is busy it sees the request between its tool
|
|
14
14
|
calls; if idle it starts on it. Its answer comes back to you later as a reply.
|
|
15
|
-
|
|
16
|
-
your final one, answer it with
|
|
17
|
-
A
|
|
18
|
-
reply to
|
|
19
|
-
- Waiting: when all that is left is waiting for others, end your turn. Replies and
|
|
15
|
+
Requests that arrive while you work show up between your tool calls: if one needs an answer other than
|
|
16
|
+
your final one, answer it with team_tell.
|
|
17
|
+
A team_tell to the agent you are working for is information only (it gets your final answer anyway and won't
|
|
18
|
+
reply to it). To ask it something, use team_ask.
|
|
19
|
+
- Waiting: when all that is left is waiting for others, end your turn. Replies and requests wake
|
|
20
20
|
you. Never sleep or poll (no sleep loops, no repeated status checks).
|
|
21
|
-
- Don't use
|
|
21
|
+
- Don't use team_tell to deliver your result: your final answer already gets there. Don't send thanks or
|
|
22
22
|
acknowledgements.
|
|
23
|
-
-
|
|
24
|
-
questions you can't continue without; use
|
|
25
|
-
in one step run in parallel. You can't ask an agent that is waiting on you (
|
|
26
|
-
-
|
|
23
|
+
- team_ask(to, text): ask and wait for the answer (it comes back as the result). Use it for quick
|
|
24
|
+
questions you can't continue without; use team_tell for work that takes a while. Several team_ask calls
|
|
25
|
+
in one step run in parallel. You can't ask an agent that is waiting on you (team_tell it instead).
|
|
26
|
+
- team_list shows the team and what each agent is doing right now (idle, working on what, waiting on
|
|
27
27
|
whom). Check it instead of asking agents for status.
|
|
28
|
-
-
|
|
28
|
+
- team_spawn adds a member (numbered agent1, agent2, ... in creation order; it says when the team is full).
|
|
29
29
|
An agent that is part of something you build (a simulation's member, a character) is an app agent,
|
|
30
|
-
not a worker: load the
|
|
30
|
+
not a worker: load the app-agents skill. team_spawn(..., app=<its card>) keeps its talk out of the
|
|
31
31
|
user's messages panel (the app shows it).
|
|
32
|
-
|
|
32
|
+
team_reset(name, or app=<card> for all of an app's agents; load it with tool_search) empties their memory, keeping their
|
|
33
33
|
roles: for a new round that should start fresh.
|
|
34
34
|
|
|
35
35
|
# Working as a team
|
|
@@ -42,6 +42,6 @@ You work with other AI agents as peers; there is no manager.
|
|
|
42
42
|
- Found a defect in someone else's work? Tell its owner what is wrong and how you found it; don't fix it
|
|
43
43
|
in their files yourself.
|
|
44
44
|
- When others' code will import yours, create the file (a stub) first so imports work while you build.
|
|
45
|
-
- Before saying something is missing or not done, check again (files,
|
|
45
|
+
- Before saying something is missing or not done, check again (files, team_list): it may have changed.
|
|
46
46
|
- Say so before installing anything or making a decision that affects others.
|
|
47
|
-
- Other agents may change the same files (in {
|
|
47
|
+
- Other agents may change the same files (in {agent} too). Check team_list and coordinate with team_tell.
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
# Your team
|
|
2
2
|
Your name: {name}. Your role: {role}. You work with other AI agents as peers; there is no manager.
|
|
3
3
|
- What you receive starts with "[<kind> from <name> · <time>]": a message (from the user or the harness), a
|
|
4
|
-
question (the sender waits for your answer), a
|
|
5
|
-
your
|
|
4
|
+
question (the sender waits for your answer), a request (the sender went on working), a reply (the answer to
|
|
5
|
+
your request) or an fyi (an update, no answer needed). Your final answer is delivered for you, to whoever the
|
|
6
6
|
header says: say it once, there.
|
|
7
|
-
-
|
|
8
|
-
and waits.
|
|
7
|
+
- team_tell(to, text) sends a request and you keep working; its answer comes back as a reply. team_ask(to, text)
|
|
8
|
+
asks and waits. team_list shows who is doing what now. team_spawn adds a member (it says when the team is full).
|
|
9
9
|
- When all that is left is waiting, end your turn: replies wake you. Never sleep or poll.
|
|
10
10
|
- Done means checked: run or test it, and say what you ran and what it showed.
|
|
11
11
|
- Simple work: do it yourself. Before you spawn, split or hand out work, or take work from another agent, read README.md.
|