pi-pocket 0.29.0-beta.6 → 0.29.0-beta.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/features.md CHANGED
@@ -133,7 +133,7 @@ The full tour. For setup, see the [README](../README.md).
133
133
  - **Uploads.** In normal chat and PiP, attach files with the picker, paste files or images, or drop them into the message box. Files are saved on the server for the conversation you send to and their paths are given to the agent; supported images also go straight to models that accept them. That includes HEIC and HEIF photos (from an iPhone, say), which the server converts to JPEG for the model in a worker, one photo at a time, with limits of 30 seconds and 64 million pixels (a photo that big needs about 1 GB of memory while it converts), so a broken or huge photo gets a notice and does not hold up the server; the original stays in the session's uploads. When the image the model gets differs from the file, its line in the message says so: the format when it changes (`[Image: shown to you as JPEG.]`, always for a HEIC photo), and, in the words of Pi's `read` tool, the original size, the shown size, and the multiplier when it is made smaller. An image that does not change gets no note. A sent HEIC photo shows in the chat and in PiP as the JPEG the model got; when the model did not get one (a model without images, or a failed conversion), or when Pi Pocket cannot be sure which image is the photo's (the message names an image with `@`, or another attached image failed), it shows as a file. An image starts to be made ready for the model as soon as it uploads, so a send seldom waits for it. In a session's message box (not PiP's), attached files stay across a reload, as the draft does, and leave with your sign-out; while a message is on its way, the box dims and holds what it sends, its words, files, and quote, which no key, dictation, or × changes until it went (in PiP's box too, which empties as it sends), also when it shows again meanwhile (you went to another place and came back), so it cannot go twice; text sent to the box meanwhile (Send to Pi) goes in once it went or failed, and so does a queued message taken back with Edit, kept meanwhile in the browser, also through a reload or a move; the same words and files sent again after a lost answer, also after a reload, find the message that went instead of making a second; once you change them, or after an answer that said it was refused or failed, they go again; if the send fails, it is all there to send again, with its quote.
134
134
  - **Code in your messages.** In a message you send, `` `code` `` and a fence of ` ``` ` lines show as code, the way Pi's replies show it, colored when the fence names a language (` ```js `). A span closes with as many backticks as opened it, a fence's lines stand on their own, and a stray or unclosed one stays text. Nothing else is formatted: `*`, `#`, lists, and spacing stay as typed, and @mentions outside code stay buttons. Only the display changes: Pi, exports, and ↑ get the message as written.
135
135
  - **Rich replies.** Code blocks in Pi's replies show their language and a Copy button, with syntax colors for the common languages (no dependencies, so it works offline), and a block of more than 40 lines folds until you ask for the rest. Some show more than code, with the code a tab away: an `html` block (or a reply that is a whole page) is previewed as a page, inline, with its scripts off and nothing loaded from anywhere; **▶ Run** runs its scripts in the artifacts' sandbox, where it can reach the internet but not the app; an `svg` block is drawn as an image (tap it for full screen); and a `diff` or `patch` block opens in the diff viewer. While Pi is still writing, they show as code. Write cards color the file they write, and the file viewer previews `.html` files the same way, with Run. **Cards in the reply.** An `artifact` call with `inline: true` (an `artifact_inline` call, in a session from before 0.28.0) shows as a card in the reply, as the `html` block above with the artifact's name and version in its bar, an **Open** button for the artifact viewer at that version, and the same Preview/Code and Run/Stop; a failed, denied, or stopped call shows as a step with its reason instead. The result records the version its call made, so each card keeps showing its own: an update by Pi adds a new card, and earlier cards, in this session and a fork, do not change. A card's page is read from the artifact's own address, so whoever can see the session sees the card, a viewer and someone invited to this session alone included, and each can Run it. Once run, its page gets what an artifact's does: a follow-up for the message box (a notice for a viewer) and saved state under the artifact's id, shared with the viewer. A preview grows to fit the page, up to 1200 px, and a running page up to 900 px, then scrolls inside. On a phone the bar takes two lines. A viewer already open on an older version stays on it and offers "latest is vN" after an update. Only a call Pi makes itself is a card: one made inside a codemode script is one of its nested calls, and in PiP's panel the call is a line among its steps.
136
- - **Images inline.** Attached images show as thumbnails (also in the composer before you send). When Pi writes `![description](path)` with a path on the server (absolute, `~/…`, or relative to the session's folder), the image shows in its reply. Images a tool returns and SVG artifacts show inline too; an image Pi reads with `read` (often one you just sent) shows only when its call is opened, so it is not on screen twice. Tap any image to see it full screen. Only real image files are shown: the server checks a file's bytes, not just its name.
136
+ - **Images inline.** Attached images show as thumbnails (also in the composer before you send). When Pi writes `![description](path)` with a path on the server (absolute, `~/…`, or relative to the session's folder), the image shows in its reply. Images a tool returns and SVG artifacts show inline too; an image Pi reads with `read` (often one you just sent) shows only when its call is opened, so it is not on screen twice. `read` shrinks an image larger than the model takes (more than 2000 pixels a side, say) and converts BMP, with Photon in a worker thread that exits after 30 seconds idle; without Photon, the server still starts and `read` refuses such an image. Tap any image to see it full screen. Only real image files are shown: the server checks a file's bytes, not just its name.
137
137
  - **Launcher.** `npm start` asks how your devices should reach Pi Pocket: this device only, the local network, a Cloudflare Tunnel, Tailscale, or Tailscale Serve. It then shows the address and a QR code to sign in your phone, and keeps running with keys to restart the server or change access.
138
138
 
139
139
  ## Limitations
@@ -1,12 +1,12 @@
1
1
  {
2
2
  "name": "pi-pocket",
3
- "version": "0.29.0-beta.6",
3
+ "version": "0.29.0-beta.7",
4
4
  "lockfileVersion": 3,
5
5
  "requires": true,
6
6
  "packages": {
7
7
  "": {
8
8
  "name": "pi-pocket",
9
- "version": "0.29.0-beta.6",
9
+ "version": "0.29.0-beta.7",
10
10
  "license": "MIT",
11
11
  "dependencies": {
12
12
  "@earendil-works/chord": "1.5.2",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-pocket",
3
- "version": "0.29.0-beta.6",
3
+ "version": "0.29.0-beta.7",
4
4
  "description": "A durable, multiplayer, mobile-first web app for Pi agents, built on Pi Durable",
5
5
  "keywords": [
6
6
  "pi",
package/src/server/app.ts CHANGED
@@ -6,6 +6,7 @@
6
6
  * What people ask of Pi lives in `commands.ts`, the people's side of a session in `collab.ts`, push notifications in
7
7
  * `alerts.ts`, provider sign-ins in `providers.ts`, and each conversation's shared live view in `room.ts`.
8
8
  */
9
+ import { createRequire } from "node:module";
9
10
  import { homedir } from "node:os";
10
11
  import { isAbsolute, join } from "node:path";
11
12
  import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context";
@@ -44,7 +45,8 @@ import {
44
45
  import type { ExecutionEnv } from "@earendil-works/pi-durable/env";
45
46
  import { NodeExecutionEnv } from "@earendil-works/pi-durable/env/node";
46
47
  import { openNodeSqliteStorage } from "@earendil-works/pi-durable/storage/sqlite/node";
47
- import { CodingTools } from "@earendil-works/pi-durable/tools";
48
+ import { createNodePhotonImages } from "@earendil-works/pi-durable/images/node";
49
+ import { CodingTools, createCodingTools } from "@earendil-works/pi-durable/tools";
48
50
  import { Alerts } from "./alerts.ts";
49
51
  import { Attribution, type Missing } from "./attribution.ts";
50
52
  import { Browsers } from "./browser.ts";
@@ -855,7 +857,7 @@ export class PocketApp {
855
857
 
856
858
  const registry = createRegistry();
857
859
 
858
- registry.install(CodingTools);
860
+ registry.install(await codingTools(this.#log));
859
861
  // Durable work of the app itself, whatever extension modules are on.
860
862
  registry.install(
861
863
  defineExtension({
@@ -4339,3 +4341,22 @@ export function installRequestShaping(
4339
4341
  });
4340
4342
  };
4341
4343
  }
4344
+
4345
+ /**
4346
+ * Pi Durable's coding tools, whose `read` shrinks and converts images with Photon, in a worker thread that exits when
4347
+ * idle. Without Photon's WebAssembly, `read` sends only images that fit the model's limits as they are.
4348
+ */
4349
+ async function codingTools(log: (line: string) => void) {
4350
+ try {
4351
+ const node = import.meta.resolve("@earendil-works/pi-durable/images/node");
4352
+
4353
+ // Where Pi Durable loads it from, so a missing package shows now and not on the first image.
4354
+ createRequire(node).resolve("@silvia-odwyer/photon/photon_rs_bg.wasm");
4355
+
4356
+ return createCodingTools({ images: await createNodePhotonImages() });
4357
+ } catch (error) {
4358
+ log(`read cannot shrink images, Photon is missing: ${describe(error)}`);
4359
+
4360
+ return CodingTools;
4361
+ }
4362
+ }