pi-orca-images 0.2.3 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +35 -4
- package/package.json +1 -1
- package/src/carousel.ts +135 -0
- package/src/commands.ts +17 -1
- package/src/detect.ts +5 -0
- package/src/index.ts +67 -6
- package/src/labels.ts +67 -1
- package/src/preview.ts +99 -0
- package/src/status.ts +3 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.0
|
|
4
|
+
|
|
5
|
+
- `[Image N]` labels are links to their image file: Cmd-click a label to check the pasted image before sending it (Orca, iTerm2, Ghostty, WezTerm, Kitty).
|
|
6
|
+
- Pasted image preview: a thumbnail of one pasted image above the prompt, with a compact `[Image N] n/total · Ctrl+Shift+←/→` legend below it (just `[Image 1]` for a single image) and no Open image link. It follows new pastes and the label under the cursor; Ctrl+Shift+←/→ steps between images. `PI_IMAGE_PREVIEW=0` turns it off.
|
|
7
|
+
- `/orca-images status` reports whether the image preview is on.
|
|
8
|
+
- `/orca-images open N` opens `[Image N]` in macOS Preview, from the prompt or the last sent message, in any terminal.
|
|
9
|
+
|
|
10
|
+
## 0.3.0
|
|
11
|
+
|
|
12
|
+
- Open image previews: each inline image gets an **Open image** link, and `/orca-images open` opens the latest image in macOS Preview. Previews are written to temporary files. Thanks to [@Ky1ezer0](https://github.com/Ky1ezer0) (#1).
|
|
13
|
+
|
|
3
14
|
## 0.2.3
|
|
4
15
|
|
|
5
16
|
- Fix: Pi crashed with "This extension ctx is stale after session replacement or reload" when an `[Image N]` label was in the prompt after `/resume`, `/new`, `/fork` or `/reload`. Pi hands extensions a Proxy over the TUI that ignores `delete`, so the old session's render hook was never removed and kept styling labels with its stale ctx.
|
package/README.md
CHANGED
|
@@ -41,24 +41,53 @@ Orca renders inline images through xterm.js's image addon. Three things keep Pi'
|
|
|
41
41
|
|
|
42
42
|
Paste an image (or drop an image file) into Pi's prompt and the path is shown as a bold, colored `[Image 1]` label, as in OpenCode. Labels are numbered per message, move and delete as one unit, and become the real path again when you send the message, so the model sees exactly what it saw before.
|
|
43
43
|
|
|
44
|
+
Each label is also a link to its image file: **Cmd-click** `[Image 1]` to check what you pasted before sending it. In Orca this opens an image tab with zoom controls; other terminals open it if they support OSC 8 links (iTerm2, Ghostty, WezTerm, Kitty). In any terminal on macOS, `/orca-images open 1` opens `[Image 1]` in Preview. It looks in the current prompt first, then in the last sent message that had labels, since typing the command replaces the prompt.
|
|
45
|
+
|
|
44
46
|
It recognizes pastes made only of existing image files (`png`, `jpg`, `gif`, `webp`, `bmp`, `tiff`, `heic`, `avif`): plain paths, `~/` paths, quoted or backslash-escaped paths, and `file://` URLs. Text that mixes a path with other words is left alone. This works in every terminal, not only Orca, including Pi's own clipboard paste. Set `PI_IMAGE_LABELS=0` to turn it off.
|
|
45
47
|
|
|
48
|
+
## Pasted image preview
|
|
49
|
+
|
|
50
|
+
While the prompt has pasted images, a small thumbnail of one of them is shown above it, with a compact navigator legend below the thumbnail:
|
|
51
|
+
|
|
52
|
+
```text
|
|
53
|
+
[Image 2] 2/3 · Ctrl+Shift+←/→
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
With a single pasted image the legend is just `[Image 1]`. The thumbnail has no **Open image** link; Cmd-click its `[Image N]` label in the prompt instead.
|
|
57
|
+
|
|
58
|
+
It shows the image you just pasted; moving the cursor onto an `[Image N]` label shows that one, and **Ctrl+Shift+←/→** steps to the previous or next image. Only one thumbnail is drawn at a time, so the prompt stays in view and every terminal that draws Pi's images draws it the same way. The preview disappears when you send the message.
|
|
59
|
+
|
|
60
|
+
In terminals without inline images the legend follows a text line with the image type, size and file name. Set `PI_IMAGE_PREVIEW=0` to turn the preview off; it is also off when `PI_IMAGE_LABELS=0`.
|
|
61
|
+
|
|
62
|
+
## Open image previews
|
|
63
|
+
|
|
64
|
+
Each rendered image gets an **Open image** file link beneath it. **Cmd-click** the link on macOS to open the preview through your terminal. In our Orca test, this opens an Orca image tab with zoom controls. Direct image clicks and hover-to-zoom are not supported.
|
|
65
|
+
|
|
66
|
+
On macOS, `/orca-images open` opens the last image rendered by Pi in **Preview**, using `open -a Preview`.
|
|
67
|
+
|
|
68
|
+
Preview copies are saved in a private `pi-orca-images-*` directory under the OS temporary directory, never in the project. Identical images reuse a file within each loaded extension instance. Files survive `/reload` and are deleted on normal Pi process exit; links expire when their files are removed. Crashes or forced termination can leave files behind, and long-running sessions can still accumulate previews. Older `.pi/image-previews/` folders are not automatically removed.
|
|
69
|
+
|
|
70
|
+
The saved bytes are those available to Pi and may already have been resized by a tool. Previews require automatic Orca detection and a writable temporary directory. They are inactive when `PI_ORCA_IMAGES=0` or an explicit `PI_IMAGE_PROTOCOL` overrides detection. The link and Preview command were tested with Pi 1.5.2 on macOS; opening through the command requires macOS Preview.
|
|
71
|
+
|
|
46
72
|
## Commands
|
|
47
73
|
|
|
48
74
|
| Command | Effect |
|
|
49
75
|
|---|---|
|
|
50
|
-
| `/orca-images` or `/orca-images status` | Shows the detection decision, Pi's active image protocol, and whether the repaint fix and image
|
|
76
|
+
| `/orca-images` or `/orca-images status` | Shows the detection decision, Pi's active image protocol, and whether the repaint fix, image labels and image preview are on |
|
|
51
77
|
| `/orca-images test` | Draws a test gradient (it is not sent to the model) |
|
|
78
|
+
| `/orca-images open` | Opens the last rendered image in macOS Preview |
|
|
79
|
+
| `/orca-images open N` | Opens the pasted `[Image N]` in macOS Preview, from the prompt or the last sent message |
|
|
52
80
|
| `/orca-images redraw` | Re-sends every visible image right now |
|
|
53
81
|
|
|
54
82
|
## Configuration
|
|
55
83
|
|
|
56
|
-
|
|
84
|
+
The rendering fixes and preview links activate only in Orca; pasted image labels can work in any terminal. Your explicit choices always win:
|
|
57
85
|
|
|
58
86
|
| Setting | Effect |
|
|
59
87
|
|---|---|
|
|
60
|
-
| `PI_ORCA_IMAGES=0` | Disables the Orca image fixes |
|
|
61
|
-
| `PI_IMAGE_LABELS=0` | Disables pasted image labels |
|
|
88
|
+
| `PI_ORCA_IMAGES=0` | Disables the Orca image fixes and preview feature |
|
|
89
|
+
| `PI_IMAGE_LABELS=0` | Disables pasted image labels (and the preview above the prompt) |
|
|
90
|
+
| `PI_IMAGE_PREVIEW=0` | Disables the pasted image preview above the prompt |
|
|
62
91
|
| `PI_IMAGE_PROTOCOL=…` | Used as is; the package does not override it |
|
|
63
92
|
| `terminal.images` in Pi settings | Applied by Pi on top of the package's choice |
|
|
64
93
|
| `terminal.showImages: false` | Pi still hides images |
|
|
@@ -69,6 +98,8 @@ It does nothing outside Orca. Your explicit choices always win:
|
|
|
69
98
|
- Pi 1.4 in its default `fullscreen` TUI mode. In `regular` mode detection still works, but the repaint fix and tab recovery do not apply.
|
|
70
99
|
- Not inside tmux or screen: image escapes do not reach Orca through them.
|
|
71
100
|
- Image labels wrap internal methods of Pi's editor (`handlePaste`, `segment`, `expandPasteMarkers`, `render`). If a Pi release changes them, pasted paths simply stay as paths.
|
|
101
|
+
- The preview above the prompt reads the pasted file when it is shown; Ctrl+Shift+←/→ does nothing if your terminal or Pi keybindings already use those keys, but moving the cursor onto a label still switches the image.
|
|
102
|
+
- Preview links read internal fields of Pi's image component. If those fields change or preview copies cannot be saved, links are omitted.
|
|
72
103
|
- The repaint fix wraps two internal methods of Pi's TUI. If a Pi release renames them, `/orca-images status` reports the fix as `unavailable` and images fall back to the first-row behavior; detection keeps working.
|
|
73
104
|
|
|
74
105
|
## Development
|
package/package.json
CHANGED
package/src/carousel.ts
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
import { readFileSync } from "node:fs";
|
|
2
|
+
import { basename } from "node:path";
|
|
3
|
+
import { Image, truncateToWidth } from "@earendil-works/pi-tui";
|
|
4
|
+
import type { ImageEntry, ImageLabels } from "./labels.ts";
|
|
5
|
+
import { NO_PREVIEW_LINK } from "./preview.ts";
|
|
6
|
+
|
|
7
|
+
/** The parts of the editor's image labels the carousel reads. */
|
|
8
|
+
export type CarouselLabels = Pick<ImageLabels, "entries" | "cursorLabel" | "pastes" | "lastPasted">;
|
|
9
|
+
|
|
10
|
+
export interface CarouselView {
|
|
11
|
+
entries: ImageEntry[];
|
|
12
|
+
selected: ImageEntry;
|
|
13
|
+
index: number;
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
export const CAROUSEL_HINT = "Ctrl+Shift+←/→";
|
|
17
|
+
/** Thumbnail size in terminal cells: small enough to keep the prompt in view. */
|
|
18
|
+
const THUMBNAIL = { maxWidthCells: 24, maxHeightCells: 6 } as const;
|
|
19
|
+
|
|
20
|
+
/**
|
|
21
|
+
* Which pasted image the preview shows. The latest event wins: a paste shows the new image,
|
|
22
|
+
* the cursor moving onto a label shows that label, and a shortcut step shows the next one.
|
|
23
|
+
*/
|
|
24
|
+
export class Carousel {
|
|
25
|
+
#selected: number | undefined;
|
|
26
|
+
#lastCursor: number | undefined;
|
|
27
|
+
#lastPastes = 0;
|
|
28
|
+
#entries: ImageEntry[] = [];
|
|
29
|
+
|
|
30
|
+
resolve(labels: CarouselLabels | undefined): CarouselView | undefined {
|
|
31
|
+
const entries = labels?.entries() ?? [];
|
|
32
|
+
this.#entries = entries;
|
|
33
|
+
if (!labels || entries.length === 0) {
|
|
34
|
+
this.#selected = undefined;
|
|
35
|
+
return undefined;
|
|
36
|
+
}
|
|
37
|
+
if (labels.pastes !== this.#lastPastes) {
|
|
38
|
+
this.#lastPastes = labels.pastes;
|
|
39
|
+
this.#selected = labels.lastPasted;
|
|
40
|
+
}
|
|
41
|
+
const cursor = labels.cursorLabel();
|
|
42
|
+
if (cursor !== this.#lastCursor) {
|
|
43
|
+
this.#lastCursor = cursor;
|
|
44
|
+
if (cursor !== undefined) this.#selected = cursor;
|
|
45
|
+
}
|
|
46
|
+
let index = entries.findIndex((entry) => entry.id === this.#selected);
|
|
47
|
+
if (index === -1) index = entries.length - 1;
|
|
48
|
+
const selected = entries[index] ?? entries[0];
|
|
49
|
+
if (!selected) return undefined;
|
|
50
|
+
this.#selected = selected.id;
|
|
51
|
+
return { entries, selected, index };
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
/** Show the next (1) or previous (-1) image, wrapping around. */
|
|
55
|
+
step(delta: 1 | -1): void {
|
|
56
|
+
const count = this.#entries.length;
|
|
57
|
+
if (count === 0) return;
|
|
58
|
+
const current = Math.max(0, this.#entries.findIndex((entry) => entry.id === this.#selected));
|
|
59
|
+
this.#selected = this.#entries[(current + delta + count) % count]?.id;
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
export interface LoadedImage {
|
|
64
|
+
data: string;
|
|
65
|
+
mimeType: string;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
const MIME_TYPES: Record<string, string> = {
|
|
69
|
+
png: "image/png", jpg: "image/jpeg", jpeg: "image/jpeg", gif: "image/gif", webp: "image/webp",
|
|
70
|
+
bmp: "image/bmp", tif: "image/tiff", tiff: "image/tiff", heic: "image/heic", heif: "image/heif",
|
|
71
|
+
avif: "image/avif",
|
|
72
|
+
};
|
|
73
|
+
|
|
74
|
+
export function loadImageFile(path: string): LoadedImage | undefined {
|
|
75
|
+
const mimeType = MIME_TYPES[path.split(".").pop()?.toLowerCase() ?? ""];
|
|
76
|
+
if (!mimeType) return undefined;
|
|
77
|
+
try {
|
|
78
|
+
return { data: readFileSync(path).toString("base64"), mimeType };
|
|
79
|
+
} catch {
|
|
80
|
+
return undefined;
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
export interface CarouselWidgetOptions {
|
|
85
|
+
labels: () => CarouselLabels | undefined;
|
|
86
|
+
carousel: Carousel;
|
|
87
|
+
theme: { muted: (text: string) => string; fallback?: (text: string) => string };
|
|
88
|
+
load?: (path: string) => LoadedImage | undefined;
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
/**
|
|
92
|
+
* A preview above the prompt: one thumbnail of the selected image with a compact navigator
|
|
93
|
+
* legend below it, `[Image 2] 2/3 · Ctrl+Shift+←/→`, or just `[Image 1]` for a single image.
|
|
94
|
+
* Terminals without inline images get Pi's text fallback (type, size and file name).
|
|
95
|
+
*/
|
|
96
|
+
export function createCarouselWidget(options: CarouselWidgetOptions) {
|
|
97
|
+
const load = options.load ?? loadImageFile;
|
|
98
|
+
const { muted } = options.theme;
|
|
99
|
+
const images = new Map<string, Image | null>();
|
|
100
|
+
return {
|
|
101
|
+
render(width: number): string[] {
|
|
102
|
+
const view = options.carousel.resolve(options.labels());
|
|
103
|
+
if (!view) {
|
|
104
|
+
images.clear();
|
|
105
|
+
return [];
|
|
106
|
+
}
|
|
107
|
+
// Keyed by label, not path: pasting the same path again (an overwritten screenshot) reads it anew.
|
|
108
|
+
const key = (entry: ImageEntry) => `${entry.id}:${entry.path}`;
|
|
109
|
+
const live = new Set(view.entries.map(key));
|
|
110
|
+
for (const cached of images.keys()) if (!live.has(cached)) images.delete(cached);
|
|
111
|
+
|
|
112
|
+
const { selected, index, entries } = view;
|
|
113
|
+
const label = `[Image ${selected.id}]`;
|
|
114
|
+
const legend = muted(entries.length === 1 ? label : `${label} ${index + 1}/${entries.length} · ${CAROUSEL_HINT}`);
|
|
115
|
+
if (!images.has(key(selected))) {
|
|
116
|
+
const loaded = load(selected.path);
|
|
117
|
+
const image = loaded
|
|
118
|
+
? new Image(loaded.data, loaded.mimeType, { fallbackColor: options.theme.fallback ?? ((s) => s) }, {
|
|
119
|
+
...THUMBNAIL,
|
|
120
|
+
filename: basename(selected.path),
|
|
121
|
+
})
|
|
122
|
+
: null;
|
|
123
|
+
// The legend below names the image; an Open image link there would crowd it.
|
|
124
|
+
if (image) Object.assign(image, { [NO_PREVIEW_LINK]: true });
|
|
125
|
+
images.set(key(selected), image);
|
|
126
|
+
}
|
|
127
|
+
const image = images.get(key(selected));
|
|
128
|
+
const body = image ? image.render(width) : [truncateToWidth(muted(`Cannot read ${selected.path}`), width)];
|
|
129
|
+
return [...body, truncateToWidth(legend, width)];
|
|
130
|
+
},
|
|
131
|
+
invalidate() {
|
|
132
|
+
for (const image of images.values()) image?.invalidate();
|
|
133
|
+
},
|
|
134
|
+
};
|
|
135
|
+
}
|
package/src/commands.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
export const SUBCOMMANDS = ["status", "test", "redraw"] as const;
|
|
1
|
+
export const SUBCOMMANDS = ["status", "test", "redraw", "open"] as const;
|
|
2
2
|
|
|
3
3
|
/**
|
|
4
4
|
* Argument completions for /orca-images. A complete subcommand yields none, so the first
|
|
@@ -10,3 +10,19 @@ export function completeSubcommand(prefix: string): Array<{ value: string; label
|
|
|
10
10
|
const matches = SUBCOMMANDS.filter((value) => value.startsWith(typed)).map((value) => ({ value, label: value }));
|
|
11
11
|
return matches.length > 0 ? matches : null;
|
|
12
12
|
}
|
|
13
|
+
|
|
14
|
+
export type ParsedCommand =
|
|
15
|
+
| { sub: (typeof SUBCOMMANDS)[number]; image?: number }
|
|
16
|
+
| { sub: "invalid" };
|
|
17
|
+
|
|
18
|
+
/** Parse /orca-images arguments: a subcommand, plus an optional image number for `open`. */
|
|
19
|
+
export function parseCommand(args: string | undefined): ParsedCommand {
|
|
20
|
+
const [sub = "status", ...rest] = (args ?? "").trim().split(/\s+/).filter(Boolean);
|
|
21
|
+
if (!(SUBCOMMANDS as readonly string[]).includes(sub)) return { sub: "invalid" };
|
|
22
|
+
if (rest.length === 0) return { sub: sub as (typeof SUBCOMMANDS)[number] };
|
|
23
|
+
const [number] = rest;
|
|
24
|
+
if (sub === "open" && rest.length === 1 && number !== undefined && /^[1-9]\d*$/.test(number)) {
|
|
25
|
+
return { sub, image: Number(number) };
|
|
26
|
+
}
|
|
27
|
+
return { sub: "invalid" };
|
|
28
|
+
}
|
package/src/detect.ts
CHANGED
|
@@ -47,3 +47,8 @@ export function decide(env: Env): Decision {
|
|
|
47
47
|
export function labelsEnabled(env: Env): boolean {
|
|
48
48
|
return !DISABLED.has((env.PI_IMAGE_LABELS ?? "").toLowerCase());
|
|
49
49
|
}
|
|
50
|
+
|
|
51
|
+
/** The preview above the prompt shows pasted labels, so it needs them; PI_IMAGE_PREVIEW=0 turns it off. */
|
|
52
|
+
export function previewEnabled(env: Env): boolean {
|
|
53
|
+
return labelsEnabled(env) && !DISABLED.has((env.PI_IMAGE_PREVIEW ?? "").toLowerCase());
|
|
54
|
+
}
|
package/src/index.ts
CHANGED
|
@@ -1,16 +1,19 @@
|
|
|
1
1
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
|
2
2
|
import { Container, getCapabilities, Image, resetCapabilitiesCache, Text } from "@earendil-works/pi-tui";
|
|
3
|
-
import {
|
|
4
|
-
import {
|
|
3
|
+
import { Carousel, createCarouselWidget } from "./carousel.ts";
|
|
4
|
+
import { completeSubcommand, parseCommand, SUBCOMMANDS } from "./commands.ts";
|
|
5
|
+
import { decide, type Decision, labelsEnabled, OWNED_ENV, previewEnabled } from "./detect.ts";
|
|
5
6
|
import { type EditorWatcher, watchEditors } from "./labels.ts";
|
|
6
7
|
import { installShim, type Shim, type ShimTarget } from "./shim.ts";
|
|
8
|
+
import { installPreviews, openInPreview } from "./preview.ts";
|
|
7
9
|
import { formatStatus } from "./status.ts";
|
|
8
10
|
import { testImagePng } from "./test-image.ts";
|
|
9
11
|
|
|
10
12
|
const COMMAND = "orca-images";
|
|
11
13
|
const TEST_ENTRY = "pi-orca-images:test";
|
|
12
14
|
const WIDGET = "pi-orca-images:probe";
|
|
13
|
-
const
|
|
15
|
+
const PREVIEW_WIDGET = "pi-orca-images:preview";
|
|
16
|
+
const USAGE = `Usage: /${COMMAND} [${SUBCOMMANDS.join("|")}], or /${COMMAND} open N for [Image N]`;
|
|
14
17
|
|
|
15
18
|
/** Apply the decision to the process environment, before Pi reads its capabilities. */
|
|
16
19
|
function applyDecision(decision: Decision, env = process.env): void {
|
|
@@ -30,15 +33,24 @@ export default function piOrcaImages(pi: ExtensionAPI) {
|
|
|
30
33
|
const decision = decide(process.env);
|
|
31
34
|
applyDecision(decision);
|
|
32
35
|
const labels = labelsEnabled(process.env);
|
|
36
|
+
const preview = previewEnabled(process.env);
|
|
37
|
+
let carousel: Carousel | undefined;
|
|
38
|
+
let previewTui: { requestRender(): void } | undefined;
|
|
33
39
|
let shim: Shim | undefined;
|
|
34
40
|
let watcher: EditorWatcher | undefined;
|
|
41
|
+
let previews: ReturnType<typeof installPreviews> | undefined;
|
|
35
42
|
|
|
36
43
|
pi.on("session_start", async (_event, ctx) => {
|
|
44
|
+
if (ctx.hasUI && decision.action === "apply") {
|
|
45
|
+
previews?.uninstall();
|
|
46
|
+
previews = installPreviews({ onError: (message) => ctx.ui.notify(message, "error") });
|
|
47
|
+
}
|
|
37
48
|
const repaint = decision.action === "apply" && decision.protocol === "kitty";
|
|
38
49
|
if (!ctx.hasUI || (!repaint && !labels)) return;
|
|
39
50
|
// The widget factory is the only public hook that hands an extension the TUI instance.
|
|
40
51
|
ctx.ui.setWidget(WIDGET, (tui) => {
|
|
41
52
|
if (repaint) {
|
|
53
|
+
// SAFETY: installShim treats every TUI member except requestRender as optional and checks it.
|
|
42
54
|
shim = installShim(tui as unknown as ShimTarget);
|
|
43
55
|
// Resend whatever Pi drew before the fix was in place (first startup render, or a
|
|
44
56
|
// session switch on a Pi that calls this factory late). Deferred: Pi may call the
|
|
@@ -47,6 +59,7 @@ export default function piOrcaImages(pi: ExtensionAPI) {
|
|
|
47
59
|
setTimeout(() => installed.recover(), 0);
|
|
48
60
|
}
|
|
49
61
|
if (labels) {
|
|
62
|
+
// SAFETY: watchEditors only wraps doRender when present and finds editors by duck typing.
|
|
50
63
|
watcher = watchEditors(tui as unknown as ShimTarget, {
|
|
51
64
|
style: (label) => ctx.ui.theme.fg("accent", ctx.ui.theme.bold(label)),
|
|
52
65
|
});
|
|
@@ -54,12 +67,45 @@ export default function piOrcaImages(pi: ExtensionAPI) {
|
|
|
54
67
|
return { render: () => [], invalidate() {} };
|
|
55
68
|
});
|
|
56
69
|
ctx.ui.setWidget(WIDGET, undefined);
|
|
70
|
+
if (preview) {
|
|
71
|
+
const shown = new Carousel();
|
|
72
|
+
carousel = shown;
|
|
73
|
+
ctx.ui.setWidget(
|
|
74
|
+
PREVIEW_WIDGET,
|
|
75
|
+
(tui, theme) => {
|
|
76
|
+
previewTui = tui;
|
|
77
|
+
return createCarouselWidget({
|
|
78
|
+
// Read through the closure, never a stale session's ctx: watcher is reset on shutdown.
|
|
79
|
+
labels: () => (carousel === shown ? watcher?.current() : undefined),
|
|
80
|
+
carousel: shown,
|
|
81
|
+
theme: {
|
|
82
|
+
muted: (text) => theme.fg("muted", text),
|
|
83
|
+
fallback: (text) => theme.fg("warning", text),
|
|
84
|
+
},
|
|
85
|
+
});
|
|
86
|
+
},
|
|
87
|
+
{ placement: "aboveEditor" },
|
|
88
|
+
);
|
|
89
|
+
}
|
|
57
90
|
});
|
|
58
91
|
|
|
92
|
+
if (preview) {
|
|
93
|
+
const step = (delta: 1 | -1) => () => {
|
|
94
|
+
carousel?.step(delta);
|
|
95
|
+
previewTui?.requestRender();
|
|
96
|
+
};
|
|
97
|
+
pi.registerShortcut("ctrl+shift+left", { description: "Previous pasted image preview", handler: step(-1) });
|
|
98
|
+
pi.registerShortcut("ctrl+shift+right", { description: "Next pasted image preview", handler: step(1) });
|
|
99
|
+
}
|
|
100
|
+
|
|
59
101
|
pi.on("session_shutdown", async (event) => {
|
|
102
|
+
previews?.uninstall();
|
|
103
|
+
previews = undefined;
|
|
60
104
|
// The watcher styles labels through this session's ctx, which goes stale on replacement.
|
|
61
105
|
watcher?.uninstall();
|
|
62
106
|
watcher = undefined;
|
|
107
|
+
carousel = undefined;
|
|
108
|
+
previewTui = undefined;
|
|
63
109
|
// On /resume, /new, /fork and /reload Pi draws the next transcript before session_start,
|
|
64
110
|
// so the repaint fix must stay in place; the next session reuses it (installShim is
|
|
65
111
|
// idempotent per TUI). Only quitting removes it.
|
|
@@ -80,11 +126,22 @@ export default function piOrcaImages(pi: ExtensionAPI) {
|
|
|
80
126
|
});
|
|
81
127
|
|
|
82
128
|
pi.registerCommand(COMMAND, {
|
|
83
|
-
description: "Inline images in Orca: status, test, or
|
|
129
|
+
description: "Inline images in Orca: status, test, redraw, open, or open N for [Image N]",
|
|
84
130
|
getArgumentCompletions: completeSubcommand,
|
|
85
131
|
handler: async (args, ctx) => {
|
|
86
|
-
const sub =
|
|
87
|
-
if (sub === "
|
|
132
|
+
const { sub, image } = { image: undefined, ...parseCommand(args) };
|
|
133
|
+
if (sub === "open" && image !== undefined) {
|
|
134
|
+
const path = watcher?.pathOf(image);
|
|
135
|
+
if (!path) {
|
|
136
|
+
ctx.ui.notify(`No [Image ${image}] in the prompt or the last sent message.`, "warning");
|
|
137
|
+
return;
|
|
138
|
+
}
|
|
139
|
+
try {
|
|
140
|
+
await openInPreview(path);
|
|
141
|
+
} catch (error) {
|
|
142
|
+
ctx.ui.notify(`Could not open [Image ${image}]: ${error instanceof Error ? error.message : String(error)}`, "error");
|
|
143
|
+
}
|
|
144
|
+
} else if (sub === "status") {
|
|
88
145
|
ctx.ui.notify(
|
|
89
146
|
formatStatus({
|
|
90
147
|
decision,
|
|
@@ -92,11 +149,15 @@ export default function piOrcaImages(pi: ExtensionAPI) {
|
|
|
92
149
|
shim,
|
|
93
150
|
orcaVersion: process.env.ORCA_APP_VERSION ?? process.env.TERM_PROGRAM_VERSION,
|
|
94
151
|
labels: labels && watcher !== undefined,
|
|
152
|
+
preview: preview && carousel !== undefined,
|
|
95
153
|
}),
|
|
96
154
|
"info",
|
|
97
155
|
);
|
|
98
156
|
} else if (sub === "test") {
|
|
99
157
|
pi.appendEntry(TEST_ENTRY, { width: 240, height: 120 });
|
|
158
|
+
} else if (sub === "open") {
|
|
159
|
+
if (previews) await previews.openLatest();
|
|
160
|
+
else ctx.ui.notify("Image previews are not active in this terminal.", "warning");
|
|
100
161
|
} else if (sub === "redraw") {
|
|
101
162
|
if (shim) shim.recover();
|
|
102
163
|
else ctx.ui.notify("pi-orca-images is not active in this terminal; nothing to redraw.", "warning");
|
package/src/labels.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { pathToFileURL } from "node:url";
|
|
1
2
|
import { isExistingFile, parseImagePaths } from "./image-paths.ts";
|
|
2
3
|
|
|
3
4
|
/** The parts of pi-tui's Editor this module wraps. */
|
|
@@ -25,8 +26,23 @@ export interface LabelOptions {
|
|
|
25
26
|
isImageFile?: (path: string) => boolean;
|
|
26
27
|
}
|
|
27
28
|
|
|
29
|
+
export interface ImageEntry {
|
|
30
|
+
id: number;
|
|
31
|
+
path: string;
|
|
32
|
+
}
|
|
33
|
+
|
|
28
34
|
export interface ImageLabels {
|
|
29
35
|
readonly count: number;
|
|
36
|
+
/** Path of `[Image N]` in the draft, or in the last sent message that had labels. */
|
|
37
|
+
pathOf(id: number): string | undefined;
|
|
38
|
+
/** Pasted labels still in the draft, in text order. */
|
|
39
|
+
entries(): ImageEntry[];
|
|
40
|
+
/** Id of the draft label the cursor touches (inside, or right before or after), if any. */
|
|
41
|
+
cursorLabel(): number | undefined;
|
|
42
|
+
/** Labels created so far; grows on every image paste, across messages. */
|
|
43
|
+
readonly pastes: number;
|
|
44
|
+
/** Id of the most recently created label. */
|
|
45
|
+
readonly lastPasted: number | undefined;
|
|
30
46
|
uninstall(): void;
|
|
31
47
|
}
|
|
32
48
|
|
|
@@ -57,7 +73,10 @@ export function installImageLabels(editor: LabelTarget, options: LabelOptions):
|
|
|
57
73
|
};
|
|
58
74
|
const ownKeys = Object.keys(original).filter((key) => Object.hasOwn(editor, key));
|
|
59
75
|
const paths = new Map<number, string>();
|
|
76
|
+
let sent = new Map<number, string>();
|
|
60
77
|
let next = 1;
|
|
78
|
+
let pastes = 0;
|
|
79
|
+
let lastPasted: number | undefined;
|
|
61
80
|
|
|
62
81
|
const known = (text: string) => [...text.matchAll(LABEL)].filter((m) => paths.has(Number(m[1])));
|
|
63
82
|
|
|
@@ -72,6 +91,8 @@ export function installImageLabels(editor: LabelTarget, options: LabelOptions):
|
|
|
72
91
|
const labels = parsed.paths.map((path) => {
|
|
73
92
|
const id = next++;
|
|
74
93
|
paths.set(id, path);
|
|
94
|
+
pastes++;
|
|
95
|
+
lastPasted = id;
|
|
75
96
|
return labelText(id);
|
|
76
97
|
});
|
|
77
98
|
const leading = parsed.leading || (spaceBefore ? " " : "");
|
|
@@ -124,20 +145,30 @@ export function installImageLabels(editor: LabelTarget, options: LabelOptions):
|
|
|
124
145
|
return original.expandPasteMarkers.call(this, expanded);
|
|
125
146
|
};
|
|
126
147
|
|
|
148
|
+
/** Labels still in the draft text; `paths` also keeps labels the user deleted. */
|
|
149
|
+
const inDraft = () => new Map(known(editor.getText()).map((m) => [Number(m[1]), paths.get(Number(m[1])) ?? ""]));
|
|
150
|
+
|
|
127
151
|
editor.submitValue = function () {
|
|
152
|
+
const submitted = inDraft();
|
|
128
153
|
try {
|
|
129
154
|
return original.submitValue.call(this);
|
|
130
155
|
} finally {
|
|
156
|
+
if (submitted.size > 0) sent = submitted;
|
|
131
157
|
paths.clear();
|
|
132
158
|
next = 1;
|
|
133
159
|
}
|
|
134
160
|
};
|
|
135
161
|
|
|
162
|
+
// Each known label is an OSC 8 link to its file: Cmd-click opens it in terminals that support links.
|
|
136
163
|
editor.render = function (width: number) {
|
|
137
164
|
const lines = original.render.call(this, width);
|
|
138
165
|
if (paths.size === 0) return lines;
|
|
139
166
|
return lines.map((line) =>
|
|
140
|
-
line.replace(LABEL, (label, id: string) =>
|
|
167
|
+
line.replace(LABEL, (label, id: string) => {
|
|
168
|
+
const path = paths.get(Number(id));
|
|
169
|
+
if (path === undefined) return label;
|
|
170
|
+
return `\x1b]8;;${pathToFileURL(path).href}\x07${options.style(label)}\x1b]8;;\x07`;
|
|
171
|
+
}),
|
|
141
172
|
);
|
|
142
173
|
};
|
|
143
174
|
|
|
@@ -145,7 +176,27 @@ export function installImageLabels(editor: LabelTarget, options: LabelOptions):
|
|
|
145
176
|
get count() {
|
|
146
177
|
return paths.size;
|
|
147
178
|
},
|
|
179
|
+
pathOf(id: number) {
|
|
180
|
+
return inDraft().get(id) ?? sent.get(id);
|
|
181
|
+
},
|
|
182
|
+
entries() {
|
|
183
|
+
return known(editor.getText()).map((m) => ({ id: Number(m[1]), path: paths.get(Number(m[1])) ?? "" }));
|
|
184
|
+
},
|
|
185
|
+
cursorLabel() {
|
|
186
|
+
const cursor = editor.getCursor?.();
|
|
187
|
+
if (!cursor || paths.size === 0) return undefined;
|
|
188
|
+
const line = editor.getText().split("\n")[cursor.line] ?? "";
|
|
189
|
+
const hit = known(line).find((m) => cursor.col >= m.index && cursor.col <= m.index + m[0].length);
|
|
190
|
+
return hit ? Number(hit[1]) : undefined;
|
|
191
|
+
},
|
|
192
|
+
get pastes() {
|
|
193
|
+
return pastes;
|
|
194
|
+
},
|
|
195
|
+
get lastPasted() {
|
|
196
|
+
return lastPasted;
|
|
197
|
+
},
|
|
148
198
|
uninstall() {
|
|
199
|
+
// SAFETY: every key restored below is one of the editor methods captured in `original`.
|
|
149
200
|
const target = editor as unknown as Record<string, unknown>;
|
|
150
201
|
for (const [key, value] of Object.entries(original)) {
|
|
151
202
|
// Restore earlier own wrappers; otherwise fall back to the prototype method.
|
|
@@ -183,6 +234,10 @@ export function findEditors(root: unknown): LabelTarget[] {
|
|
|
183
234
|
|
|
184
235
|
export interface EditorWatcher {
|
|
185
236
|
readonly editors: number;
|
|
237
|
+
/** Path of `[Image N]` in any watched editor, as in {@link ImageLabels.pathOf}. */
|
|
238
|
+
pathOf(id: number): string | undefined;
|
|
239
|
+
/** Labels of the editor that has pasted images in its draft, else of the first watched editor. */
|
|
240
|
+
current(): ImageLabels | undefined;
|
|
186
241
|
uninstall(): void;
|
|
187
242
|
}
|
|
188
243
|
|
|
@@ -215,6 +270,17 @@ export function watchEditors(
|
|
|
215
270
|
get editors() {
|
|
216
271
|
return installed.size;
|
|
217
272
|
},
|
|
273
|
+
pathOf(id: number) {
|
|
274
|
+
for (const labels of installed.values()) {
|
|
275
|
+
const path = labels.pathOf(id);
|
|
276
|
+
if (path !== undefined) return path;
|
|
277
|
+
}
|
|
278
|
+
return undefined;
|
|
279
|
+
},
|
|
280
|
+
current() {
|
|
281
|
+
const all = [...installed.values()];
|
|
282
|
+
return all.find((labels) => labels.count > 0) ?? all[0];
|
|
283
|
+
},
|
|
218
284
|
uninstall() {
|
|
219
285
|
active = false;
|
|
220
286
|
for (const labels of installed.values()) labels.uninstall();
|
package/src/preview.ts
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
import { execFile } from "node:child_process";
|
|
2
|
+
import { createHash } from "node:crypto";
|
|
3
|
+
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
|
4
|
+
import { tmpdir } from "node:os";
|
|
5
|
+
import { join } from "node:path";
|
|
6
|
+
import { pathToFileURL } from "node:url";
|
|
7
|
+
import { promisify } from "node:util";
|
|
8
|
+
import { Image, truncateToWidth } from "@earendil-works/pi-tui";
|
|
9
|
+
|
|
10
|
+
const run = promisify(execFile);
|
|
11
|
+
const EXTENSIONS: Record<string, string> = {
|
|
12
|
+
"image/png": "png", "image/jpeg": "jpg", "image/gif": "gif",
|
|
13
|
+
"image/webp": "webp", "image/bmp": "bmp", "image/tiff": "tiff",
|
|
14
|
+
"image/avif": "avif", "image/heic": "heic",
|
|
15
|
+
};
|
|
16
|
+
|
|
17
|
+
let previewDirectory: string | undefined;
|
|
18
|
+
|
|
19
|
+
/** Retain temporary previews until this module's process exits, including across /reload. */
|
|
20
|
+
export function savePreview(data: string, mimeType: string): string {
|
|
21
|
+
const extension = EXTENSIONS[mimeType];
|
|
22
|
+
if (!extension) throw new Error(`Unsupported preview format: ${mimeType}`);
|
|
23
|
+
if (!previewDirectory) {
|
|
24
|
+
previewDirectory = mkdtempSync(join(tmpdir(), "pi-orca-images-"));
|
|
25
|
+
const directory = previewDirectory;
|
|
26
|
+
process.once("exit", () => {
|
|
27
|
+
try { rmSync(directory, { recursive: true, force: true }); } catch { /* OS temp cleanup is the fallback. */ }
|
|
28
|
+
});
|
|
29
|
+
}
|
|
30
|
+
const directory = previewDirectory;
|
|
31
|
+
const bytes = Buffer.from(data, "base64");
|
|
32
|
+
const path = join(directory, `${createHash("sha256").update(bytes).digest("hex")}.${extension}`);
|
|
33
|
+
writeFileSync(path, bytes, { mode: 0o600 });
|
|
34
|
+
return path;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export async function openInPreview(path: string): Promise<void> {
|
|
38
|
+
if (process.platform !== "darwin") throw new Error("Opening previews requires macOS Preview.");
|
|
39
|
+
await run("/usr/bin/open", ["-a", "Preview", path], { timeout: 10_000 });
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
const PREVIEW = Symbol.for("pi-orca-images.preview");
|
|
43
|
+
/** Set on an Image to keep it without an Open image link, e.g. the pasted-image thumbnail. */
|
|
44
|
+
export const NO_PREVIEW_LINK = Symbol.for("pi-orca-images.no-preview-link");
|
|
45
|
+
type PreviewHost = typeof Image.prototype & { [PREVIEW]?: { uninstall(): void } };
|
|
46
|
+
|
|
47
|
+
export function installPreviews(options: {
|
|
48
|
+
open?: (path: string) => Promise<void>;
|
|
49
|
+
save?: (data: string, mimeType: string) => string;
|
|
50
|
+
onError: (message: string) => void;
|
|
51
|
+
}) {
|
|
52
|
+
const prototype = Image.prototype as PreviewHost;
|
|
53
|
+
prototype[PREVIEW]?.uninstall();
|
|
54
|
+
const originalRender = prototype.render;
|
|
55
|
+
const paths = new WeakMap<Image, string | null>();
|
|
56
|
+
let latest: string | undefined;
|
|
57
|
+
let active = true;
|
|
58
|
+
const open = async (path: string) => {
|
|
59
|
+
try {
|
|
60
|
+
await (options.open ?? openInPreview)(path);
|
|
61
|
+
} catch (error) {
|
|
62
|
+
if (active) options.onError(`Could not open image in Preview: ${error instanceof Error ? error.message : String(error)}`);
|
|
63
|
+
}
|
|
64
|
+
};
|
|
65
|
+
prototype.render = function (width) {
|
|
66
|
+
const lines = originalRender.call(this, width);
|
|
67
|
+
if (!active || (this as { [NO_PREVIEW_LINK]?: boolean })[NO_PREVIEW_LINK]) return lines;
|
|
68
|
+
if (!paths.has(this)) {
|
|
69
|
+
// Pi exposes no image-data accessor; fail closed if its internal fields change.
|
|
70
|
+
// SAFETY: both fields are type-checked below before use.
|
|
71
|
+
const source = this as unknown as { base64Data?: string; mimeType?: string };
|
|
72
|
+
try {
|
|
73
|
+
paths.set(this, typeof source.base64Data === "string" && typeof source.mimeType === "string"
|
|
74
|
+
? (options.save ?? savePreview)(source.base64Data, source.mimeType) : null);
|
|
75
|
+
} catch {
|
|
76
|
+
paths.set(this, null);
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
const path = paths.get(this);
|
|
80
|
+
if (!path) return lines;
|
|
81
|
+
latest = path;
|
|
82
|
+
const label = truncateToWidth("Open image", width);
|
|
83
|
+
return [...lines, `\x1b]8;;${pathToFileURL(path).href}\x07\x1b[4m${label}\x1b[24m\x1b]8;;\x07`];
|
|
84
|
+
};
|
|
85
|
+
const preview = {
|
|
86
|
+
async openLatest() {
|
|
87
|
+
if (latest) await open(latest);
|
|
88
|
+
else options.onError("No image has been rendered yet. Try /orca-images test first.");
|
|
89
|
+
},
|
|
90
|
+
uninstall() {
|
|
91
|
+
if (!active) return;
|
|
92
|
+
active = false;
|
|
93
|
+
prototype.render = originalRender;
|
|
94
|
+
if (prototype[PREVIEW] === preview) delete prototype[PREVIEW];
|
|
95
|
+
},
|
|
96
|
+
};
|
|
97
|
+
prototype[PREVIEW] = preview;
|
|
98
|
+
return preview;
|
|
99
|
+
}
|
package/src/status.ts
CHANGED
|
@@ -7,9 +7,10 @@ export interface StatusInput {
|
|
|
7
7
|
shim: Pick<Shim, "renderPatched" | "focusPatched" | "recoveries"> | undefined;
|
|
8
8
|
orcaVersion: string | undefined;
|
|
9
9
|
labels: boolean;
|
|
10
|
+
preview: boolean;
|
|
10
11
|
}
|
|
11
12
|
|
|
12
|
-
export function formatStatus({ decision, images, shim, orcaVersion, labels }: StatusInput): string {
|
|
13
|
+
export function formatStatus({ decision, images, shim, orcaVersion, labels, preview }: StatusInput): string {
|
|
13
14
|
const action = decision.action === "apply" ? `apply ${decision.protocol}` : decision.action;
|
|
14
15
|
const repaint = shim
|
|
15
16
|
? `repaint fix: ${shim.renderPatched ? "on" : "unavailable"}, focus recovery: ${
|
|
@@ -22,6 +23,7 @@ export function formatStatus({ decision, images, shim, orcaVersion, labels }: St
|
|
|
22
23
|
` Pi image protocol: ${images ?? "none"}`,
|
|
23
24
|
` ${repaint}`,
|
|
24
25
|
` image labels: ${labels ? "on" : "off (PI_IMAGE_LABELS)"}`,
|
|
26
|
+
` image preview: ${preview ? "on" : "off (PI_IMAGE_PREVIEW)"}`,
|
|
25
27
|
` Orca: ${orcaVersion ?? "not detected"}`,
|
|
26
28
|
].join("\n");
|
|
27
29
|
}
|